8379  Reviews star_rate star_rate star_rate star_rate star_half

ETL and Data Orchestration using Azure Data Factory

In this Azure course, participants learn how to design, build, and manage data pipelines using Azure Data Factory. They explore ETL (Extract, Transform, Load) concepts, data movement, transformation...

Read More
By Request
Contact Us For Pricing
Duration 1 day
Course Code WA3715
Available Formats Classroom

Overview

In this Azure course, participants learn how to design, build, and manage data pipelines using Azure Data Factory. They explore ETL (Extract, Transform, Load) concepts, data movement, transformation techniques, and workflow orchestration. Through hands-on exercises, they connect to various data sources, implement dynamic pipelines, and optimize performance for efficient data integration. The course also covers monitoring, debugging, and best practices to ensure robust data processing workflows.

Prerequisites

  • Basic understanding of data integration and ETL concepts
  • Familiarity with SQL and data transformation principles (recommended)
  • Some experience with cloud platforms, preferably Microsoft Azure (helpful but not required)

Course Details

Setup Requirements

  • A computer with an internet connection
  • An active Azure subscription (or a trial account)
  • Azure Data Factory instance set up in the Azure portal
  • Sample datasets for hands-on exercises (provided during the course)

Azure Data Factory Overview

  • Understanding ETL and its role in modern data integration
  • Introduction to Azure Data Factory and its core capabilities
  • Understanding data pipelines and data flows
  • Key concepts: pipelines, activities, datasets, triggers, and linked services
  • Differences between Mapping Data Flow and Wrangling Data Flow

Building Data Pipelines in Azure Data Factory

  • Introduction to data movement activities
  • Connecting to Azure SQL, on-prem SQL Server, and Blob storage
  • Using the Copy Data activity for data migration
  • Implementing the Data Flow and Custom activities
  • Introduction to data transformation activities
  • Working with variables, lookups, and the wait activity

Data Transformation Techniques

  • Handling multiple inputs and outputs in data flows
  • Using schema modifiers for data structuring
  • Applying formatters for data standardization
  • Implementing data cleansing, enrichment, and aggregation
  • Leveraging Mapping Data Flow for advanced transformations
  • Utilizing Wrangling Data Flow for code-free data preparation

Data Orchestration and Workflow Management

  • Managing dependencies and scheduling in Azure Data Factory
  • Implementing parameterized pipelines for dynamic data processing
  • Configuring triggers for automated workflow execution
  • Using parameters and dynamic content for flexible pipeline design
  • Handling errors and retries in workflow automation

Working with Iteration & Conditionals

  • Using ForEach and Until loops for iterative processing
  • Applying filters to refine dataset selection
  • Implementing If Condition and Switch activities for logic branching
  • Handling conditional transformations within data pipelines
  • Best practices for efficient loop execution in ADF

Monitoring and Management in Azure Data Factory

  • Monitoring pipeline execution using the Azure Data Factory Monitoring Hub
  • Understanding pipeline run status, triggers, and alerts
  • Managing resources such as linked services, datasets, and pipelines
  • Implementing best practices for performance optimization and cost management
  • Debugging pipeline failures and optimizing resource utilization
Read Less

Schedule

Upcoming Course Dates

There are currently no scheduled dates for this course. If you are interested in this course, contact us with the form below.

Contact Information

FAQ

How do I get a Microsoft exam voucher?

Pearson Vue Exam vouchers can be requested and ordered with your course purchase or can be ordered separately by clicking here.

  • Vouchers are non-refundable and non-returnable. Vouchers expire 12 months from the date they are issued unless otherwise specified in the terms and conditions.
  • Voucher expiration dates cannot be extended. The exam must be taken by the expiration date printed on the voucher.

Do Microsoft courses come with post lab access?

Most Microsoft official courses will include post-lab access ranging from 30 to 180 calendar days after instructor led course delivery. A lab training key in class will be provided that can be leveraged to continue connecting to a remote lab environment for the individual course attendee.

Does the course schedule include a Lunchbreak?

Lunch is normally an hour-long after 3-3.5 hours of the class day.

What languages are used to deliver training?

Microsoft courses are conducted in English unless otherwise specified.

Reviews

This was effective way to provide a ton of information in a short time period.

This was a good program to get prepared for the solutions architect associate exam.

Both course material and instructor demonstrated a sound foundation on Maximo material

You get detailed labs to guide you through the technical material giving you a hands on method of learning otherwise difficult material.

The exit certified aws course provided a good introduction to the tools available on aws.