Home › Apps › Azure Data Engineering Course
Azure Data Engineering Course: Basic to Advanced
A free course that takes you from zero to job-ready Azure data engineer in three levels and about 110 videos. Every module adds a piece to one retail project, so you finish with a real portfolio.
Videos are released level by level. Subscribe to get each new lesson.
Before you start
- Basic: comfortable with a computer and Excel. No cloud or coding needed.
- Intermediate: SQL joins and grouping, basic Python, or the Basic level.
- Advanced: a year of hands-on pipelines, Spark and Git basics.
You need an Azure free account or pay-as-you-go subscription with a budget alert, a Microsoft Fabric trial, Databricks (Free Edition is fine for practice), VS Code, Git and Python 3.11 or newer.
Level 1: Basic
For: Freshers, career switchers, and BI or SQL people new to the cloud. Time: about 6 weeks, 8 modules, about 35 videos. Exam: DP-900 (Azure Data Fundamentals).
Load CSV and API data into a data lake and a SQL database with a scheduled Azure Data Factory pipeline.
| # | Module and lessons | Hands-on lab | Videos |
|---|---|---|---|
| B1 | Data engineering and cloud foundations What a data engineer does; OLTP vs OLAP; batch vs streaming; ETL vs ELT; warehouse vs lake vs lakehouse; IaaS, PaaS, SaaS | Draw the course project architecture | 4 |
| B2 | Azure essentials for data people Free account, subscriptions, resource groups, regions; portal, Cloud Shell, Azure CLI; pricing calculator and budget alerts; Entra ID and RBAC basics | Create a resource group, budget alert and role assignment | 5 |
| B3 | SQL for data engineers SELECT, filters, joins, GROUP BY; window functions; CTEs; DDL vs DML; indexes and keys | Write 20 queries on the retail sample data | 6 |
| B4 | Python for data engineers Variables, lists, dicts, loops, functions; reading CSV and JSON; pandas basics; calling a REST API | Clean a messy CSV and pull data from a public API | 5 |
| B5 | Azure Storage and ADLS Gen2 Blob vs ADLS Gen2; containers, folders, access tiers; CSV, JSON, Parquet, Delta; keys vs SAS vs managed identity | Build raw, processed and curated folders | 4 |
| B6 | Azure SQL Database SQL DB vs Managed Instance vs SQL on VM; DTU vs vCore, serverless; firewall and Entra sign-in; star schema | Create a serverless Azure SQL DB with a star schema | 4 |
| B7 | Azure Data Factory fundamentals Pipelines, activities, datasets, linked services, integration runtimes; Copy activity; triggers; monitoring and alerts | Copy CSV from ADLS to Azure SQL every day | 5 |
| B8 | Basic mini project End-to-end walkthrough; parameters and simple error handling; cost cleanup; showing it on GitHub and LinkedIn | Retail sales batch pipeline | 2 |
You are ready for the next level when you can: Explain the architecture in 2 minutes, run the pipeline, and fix a failed Copy activity from the monitor logs.
Level 2: Intermediate
For: Basic graduates and junior data engineers. Time: about 8 weeks, 9 modules, about 40 videos. Exam: DP-700 (Fabric Data Engineer) part 1, Databricks Data Engineer Associate.
Build a bronze, silver and gold lakehouse in Azure Databricks and Microsoft Fabric, run by Data Factory.
| # | Module and lessons | Hands-on lab | Videos |
|---|---|---|---|
| I1 | Data modelling for analytics Star and snowflake schemas; slowly changing dimensions (SCD 1 and 2); surrogate keys; data vault overview; medallion architecture | Model the retail gold layer | 4 |
| I2 | Advanced Data Factory Parameters and expressions; Lookup, ForEach, If, Until; metadata-driven pipelines; watermark and CDC incremental loads; Mapping Data Flows; self-hosted IR | Incremental load of 10 tables from metadata | 6 |
| I3 | Apache Spark fundamentals Driver, executors, partitions; lazy evaluation; DataFrames and Spark SQL; joins, aggregations, UDFs | Transform sales data with PySpark | 5 |
| I4 | Azure Databricks Workspaces, clusters, serverless; notebooks and Git folders; secrets with Key Vault; Unity Catalog external locations; Jobs | Bronze-to-silver notebooks as a Databricks job | 5 |
| I5 | Delta Lake ACID and the transaction log; MERGE, time travel, schema evolution; OPTIMIZE, liquid clustering, VACUUM; SCD 2 with MERGE | SCD 2 customer dimension in Delta | 4 |
| I6 | Microsoft Fabric for data engineers OneLake, workspaces, capacities; Lakehouse vs Warehouse; shortcuts and mirroring; Fabric notebooks; Dataflows Gen2; Fabric pipelines | Rebuild the medallion pipeline in Fabric | 6 |
| I7 | Synapse and Fabric Warehouse SQL Dedicated vs serverless SQL pools (legacy); T-SQL in Fabric Warehouse; COPY INTO, CTAS; distribution and statistics | Load gold tables into Fabric Warehouse | 3 |
| I8 | Serving and reporting Power BI basics for engineers; semantic models; Direct Lake mode; row-level security handoff | Direct Lake report on the gold layer | 3 |
| I9 | Data quality and testing Null, duplicate and range checks; Great Expectations or Databricks expectations; logging and alerts; reprocessing failed loads | Add quality checks and an alert email | 4 |
You are ready for the next level when you can: Explain when to use Databricks vs Fabric, write an incremental MERGE, and debug a slow Spark job in the Spark UI.
Level 3: Advanced
For: Working data engineers aiming for senior roles. Time: about 8 weeks, 9 modules, about 35 videos. Exam: DP-700 part 2, Databricks Data Engineer Professional.
Run a production-grade platform: streaming and batch, governed, deployed by CI/CD, cost-controlled and ready for AI.
| # | Module and lessons | Hands-on lab | Videos |
|---|---|---|---|
| A1 | Real-time streaming Event Hubs and the Kafka endpoint; Structured Streaming, watermarks, checkpoints; Auto Loader; Fabric Real-Time Intelligence (Eventstream, Eventhouse, KQL) | Live order stream to a real-time dashboard | 5 |
| A2 | Declarative pipelines Lakeflow Declarative Pipelines (formerly Delta Live Tables); expectations; streaming tables and materialized views; Change Data Feed; dbt overview | Rebuild silver and gold as a declarative pipeline | 4 |
| A3 | Governance and security Unity Catalog grants and lineage; Microsoft Purview scans and classification; private endpoints; managed identities; Key Vault; column masks and row filters | Lock down the lakehouse with least privilege | 5 |
| A4 | Performance tuning Partitioning, file sizes, the small-file problem; broadcast joins, skew, AQE; caching; Photon; Spark UI deep dive; Fabric capacity metrics | Make a slow job 5x faster, measured | 4 |
| A5 | CI/CD and DevOps for data Git branching for data teams; Azure DevOps and GitHub Actions; ADF publishing; Databricks Asset Bundles; Fabric deployment pipelines; pytest | Automated dev to test to prod deployment | 5 |
| A6 | Infrastructure as code Terraform and Bicep for ADLS, ADF, Databricks and Key Vault; environments and state; policy and tagging | Deploy the whole platform from one repo | 3 |
| A7 | Monitoring, reliability and cost Azure Monitor, Log Analytics, KQL alerts; Databricks system tables; retries and SLAs; cluster policies, auto-termination, reservations, capacity sizing | Cost and failure dashboard with alerts | 3 |
| A8 | AI-ready data engineering Feature tables; embeddings and vector search pipelines; RAG data preparation; Azure AI Foundry and Fabric data agents; agentic AI over the lakehouse | Product-docs embedding pipeline | 4 |
| A9 | System design and interview prep Design cases (e-commerce, IoT, banking); trade-offs; 50 common interview questions; resume and portfolio review | Mock design interview | 2 |
You are ready for the next level when you can: Whiteboard a full platform, defend Databricks vs Fabric vs Synapse choices, and deploy changes without touching the portal.
Projects
Each level ends with a portfolio project that grows the same retail platform.
| Level | Project | Tools |
|---|---|---|
| Basic | Retail sales batch pipeline: daily store CSVs and a currency API land in ADLS and load into an Azure SQL star schema | ADLS Gen2, ADF, Azure SQL, Python |
| Intermediate | Retail lakehouse: incremental loads into bronze, silver and gold Delta tables, SCD 2 customers, Power BI Direct Lake report | ADF or Fabric pipelines, Databricks or Fabric Spark, Delta Lake, Power BI |
| Advanced | Real-time retail platform: live order stream, governed lakehouse, CI/CD with infrastructure as code, cost dashboard, product Q&A embeddings | Event Hubs, Structured Streaming or Fabric RTI, Unity Catalog, Purview, Terraform, Azure DevOps or GitHub Actions |
Certifications this course prepares you for
- Microsoft DP-900 Azure Data Fundamentals (Basic)
- Microsoft DP-700 Fabric Data Engineer Associate (Intermediate and Advanced); it replaced DP-203
- Databricks Certified Data Engineer Associate (Intermediate) and Professional (Advanced)
Exam names and content change. Always check the official Microsoft Learn and Databricks exam pages before you book.