DP-203 Data Engineering on Microsoft Azure

Module 1: Explore compute and storage options for data engineering workloads

• Introduction to Azure Synapse Analytics.
• Describe Azure Databricks.
• Introduction to Azure Data Lake storage.
• Describe Delta Lake architecture.
• Work with data streams by using Azure Stream Analytics.

Lab: Explore compute and storage options for data engineering workloads.

Module 2: Run interactive queries using Azure Synapse Analytics serverless SQL pools

• Explore Azure Synapse serverless SQL pools capabilities.
• Query data in the lake using Azure Synapse serverless SQL pools.
• Create metadata objects in Azure Synapse serverless SQL pools.
• Secure data and manage users in Azure Synapse serverless SQL pools.

Lab: Run interactive queries using serverless SQL pools.

Module 3: Data exploration and transformation in Azure Databricks

• Describe Azure Databricks.
• Read and write data in Azure Databricks.
• Work with DataFrames in Azure Databricks.
• Work with DataFrames advanced methods in Azure Databricks.

Lab: Data Exploration and Transformation in Azure Databricks.

Module 4: Explore, transform, and load data into the Data Warehouse using Apache Spark

• Understand big data engineering with Apache Spark in Azure Synapse Analytics.
• Ingest data with Apache Spark notebooks in Azure Synapse Analytics.
• Transform data with DataFrames in Apache Spark Pools in Azure Synapse Analytics.
• Integrate SQL and Apache Spark pools in Azure Synapse Analytics.

Lab: Explore, transform, and load data into the Data Warehouse using Apache Spark.

Module 5: Ingest and load data into the data warehouse

• Use data loading best practices in Azure Synapse Analytics.
• Petabyte-scale ingestion with Azure Data Factory.

Lab: Ingest and load Data into the Data Warehouse.

Module 6: Transform data with Azure Data Factory or Azure Synapse Pipelines

• Data integration with Azure Data Factory or Azure Synapse Pipelines.
• Code-free transformation at scale with Azure Data Factory or Azure Synapse Pipelines.

Lab: Transform Data with Azure Data Factory or Azure Synapse Pipelines.

Module 7: Orchestrate data movement and transformation in Azure Synapse Pipelines

• Orchestrate data movement and transformation in Azure Data Factory.

Lab: Orchestrate data movement and transformation in Azure Synapse Pipelines.

Module 8: End-to-end security with Azure Synapse Analytics

• Secure a data warehouse in Azure Synapse Analytics.
• Configure and manage secrets in Azure Key Vault.
• Implement compliance controls for sensitive data.

Lab: End-to-end security with Azure Synapse Analytics.

Module 9: Support Hybrid Transactional Analytical Processing (HTAP) with Azure Synapse Link

• Design hybrid transactional and analytical processing using Azure Synapse Analytics.
• Configure Azure Synapse Link with Azure Cosmos DB.
• Query Azure Cosmos DB with Apache Spark pools.
• Query Azure Cosmos DB with serverless SQL pools.

Lab: Support Hybrid Transactional Analytical Processing (HTAP) with Azure Synapse Link.

Module 10: Real-time Stream Processing with Stream Analytics

• Enable reliable messaging for Big Data applications using Azure Event Hubs.
• Work with data streams by using Azure Stream Analytics.
• Ingest data streams with Azure Stream Analytics.

Lab: Real-time Stream Processing with Stream Analytics.

Module 11: Create a Stream Processing Solution with Event Hubs and Azure Databricks

• Process streaming data with Azure Databricks structured streaming.

Lab: Create a Stream Processing Solution with Event Hubs and Azure Databricks.