Azure Data Engineering Cookbook - Second Edition: Get well versed in various data engineering techniques in Azure using this recipe-based guide

Nearly 80 recipes to help you collect and transform data from multiple sources into a single data source, making it way easier to perform analytics on the data

Key Features:

• Build data pipelines from scratch and find solutions to common data engineering problems
• Learn how to work with Azure Data Factory, Data Lake, Databricks, and Synapse Analytics
• Monitor and maintain your data engineering pipelines using Log Analytics, Azure Monitor, and Azure Purview

Book Description:

The famous quote 'Data is the new oil' seems more true every day as the key to most organizations' long-term success lies in extracting insights from raw data. One of the major challenges organizations face in leveraging value out of data is building performant data engineering pipelines for data visualization, ingestion, storage, and processing. This second edition of the immensely successful book by Ahmad Osama brings to you several recent enhancements in Azure data engineering and shares approximately 80 useful recipes covering common scenarios in building data engineering pipelines in Microsoft Azure.

You'll explore recipes from Azure Synapse Analytics workspaces Gen 2 and get to grips with Synapse Spark pools, SQL Serverless pools, Synapse integration pipelines, and Synapse data flows. You'll also understand Synapse SQL Pool optimization techniques in this second edition. Besides Synapse enhancements, you'll discover helpful tips on managing Azure SQL Database and learn about security, high availability, and performance monitoring. Finally, the book takes you through overall data engineering pipeline management, focusing on monitoring using Log Analytics and tracking data lineage using Azure Purview.

By the end of this book, you'll be able to build superior data engineering pipelines along with having an invaluable go-to guide.

What You Will Learn:

• Process data using Azure Databricks and Azure Synapse Analytics
• Perform data transformation using Azure Synapse data flows
• Perform common administrative tasks in Azure SQL Database
• Build effective Synapse SQL pools which can be consumed by Power BI
• Monitor Synapse SQL and Spark pools using Log Analytics
• Track data lineage using Microsoft Purview integration with pipelines

Who this book is for:

This book is for data engineers, data architects, database administrators, and data professionals who want to get well versed with the Azure data services for building data pipelines. Basic understanding of cloud and data engineering concepts will help in getting the most out of this book.

近80個食譜，幫助您從多個來源收集和轉換數據到單一數據源，使數據分析變得更加容易。

主要特點：
- 從頭開始構建數據管道，並找到常見數據工程問題的解決方案
- 學習如何使用Azure Data Factory、Data Lake、Databricks和Synapse Analytics
- 使用Log Analytics、Azure Monitor和Azure Purview監控和維護數據工程管道

書籍描述：
著名的語錄“數據是新的石油”似乎每天都更加真實，因為大多數組織長期成功的關鍵在於從原始數據中提取洞察力。組織在從數據中獲取價值方面面臨的主要挑戰之一是構建高效的數據工程管道，用於數據可視化、輸入、存儲和處理。這本由Ahmad Osama撰寫的極其成功的書籍的第二版為您帶來了Azure數據工程的幾項最新增強功能，並分享了約80個有用的食譜，涵蓋了在Microsoft Azure中構建數據工程管道的常見情景。

您將探索Azure Synapse Analytics工作區Gen 2的食譜，並熟悉Synapse Spark池、SQL Serverless池、Synapse集成管道和Synapse數據流。在本書的第二版中，您還將了解Synapse SQL池的優化技巧。除了Synapse的增強功能外，您還將發現有關管理Azure SQL Database的有用提示，並了解安全性、高可用性和性能監控。最後，本書將帶您了解整體數據工程管道管理，重點關注使用Log Analytics進行監控以及使用Azure Purview追踪數據來源。

通過閱讀本書，您將能夠構建優秀的數據工程管道，並擁有一本寶貴的指南。

學到什麼：
- 使用Azure Databricks和Azure Synapse Analytics處理數據
- 使用Azure Synapse數據流進行數據轉換
- 在Azure SQL Database中執行常見管理任務
- 構建可供Power BI使用的有效Synapse SQL池
- 使用Log Analytics監控Synapse SQL和Spark池
- 使用Microsoft Purview集成管道追踪數據來源的數據來源

適合對象：
本書適合數據工程師、數據架構師、數據庫管理員和數據專業人士，他們希望熟悉用於構建數據管道的Azure數據服務。對雲和數據工程概念的基本理解將有助於充分利用本書的內容。