Skip to content

⚠️ Disclaimer
This is the reference implementation and should be used as a guide or for learning purposes only.

Go to Playbook Main Page

Introduction - Delta Live Tables Pipeline#

Welcome to the Delta Live Tables Pipeline playbook! This section of the playbook is designed to give an overview of Delta Live tables (DLT) and outline the core data flow and the main processing stages between the source to the Silver Layer emphasizing the transformation steps, quality controls, and business logic implementation at each stage, showing how data evolves from raw format to business-ready state.

  1. Delta Live Table * Defines what Delta Live Tables are and how they are used

  2. Architecture and flow * Outlines high-level overview of the Delta Live Tables pipeline Below are the POC Implemented Data Flows :

    • User Profile Pipeline

      • This section explains the end-to-end process of file ingestion using Delta Live Tables (DLT) for user profile data by managing patient demographic and health information.

    • Insulin Data Pipeline

      • This section explains the end-to-end process of file ingestion using Delta Live Tables (DLT) for insulin data which records insulin administration events and Tracks dosage and timing information.

  3. Modularization and Error handling