Skip to content

Go to Playbook Main Page
Next: File Ingestion Using Autoloader(Python)

Landing to Bronze Ingestion#

Welcome to the Landing to Bronze Ingestion playbook! This section of the playbook is designed to outline the strategies, methods, and tools used to move data from the Landing Zone to the Bronze Layer in our data architecture.

Note: Currently, this playbook focuses on file-based ingestion using Databricks Autoloader. Additional ingestion methods will be documented in future releases.

  1. File Ingestion Overview - File Ingestion from landing zone to Bronze layer design document.
  2. File Ingestion Using Autoloader(Python) - Comprehensive documentation of Databricks Autoloader usage for Bronze layer data ingestion and validation.
  3. File Ingestion Using Autoloader(Python)-Tutorial - - Step-by-step tutorial with screenshots demonstrating how to implement and execute the Autoloader ingestion process in Databricks.

Click here to access the Modular Components

Click here to access official Databricks Autoloader Documentation

Click here to access the repository to view the example: DHO-PoC1/src/dhopoc1/landing_to_bronze.py

Click here to access the repository to view the example: DHO-PoC1/src/dhopoc1/quality_bronze.py

Click here to access the repository to view the example: DHO-PoC1/src/dhopoc1/databricks_utils_bronze.py