Data Engineering

– Designed end-to-end data pipelines using Microsoft Fabric, Azure Data Factory, and PySpark across Medallion Architecture (Bronze, Silver, Gold), integrating 5+ source systems.

– Built metadata-driven ETL framework with configuration tables to automate ingestion and transformation, improving scalability and reducing manual effort.

– Developed incremental loading and data validation workflows (PySpark, SQL) to optimize processing and minimize full reloads.

– Configured Azure DevOps for source control, CI/CD, and deployment across Dev/Test/Prod environments.