Having 6+ years of experience in Azure Databricks, Apache Nifi, Apache spark for performing transformation, validation, profiling on ETL jobs with end-to-end workflow along with working of large amount of structured and unstructured data on azure data lake and Datawarehouse using ADLS gen2 and power CMS technology. Building end to end data pipelines for data processing, transformation and performing validation along with addition of delta load and incremental load along with hands on experience building microservice architecture using python Django , Flask based web frameworks