Passionate Data Engineer with a demonstrated history of working at a Fortune 500 company, Morgan Stanley (Client) providing solution in various big data related problem optimizing, ETL pipelines Have Good hold on technologies like (Apache Spark, Databricks, Snowflake, Greenplum , Python, SQL, Git,Bash scripting). Always willing to work directly with Businesses and connect dots between howto use particular technology in helping business user to solve their problems. Project Experience * ) Cloud Data Migration As part of modernization in firm (Morgan Stanley) we had to move all the data/data process which involved sourcing, integrating, transforming, and securely storing data to cloud . --> Developed in house (Spark based) ETL framework which gives more control over other etl tools currently in market to move data from on premise database (Hive,db2,Greenplum) to (Snowfalke/AzureSQL). --> Optimized data pipelines for performance and scalability, reducing data processing time by 40% and reduced the resources by 60% *) Providing timely Dataset for Consolidated Audit Trail (CAT) reporting -->Explored and Identified the power of Databricks in integrating, transforming, and securely storing data to meet regulatory requirements --> With the help of Databricks helped team to move out of Hadoop Ecosystem, and provide timely datasets for downstream team to generate reports for Consolidated Audit Trail (CAT)