7+ years of IT experience in engineering, analysis, design, development, implementation, maintenance, and support; expertise in creating strategic deployment plans for big data technologies to effectively address the needs for processing big data. Experienced data professional with expertise in data science, analytics, and machine learning. Skilled in cloud technologies, ETL processes, and optimizing data processing. Proficient in database modeling, relational, and NoSQL databases. Strong background in implementing ETL processes using Apache Airflow, SSIS, and AWS services. Five years' worth of experience using Snowflakes, SQL, Databricks, Spark and AWS Cloud services, such as S3, EC2, EMR, Lambda, Athena, Redshift, Glue, Kinesis, Cloud watch, Step Functions etc. Prolonged work on Python data engineering and Spark programming. Utilizes Python and advanced libraries for data extraction, transformation, and automation. Managed projects using industry-standard tools like Jira, Bit Bucket, GitLab, and Confluence. Expertise in creating data streaming with Apache Spark, Spark Streaming, Kafka, and Apache Airflow. Demonstrated proficiency in big data technologies, deploying and managing large-scale data processing systems. Extensive experience in Spark programming and Python data engineering for optimized data processing and transformations. Proven ability to train and mentor teams on big data and cloud technologies. Skilled in data migration, cleansing, and profiling for data accuracy and integrity. Brief exposure to Azure services and successful implementations using AWS Glue. Developed Tableau reports for digital platform insights and performed statistical analysis. Delivered recurring production reports and collaborated on feature enhancements. Automated infrastructure configurations using Terraform for data science and tech-Mobile applications. Troubleshooted data issues using Elastic search in Kibana. Supervised a team of ETL developers in an Agile-scrum setup. Leveraged technical expertise for data-related aspects of BI applications. Improved ETL solutions in SSIS for optimized data retrieval and processing. Optimized SQL queries and Tableau dashboards for performance improvement. Streamlined data collection, processing, and analysis through data pipelines using AWS Lambda and Python scripts. Leveraged Spark and Databricks for large-scale data transformations. Implemented self-service EMR clusters for efficient testing. Maintained Snowflake databases for data consistency and accuracy throughout the pipeline.