Have questions that need expert answers? Look no further! As a Data Engineer and Big Data Developer with over 5 years of experience, I'm here to offer personalized, insightful responses to any queries within my field of expertise.

Why Choose My Expertise?

Comprehensive knowledge in Apache Spark, PySpark, Python, SQL, and various Azure services
Proven track record in designing and implementing efficient ETL processes
Experience with Batch and Streaming pipelines
Skills in data pipelines, data modeling, data warehousing, and performance optimization
Ability to integrate machine learning models with ADF batch pipelines

Professional Journey

Wipro Technologies: Developed Big Data solutions for the Retail sector, managing the ingestion and processing of daily sales data using PySpark and Hadoop. Achieved a 30% time-saving through Spark SQL optimization by developing an end-to-end big data pipeline.

Capgemini India: Led the development of an entity management tool for deduplication of customer and policy records. Improved performance by 10% through join optimization, earning accolades from ADNIC CIO and stakeholders.

Michelin India: Deputy Manager - Data Engineer, specializing in Spark-based applications using Databricks for data extraction and transformation. Successfully reduced processing time by integrating ML models and tuning SQL queries. Saved 30% on storage costs by migrating projects from ADLS Gen1 to Gen2.

Technical Skills

Big Data Tools: Spark, PySpark, Performance Tuning, Spark SQL, Hadoop, Hive, Spark Streaming
Languages: SQL, Python
Azure Services: Azure Data Lake Gen2, Azure Blob, Azure Data Factory (ADF), Azure Databricks (ADB)
Databricks Specific: Delta Lake, Delta Tables, Unity Catalog, Autoloader
AWS Services: Redshift, S3, Athena, EMR, Glue
Databases: SQL, Advanced SQL, MySQL, MS SQL Server SSIS, Azure SQL
Data Warehousing: Snowflake Cloud Data Warehouse
Version Control: Git, GitLab, GitHub, Azure DevOps
Methodologies & Tools: Jira, Jenkins, PyCharm, VS Code, pytest, Agile, Scrum, CI/CD
Operating Systems: Windows, Linux
Soft Skills: Communication, Teamwork, Leadership, Analytical Skills

Ready to Get the Answers You Need?

Whether you're tackling complex ETL processes, optimizing performance, or integrating advanced data solutions, I'm here to provide the clarity and guidance you need. Connect with me directly via email at siddharthnahata2396@gmail.com and let's unlock the potential of your data projects together.

Keep Learning, Keep Growing.

Discovery Call( Intro to your Query Only)

Siddharth Nahata