I am a Senior Data Engineer with extensive experience in building Big Data systems to provide a Unified Analytics Platform (Batch & Streaming platforms). With expertise in conceptualizing and implementing data pipelines, I am responsible for converting data into informational insights thus helping the organization to make data-driven decisions. My experience in various industries includes Retail, Networking, Manufacturing Logistics Energy & Utilization domains. ☑️ Key Competencies: ➡️ Designing Big Data ETL Pipelines ➡️ Refine data lake for Business Reporting ➡️ Building a Unified Analytics Platform ➡️ Design Thinking ➡️ Optimize the Job Execution Time ➡️ Strategy Planning and Implementation ➡️ Communication ☑️ Technologies: ➡️ Microsoft Azure, Spark, Python, Azure Data Lake Gen2, Azure Databricks, Azure Data Factory, Kubernetes, Kafka, Spark Streaming, Event Hubs, IoT, Apache Druid, REST APIs ➡️ HDFS, SQOOP, Hive, GitLab, Azure Datawarehouse, Scala, Bitbucket, Jenkins, Control-M, Maven, Agile ☑️ Key Achievements: ➡️ Minimized the execution time of jobs from 8 hours to 2 hours ➡️ Proactively proposing & implementing Job Failure Message User Story which approximately saved 70% of debugging time to analyze the failure. ➡️ Technical Writer at Medium for freeCodeCamp, PlumbersofDatascience, Towards Data Science on topics related to Big Data Internal working and Optimizations. ➡️ Worked on the optimization of Spark-based Data Integration framework performance using best practices which resulted in 40% performance improvement. ➡️ Fluent in English, German Please contact me at jayvardhan.reddy.v@gmail.com to find out how I might consult, contribute and add value to your organization.