Services
Video meeting . 15 mins
Priority DM . 2 days reply
Video meeting . 30 mins
Video meeting . 30 mins
Video meeting . 30 mins
Priority DM . 2 days reply
Video meeting . 60 mins
Video meeting . 30 mins
Video meeting . 30 mins
About me
I am a Lead Data Engineer with comprehensive experience in the development of Big Data systems for the provision of a Unified Analytics Platform (Batch & Streaming platforms).
I am responsible for transforming data into informative insights and assisting the company in making data-driven choices, and I have expertise in developing and executing data pipelines.
My experience spans the Health, Insurance, Finance, Reverse Logistics, and FMCG sectors of the economy.
Key Qualifications:
• Creating Big Data ETL Pipelines
• Refine the data lake in preparation for business reporting.
• Establishing a Unified Analytics Platform
• Consider Design Thinking.
• Improve the job execution time.
• Strategy Development and Implementation
• Communication
Technologies:
• MongoDB, Microsoft Azure, Spark, Python, Azure Data Lake Gen2, Azure Databricks, Azure Data Factory, Kafka, Spark Streaming, REST APIs, Snowflake, Django, Google Cloud Plateform
• HDFS, SQOOP, Hive, GitLab, Azure Datawarehouse, Scala, Bitbucket, Jenkins, Agile
Key accomplishments:
• Job execution time was reduced from 8 hours to 2 hours.
• Proactively designing and implementing the Job Failure Message User Story, which reduced roughly 70% of the debugging time required to investigate the failure.
• Technical writer for freeCodeCamp, PlumbersofDatascience, and Towards Data Science on Big Data Internal Working and Optimizations on Medium.
• Worked on optimising the performance of the Spark-based Data Integration framework utilising best practises, resulting in a 40% performance boost.
• English fluency