About me
I am a Data Engineer with a good history of working with Hive, Apache
Airflow, Prestosql(trino), Argo workflow engine. Exposure to avid cloud
technologies from major cloud service providers like AWS. Understanding of
CI/CD pipelines, GitOps, Containerization, and cluster architecture over
Kubernetes. Maintaining core logging pipelines using, Fluentd, Prometheus,
Statsd, Logstash, and powered production analytics with Kibana and Grafana.
Experience in Python and GoLang has helped me perform well with the above tools.
Here is a list of work that I have done in past 4 years.
- Built a load aware scaling interface for resource intensive python workloads using AWS EKS .
This helped customers to run data science use cases under existing ETL pipelining architecture in a cost effective manner
- Built CLI based internal tooling which helped to avoid
cascading failures, fix issues with data, and restore business as usual in case
of failures.
-Built metadata recommendation bots, built over Argo
orchestration layer.
- Setup ETL pipelines, data warehouse and BI layer at Animall
- Built MLOps pipelines for Data Science team
- Mentored 5 SDE-1 and 2 SDE-2 and many interns, helped them achieve there KPIs and have exponetial growth