About me
I am a software engineer who does data engineering, data science, devops, backend and frontend development. Focus is always on programmatic problem solving. My interest lies in high impact problems that requires combination of data engineering, data science, domain expertise.
At my current job, I head data engineering and data science teams at Goibibo. Before I moved to data engineering and science, I was a C/C++/ARM-Assembly/JS developer working primarily on Webkit ( browser engine ) and built app-development frameworks based on Webkit for General Motors cars.
■ ReBuilt realtime and near realtime >100 TB dataplatform at my current Job.
■ Led a team to build near realtime, reinforcement learning based dynamic pricing system that optimizes discounts to give >20%-50% more business while giving 10%-40% less discounts. It required understanding of economics, data engineering, business domain expertise and data science to solve this problem.
■ Led a team to build realtime Fraud prevention and detection system.
■ Lead a team to do the highly efficient Devops for Kafka cluster( upto 1 GB/Sec I/O), Redshift, Redshift Spectrum, Athena, Glue, Yarn.
■ Built Intelligent invite to increase invite success-rate several times.
■ Cloudera certified Apache Hadoop Developer (CCD-410)
■ Attended "Tracking Challenges of BigData" at Massachusetts Institute of Technology.
■ Designed, Developed and led large scale projects for Automotive industry, Consumer electronics and E-Learning industry.
■ 10+ years in development of Data engineering, data science and embedded software design and development. Strong customer facing skills.
►► Technologies worked on: Hadoop, Spark, Kafka Streams, Kafka, Redshift, Athena, Dynamodb, Impala, Oozie, Sqoop, HBase, Hive, Scala, Python, SQL ( Yeah, It's a very powerful programming language ), Java, R, MySql, C++, Qt, Javascript, D3.js, Node.js, Angular.js, Posgresql.