As a Senior Data Engineer at American Express, I design and optimize scalable, secure, and high-performance data solutions that drive business insights and innovation. With a passion for solving complex data challenges, I specialize in building modern data pipelines and cloud-native architectures on AWS and Google Cloud Platform (GCP), ensuring data reliability, efficiency, and security across global platforms.
🔹 What I Do:
Architect and implement large-scale distributed data systems using tools like Hadoop, Hive, PySpark, and Airflow.
Design highly available and resilient data pipelines that support real-time and batch processing needs.
Leverage SQL, Python, and PostgreSQL for data modeling, transformation, and advanced analytics.
Build and optimize cloud-based solutions to accelerate data access, reduce costs, and enhance system performance on AWS and GCP.
Collaborate closely with cross-functional teams to align data strategies with business objectives and drive innovation.
🔹 My Technical Toolkit:
Cloud Platforms: AWS (S3, Redshift, Glue, EMR, Lambda), GCP (BigQuery, Dataflow, Pub/Sub)
Big Data & Distributed Systems: Hadoop, Hive, PySpark, Kafka
Data Orchestration: Apache Airflow
Databases: PostgreSQL, Oracle, Snowflake
Programming: Python, Shell Scripting, SQL
DevOps & CI/CD: Docker, Kubernetes, Jenkins, Terraform