Data Architect & Sr Data Engineer · Capgemini India Pvt. Ltd.
Apr 2024 – PresentHyderabad, India
Global Insurer Apr 2024 – Present
- Designed a multi-entity, multi-region lakehouse on Databricks on AWS, keeping each entity's data in its approved region to meet GDPR, local data-protection and data-residency rules.
- Architected a fully metadata-driven ingestion framework: one generic PySpark job reads source and table configuration (keys, load type, SCD type, PII columns), so new sources are onboarded through config, not new code.
- Built medallion pipelines (bronze → silver → gold) with Delta Lake and Auto Loader, including Delta MERGE upserts and SCD Type 2 history for Party, Contract and Claims data.
- Defined personal-data governance in Unity Catalog: PII classification tags, column masks, row filters, group-based grants and lineage for audit.
- Built real-time stream processing with Kafka and PyFlink on EKS, with MongoDB as the operational store and S3 checkpointing for fault-tolerant, replayable processing.
- Set up CI/CD with Jenkins, GitHub and Databricks Asset Bundles across dev, test and prod, and kept production costs in check with job clusters, autoscaling, Photon and Delta table maintenance.
Big Data Practice Leadership Apr 2024 – Present
- Contribute to and manage the Big Data practice in India: analysing demand-fulfilment gaps, forecasting demand and keeping resource utilisation (ARVE) above 90%.
- Partner with recruiters and interview panels on selecting the right people, and run day-to-day resourcing, reskilling, training and project readiness.
- Work with the presales team on proposal management and estimation, preparing RFP/RFI responses and solution architectures.