Senior Data Engineer- (Big data and Data Pipelines)- Delhi, India

📋 Description

• • Architect and own the next generation of Findem’s 3D talent-data lake, ingesting billions of career, company, and market events from MongoDB, S3, Kafka, and server-to-server APIs to power AI-driven talent insights that no competitor can replicate.
• • Design and implement highly-scalable batch and streaming ETL pipelines in Spark, Hadoop, Kafka, and Airflow that transform raw, semi-structured data (JSON, Avro, Parquet, ORC) into analytics-ready models within strict SLAs, ensuring 99.9 % uptime for both internal BI and customer-facing APIs.
• • Leverage AI-assisted coding tools (Cline, Cursor, or similar) to accelerate development—crafting precise prompts, reviewing AI-generated code, and enforcing rigorous testing and documentation standards so every commit is clean, secure, and maintainable.
• • Build self-serve, interactive query engines on Athena/Presto and Redshift that allow data scientists, product managers, and customers to explore 3D talent graphs in seconds, turning ad-hoc questions into actionable hiring strategies without engineering bottlenecks.
• • Create end-to-end data quality and observability frameworks that automatically detect schema drift, duplicate records, and anomalous metrics, pushing alerts to Slack and PagerDuty before downstream consumers are impacted.
• • Partner with machine-learning engineers to productionize feature stores and real-time inference pipelines that feed Findem’s AI ranking models, ensuring sub-second latencies for candidate recommendations across millions of profiles.
• • Establish data governance, lineage, and cataloging practices that satisfy SOC-2 and GDPR requirements while empowering analysts to trace any metric back to its raw source with a single click.
• • Continuously research emerging big-data technologies (Delta Lake, Flink, Iceberg, etc.), run proof-of-concepts, and champion adoption when ROI is proven—keeping Findem two steps ahead of the market.
• • Mentor junior engineers through code reviews, design sessions, and pair programming, raising the technical bar and fostering a culture of experimentation, ownership, and customer obsession.
• • Translate complex business questions from talent acquisition leaders into elegant data solutions that reduce time-to-hire by 30 % and increase pipeline diversity by measurable margins, directly impacting Findem’s growth trajectory.