Senior Data Engineer

📋 Description

• • Architect and own the upstream data platform that fuels every strategic decision at PURE Insurance. You will design resilient ingestion pipelines that pull change-data-capture (CDC) feeds from core policy, claims, and billing systems, stream events from Salesforce and Workday, and batch-load third-party catastrophe and telematics data—delivering terabytes of clean, timely data to the silver and gold layers of our lakehouse.
• • Build and tune distributed, production-grade Spark jobs on Databricks, leveraging Delta Lake, DLT pipelines, and Unity Catalog to guarantee exactly-once semantics, schema evolution, and millisecond-level SLAs for real-time pricing models and fraud-detection algorithms.
• • Establish enterprise-class data quality and governance frameworks: write automated tests in Great Expectations, enforce row/column-level security with Unity Catalog and Lakehouse Federation, and maintain end-to-end lineage that auditors and data scientists can trust.
• • Partner daily with Analytics Engineers who model in dbt; you provide rock-solid, well-documented datasets so they can focus on business logic instead of plumbing. Together you iterate on medallion architecture patterns, ensuring raw → curated → analytics-ready data flows are optimized for both cost and performance.
• • Champion software-engineering best practices: peer-review every PR in GitHub, containerize micro-services with Docker, deploy via GitHub Actions + Terraform to AWS (S3, Glue, Lambda, Kinesis), and monitor with Datadog dashboards and SLO alerts.
• • Mentor a growing team of mid-level and junior engineers through pair programming, brown-bag sessions, and architecture guilds. Your code reviews set the bar for readability, testability, and performance, and you actively sponsor teammates to speak at meetups and conferences.
• • Experiment fearlessly: prototype Iceberg tables, evaluate Flink for ultra-low-latency streams, or spin up a Rust-based data-quality micro-service when the use case demands it. We reward curiosity and measure success by business impact, not buzzwords.
• • Translate complex technical trade-offs into plain English for stakeholders in Claims, Underwriting, Actuarial, and Marketing. You’ll run quarterly “data days” to showcase new datasets, gather feedback, and co-create the next quarter’s roadmap.
• • Ensure disaster-recovery and compliance posture by automating cross-region replication, encryption at rest/in transit, and SOC-2 evidence collection. You treat security as a feature, not an afterthought.
• • Contribute to open-source—whether it’s a dbt package for insurance-specific transformations or a Spark connector enhancement—and represent PURE at industry events, reinforcing our reputation as a destination for elite data talent.