Sr. Data Engineer

📋 Description

• • Architect and own end-to-end data pipelines that ingest, cleanse, and transform multi-terabyte clinical and device data streams generated by Abbott’s continuous glucose-monitoring ecosystem, ensuring 99.9 % uptime and sub-minute latency for real-time patient insights.
• • Design, implement, and continuously optimize Apache Spark jobs (PySpark, Spark SQL, RDD, Dataset APIs) to process both batch and streaming workloads, driving a 3× performance improvement in data-refresh cycles while maintaining strict HIPAA and GDPR compliance.
• • Build resilient, low-latency streaming architectures using Apache Kafka, AWS Kinesis, or Azure Event Hubs—complete with schema registry, KSQL, and Kafka Connect—to enable real-time alerting for patients and clinicians when glucose levels breach critical thresholds.
• • Develop and maintain stream-processing applications with Apache Flink or Spark Structured Streaming, guaranteeing millisecond-level data availability for downstream AI/ML models that predict hypoglycemic events up to 60 minutes in advance.
• • Collaborate with data scientists to productionize large-scale machine-learning algorithms (gradient boosting, neural networks, time-series forecasting) on Spark clusters, packaging models as RESTful micro-services and deploying them via CI/CD pipelines on AWS or Azure.
• • Create intuitive, interactive visualizations in Tableau, Power BI, or custom React/D3 dashboards that translate complex glucose trends into actionable insights for patients, caregivers, and R&D teams, accelerating product iteration cycles by 25 %.
• • Design and expose secure, versioned REST and GraphQL APIs that allow internal and third-party applications to query live metrics, ensuring backward compatibility and OAuth2-based authentication.
• • Partner with cross-functional squads—product managers, regulatory affairs, cybersecurity, and software engineers—to translate business requirements into scalable data solutions that support Abbott’s mission of helping people with diabetes live fuller lives.
• • Establish data-quality frameworks and automated testing suites (unit, integration, performance) that detect anomalies in glucose readings within seconds, reducing false alarms and improving patient trust.
• • Mentor junior engineers and analysts through code reviews, design sessions, and lunch-and-learns, fostering a culture of continuous learning and innovation across the global Diabetes Care division.
• • Drive cloud-cost optimization initiatives by rightsizing Spark clusters, leveraging spot instances, and implementing auto-scaling policies, achieving a 20 % reduction in annual cloud spend without compromising performance.
• • Contribute to Abbott’s open-source community by publishing reusable Spark libraries, Kafka connectors, and data-quality tools, enhancing the company’s reputation as a technology leader in digital health.