Senior Data Engineer

📋 Description

• • As a Senior Data Engineer with a focus on AI/ML at Tebra, you will be instrumental in architecting, building, and maintaining the specialized data infrastructure that underpins Tebra’s intelligent features, driving innovation in healthcare technology.
• • You will serve as a key technical subject matter expert in data systems, collaborating closely with Machine Learning Engineers to transform raw, often complex, healthcare data into high-quality training sets and real-time inference features essential for advanced AI models.
• • This is a highly hands-on role where you will take ownership of significant data sub-systems, translating intricate business requirements into robust software solutions that accelerate the deployment of AI capabilities across the organization.
• • You will be tasked with tackling complex technical challenges, ranging from sophisticated data versioning strategies to efficient feature serving mechanisms, ensuring that our Machine Learning models are consistently fed by reliable, scalable, and high-performance data pipelines.
• • Your primary responsibilities will include architecting and writing software solutions designed to solve challenging business problems, with a specific emphasis on creating scalable pipelines for feature extraction, generating comprehensive training data, and managing model monitoring logs.
• • You will own and act as the Subject Matter Expert (SME) for critical large-scale software systems, such as the organization's Feature Store or Data Lakehouse, ensuring seamless and efficient data availability for both experimental research and production inference environments.
• • A key aspect of your role will involve continuously monitoring data pipelines in production, proactively detecting data drift or anomalies in data quality, and implementing automated recovery systems to guarantee the ongoing reliability and freshness of features and training data.
• • You will lead Engineering Design Reviews, articulating well-reasoned explanations for architectural decisions, such as choosing between batch processing for model training versus real-time streaming for inference, ensuring optimal performance and scalability.
• • You will develop and implement software frameworks that are designed for extensibility, enabling other team members to leverage them for tasks like automated data quality checks and schema validation tools, thereby preventing common issues like training-serving skew.
• • A core function will be translating high-level business requirements into concrete software solutions, effectively bridging the gap between diverse raw data sources and the precisely structured inputs required for sophisticated Machine Learning models.
• • You will possess a deep understanding of when and how to optimize complex code, specifically tuning performance-critical components like Spark jobs or SQL queries to efficiently handle the massive datasets often required for advanced applications such as Large Language Model (LLM) fine-tuning or deep learning models.
• • You will foster cross-functional collaboration, working closely with ML engineers and other stakeholders to implement and champion MLOps best practices, including robust data versioning, comprehensive lineage tracking, and ensuring reproducibility of ML experiments and deployments.
• • You will demonstrate expertise in scoping tasks effectively, breaking down complex data infrastructure initiatives into manageable, actionable deliverables for the engineering squad, ensuring project clarity and efficient execution.
• • This role is pivotal in ensuring Tebra can leverage its vast healthcare data assets to build cutting-edge AI features that enhance practice well-being and patient care, making a tangible impact on the future of healthcare delivery.
• • You will contribute to the strategic direction of Tebra's data platform, influencing technology choices and best practices that will shape the company's data-driven future.
• • By ensuring the quality and accessibility of data, you will directly enable faster iteration cycles for ML models, leading to quicker delivery of valuable AI-powered features to Tebra's customers.
• • You will play a crucial role in maintaining the integrity and performance of production data systems, minimizing downtime and ensuring the consistent availability of critical data for both operational and analytical needs.
• • Your work will involve staying abreast of the latest advancements in data engineering and AI/ML infrastructure, bringing innovative solutions and approaches to Tebra's data challenges.
• • You will mentor junior engineers and share your expertise, contributing to the overall growth and technical excellence of the data engineering team.
• • The ability to work independently and take initiative on complex problems will be essential for success in this role, driving forward critical data initiatives with minimal oversight.
• • You will be a key contributor to building a scalable, reliable, and performant data foundation that supports Tebra's ambitious growth and innovation goals in the healthcare technology sector.