Staff AI Engineer (Orchestration)

📋 Description

• • As a Staff AI Engineer on the Orchestration team at Heidi Health, you will be at the forefront of revolutionizing healthcare by developing and deploying a clinician-grade AI Care Partner. This pivotal role involves owning the end-to-end design and delivery of a sophisticated retrieval and question-answering stack, ensuring its reliability, safety, and effectiveness in critical clinical environments.
• • You will architect and implement a comprehensive data ingestion and indexing pipeline, meticulously handling literature and clinical guidelines. This includes defining the architecture for ingestion, normalization, metadata extraction, de-duplication, and versioning of vast amounts of medical information, ensuring data integrity and accessibility.
• • A core responsibility will be building and optimizing a hybrid search and retrieval system. This advanced system will integrate lexical search, vector embeddings, and sophisticated re-ranking mechanisms, all while adhering to strict latency budgets and cost controls to ensure efficient operation.
• • You will be instrumental in designing the grounding and answer synthesis components of the AI Care Partner. This involves ensuring that all generated answers are accurately cited, preserve provenance, and clearly communicate confidence levels and abstention when necessary, fostering trust and transparency with clinicians.
• • Leading model development efforts will be a key aspect of your role. You will work on advanced techniques such as prompting, fine-tuning, distillation, and tool use to enhance the faithfulness, coverage, and overall utility of the AI models, making them more accurate and helpful for healthcare professionals.
• • Establishing and maintaining gold-standard evaluation methodologies is crucial. This includes implementing offline Information Retrieval (IR) metrics like nDCG and MAP, conducting rigorous factuality and faithfulness audits, and overseeing human review processes with expert adjudication to ensure the highest quality outputs.
• • You will drive the execution of online experiments at scale, defining critical guardrails and Key Performance Indicators (KPIs). By shipping A/B tests, you will directly measure the impact of AI interventions on clinician workflows, continuously optimizing for user experience and clinical efficiency.
• • Productionizing these complex AI services requires a strong focus on reliability and observability. You will implement robust monitoring, tracing, canary deployments, rollback strategies, and comprehensive incident playbooks to ensure seamless operation and rapid issue resolution.
• • A significant part of your role will involve defining and enforcing data governance policies for sensitive medical content. This includes establishing access control mechanisms, ensuring compliant handling of Protected Health Information (PHI), maintaining detailed audit logs, and implementing appropriate retention policies.
• • You will collaborate closely with clinical experts to translate their needs into actionable technical specifications. This involves defining user intents, data schemas, and acceptance criteria, converting ambiguous clinical questions into precise, testable requirements for the AI system.
• • As a Staff+ engineer, you will also play a vital role in mentoring and coaching other engineers and scientists. You will elevate the team's technical capabilities through detailed design documentation, thorough code reviews, and the development of reusable components and best practices.
• • This role offers a unique opportunity to bridge the gap between cutting-edge AI research and real-world clinical application, transforming how healthcare is delivered and experienced. You will be empowered to set technical direction, establish quality standards, and lead cross-functional teams to achieve ambitious goals in a fast-paced, impactful startup environment.