Job Overview
Location
Remote USA
Job Type
Full-time
Category
Data Science
Date Posted
May 22, 2026
Full Job Description
đź“‹ Description
- • As a Senior Manager, AI Performance & Operations at Devoted Health, you will own the production performance and continuous improvement of the company’s core agentic workflows for service and experience, ensuring every general-availability workflow has clear performance thresholds, reliable monitoring, explicit fallback paths, and a measurable improvement plan.
- • You will serve as the accountable operator for Guide’s highest-volume and highest-risk AI workflows once live, maintaining a portfolio view of each workflow’s status, detecting degradation early, diagnosing root causes, driving corrective action, and verifying that implemented changes improve outcomes.
- • You will support early-stage projects by helping teams define instrumentation, evaluation design, launch criteria, and safe handoff into general availability, acting as the in-house expert for safe and effective AI deployment.
- • You will translate technical changes into plain language for leadership, explaining what broke, why it matters, and who is fixing it, ensuring executive alignment and informed decision-making.
🎯 Requirements
- • Strong production experience with LLMs, RAG, voice AI, agentic orchestration, or equivalent ML systems.
- • Demonstrated experience designing or operating rubrics, scoring frameworks, regression tests, and production-sampling workflows for AI evaluation.
- • Ability to distinguish failure modes such as model reasoning, tool failure, retrieval issues, and workflow logic through diagnostic acumen.
- • Comfort with active experimentation including prompt/context tuning, retrieval redesign, and safe model migrations beyond basic triage.
- • Excellent cross-functional communication skills to convey technical changes clearly in high-accountability, ambiguous environments.
- • Deep personal accountability with a drive to own results rather than just producing recommendations.
🏖️ Benefits
- • Employer-sponsored health, dental, and vision plans with low or no premium.
- • Generous paid time off.
- • $100 monthly mobile or internet stipend.
- • Stock options for all employees.
- • Bonus eligibility for all roles excluding Director and above.
- • Parental leave program and 401K plan.
Skills & Technologies
Senior
Remote
$110k-172k
About Devoted Health, Inc.
Devoted Health, Inc. operates as a Medicare Advantage health plan provider, offering comprehensive coverage, personalized care navigation, and integrated technology to seniors across the United States. The company combines clinical expertise, data analytics, and member support services to coordinate physician visits, prescription management, and preventive care, aiming to improve health outcomes and reduce unnecessary costs for its members.
Get more remote jobs like this
Subscribe to the weekly newsletter for similar remote roles and curated hiring updates.
Newsletter
Weekly remote jobs and featured talent.
No spam. Only curated remote roles and product updates. You can unsubscribe anytime.
