Software Engineer - GPU Inference

📋 Description

• • As a Software Engineer - GPU Inference at BaseTen Inc., you will be the primary owner of the Voice AI inference stack, responsible for bringing state-of-the-art open-source models into production for real-time voice applications across industries such as productivity, customer service, clinical conversation, and education.
• • Your day-to-day work will involve owning end-to-end product areas — from architecture and system design through implementation, rollout, and long-term production operations — designing and operating real-time, large-scale, high-performance model serving systems for STT, TTS, and voice agent workloads with strict SLOs, driving cross-team collaboration with Forward Deployed Engineers, Model Performance Engineers, and Core Product and Training Platform teams, and mentoring teammates through code reviews, design docs, and technical leadership.
• • You will join a small, high-impact founding team focused on Voice AI at Baseten, a rapidly growing AI infrastructure company powering mission-critical inference for innovative AI companies like Cursor, Notion, OpenEvidence, Abridge, Clay, Gamma, and Writer, backed by top-tier investors including BOND, IVP, Spark Capital, Greylock, and Conviction.
• • In this role, you will have the opportunity to shape the future of Voice AI by building the world’s fastest Whisper with streaming and diarization, contributing to TTS inference for models like Orpheus, designing ergonomic APIs and SDKs for self-serve adoption, and enabling continuous training of voice models — all while gaining deep expertise in ML infrastructure, GPU-optimized serving, and production-grade AI systems at scale.