Senior Software Engineer II - Applied AI and Evaluations (Remote Eligible)

📋 Description

• • Senior Software Engineer II - Applied AI and Evaluations role focused on owning agent quality end-to-end for Smartsheet's SmartAssist intelligent agent platform, ensuring AI-powered work management tools deliver reliable, high-quality outputs.
• • Day-to-day responsibilities include diagnosing agent failures across factual accuracy, completeness, tone, actionability, and latency; designing evaluation systems; improving quality through prompt/context engineering and RAG tuning; extending evaluation frameworks with scorers, golden datasets, and regression gates; and collaborating with Agent Architecture leads to distinguish quality vs. structural issues.
• • Smartsheet is a leader in work management software, building the next generation of AI-powered tools through SmartAssist, with a mission to empower teams to automate manual work, uncover insights, and scale smarter by creating space for meaningful work.
• • The role offers the opportunity to establish repeatable quality improvement methodologies, drive measurable impact on production AI agents, and grow expertise in LLM evaluation, agent systems, and applied AI at the forefront of enterprise AI innovation.