Sr. Software Development Engineer in Test

Dialpad, Inc.

Job Overview

Location

Vancouver, Canada

Job Type

Full-time

Full Job Description

📋 Description

• As a Sr. Software Development Engineer in Test (SDET) in Agentic QA at Dialpad, you will own the test automation and quality frameworks that support Dialpad’s AI Voice Agent services, ensuring the reliability and performance of AI-driven communication products used by over 70,000 companies globally.
• You will design and build automation tooling for AI/LLM-driven systems, develop evaluation frameworks to measure response quality and hallucination rates, integrate AI quality checks into CI/CD pipelines with sub-15-minute feedback, and drive automation coverage of 80%+ for critical AI workflows using deterministic and probabilistic validation.
• You will partner with Applied AI teams on prompt engineering and model selection, build tooling for LLM observability and debugging, design and execute performance and load tests for AI services, and define and track AI quality KPIs such as task success rates, precision/recall, and latency.
• You will mentor engineers, participate in design and architecture reviews to ensure testability and resilience, and contribute to raising the bar on AI quality engineering practices while working in a collaborative, distributed team environment across time zones.
• Dialpad is an AI-native business communications platform leading the shift to Agentic AI, where AI is embedded in every employee’s workflow to move faster, think bigger, and achieve more, guided by core traits of being Scrappy, Curious, Optimistic, Persistent, and Empathetic.

🎯 Requirements

• 5+ years of experience in software engineering or SDET roles with emphasis on software development
• Strong programming skills in Python (preferred), Java, or JavaScript
• Experience testing distributed, cloud-native SaaS systems and APIs
• Hands-on exposure to LLMs or AI/ML systems (e.g., OpenAI, Claude, Gemini)
• Understanding of non-deterministic systems and probabilistic testing approaches
• Experience building test frameworks and scalable automation systems
• Familiarity with AI evaluation techniques (benchmarking, golden datasets, human-in-the-loop validation)
• Experience with CI/CD pipelines (e.g., Jenkins, GitHub Actions)
• Strong collaboration skills for working across distributed teams and time zones
• Bachelor’s degree in Computer Science or equivalent practical experience

🏖️ Benefits

• Competitive salary range of $150,500–$175,250 CAD for British Columbia, Canada
• Comprehensive benefits and real opportunities for growth
• Access to cutting-edge AI tools and a robust training program to reach full potential
• Inclusive, vibrant office environment designed to cultivate collaboration and connection
• Recognition as a Great Place to Work, with 91% of employees feeling valued and empowered

Skills & Technologies

Python

JavaScript

Java

React

Redis

Senior

Onsite

Degree Required

Ready to Apply?

Apply Externally

You will be redirected to an external site to apply.

Dialpad, Inc.

Visit Website

About Dialpad, Inc.

Dialpad, Inc. provides cloud-based business communications software combining voice, video, messaging, and meetings in a unified platform. The company’s AI-driven solutions include real-time transcription, sentiment analysis, and automated call routing, delivered through desktop and mobile applications for enterprises and contact centers worldwide.

View Company Profile

Get more remote jobs like this

Subscribe to the weekly newsletter for similar remote roles and curated hiring updates.

Weekly remote jobs and featured talent.

No spam. Only curated remote roles and product updates. You can unsubscribe anytime.