Job Overview
Location
United States
Job Type
Full-time
Category
Software Engineering
Date Posted
May 28, 2026
Full Job Description
đź“‹ Description
- • Lead QA Engineer at Titan AI Inc., responsible for establishing the first formal QA function as the company scales from a handful to hundreds of banking customers, where QA failures pose operational and regulatory risks, not just UX issues.
- • Design and execute evaluation frameworks for LLM and agentic AI outputs across Foundry, Agent Builder, and client-deployed instances; write assertions, define behavioral contracts, and own regression baselines for non-deterministic AI behavior using distributions and confidence intervals.
- • Build and maintain automated test suites (end-to-end, integration, regression) for backend APIs, document ingestion pipelines, AI inference workflows, and frontend surfaces; enforce quality gates in CI/CD, triage production bugs, and own regression tests to prevent recurrence.
- • Produce SOC 2 Type II-compliant test artifacts, audit logs, and process documentation; collaborate with Forward Deployed Engineering on client-side validation and production issue reproduction to meet banking examiner scrutiny.
- • Join a mission-driven AI company building trustworthy small language models and AI bankers for financial institutions, backed by leading fintech investors and operating under strict compliance, audit, and model-risk standards.
- • Opportunity to shape and scale a QA practice from the ground up, with your individual contributions forming the foundation for future team expansion and becoming a first-class input in deployment decisions.
🎯 Requirements
- • Seven or more years in software QA engineering, with at least two years personally testing AI or ML systems.
- • Experience writing test cases against LLM outputs and building evaluation pipelines from scratch using tools like RAGAS, DeepEval, LangSmith, or comparable.
- • Fluency in Python and hands-on experience with automated testing frameworks such as pytest, Playwright, or Selenium.
- • Proven ability to trace failures from application layer to infrastructure in Azure, async systems, and REST APIs without engineer dependency.
- • Experience integrating QA gates into CI/CD pipelines and owning the process end to end.
- • Familiarity with document processing pipelines, multi-agent architectures, RAG validation, or observability tooling such as Arize or Langfuse is advantageous.
🏖️ Benefits
- • Competitive base salary and meaningful equity compensation.
- • Fully remote work arrangement within the United States.
- • Occasional travel to client sites and team offsites for collaboration and engagement.
Skills & Technologies
Python
Azure
REST
Selenium
Pytest
Senior
Remote
About Titan AI Inc.
Titan AI is positioning itself as an “AI operating system for financial services.” It develops AI-driven tools and infrastructure aimed at banking and financial institutions, purportedly enabling automation of workflows, analytics, risk management, and operational processes in finance.
Get more remote jobs like this
Subscribe to the weekly newsletter for similar remote roles and curated hiring updates.
Newsletter
Weekly remote jobs and featured talent.
No spam. Only curated remote roles and product updates. You can unsubscribe anytime.
