Job Overview
Location
Atlanta; Boston; Charlotte; Chicago; Dallas; Los Angeles; New York; San Francisco
Job Type
Full-time
Category
Data Science
Date Posted
June 20, 2026
Full Job Description
đź“‹ Description
- • Design and execute structured evaluations for AI-enabled features and workflows to assess output quality across groundedness, instruction adherence, consistency, usefulness, tone, control compliance, and risk.
- • Identify AI hallucinations, unsupported assertions, missing logic, and unsafe recommendations in generated outputs.
- • Build and maintain golden datasets, benchmark prompts, comparison sets, and scorecards to standardize AI quality measurement across releases.
- • Develop and refine scoring rubrics that enable consistent, repeatable evaluation of AI outputs against nuanced, non-binary criteria.
- • Compare performance across prompt versions, workflow revisions, tools, and AI models to detect quality regression or improvement.
- • Support release decisions with evidence-based analysis of AI performance changes and quality trends.
- • Collaborate closely with Finance SMEs, product managers, and engineers to define what acceptable AI output looks like in real business contexts, particularly within finance, accounting, FP&A, and transaction services.
- • Help establish human-review thresholds and escalation patterns for high-risk AI use cases involving financial data or decision-making.
- • Analyze live production feedback, reviewer overrides, and quality signals to continuously improve evaluation coverage and accuracy.
- • Work in a hybrid model with flexibility to work remotely up to two days per week, based in one of Accordion’s U.S. offices: Atlanta, Boston, Charlotte, Chicago, Dallas, Los Angeles, New York, or San Francisco.
- • Document evaluation findings with precision and maintain rigorous standards for evidence-based reviews.
- • Operate effectively in ambiguous environments while maintaining disciplined scoring, documentation, and process adherence.
- • Communicate complex quality issues and recommendations clearly to cross-functional teams including Engineering and business stakeholders.
- • Contribute to a growing team focused on driving future growth in AI quality assurance within private equity finance consulting.
- • Demonstrate entrepreneurial spirit and comfort in a fluid, flat organizational structure with high autonomy and ownership.
- • Manage multiple complex workstreams simultaneously while maintaining focus on individual deliverables.
- • Lead by example, mentor others, and foster a collaborative team environment centered on trust, business reality, and client service excellence.
🎯 Requirements
- • 4+ years of experience in QA, analytics, business process validation, AI evaluation, operations, or similar roles
- • Strong writing, analysis, and pattern-recognition skills
- • Experience evaluating outputs against nuanced criteria rather than only binary correctness
- • Experience with finance, accounting, FP&A, transaction services, or business process design preferred
- • Ability to work with structured rubrics, scenario libraries, and evidence-based reviews – thoughtful, precise, and highly discerning
- • Ability to spot subtle output problems others miss
🏖️ Benefits
- • Annual salary range of $110,000 to $145,000
- • Bonus compensation
- • Hybrid work model with up to two remote days per week
- • Opportunity to work with 1,600+ finance and technology experts in a collaborative environment
- • Exposure to private equity clients and CFO-focused value creation initiatives
- • Equal opportunity employer with commitment to diversity, equity, and inclusion
Skills & Technologies
See exactly how your profile matches this role — strengths, skill gaps, and what to do about them.
About Accordion Partners, LLC
Accordion Partners is a private equity-focused financial consulting firm providing portfolio company value creation services. It delivers interim management, financial planning & analysis, operational improvement, and transaction support to middle-market PE-backed businesses across North America. Founded by ex-Bain and Deloitte professionals, the company embeds senior-level talent within client organizations to accelerate EBITDA growth, optimize cash flow, and prepare assets for exit. Services span strategic finance, accounting, treasury, pricing, supply chain, and M&A integration, delivered through a flexible staffing model that scales with deal timelines and investment horizons.
Subscribe to the weekly newsletter for similar remote roles and curated hiring updates.
Newsletter
Weekly remote jobs and featured talent.
No spam. Only curated remote roles and product updates. You can unsubscribe anytime.
Similar Opportunities
1 month ago
2 months ago
Hangar Aviation Technologies, Inc.
2 months ago
Singular Intelligence Ltd.
17 days ago