This job has expired

This position was posted on March 18, 2026 and is likely no longer accepting applications. We've kept it here for historical reference. Check out the similar jobs below!

Technical Data Delivery Lead

Pareto AI, Inc.

Job Overview

Location

US Remote

Job Type

Full-time

Full Job Description

📋 Description

• As a Technical Data Delivery Lead at Pareto AI, you will sit at the core of building human training data pipelines for frontier AI labs, owning the full lifecycle of complex data collection and evaluation workflows from initial scoping to final delivery, directly impacting the quality and performance of cutting-edge AI models developed in collaboration with leading institutions like Anthropic, Stanford, and Character.AI.
• You will design end-to-end data pipelines for advanced AI training methodologies including RLVR, RLHF, SFT, red-teaming, and model evaluation, defining expert sampling strategies, annotation schemas, rubric structures, and QA systems while prototyping novel workflows and making confident tradeoff decisions under evolving requirements.
• You will build, test, and iterate on AI agents that automate critical pipeline tasks such as quality gate review, expert matching, output flagging, and throughput anomaly detection, working closely with engineering to scope capabilities, write reliable prompts and evaluation logic, and monitor performance in production environments.
• You will establish and enforce data quality standards across annotation and evaluation processes by designing audits using inter-rater reliability metrics, calibration sets, and statistical sampling, while creating automated checks, structured output validation, and model-assisted review layers to prevent issues before they occur.
• You will serve as the primary technical interface with AI researchers and technical program managers at client organizations, translating research-driven requirements into operational workflows, communicating pipeline performance clearly, escalating technical risks early, and contributing to project scoping and pricing decisions.
• You will stay current with advancements in LLM post-training, evaluation methodology, and data tooling, evaluating and integrating innovations like model-assisted annotation and automated calibration into active pipelines to improve quality and efficiency based on domain-specific applicability.
• You will lead and mentor a team of project managers responsible for day-to-day execution tracking, ensuring alignment between technical delivery and client expectations while fostering a culture of ownership, precision, and continuous improvement.
• You will operate in a highly ambiguous, fast-moving environment where model requirements and client priorities shift frequently, requiring comfort with uncertainty and the ability to drive outcomes without rigid guidelines.

🎯 Requirements

• Proficiency in Python and SQL for data manipulation, pipeline monitoring, and quality analysis, including writing scripts to parse formats, run statistical checks, and build lightweight tooling
• Working knowledge of LLM internals such as RLHF/SFT training loops, prompt structure effects on output distribution, and RL environment setup qualities for tool use in agentic data collection and evaluation projects
• Hands-on experience with at least one agentic or LLM workflow framework (LangChain, DSPy, AutoGen, direct tool-use via API, or equivalent)
• Demonstrated ownership of a data or ML pipeline from scoping through delivery, including quality design beyond mere throughput tracking
• Strong written communication skills to create technical guidelines and rubrics for distributed expert workers and brief senior researchers on pipeline performance
• Comfort operating with ambiguity in a fast-moving environment where model requirements evolve and client priorities shift

🏖️ Benefits

• Fully remote work arrangement based in the US, enabling flexibility and work-life balance while collaborating with a distributed team and global AI research partners
• Opportunity to work at the forefront of AI development with direct collaboration with leading AI labs and research institutions including Anthropic, Character.AI, Stanford, and University of Pennsylvania
• High-impact role where you design and operate agentic systems that automate data pipeline tasks, shaping the future of ethical AI training and model evaluation
• Professional growth in a fast-paced, innovative environment that values demonstrated ability over credentials, encouraging ownership, technical depth, and systems thinking
• Access to cutting-edge tools and methodologies in LLM post-training, data tooling, and agentic AI, with support to experiment and integrate novel approaches into production pipelines
• Collaborative culture focused on fairness and ethics in AI development, where your work contributes to building equitable opportunities for global talent in AI training

Skills & Technologies

Python

Senior

Remote

Ready to Apply?

Apply Externally

You will be redirected to an external site to apply.

Pareto AI, Inc.

Visit Website

About Pareto AI, Inc.

Pareto AI, Inc. develops data-science software that automates lead research and outbound sales targeting for B2B companies. Its platform aggregates public and proprietary datasets, applies machine-learning models to identify high-intent prospects, and delivers ranked lead lists directly to CRMs. Customers configure ideal customer profiles and receive continuously refreshed contacts, firmographics, and buying signals, reducing manual research time and improving campaign conversion rates. The company serves SaaS, fintech, and professional-services teams seeking scalable, data-driven pipeline growth.

View Company Profile

Get more remote jobs like this

Subscribe to the weekly newsletter for similar remote roles and curated hiring updates.

Weekly remote jobs and featured talent.

No spam. Only curated remote roles and product updates. You can unsubscribe anytime.