Data Scientist, Core Experimentation

OpenAI, Inc.

Job Overview

Location

Seattle

Job Type

Full-time

Full Job Description

📋 Description

• Lead the statistical direction and technical strategy for OpenAI’s core experimentation platform, ensuring rigorous, scalable, and trustworthy methods for product and research decision-making.
• Design and improve advanced experimentation methodologies including variance reduction techniques (e.g., CUPED), sequential testing, sample ratio mismatch detection, bias mitigation, and heterogeneous treatment effect analysis.
• Build pragmatic, production-grade analytical systems and pipelines in Python and distributed computing environments (e.g., Spark) to support high-volume experimentation across AI products.
• Improve the reliability of experiment results by identifying and preventing data quality failures, logging errors, metric misalignment, and systemic biases in experimentation workflows.
• Partner closely with engineering, product, and infrastructure teams to embed statistical best practices into experiment design, metric definition, and decision-making processes.
• Investigate and resolve complex anomalies in experiment outcomes, conducting root cause analyses of measurement failures and systemic measurement drift.
• Develop scalable data models and instrumentation pipelines that enable accurate tracking of treatment effects in large-scale machine learning systems, ranking architectures, and marketplace platforms.
• Establish and evangelize experimentation governance frameworks, statistical standards, and interpretation guidelines to ensure consistency and integrity across all teams.
• Translate advanced causal inference and statistical concepts into accessible product features and tooling that empower non-statisticians to design, run, and interpret experiments confidently.
• Mentor junior data scientists and raise the overall technical bar for experimentation and causal reasoning across the organization.
• Contribute to the architectural evolution of the experimentation platform, balancing statistical rigor with operational simplicity and usability for diverse technical users.
• Drive innovation in experimentation for frontier AI systems, including novel challenges in online learning, adaptive systems, and high-dimensional treatment spaces.
• Collaborate cross-functionally to define and validate key performance indicators for AI product experiments, ensuring metrics align with both business outcomes and scientific validity.
• Maintain deep awareness of state-of-the-art research in online experimentation, causal inference, and platform scaling, and integrate applicable advancements into OpenAI’s infrastructure.
• Communicate statistical insights and technical trade-offs clearly to technical and non-technical stakeholders through documentation, presentations, and workshops.
• Ensure experimentation systems are robust to real-world production complexities such as non-stationary user behavior, infrastructure variability, and concurrent experiment interference.
• Own the end-to-end lifecycle of critical experimentation initiatives—from hypothesis formulation and design to analysis, deployment, and impact measurement.
• Support product and research teams in designing experiments for AI models with non-traditional outcomes, including those involving generative systems, reinforcement learning, and human feedback.

🎯 Requirements

• Experience building, scaling, or operating experimentation platforms at a large technology company
• Deep expertise in statistics, causal inference, and online experimentation methodology
• Strong coding and systems skills in Python and large-scale data processing frameworks (e.g., Spark)
• Experience with variance reduction, CUPED, sequential testing, SRM detection, metric design, or heterogeneous treatment effects
• Proven track record of influencing technical strategy through hands-on technical leadership
• Experience in large-scale product experimentation, ML experimentation, ranking systems, marketplace systems, or similar high-scale experimentation domains

🏖️ Benefits

• Compensation range of $293K - $325K USD
• Hybrid work model based in Bellevue with emphasis on in-person collaboration for technical design and cross-functional partnership
• Opportunity to shape the future of experimentation in the AI era at a leading AI research and deployment company
• Work alongside world-class engineers and researchers on frontier AI systems with global impact

Skills & Technologies

Python

Apache Spark

Data Science

Hybrid

$293k-325k

Ready to Apply?

Apply Externally

You will be redirected to an external site to apply.

OpenAI, Inc.

Visit Website

About OpenAI, Inc.

OpenAI is a San Francisco-based artificial intelligence research and deployment company founded in 2015. It develops large-scale AI models such as GPT, DALL-E, and Codex, providing cloud APIs and consumer applications like ChatGPT. Originally established as a non-profit, it later created a capped-profit subsidiary to attract capital while maintaining its mission to ensure artificial general intelligence benefits all of humanity.

View Company Profile

Get more remote jobs like this

Subscribe to the weekly newsletter for similar remote roles and curated hiring updates.

Weekly remote jobs and featured talent.

No spam. Only curated remote roles and product updates. You can unsubscribe anytime.