Data Scientist

📋 Description

• • Join a pioneering AI evaluation platform that is fundamentally changing how the world's leading AI laboratories assess and benchmark the performance of their cutting-edge models. This is a unique opportunity to be at the forefront of AI development, contributing to a product that has become the industry standard for understanding real-world AI performance.
• • As a Data Scientist, you will be instrumental in driving data-informed decision-making across the organization. Your primary focus will be on optimizing user engagement and product growth through rigorous experimentation, sophisticated causal inference, and in-depth retention analytics.
• • You will be responsible for the end-to-end design, implementation, and analysis of experiments. This includes traditional A/B tests, multi-armed bandits for dynamic optimization, and quasi-experimental methods to understand the impact of product changes in various contexts. You will collaborate closely with product managers, engineers, and marketing teams to meticulously define hypotheses, establish key success metrics, and ensure adequate statistical power for all tests.
• • A significant part of your role will involve applying advanced causal inference techniques. This means going beyond simple correlations to accurately estimate treatment effects, even in non-randomized settings. Methods such as difference-in-differences, propensity score matching, synthetic control, and regression discontinuity will be part of your analytical toolkit. Ensuring the statistical validity of your findings, including controlling for biases, applying multiple testing corrections, and providing robust confidence intervals, will be paramount.
• • You will also own the development and refinement of comprehensive retention measurement frameworks. This involves leveraging techniques like cohort analysis, survival analysis, and predictive modeling to understand user lifecycle and identify drivers of churn. You will define, track, and diagnose trends in core engagement metrics such as Daily Active Users (DAU), Weekly Active Users (WAU), Monthly Active Users (MAU), and various forms of rolling and N-day retention.
• • Identifying the key factors influencing user retention will be a critical objective. You will achieve this through detailed segmentation, funnel analysis, and the application of predictive modeling. Your insights will directly inform growth teams, helping them to optimize critical user journeys, including onboarding flows, engagement loops, and monetization strategies.
• • To support these analytical efforts, you will contribute to building and maintaining scalable data pipelines. Proficiency in tools like PySpark, SQL, and other big data technologies will be essential for processing and analyzing vast datasets efficiently. You will also play a role in developing automated dashboards and reports using tools such as Tableau, Looker, or Metabase to provide real-time visibility into experiment performance and retention trends.
• • Data quality and consistency are foundational. You will ensure that metric definitions are standardized across teams and that the data used for analysis is accurate and reliable. Furthermore, you will focus on optimizing queries and computations for performance and cost-efficiency within distributed computing environments like Databricks, AWS EMR, or GCP BigQuery.
• • Cross-functional collaboration is key to success in this role. You will act as a trusted advisor, partnering with product, engineering, and marketing teams to translate complex business questions into actionable data-driven analyses. Presenting your findings and strategic recommendations to executive stakeholders in a clear, concise, and compelling manner will be a regular part of your responsibilities.
• • You will also have the opportunity to mentor junior data scientists and analysts, sharing your expertise and promoting best practices in experimentation and retention analytics within the team. This role offers a chance to make a tangible impact on a product that is shaping the future of artificial intelligence, working alongside a passionate and mission-driven team backed by top-tier investors.