People Scientist

📋 Description

• • At Maki People, the Science team is at the forefront of revolutionizing the hiring landscape, driven by a commitment to innovation, scientific rigour, and collaborative spirit. As a People Scientist, you will play a pivotal role within this dynamic team, contributing directly to the development and enhancement of our cutting-edge assessment platform. This role sits at the critical intersection of psychometrics, artificial intelligence, and advanced research, ensuring that our automated scoring systems are not only scientifically sound but also fair, transparent, and continuously evolving.
• • Your primary responsibility will be to ensure the scientific robustness and ethical integrity of Maki's automated scoring systems. This involves a deep dive into the selection, evaluation, and ongoing refinement of both AI and psychometric models. You will meticulously assess the statistical accuracy, reliability, and validity of Large Language Models (LLMs) employed for automated scoring across various assessment types, including structured grids, job-specific skills evaluations, and multi-lingual proficiency tests in both written and spoken formats.
• • A significant part of your work will involve comparing and validating Speech-to-Text (STT) and Text-to-Speech (TTS) models, understanding their downstream impact on candidate scores and overall assessment fairness. You will be tasked with continuously identifying and evaluating emerging LLM, STT, and TTS models to optimize scoring precision, efficiency, and to stay ahead of technological advancements.
• • Furthermore, you will be instrumental in evaluating and calibrating established psychometric models, such as Classical Test Theory (CTT), Item Response Theory (IRT), and Confirmatory Factor Analysis (CFA). This ensures the scientific validity, reliability, and comparability of AI-scored assessments across diverse populations and different test forms, maintaining the highest standards of psychometric practice.
• • A key area of focus will be the comparison between AI-scored assessments and expert human judgments. You will design and execute research studies to validate AI scoring against human benchmarks, ensuring alignment and trustworthiness. This includes benchmarking semantic and embedding models like BERT, GPT-4, MPNet, and DeepSeek for their efficacy across a wide range of assessment types. You will also contribute to the development of hybrid scoring pipelines that effectively integrate human oversight with AI-driven analytics, leveraging the strengths of both.
• • Addressing bias and ensuring fairness is paramount. You will be responsible for detecting and meticulously analyzing potential biases in AI-generated or psychometric scores across various demographic groups. This involves applying advanced fairness and bias-mitigation techniques, such as reweighting, calibration, and subgroup analysis, while rigorously maintaining model performance integrity. Your contributions will extend to developing and maintaining internal fairness dashboards and comprehensive compliance documentation, fostering transparent model governance and adherence to ethical AI principles.
• • You will also engage in predictive analytics to derive meaningful performance insights. Working with large-scale datasets encompassing assessment results and actual job performance data, you will model the complex relationships between candidate scores, job performance, and employee retention outcomes. This will involve developing and rigorously testing predictive models designed to estimate success probabilities and identify key behavioural and linguistic predictors of high performance.
• • Collaboration is key to translating these insights into tangible value. You will work closely with our data science, implementation, and customer success teams to translate complex analytical findings into actionable recommendations for our clients and internal stakeholders, enabling them to make more informed talent decisions.
• • The role also encompasses ongoing model monitoring and issue resolution. You will investigate anomalies and discrepancies identified by clients or through internal Quality Assurance processes. This requires conducting thorough diagnostic analyses and recommending evidence-based improvements to ensure the continued accuracy and reliability of our assessment tools.
• • As a People Scientist, you will also be involved in technical research that bridges the gap between AI and psychometrics. This includes exploring advanced techniques such as fine-tuning, prompt engineering, and novel evaluation methods to further enhance model performance. A crucial aspect of this research is the ability to translate complex technical findings into clear, actionable insights for non-technical stakeholders, ensuring widespread understanding and adoption of our scientific advancements.
• • You will also contribute to the dissemination of research findings through internal reports, academic publications, or presentations at industry conferences, positioning Maki People as a thought leader in the field. As one of the early employees at Maki People, you will have a unique opportunity to shape the future direction of the Science team and the company's product, sharing ownership and contributing to our collective success.