Member of Technical Staff - Safety Lead

📋 Description

• • As a Member of Technical Staff - Safety Lead at ReflectionAI Inc., you will be at the forefront of ensuring the responsible development and deployment of cutting-edge open superintelligence models. Our mission is to build open superintelligence and make it accessible to all, and your role is pivotal in safeguarding this endeavor. You will own the entire red-teaming and adversarial evaluation pipeline, meticulously probing our models for potential failure modes across critical areas such as security vulnerabilities, misuse potential, and alignment gaps. This is a high-impact role where you will act as a crucial gatekeeper, ensuring that every model release meets stringent safety and risk thresholds before it is made available to the public.
• • Your responsibilities will extend to working hand-in-hand with our dedicated Alignment team. You will translate your safety findings and identified risks into concrete, actionable guardrails and mitigation strategies. This collaborative effort is essential to guarantee that our models behave reliably and predictably, even under stress, and strictly adhere to our deployment policies. You will be instrumental in building trust and confidence in our open-weight releases, a cornerstone of our mission.
• • A significant part of your role will involve developing and refining scalable, automated safety benchmarks. These benchmarks will not be static; they will evolve dynamically alongside our model capabilities, moving beyond traditional, fixed datasets to embrace sophisticated, dynamic adversarial testing methodologies. This proactive approach ensures we are continuously identifying and addressing emerging safety challenges.
• • You will be tasked with researching and implementing state-of-the-art jailbreaking techniques and their corresponding defenses. Staying ahead of potential vulnerabilities in the wild is paramount, and your expertise will be critical in anticipating and neutralizing threats before they can be exploited.
• • This role demands a deep technical understanding of Large Language Model (LLM) safety, encompassing a thorough knowledge of adversarial attacks, established red-teaming methodologies, and model interpretability techniques. You will leverage this expertise to design and execute comprehensive safety evaluations.
• • Beyond technical prowess, you will need strong software engineering capabilities. This includes a proven track record in building automated evaluation pipelines or developing large-scale Machine Learning systems. Your ability to engineer robust and efficient systems will be key to scaling our safety efforts.
• • Experience with Reinforcement Learning from Human Feedback (RLHF) and Reinforcement Learning from AI Feedback (RLAIF) and their impact on model safety and alignment is a significant plus. Understanding how these training paradigms influence model behavior under safety constraints will be highly valuable.
• • You will thrive in a fast-paced, high-agency startup environment where a bias toward action is essential. This role requires you to make high-stakes decisions regarding model release and the establishment of safety thresholds, demonstrating both technical judgment and leadership.
• • Your passion for advancing the frontier of artificial intelligence, coupled with a commitment to ethical and safe AI development, will drive your success. You will be part of a small, talent-dense team dedicated to building foundational models that will shape the future of AI. Joining ReflectionAI Inc. means building from the ground up, defining our company's future, and pushing the boundaries of open foundational models. We are committed to providing an environment where you can do the most impactful work of your career, with the assurance that you and your loved ones are well-supported.