AI Red Tester

📋 Description

• • As a Global AI Red Tester at Distro, Inc., you will be at the forefront of securing the future of artificial intelligence. This pivotal role involves a deep dive into the vulnerabilities of AI models, including large language models (LLMs), multimodal systems, and autonomous agents. You will be instrumental in designing and executing sophisticated adversarial testing campaigns, meticulously probing for weaknesses that could be exploited in real-world scenarios. Your work will directly contribute to enhancing the security posture of our AI deployments and safeguarding against emerging threats.
• • A significant portion of your responsibility, approximately 50%, will be dedicated to conducting comprehensive red team exercises. These exercises will simulate realistic deployment scenarios and explore edge cases, ensuring that our AI systems are robust and resilient. You will systematically investigate AI models for bias, toxicity, misinformation generation, and other harmful outputs across a diverse range of contexts and demographic considerations. This proactive approach is crucial for identifying and mitigating potential risks before they impact users or the business.
• • You will play a key role in researching and identifying novel attack vectors and prompt injection techniques. This involves staying ahead of the curve in understanding how AI models can be manipulated or compromised. By identifying these vulnerabilities, you will contribute to the development of advanced defense strategies and help prevent jailbreaks and unintended model behaviors. Furthermore, you will evaluate and recommend the adoption of AI security tools designed to significantly reduce the mean time to detect and respond to AI-specific threats, thereby strengthening our overall security infrastructure.
• • Another critical aspect of your role (10%) will be the creation of adversarial datasets and benchmarks. These resources will be essential for rigorously evaluating the robustness of AI models under various simulated attack conditions. By developing standardized testing methodologies and datasets, you will ensure consistent and effective evaluation of AI security across different models and applications.
• • Collaboration is at the heart of this position. You will work closely with global security teams, including the Global Security Operations Center (GSOC), member firm security teams, and development teams, to conduct red teaming activities. This collaborative effort ensures a holistic approach to AI security. A key deliverable will be the clear and concise documentation and presentation of your findings, including identified vulnerabilities and actionable remediation recommendations. Your ability to communicate complex technical risks to stakeholders across global teams and our network of member firms will be paramount in driving the mitigation of identified risks.
• • You will also be responsible for developing and maintaining essential process documentation related to AI red teaming. This includes creating detailed reports on testing activities, tracking key operational metrics, and performing other programmatic tasks necessary to support the efficient functioning of the AI Red Team. This ensures that our testing efforts are well-organized, repeatable, and measurable.
• • This role offers a unique opportunity to shape the security landscape of AI within Distro, Inc. You will be a trusted advisor, providing critical insights into the security of our AI investments and helping to build a more secure and trustworthy AI ecosystem. Your expertise will be vital in navigating the evolving challenges of AI security and ensuring that our organization remains at the cutting edge of defensive capabilities.
• • The position requires a proactive and analytical mindset, with a passion for uncovering vulnerabilities and a commitment to improving security. You will have the autonomy to explore new testing methodologies and contribute to the strategic direction of AI security within the company. This is an exciting opportunity for a seasoned security professional to make a significant impact in a rapidly growing field.