Job Overview
Location
Indiana, USA
Job Type
Full-time
Category
Machine Learning Engineer
Date Posted
February 27, 2026
Full Job Description
đź“‹ Description
- • As a Senior Applied AI Engineer at Drata, you will be at the forefront of innovation, driving the quality and effectiveness of our AI systems. This role is deeply rooted in research, experimentation, and rigorous evaluation, focusing on optimizing retrieval strategies, building robust evaluation frameworks, and establishing the scientific bedrock that ensures our AI features deliver accurate, trustworthy, and impactful results.
- • You will play a pivotal role in advancing Drata's mission to redefine compliance automation through the strategic application of AI and General AI technologies. Your work will directly influence how our AI understands and navigates the complex landscape of compliance artifacts, including VRM Agent, AIQA, Trust Agent, policy-to-control mappers, and regulatory summarization.
- • This is a research-intensive position where experimentation and scientific rigor are paramount. You will collaborate closely with AI Engineers, validating and refining approaches before handing them off for productionization. Your ownership will extend to the critical quality metrics and evaluation systems that guarantee our AI performs as promised, ensuring reliability and trust.
- • Your responsibilities will encompass designing and evaluating sophisticated information access and reasoning strategies. This includes exploring and optimizing techniques across Retrieval Augmented Generation (RAG), agentic systems, and traditional Machine Learning approaches. You will delve into areas such as advanced chunking strategies, the selection and tuning of embedding models, hybrid search methodologies, metadata filtering, structured retrieval, effective tool use within AI agents, and the orchestration of multi-step reasoning workflows.
- • You will be instrumental in prototyping cutting-edge Generative AI workflows, including complex agentic systems. These systems will be designed to map and reason over intricate compliance objects, understanding the relationships between controls, risks, and regulatory requirements.
- • Beyond GenAI, you will explore and implement Machine Learning and probabilistic approaches where they offer superior solutions. This may involve developing classifiers, ranking models, graph/link prediction algorithms, calibration techniques, and leveraging weak supervision to enhance model performance and efficiency.
- • A core aspect of your role will be building and maintaining comprehensive evaluation frameworks. This includes curating high-quality golden datasets, defining and implementing automated quality metrics, and establishing robust mechanisms for detecting regressions to ensure continuous improvement and stability.
- • You will implement and fine-tune advanced ranking and reranking systems. This involves working with cross-encoders, LLM-based rerankers, learning-to-rank algorithms, and developing custom scoring functions to optimize the relevance and accuracy of information retrieval.
- • You will conduct rigorous experiments to validate hypotheses, quantify the impact of proposed improvements, and provide data-driven evidence before any changes are rolled out to production. This experimental approach ensures that our AI advancements are both effective and reliable.
- • Debugging failure modes will be a critical part of your work. You will meticulously analyze issues across retrieval, reasoning, and generation stages, developing detailed error taxonomies to systematically address and prevent future problems.
- • Collaboration is key. You will work closely with AI and Software Engineers, ensuring a smooth and effective handoff of validated approaches for productionization, fostering a strong partnership between research and development.
- • Staying at the cutting edge of applied research is essential. You will continuously monitor advancements in RAG, agent systems, LLM evaluation, and relevance modeling, proactively bringing innovative ideas and techniques into the product to maintain Drata's technological leadership.
- • Your contributions will directly shape the future of compliance and security automation, reinforcing Drata's position as a pioneer in the field and driving our mission forward with state-of-the-art technology.
🎯 Requirements
- • 6+ years of experience in applied research, data science, or Machine Learning, with a specialization in Natural Language Processing (NLP), Information Retrieval (IR), or knowledge systems.
- • 2+ years of hands-on experience building or contributing to production AI/ML systems.
- • Strong foundational knowledge in information retrieval, including dense and sparse retrieval techniques, embedding models, and search relevance.
- • Proven experience with Retrieval Augmented Generation (RAG) systems, encompassing chunking strategies, vector databases, and retrieval optimization.
- • Proficiency in evaluation methodologies, including metrics design, golden dataset creation, A/B testing, and statistical analysis.
- • Strong Python programming skills and demonstrated comfort with notebook-driven research workflows.
- • Experience effectively communicating research findings to engineering teams and translating complex insights into actionable recommendations.
- • Bonus: Experience within compliance, legal, or other document-heavy domains.
- • Bonus: Publications or significant contributions in IR, NLP, or RAG evaluation.
🏖️ Benefits
- • Shared Success: Stock equity (RSUs) providing employees with a direct stake in the company's growth and success.
- • Health & Wellness: Comprehensive medical, dental, and vision coverage with up to 100% employer-paid premiums for employees and dependents, complemented by wellness benefits and healthcare concierge services.
- • Financial Well-being: A robust package including a 401(k) plan, company-paid life and disability insurance, tax-advantaged spending accounts, and discounted voluntary benefits.
- • Family Support: Paid Parental Leave after six months of employment, access to Kindbody fertility and family-building benefits, and dedicated leave specialists.
- • Growth & Development: Generous annual stipends for professional and personal development, alongside access to a wide range of internal learning opportunities.
- • Time Off & Flexibility: Flexible vacation policy, paid holidays, and other benefits designed to support rest, rejuvenation, and work-life balance.
Skills & Technologies
Python
REST
Data Science
Senior
Remote
$192k-259k
About Drata, Inc.
Drata is a compliance automation platform that continuously monitors and collects evidence for security frameworks such as SOC 2, ISO 27001, HIPAA, and PCI DSS. It integrates with cloud services, identity providers, and SaaS tools to track controls, manage risks, and generate audit-ready documentation. The company serves technology startups and enterprises seeking to streamline security compliance workflows and maintain continuous monitoring without manual spreadsheets.
