Job Overview
Location
United States
Job Type
Full-time
Category
Software Engineering
Date Posted
July 2, 2026
Full Job Description
📋 Description
- • Build LLM-powered features, workflows, and agentic systems that solve real customer problems in security operations.
- • Design and implement evaluation frameworks to measure agent quality, reliability, and business impact in production environments.
- • Create automated benchmarks, regression tests, and datasets for evaluating AI behavior across security workflows.
- • Investigate agent failures and develop systematic approaches to improve performance, accuracy, and trustworthiness.
- • Experiment with prompting, tool use, retrieval, memory, planning, and reasoning strategies to enhance agent capabilities.
- • Build infrastructure that supports rapid experimentation, evaluation, deployment, and monitoring of AI systems.
- • Work closely with customers and internal engineering and product teams to understand security workflows and identify automation opportunities.
- • Contribute to engineering best practices for testing, observability, and production reliability of AI applications.
- • Stay current with advances in large language models, agent architectures, evaluation methodologies, and AI engineering practices.
- • Design and maintain observability systems that detect regressions before customers experience them.
- • Answer critical questions about agent performance: whether agents produce accurate investigations, if changes to prompts or models improve outcomes, and which failure modes matter most.
- • Develop datasets and benchmarks that reflect real customer workflows and security investigation scenarios.
- • Build feedback loops that continuously improve production agent performance based on real-world usage.
- • Contribute to the development of automated testing pipelines for AI systems operating in high-stakes security environments.
- • Ship improvements to production AI systems used by customers within the first six months of employment.
- • Help establish rigorous evaluation practices across the company to ensure reliability and trustworthiness of AI agents.
- • Become a trusted contributor across engineering, product, and AI initiatives by deeply understanding customer workflows and security operations.
🎯 Requirements
- • Strong programming skills, preferably in Python
- • Solid software engineering fundamentals, including testing, debugging, and system design
- • Experience building applications, projects, or products using LLMs and modern AI tools
- • Ability to design experiments, interpret results, and make data-driven decisions
- • Strong communication skills and willingness to collaborate across disciplines
- • Curiosity, ownership, and a desire to learn quickly
🏖️ Benefits
- • Work on some of the most challenging problems in applied AI
- • Help define how agentic systems are evaluated and deployed in production
- • Join a small, highly collaborative team with significant ownership and impact
- • Learn quickly while working alongside experienced engineers, researchers, and security experts
- • Shape the future of AI-powered security operations
Skills & Technologies
See exactly how your profile matches this role — strengths, skill gaps, and what to do about them.
About Crogl Inc.
Crogl is a B2B SaaS platform that provides automated website monitoring, testing, and alerting for uptime, performance, and user-experience issues. It continuously crawls sites, simulates user journeys, and delivers real-time notifications via Slack, email, and webhooks so engineering and product teams can detect and fix problems before customers notice. The service supports synthetic monitoring, transaction checks, page-speed tracking, and detailed analytics dashboards to help organizations maintain reliable web applications and improve overall digital quality.
Subscribe to the weekly newsletter for similar remote roles and curated hiring updates.
Newsletter
Weekly remote jobs and featured talent.
No spam. Only curated remote roles and product updates. You can unsubscribe anytime.
Similar Opportunities
Distro, Inc.
3 months ago
2 months ago
Equifax Inc.
20 days ago
Springer Nature Limited
1 month ago