Staff AI Systems Engineer

📋 Description

• • Flock Safety is at the forefront of public safety technology, building a nationwide network that connects communities, law enforcement, businesses, and schools to proactively prevent crime and enhance security. We are a rapidly growing company, backed by significant venture funding and a substantial valuation, driven by a high-performance, low-ego team committed to urgency, collaboration, and bold thinking. Our mission is to build the impossible, and we are seeking exceptional talent to join us in this endeavor.
• • As a Staff AI Systems Engineer, you will play a pivotal role in shaping the future of our emerging product, Night Shift. This innovative AI research assistant is designed to significantly amplify the impact of investigators by automating the time-consuming and repetitive tasks associated with case management. You will be an integral part of the Machine Learning team, collaborating closely with Engineering (Backend, Frontend, Design) in a dynamic, fast-paced environment.
• • This is a unique opportunity to be one of the earliest technical contributors to our agentic AI system architecture. You will have direct ownership of our AI evaluation framework, a critical component for ensuring the quality, reliability, and effectiveness of our AI systems. The ultimate goal is to deliver measurably faster and more accurate leads to law enforcement officers, revolutionizing how cases are worked.
• • Your responsibilities will encompass the design, development, and implementation of advanced AI systems, with a strong emphasis on LLM agents and their integration into production environments. You will be instrumental in defining and refining the architectural patterns for multi-agent systems, ensuring seamless interaction, effective context management, and robust tool utilization.
• • A key aspect of this role involves deep engagement with Retrieval Augmented Generation (RAG) techniques. You will be responsible for implementing and optimizing vector and hybrid search solutions, leveraging technologies like pgvector or turbopuffer, and integrating rerankers to enhance retrieval accuracy. This includes ensuring grounding and attribution for AI-generated claims, building essential guardrails to maintain system integrity and safety.
• • You will also be responsible for developing and maintaining our AI evaluation framework at scale. This includes building both offline and online evaluation harnesses to rigorously measure various aspects of system performance. Key metrics will include search and retrieval performance, recommendation accuracy, system safety and robustness (including security, compliance, and red-teaming), and the critical trade-offs between cost, performance, and latency.
• • Furthermore, you will contribute to the ML platform's evolution, working with backend technologies such as Python and JavaScript (Typescript/Golang are a plus). Experience with web services (Express/FastAPI, REST, SSE, JWTs), cloud infrastructure (AWS, Terraform, VPC, Networking), and backend databases (Postgres, Redis) is essential.
• • Observability will be a core focus, utilizing tools like Prometheus, Grafana, OpenTelemetry, and LangSmith/Langfuse to monitor system health, performance, and identify areas for improvement. Experience with durable execution frameworks (Temporal, Hatchet) and OLAP databases (ClickHouse, Bigquery) is preferred.
• • For candidates with a strong ML inference background, experience with PyTorch, TensorRT, and NVIDIA Triton, particularly in multimodal domains (text, image, video), will be highly valued. Similarly, expertise in compute orchestration using Kubernetes, Prefect, or Ray is a plus.
• • The role demands a proactive approach to identifying and mitigating risks, ensuring the AI systems are not only effective but also secure, compliant, and reliable. You will be instrumental in establishing best practices for agentic AI development and deployment.
• • Your work will directly impact the efficiency and effectiveness of law enforcement agencies, helping them solve crimes faster and more accurately. This is an opportunity to work on cutting-edge AI technology with a tangible, positive impact on public safety.
• • You will be empowered to own the roadmap for the agent evaluation platform, driving its continuous improvement and expansion. This includes leading research and development efforts into areas like lightweight fine-tuned projection layers, specialized embeddings, and multimodal understanding to further enhance system performance on core metrics.
• • The 90-day plan outlines a clear path to success, starting with immersion in the current system, contributing quick wins, establishing foundational evaluation and observability scaffolding, and proposing a robust architecture for the evaluation framework. By 60 days, you will deliver an MVP evaluation harness and take on system features for improvement. Beyond 90 days, you will productionize the platform, making it the single source of truth for quality and safety, and lead deeper R&D threads.
• • If you are passionate about building AI that makes a real-world difference in public safety and thrive on making complex systems measurable, dependable, and fast, we encourage you to apply.