Senior Staff Engineer

📋 Description

• • As a Senior Staff Full Stack Software Engineer at Cloudera, you will be instrumental in shaping the future of AI-powered data workflows. You will play a pivotal role in the Anywhere Cloud team, driving the innovation that delivers a seamless "build your own pipeline, bring your own engine" experience. This means enabling data and AI workloads to operate fluidly across any environment – public clouds, private data centers, and even the edge – without the constraints of vendor lock-in. By leveraging the power of Kubernetes, our hybrid architecture intelligently separates compute and storage, offering customers unparalleled flexibility and optimized infrastructure utilization.
• • Your primary responsibility will be to lead the architecture and delivery of sophisticated AI-powered workflows that form the very core of our product offerings. This leadership extends to defining the overarching technical strategy, establishing rigorous quality and reliability standards, and ensuring the successful end-to-end delivery of systems that translate complex, often ambiguous customer needs into robust, measurable, and privacy-conscious AI experiences. You will be a key collaborator, working hand-in-hand with Product, Design, Data Science, and Go-To-Market teams to bring high-impact features to life at scale.
• • You will own the architectural vision for the AI workflow stack. This involves designing, continuously evolving, and meticulously documenting the entire end-to-end system. Your design will encompass critical components such as prompting mechanisms, retrieval systems, tool/function-calling capabilities, agent orchestration, evaluation frameworks, observability tools, and safety protocols. A key aspect of this ownership is defining clear interfaces, Service Level Agreements (SLAs), and robust versioning strategies to ensure maintainability and scalability.
• • A significant part of your role will be shipping production-ready systems. This means building highly reliable, low-latency services that seamlessly integrate both hosted and self-hosted foundation models, alongside traditional microservices. You will be responsible for the complete lifecycle of features, from the user-facing interface (UI) all the way through to the backend services, ensuring a cohesive and high-quality user experience.
• • To guarantee the predictable and deterministic behavior of our AI-powered data platform, you will implement comprehensive testing frameworks. This includes rigorous unit testing, regression testing, and end-to-end testing. Furthermore, you will establish critical safety guardrails and human-in-the-loop processes. These measures are essential for maintaining accuracy, ensuring ethical AI outputs, and preventing the generation of toxic or harmful content.
• • Optimization for cost and performance will be a continuous focus. You will be responsible for instrumenting systems, analyzing performance metrics, and optimizing unit economics. This includes managing token usage, implementing effective caching strategies, leveraging batching techniques, and exploring model distillation. Performance optimization will involve monitoring and improving p95 latency, throughput, and autoscaling capabilities.
• • Driving data excellence is paramount. You will actively shape data contracts, establish feedback loops for continuous improvement, define labeling strategies, and contribute to feature stores. These efforts are crucial for the ongoing enhancement of model and workflow quality, ensuring our AI capabilities remain cutting-edge and effective.
• • As a Senior Staff Engineer, you will provide significant technical leadership across multiple teams. Your role will involve unblocking complex projects, elevating code and design standards through best practices, and actively mentoring senior engineers, fostering their growth and technical expertise.
• • You will foster strong partnerships across various functions. This includes translating product vision into actionable technical plans, influencing product roadmaps with data-driven insights, and effectively communicating technical trade-offs and strategic decisions to executives and stakeholders.