Staff Software Engineer - Golang

📋 Description

• • As a Staff Software Engineer specializing in Golang within Cloudera's Anywhere Cloud team, you will be at the forefront of cloud innovation, driving the development and enhancement of a cutting-edge platform designed to deliver a seamless "build your own pipeline, bring your own engine" experience for data and AI workloads.
• • Your primary focus will be on architecting, building, and refining scalable backend systems and APIs that are critical to customer adoption and expansion, ensuring these services are robust, performant, and enterprise-ready.
• • You will play a pivotal role in driving the performance, reliability, and security of our Kubernetes-based backend services, contributing directly to the stability and efficiency of the entire platform.
• • A key responsibility will be the implementation of comprehensive testing frameworks, encompassing unit, regression, and end-to-end tests. This rigorous approach will guarantee deterministic and predictable behavior from our AI-powered data platform, establishing essential safety guardrails and human-in-the-loop processes to maintain accuracy and ensure the production of ethical, responsible, and non-toxic outputs.
• • You will be instrumental in optimizing for cost and performance by instrumenting, analyzing, and refining unit economics, including token usage, caching strategies, batch processing, and distillation techniques. Furthermore, you will focus on performance metrics such as p95 latency, throughput, and autoscaling mechanisms.
• • Driving data excellence is paramount; you will actively shape data contracts, feedback loops, labeling strategies, and feature stores to foster continuous improvement in model and workflow quality, ensuring our AI capabilities remain state-of-the-art.
• • As a Staff Engineer, you will provide significant technical leadership across multiple teams, actively unblocking complex projects, elevating code and design standards, and mentoring other senior engineers to foster a culture of technical growth and excellence.
• • You will collaborate closely with cross-functional teams, translating product intent into actionable technical plans, influencing roadmaps with data-driven insights, and effectively communicating complex technical trade-offs to executives and stakeholders.
• • This role involves a deep dive into the AI/ML ecosystem, requiring an understanding of LLMs, vector databases, RAG, and prompt engineering, with a focus on integrating these technologies into our cloud platform.
• • You will contribute to the security and privacy posture of the platform by applying a security-first mindset, with familiarity in data governance, PII handling, tenant isolation, and compliance considerations.
• • The position offers the opportunity to design reusable AI workflow primitives, SDKs, or internal platforms that will be leveraged by numerous product teams, amplifying your impact across the organization.
• • You will be involved in building and maintaining robust tracing, metrics, and logging systems for AI applications, and gain familiarity with quality dashboards and prompt diff tooling.
• • A significant aspect of the role involves advanced Kubernetes design and operation, including optimizing performance through techniques like node affinity, resource limits, and horizontal pod autoscaling, as well as implementing service mesh solutions and developing custom resource definitions (CRDs).
• • You will also gain experience in managing machine learning workloads on container orchestration platforms like Kubernetes, which includes setting up GPU resources, orchestrating distributed training jobs, and deploying models at scale.
• • The Anywhere Cloud team's mission is to bring the best of public cloud capabilities—cost efficiency, scalability, elasticity, and agility—to wherever data resides, whether that's public clouds, private data centers, or the edge, all powered by Kubernetes and a hybrid architecture that separates compute and storage for maximum flexibility and optimized infrastructure usage.
• • By simplifying Kubernetes and cloud management, especially with the integration of Taikun, Cloudera is creating a unified, scalable, and future-ready platform, and your contributions will be central to this evolution.
• • This role is ideal for an engineer passionate about building and managing complex distributed systems at the core of hybrid cloud environments, with a strong emphasis on performance and DevOps principles.