Kubernetes & Bare Metal Engineer, ISS

📋 Description

• • As a senior individual contributor, you will be instrumental in the design, deployment, and ongoing operation of large-scale bare-metal Kubernetes clusters and their associated platform services within our on-premise data centers.
• • You will take the lead in architecting and implementing new features and capabilities for our Kubernetes clusters, ensuring they meet the evolving needs of our engineering teams.
• • You will assume ownership of critical components of the platform, which may include cluster lifecycle management, sophisticated networking configurations, robust storage integrations, comprehensive observability solutions, or advanced multi-tenancy frameworks.
• • A key aspect of your role will be to champion and drive improvements in the reliability, performance, and security of the Kubernetes platform, which serves as a foundational element for numerous business units across the company.
• • You will act as a technical mentor to other engineers, sharing your expertise and influencing the adoption of best practices within the Infrastructure Shared Services (ISS) team and with our partner engineering organizations.
• • In terms of Platform Design & Architecture, you will be responsible for designing and continuously evolving bare-metal Kubernetes architectures. This includes the intricate details of the control plane, worker nodes, and the seamless integration of networking and storage solutions, specifically mentioning Portworx on Pure Storage arrays (FlashArray/FlashBlade).
• • You will define and standardize the cluster lifecycle management processes, encompassing provisioning, seamless upgrades, and efficient decommissioning. This will involve leveraging tools such as Kubespray, Foreman, and our internal Continuous Deployment (CD) pipelines.
• • A significant contribution will be in the design of multi-tenant, secure clusters. This involves implementing robust Role-Based Access Control (RBAC), integrating with OIDC/SSO for authentication, ensuring namespace isolation, and defining effective quota and limit strategies to manage resource consumption.
• • For Implementation & Operations, you will be hands-on in deploying, operating, and continuously enhancing large-scale bare-metal Kubernetes clusters deployed across multiple data centers, covering development, staging, and production environments.
• • You will implement and maintain the complex cluster networking infrastructure, including Container Network Interface (CNI) plugins (with a preference for Cilium), Border Gateway Protocol (BGP), load balancers, ingress controllers, and multi-rack/Top-of-Rack (ToR) topologies.
• • You will build and maintain GitOps-based workflows, utilizing tools like ArgoCD, and develop CI/CD pipelines to automate the management of cluster add-ons, platform services, and tenant workloads, ensuring consistency and repeatability.
• • Ensuring comprehensive observability of the platform is crucial. This involves implementing and maintaining monitoring solutions using Prometheus, the Elastic stack (ELK), Grafana, and related tooling for metrics, logs, and traces. You will also collaborate with Site Reliability Engineering (SRE) teams to define Service Level Objectives (SLOs) and configure effective alerts.
• • You will participate in a "follow the sun" on-call rotation to ensure the continuous availability of the production system, taking a leading role in incident management and conducting thorough incident postmortems to identify root causes and implement preventative measures.
• • Regarding Reliability, Security & Compliance, you will own and actively improve the reliability and performance of the clusters and their constituent platform components. This includes leading root cause analysis for complex incidents and developing long-term solutions.
• • You will be responsible for implementing and enforcing stringent security best practices for Kubernetes environments, covering secure default configurations, granular RBAC policies, network policies, and robust secrets management.
• • Close collaboration with SRE, Security, and Network Engineering teams will be essential to meet agreed-upon Service Level Indicators (SLIs) and SLOs, and to establish effective support models for our on-premise Kubernetes infrastructure.
• • In terms of Collaboration & Leadership, you will partner closely with Business Unit (BU) engineering teams to facilitate the onboarding and operation of their production use cases on the bare-metal clusters. This includes supporting specific workloads like on-prem GitHub Actions runners, ELK stacks, and KubeVirt deployments.
• • You will provide significant technical leadership on cross-team projects, leading design reviews, authoring comprehensive design documents, and driving decisions that effectively balance the critical factors of reliability, cost-efficiency, and user experience.
• • You will actively mentor junior and mid-level engineers, fostering their growth by sharing best practices in Kubernetes, automation, and production operations.