Senior Software Engineer - Platform Operations

📋 Description

• • As PhysicsX Limited scales its innovative AI-driven simulation software stack, the operational health, cost efficiency, and developer experience of our platform are paramount. We are seeking a Senior Software Engineer to join our dedicated Platform Operations team, a critical function responsible for ensuring our platform is observable, governable, and a pleasure for our engineers to work with.
• • This role is ideal for an experienced engineer with a robust DevOps background who possesses a deep commitment to operational excellence, enhancing developer productivity, and managing the sustainable costs of running infrastructure at scale. You will be instrumental in collaborating with Platform SRE, Core Infrastructure, and product engineering teams to guarantee the platform remains consistently observable, highly efficient, and easily consumable by all users.
• • A core responsibility will be to own and continuously evolve our comprehensive platform monitoring and observability stack. This includes hands-on management of tools like Grafana, Loki for log aggregation, and Mimir (or Prometheus) for metrics, alongside refining associated alerting strategies and dashboarding practices to provide clear, actionable insights.
• • You will be tasked with defining, implementing, and maintaining Service Level Objective (SLO) dashboards and sophisticated alerting policies. These will provide both platform and product teams with transparent and immediate visibility into the health and performance of our systems, enabling proactive issue resolution.
• • A significant part of your role will involve leading the adoption and ongoing operation of Backstage as our internal developer portal. This includes developing and maintaining custom plugins, enriching service catalogue entries, and creating 'golden path' templates designed to streamline workflows and significantly improve the developer experience across the entire PhysicsX engineering organization.
• • You will drive the implementation and refinement of FinOps practices across our platform. This encompasses establishing robust cloud cost attribution mechanisms, developing effective showback and chargeback reporting, and implementing advanced cost anomaly detection across our multi-cloud environments to ensure fiscal responsibility and optimize spending.
• • A key duty will be to implement and maintain stringent Identity and Access Management (IAM) and permissions policies across major cloud providers such as AWS, GCP, and Azure. The focus will be on consistently enforcing the principle of least privilege access to safeguard our infrastructure and data.
• • You will contribute to the development and integration of lightweight yet effective security practices. This includes enhancing audit logging capabilities, integrating vulnerability scanning into our CI/CD pipelines, and implementing policy enforcement using tools like Open Policy Agent (OPA) or Kyverno to ensure compliance and security posture.
• • Collaboration is central to this role. You will work closely with the Core Infrastructure and Runtime teams to surface critical operational insights and ensure that learnings from operational challenges are fed back into the design and architecture of our infrastructure and runtime environments.
• • You will play a vital role in driving our incident response practices. This involves creating comprehensive runbooks, facilitating thorough post-mortem analyses to identify root causes, and spearheading reliability improvement initiatives based on incident learnings.
• • Automation and tooling development are essential. You will contribute by writing scripts and building tools, primarily in Python and potentially Golang, to automate repetitive tasks, reduce operational toil, and enhance the overall operability of the platform.
• • The role requires a proactive approach to identifying and addressing potential bottlenecks or areas for improvement within the platform's operational framework, ensuring scalability and resilience as PhysicsX continues its rapid growth.