Principal Engineer, Distributed Data Systems

📋 Description

• • CloudZero is seeking a Principal Engineer, Distributed Data Systems to join our innovative and rapidly growing team. This is a pivotal role for an individual with a profound passion for architecting data pipelines at an unprecedented scale. We are not just talking about systems that handle millions of records; we are designing and building solutions that ingest and process the AI workload cost and usage data from the world's leading engineering organizations, ensuring our systems operate flawlessly under immense pressure. Our platform processes billions of cost and usage events daily, spanning across major cloud providers like AWS, Azure, and GCP, alongside an ever-expanding list of SaaS and AI platforms. The accuracy and real-time nature of this data are paramount, as our customers rely on it for critical financial planning and strategic decision-making. Any disruption in our data pipeline directly impacts their ability to plan and operate effectively.
• • We have engineered a unique, fully event-driven, serverless architecture that efficiently processes terabytes of cost data while maintaining exceptional sub-second query performance. However, our rapid growth trajectory necessitates a continuous evolution of our infrastructure. The architecture that has brought us this far will need to be reimagined and scaled to meet the demands of the future. This role requires an engineer who has not only pushed traditional database architectures to their absolute limits but has also thrived in overcoming those challenges. You should possess an innate ability to recognize the elegance in our current design while simultaneously identifying and addressing the architectural boundaries we need to surpass.
• • As a Principal Engineer, your responsibilities will extend far beyond mere system maintenance. You will be instrumental in redefining how data at scale should function. The challenges are substantial, the expectations are exceptionally high, and the opportunity to build a truly enduring and impactful system is immense. Your primary focus will be on architecting for hyper-scale, designing data pipelines capable of supporting customers 5 to 10 times larger than our current biggest clients, all without requiring fundamental architectural rewrites every 18 months. This involves a relentless pursuit of simplicity, building for ultimate reliability by stripping away unnecessary complexity to its core, resulting in fewer moving parts, clearer data flows, and more predictable performance.
• • A significant aspect of this role involves mentoring and influencing other engineers across the organization. You will be expected to share your hard-won knowledge, including lessons learned from performance bottlenecks, architectural dead ends, and the clever solutions that ultimately succeeded. Your experience will be invaluable in leveling up the technical capabilities of our engineering teams. Furthermore, you will drive innovation across all engineering teams, acting as a key voice that challenges conventional thinking about scale, reliability, and performance, not just within the data team but across the entire engineering department.
• • You will play a crucial role in translating complex technical concepts into clear, actionable insights for leadership. Your technical tradeoffs and strategic recommendations will not only shape implementation details but will also directly influence product and business strategy, ensuring that our technical direction aligns with our overarching company goals. This requires a deep understanding of both the technical landscape and the business implications of architectural decisions.
• • We are looking for someone who views code as a potential liability and champions simplicity as a core superpower. You should be adept at explaining complex system behaviors under scale to non-technical stakeholders, such as a CEO, without sacrificing essential technical nuance. A profound ability to think critically about what *not* to build, and to articulate the tradeoffs made during the development process, is essential. You should also possess a natural curiosity for emerging technologies, experimenting with them not for their novelty but for their potential to solve complex problems and push the boundaries of what's possible. Architectural constraints should be viewed as stimulating puzzles to be solved, rather than insurmountable roadblocks.
• • This role is ideal for a seasoned engineer who has a proven track record of building and scaling complex data systems that have reached the limits of their existing tooling. Your experience in early- or growth-stage SaaS companies will be highly valued, providing a strong foundation in navigating the unique challenges and opportunities of a fast-paced environment. A deep, hands-on understanding of large-scale data pipelines and distributed systems is critical, coupled with a robust grasp of system design patterns and the practical, real-world tradeoffs involved in their implementation. Finally, you must be a collaborative and clear communicator, capable of engaging effectively with both technical and non-technical audiences.