DevOps Engineer - Event Streaming Platform (m/f/d)

📋 Description

• • As a DevOps Engineer at Statista, you will be instrumental in shaping the future of our data infrastructure by designing, building, and operating a company-wide event streaming platform. This is a pivotal role where you will transform our existing real-time event monitoring system into a robust, scalable, and self-serviceable infrastructure. Your work will empower engineering teams across the organization to leverage real-time data effectively, enabling them to make more informed, fact-based decisions.
• • Your primary responsibility will be to architect and manage a sophisticated event bus built on Apache Kafka. This involves not just setting up the core infrastructure but also continuously evolving it to meet the growing demands of a global data platform. You will delve deep into Kafka's intricacies, including consumer group management, optimal topic design, effective partitioning strategies, and proactive operational troubleshooting to ensure high availability and performance.
• • A significant part of your role will involve defining and maintaining our infrastructure as code (IaC) using AWS Cloud Development Kit (CDK). You will be responsible for provisioning and managing a wide array of AWS services, including Elastic Container Service (ECS) for container orchestration, Lambda for serverless compute, DynamoDB for NoSQL data storage, Virtual Private Cloud (VPC) for network configuration, Identity and Access Management (IAM) for security, and CloudWatch for monitoring and logging. This IaC approach will ensure consistency, repeatability, and auditability across our cloud environment.
• • You will be tasked with building and maintaining cutting-edge CI/CD pipelines. The focus will be on implementing zero-downtime deployment strategies and ensuring that robust rollback capabilities are in place. This will minimize service disruptions and provide confidence in our deployment processes, allowing for rapid iteration and delivery of new features and improvements.
• • A key objective is to create intuitive self-service tooling and comprehensive documentation. This will empower other engineering teams to independently onboard onto the event streaming platform, reducing dependencies and accelerating their ability to integrate with our data streams. Your efforts will foster a culture of autonomy and efficiency within our engineering department.
• • You will be responsible for instrumenting the platform with Datadog, a leading monitoring and analytics platform. This includes defining Service Level Objectives (SLOs) to measure performance and reliability, and building detailed runbooks. These runbooks will serve as essential guides for managing operations at scale, ensuring that issues can be quickly identified, diagnosed, and resolved.
• • Security will be a foundational element of your work. You will embed security best practices from the very inception of the platform. This includes implementing the principle of least privilege for IAM roles, establishing secure secrets management processes, and ensuring proper network segmentation to protect our data and infrastructure.
• • This role offers a unique opportunity to work with a world-leading business data platform, contributing directly to its core technological capabilities. You will be at the forefront of adopting and implementing modern DevOps practices and technologies, driving innovation and efficiency.
• • You will collaborate closely with various engineering teams, understanding their data needs and translating them into scalable platform solutions. This cross-functional collaboration is vital for the success of the event streaming platform and its adoption across the company.
• • The position requires a proactive approach to identifying potential issues and implementing preventative measures. You will be expected to continuously monitor system performance, identify bottlenecks, and optimize resource utilization to ensure cost-effectiveness (FinOps principles).
• • By building this self-service event streaming platform, you will directly contribute to Statista's mission of empowering people worldwide to make fact-based decisions, by providing the underlying data infrastructure that makes this possible.
• • You will have the autonomy to make architectural decisions and drive the technical direction of the event streaming platform, with the support of a talented and collaborative team.
• • This role is perfect for an engineer who thrives in a fast-paced environment, enjoys solving complex technical challenges, and is passionate about building scalable, reliable, and secure infrastructure.
• • Your expertise will be crucial in ensuring that Statista's data infrastructure can keep pace with our rapid global growth and evolving business needs.
• • You will be a key player in migrating and enhancing our existing event monitoring capabilities, ensuring a seamless transition and significant improvements in performance and scalability.
• • The opportunity to work with a diverse and inclusive team, celebrating different perspectives and backgrounds, is a core aspect of Statista's culture, and this role is no exception.
• • You will be instrumental in fostering a culture of operational excellence and continuous improvement within the engineering organization.
• • This role offers a chance to leave a significant technical footprint on a globally recognized data platform, impacting how data is consumed and utilized across the company.
• • You will be responsible for the end-to-end lifecycle of the event streaming platform, from design and implementation to ongoing operation and optimization.
• • The successful candidate will be a hands-on engineer, comfortable with both high-level design and low-level implementation details, ensuring a well-rounded and effective platform.
• • You will contribute to defining and upholding engineering standards and best practices within the DevOps and platform engineering domains.
• • This role is an exciting opportunity to grow your career within a dynamic and data-driven company that is committed to empowering its employees and its users with reliable information.