Data Platform Engineer

📋 Description

• • As a Data Platform Engineer at Adyen, you will build and maintain the foundational tooling and processes for Adyen’s on-premise Big Data Platforms, which support tens of products, hundreds of developers, and thousands of daily jobs across a massive infrastructure of 500+ nodes, 25k+ CPU cores, 240+ TBs of RAM, and 23+ PBs of storage.
• • Your day-to-day responsibilities include developing and maintaining scalable big data platforms, working with distributed systems (databases, filesystems, compute), improving continuous release and deployment environments, building tools for data discoverability and quality, introducing technologies to reduce time to insight for analysts and data scientists, and developing streaming processing applications and frameworks.
• • You will collaborate closely with data engineers, ML scientists, and DataOps Engineers to operate and enhance big data platforms, with immediate focus on the OLAP ecosystem (Druid, Clickhouse, Pinot) or the Data Engineering ecosystem (data contracts, validations, optimization tooling, Kafka ingestion tooling).
• • Adyen is a global financial technology platform powering payments, data, and financial products for industry leaders like Meta, Uber, H&M, and Microsoft. The company fosters an environment where individuals are empowered to own their careers, tackle unique technical challenges at scale, and deliver innovative, ethical solutions that help businesses achieve their ambitions faster.
• • In this role, you will deepen your expertise in large-scale private cloud systems, distributed data and compute systems (Spark, Trino, Druid), DevOps pipelines, real-time and batch data pipelines (Kafka, Spark streaming), and Kubernetes or Hadoop ecosystems, while contributing to a platform that processes tens of thousands of Spark jobs daily.