Senior Software Engineer - Data Platform

📋 Description

• • Shape the future of TV advertising data at Vibe. You will own the architecture, performance and cost profile of a platform that ingests billions of events per day from our real-time bidding engine, 3rd-party ad-servers, Smart-TV OEMs and MVPDs, turning raw logs into the insights that power campaign optimisation, billing, identity resolution and predictive modelling.
• • Architect for 10× growth. We crossed profitability in 18 months and revenue is compounding 30 % every quarter. Your designs must sustain this trajectory while keeping cloud spend predictable. You will evaluate when to stay on managed services (Athena, MSK, Kinesis) versus when to build in-house (custom Rust/Go services, ClickHouse clusters) and document trade-offs for leadership.
• • Build bullet-proof data pipelines. You will design both batch (Spark on Kubernetes) and streaming (Kafka → Flink → ClickHouse) workflows that guarantee exactly-once semantics, sub-minute freshness for real-time dashboards and hourly roll-ups for finance. You will institute SLAs (99.9 % availability, <0.5 % data loss) and back them with automated canary deploys, chaos testing and cost-aware autoscaling policies.
• • Democratise data access. Partner with Product Managers to expose curated datasets through REST/GraphQL APIs, dbt-generated marts and self-service notebooks. You will create a “data-as-a-product” culture: clear schemas, semantic layer, lineage docs and an internal data catalog so that any engineer can answer “where did this metric come from?” in under two minutes.
• • Drive identity resolution at scale. Leverage probabilistic and deterministic techniques to link CTV, mobile and web identifiers in a privacy-safe way. You will collaborate with Data Science to productionise new identity graphs, optimise storage formats (Parquet/ORC vs. ClickHouse dictionaries) and ensure GDPR/CCPA compliance through differential privacy and user-deletion pipelines.
• • Champion reliability and cost excellence. Implement SLOs, golden signals and anomaly detection (Prometheus, Grafana, Alertmanager). You will run weekly cost reviews, identify hot partitions, right-size clusters and negotiate reserved capacity. Expect to cut infra spend per 1 k events by 20 % within your first year while improving p95 query latency.
• • Mentor and level-up the team. We are a remote-first group of polyglot engineers (Python, Go, Rust). You will conduct architecture reviews, pair program and write internal RFCs that raise the bar for code quality, testing and observability. You will also interview future teammates and help define our engineering ladder.
• • Stay technology-agnostic but opinionated. Today we run Kubernetes on AWS, store 100 TB in S3, query with Athena/ClickHouse and orchestrate with Airflow. Tomorrow we might adopt Iceberg, DuckDB or serverless Spark. You will run proof-of-concepts, write decision records and roll out changes with zero downtime migrations.
• • Influence the product roadmap. Your insights on attribution, reach/frequency capping and audience forecasting will directly shape what we build next. You will present findings to the exec team and see your work on the screen during every major streaming event.