Senior Data Engineer

📋 Description

• • Join Zartis as a Senior Data Engineer and become the technical backbone of a high-impact project in the fast-growing simulation-based training industry. You will design, build, and scale the data infrastructure that powers critical decision-making for thousands of end-users worldwide.
• • Own the end-to-end architecture of data ingestion, ETL/ELT, and processing workflows that transform raw, high-volume datasets into clean, analytics-ready assets. Your pipelines will feed dashboards, ML models, and real-time simulations used by trainers, analysts, and product managers.
• • Translate complex analytical logic—often prototyped by data scientists—into production-grade Python services that are fault-tolerant, observable, and cost-efficient. You will refactor notebooks into micro-services, add CI/CD gates, and ensure reproducibility across dev, staging, and production environments.
• • Leverage Azure cloud services (Data Factory, Synapse, Databricks, Cosmos DB, Blob Storage) to build a modern, cloud-native lakehouse that balances performance, security, and cost. You will tune clusters, choose the right storage tiers, and implement autoscaling policies that keep cloud spend predictable.
• • Containerize every workflow with Docker and orchestrate them on Kubernetes (AKS). You will define Helm charts, set up namespace-level resource quotas, and integrate secrets management so that data jobs spin up in seconds, not minutes.
• • Establish rigorous data governance: schema registries, lineage tracking, quality checks, and PII masking. You will partner with Legal & Compliance to meet GDPR and industry-specific standards without slowing delivery velocity.
• • Evaluate and onboard external data feeds—ranging from telemetry streams to third-party APIs—validating contracts, SLAs, and freshness guarantees. When vendors change schemas at 2 a.m., your monitoring alerts fire before stakeholders notice.
• • Champion workflow orchestration best practices using Apache Airflow or Dagster. You will write DAGs that are idempotent, backfill-friendly, and version-controlled, turning chaotic cron jobs into a transparent, debuggable system.
• • Stay on the cutting edge of data engineering and MLOps. Experiment with emerging tools (e.g., MLflow for experiment tracking, HEAT for data-centric ML pipelines) and run internal tech talks to share findings with 200+ engineers across Zartis hubs.
• • Collaborate daily with cross-functional squads—product managers, UX researchers, DevOps, and QA—to prioritize features, estimate effort, and ship increments every two weeks. Your voice shapes not just the “how” but the “why” behind every technical decision.
• • Mentor mid-level engineers through pair programming, design reviews, and lunch-and-learn sessions. Contribute to Zartis’ internal guilds, publish blog posts, and speak at meetups to grow our collective expertise and employer brand.
• • Enjoy the autonomy of a 100 % remote setup while staying connected via Slack, Miro, and Zoom. We default to asynchronous communication and record demos so teammates in Madrid, Kraków, or São Paulo can catch up on their own schedule.