Data Engineer

📋 Description

• • Own the end-to-end lifecycle of a high-stakes, multi-terabyte data migration that will underpin the next generation of our client’s case-management platform. You will architect repeatable ETL pipelines, tune them for performance, and guarantee zero-loss, zero-downtime cut-over events that keep 24/7 operations running.
• • Translate complex business rules into deterministic SQL logic. You will interrogate legacy MySQL schemas, stored procedures, and ad-hoc scripts, reverse-engineer undocumented data flows, and craft elegant transformations that preserve referential integrity while unlocking new analytics capabilities.
• • Champion a rigorous data-quality culture. Apply the full six-pillar framework (accuracy, completeness, consistency, timeliness, validity, uniqueness, integrity) on every batch. Build automated validation suites that surface anomalies before they reach production, and publish transparent quality dashboards for executives and end-users alike.
• • Partner shoulder-to-shoulder with business analysts, solution architects, and QA testers in daily stand-ups and sprint reviews. You will turn vague requirements into precise mapping documents, peer-review code, and provide concise feedback that keeps the entire delivery squad aligned and unblocked.
• • Anticipate and neutralise risk. Maintain a living RAID log that captures schema drift, volume spikes, encryption-key rotations, and regulatory changes. Escalate early, propose mitigation options, and own the communication plan so stakeholders never face surprises.
• • Design for security and compliance from day one. Ensure every pipeline enforces field-level encryption, PII masking, and retention policies that satisfy UK GDPR and sector-specific regulations. Participate in quarterly audits with confidence and documentation to match.
• • Create self-service knowledge assets. Produce run-books, ER diagrams, data dictionaries, and post-migration playbooks that empower support teams long after you have moved on to the next challenge.
• • Optimise for speed and cost. Profile every query, leverage MySQL partitioning, indexing, and parallel load strategies, and benchmark against cloud-native targets to deliver migrations that finish ahead of schedule and under budget.
• • Support User Acceptance Testing as the authoritative data expert. Reproduce issues in lower environments, trace lineage from source to target, and deliver fixes within hours—not days—so business sign-off stays on track.
• • Stay curious. Evaluate emerging ETL tools, experiment with Infrastructure-as-Code for pipeline deployment, and feed lessons learned back into the guild so the entire engineering organisation levels up.