Data Engineer

📋 Description

• • Architect, build, and maintain scalable, secure, and highly-available data pipelines that ingest, transform, and serve mission-critical datasets for federal agencies—your code will directly influence policy decisions and public services that affect millions of citizens.
• • Design and implement cloud-native ETL/ELT workflows on AWS using services such as Glue, Lambda, Kinesis, EMR, Step Functions, and Redshift; optimize for cost, performance, and compliance with FedRAMP, FISMA, and agency-specific security controls.
• • Partner with data scientists, analysts, and policy experts to translate complex business requirements into robust data models, ensuring that every table, view, and API endpoint is accurate, timely, and well-documented for downstream consumption.
• • Establish and enforce data governance standards—including lineage tracking, schema evolution, PII handling, and automated quality checks—so that federal stakeholders can trust the integrity of every metric and report they receive.
• • Implement Infrastructure-as-Code (Terraform or CloudFormation) to provision repeatable, auditable data environments across dev, test, staging, and production; participate in peer reviews to uphold best practices for version control, CI/CD, and automated testing.
• • Monitor pipeline health with CloudWatch, Datadog, or similar tools; set SLAs, SLOs, and alerting thresholds that balance system resilience with taxpayer value, and respond to incidents with clear post-mortems and continuous-improvement action items.
• • Collaborate in cross-functional agile squads that include product owners, UX researchers, and cybersecurity engineers; present technical trade-offs in plain language so non-technical stakeholders can make informed decisions.
• • Contribute to internal knowledge bases and brown-bag sessions, mentoring junior engineers and leveling up the entire team’s cloud and data engineering expertise.
• • Evaluate emerging AWS features and open-source frameworks (e.g., Apache Iceberg, Delta Lake, dbt, Airflow 2.x) to keep 540’s data stack modern, maintainable, and aligned with federal cloud-first mandates.
• • Champion a culture of experimentation and ethical data use—run proof-of-concepts, document findings, and ensure that every dataset is handled with the highest standards of privacy and civil liberties.
• • Own the end-to-end lifecycle of data products—from initial ingestion of raw feeds (SFTP, API, streaming logs) to curated analytics-ready datasets—ensuring that each stage is reproducible, versioned, and compliant with NIST 800-53 controls.
• • Work closely with security teams to implement encryption in transit and at rest, role-based access controls, and continuous vulnerability scanning, turning compliance requirements into elegant technical solutions rather than bureaucratic obstacles.
• • Provide on-call support on a rotating basis, using blameless post-mortems to turn outages into learning opportunities that strengthen both systems and team culture.
• • Translate complex data insights into compelling visual stories for senior government officials, helping them understand not just what the numbers say, but why they matter for national priorities like climate resilience, public health, and economic equity.
• • Drive cost-optimization initiatives—leverage spot instances, auto-scaling, and tiered storage strategies to reduce cloud spend without compromising performance or security, delivering measurable ROI to federal clients.