Technology and Innovation - Azure Data Engineer

📋 Description

• • Architect and deliver end-to-end cloud data solutions on Microsoft Azure that power critical business decisions for Fortune 500 and high-growth clients. You will own the full lifecycle—from ingestion of multi-terabyte data sets through to curated, analytics-ready models—leveraging Azure Data Factory, Azure Databricks, and Azure Data Lake Gen2 to ensure scalability, security, and cost-efficiency.
• • Design resilient, low-latency data pipelines that process both batch and streaming workloads. Employ advanced SQL, Python, and PySpark to transform raw data into high-value assets, applying partitioning, indexing, and caching strategies that cut query times by up to 70 % and reduce compute spend.
• • Build and maintain enterprise-grade data models (relational, dimensional, and big-data) that serve BI dashboards, AI/ML experiments, and regulatory reporting. Collaborate closely with data scientists, finance analysts, and product owners to translate complex requirements into elegant, extensible schemas.
• • Implement rigorous data-quality frameworks, automated testing suites, and CI/CD pipelines using Azure DevOps and Git. Champion engineering best practices such as Infrastructure-as-Code, peer code reviews, and semantic versioning to guarantee repeatability and rapid rollback.
• • Optimize performance across every layer of the stack—storage, compute, and network. Diagnose bottlenecks with Azure Monitor, Databricks Ganglia, and custom logging, then tune clusters, rewrite queries, and redesign partitions to hit aggressive SLAs (sub-second for dashboards, <15 min for daily loads).
• • Serve as a trusted advisor during client workshops and sprint reviews, articulating trade-offs between Synapse serverless pools, dedicated SQL pools, and on-demand Databricks jobs. Translate technical outcomes into business impact stories that resonate with CFOs and CTOs alike.
• • Contribute to Riveron’s internal knowledge base and open-source-style guilds, publishing reusable notebooks, Terraform modules, and architectural blueprints. Mentor junior engineers through pair programming and brown-bag sessions, fostering a culture of continuous learning aligned with our DELTA values.
• • Stay ahead of the Azure roadmap—preview features like Fabric OneLake, Mirrored Databases, and Copilot for Synapse—running proof-of-concepts that position Riveron as an early adopter and thought leader in the data engineering space.
• • Ensure compliance with SOC 2, GDPR, and industry-specific regulations by embedding encryption, row-level security, and audit logging into every pipeline. Partner with InfoSec to conduct threat modeling and penetration-test remediations.
• • Participate in Agile ceremonies (daily stand-ups, sprint planning, retrospectives) while maintaining a pragmatic balance between innovation and delivery commitments. Use story-pointing and burndown metrics to forecast capacity and manage stakeholder expectations.