Azure Data Engineer

📋 Description

• • **Shape the Future of Data at Bright Vision Technologies:** Join a forward-thinking software development company as an Azure Data Engineer and play a pivotal role in building innovative solutions that automate and optimize business operations. At Bright Vision Technologies, we are dedicated to leveraging cutting-edge technologies to create scalable, secure, and user-friendly applications. This is a significant opportunity to contribute to our mission of transforming business processes through technology and to grow your career within an established and well-respected organization.
• • **Design and Implement Robust Data Architectures:** You will be responsible for designing, building, and optimizing enterprise-grade data solutions on the Microsoft Azure platform. This includes architecting scalable data solutions using Azure Synapse Analytics, Azure SQL Database, and Azure Data Lake. Your expertise will be crucial in developing logical and physical data models, ensuring optimal normalization, dimensional modeling, and schema design for performance and efficiency. Furthermore, you will implement comprehensive data governance, lineage tracking, and cataloging solutions using Azure Purview, ensuring data quality and discoverability across the organization.
• • **Develop and Optimize Data Integration Pipelines:** A core aspect of this role involves building and managing sophisticated ETL/ELT pipelines using Azure Data Factory (ADF). You will integrate diverse data sources, including structured, semi-structured, and unstructured data from on-premises systems, cloud environments, and various APIs. A key focus will be on optimizing ingestion pipelines for both batch and real-time streaming data, leveraging services like Azure Event Hub and Azure Stream Analytics to ensure timely and efficient data flow.
• • **Drive Big Data Processing and Advanced Analytics:** You will work extensively with distributed data processing frameworks such as Databricks, Spark, and Hadoop to handle large-scale data challenges. This role extends to implementing advanced analytics solutions, including the development and deployment of machine learning pipelines within Azure ML. You will also play a key role in enabling self-service Business Intelligence (BI) by creating and managing Power BI dashboards and semantic models, empowering stakeholders with actionable insights.
• • **Champion Cloud and DevOps Best Practices:** Deploy and manage data solutions efficiently using Infrastructure as Code (IaC) principles with tools like ARM templates, Terraform, or Bicep. You will integrate robust CI/CD pipelines for data workflows using Azure DevOps or GitHub Actions, ensuring seamless and automated deployments. A critical responsibility will be ensuring the high availability, disaster recovery capabilities, and optimal performance tuning of all data systems.
• • **Ensure Data Security and Compliance:** Implement stringent security measures, including role-based access control (RBAC), encryption, and data masking techniques, to protect sensitive data. You will ensure adherence to critical compliance standards such as GDPR, HIPAA, and other enterprise security policies. Continuous monitoring and auditing of data usage will be performed using Azure Monitor and Log Analytics to maintain a secure data environment.
• • **Collaborate and Lead for Impact:** Partner closely with data scientists, data analysts, and business stakeholders to understand their needs and deliver data-driven insights that support strategic decision-making. You will have the opportunity to mentor junior data engineers, fostering a culture of learning and enforcing high standards for coding and data practices. This role requires driving best practices in data engineering and promoting cloud adoption across the organization.
• • **Location and Work Model:** This position is based in Bridgewater, New Jersey (08807) and offers a flexible work arrangement, supporting onsite, hybrid, and remote work models to accommodate diverse needs and preferences. We believe in empowering our team members with the flexibility to work in an environment where they can be most productive.