(Senior) Data Engineer (m/w/d)

📋 Description

• • Join GALVANY as a (Senior) Data Engineer and play a pivotal role in shaping the future of sustainable living. Our mission is to make climate-neutral living accessible to everyone, and we achieve this through tangible, intelligent solutions that simplify the adoption of heat pumps, battery storage, and smart metering. GALVANY-Tech is at the forefront of the energy transition, powered by sophisticated software. We are developing an AI-based Operating System designed to revolutionize the sales, planning, installation, and operation of heat pumps and integrated energy systems. This system spans from individual single-family homes to large multi-unit buildings, and extends to our innovative Energy Community, functioning as a Virtual Power Plant.
• • As a Data Engineer, you will be instrumental in architecting, building, and maintaining the robust data infrastructure that underpins our cutting-edge AI-based Operating System. Your responsibilities will encompass the intricate processing of real-time energy data streams originating from a diverse array of heat pumps and integrated energy systems. You will be responsible for constructing and optimizing data pipelines that serve as the lifeblood for our advanced AI models and comprehensive analytics platforms. A critical aspect of your role will be to meticulously ensure the highest standards of data quality, consistency, and freshness across our entire data ecosystem, from individual household data points to the aggregated insights from our Virtual Power Plant.
• • You will be tasked with designing and implementing scalable data pipelines, leveraging modern technologies such as Kafka, Benthos, and ClickHouse, along with their associated tools. This involves creating efficient and reliable data flows that can handle high volumes of real-time data. A core focus will be on establishing and maintaining data quality, ensuring that data is accurate, consistent, and readily available across both the energy ecosystem and our internal operational processes. This includes developing strategies for data validation, cleansing, and anomaly detection.
• • Collaboration is key in this role. You will work closely with our Machine Learning Engineers to ensure that the data provided is in the optimal format and structure required for training and deploying sophisticated AI models. This requires a deep understanding of both data engineering principles and the needs of machine learning applications.
• • Proactive monitoring and validation will be built into the very fabric of the pipelines you create. You will implement comprehensive monitoring systems and validation checks from the outset to guarantee the integrity and reliability of the data flowing through our systems. This proactive approach minimizes potential issues and ensures data trustworthiness.
• • You will act as a crucial bridge between business needs and technical solutions. This involves translating complex business questions into specific data requirements and, conversely, communicating the capabilities and limitations of our data infrastructure back to business stakeholders. This ensures that our data efforts are aligned with strategic objectives.
• • Innovation is encouraged. You will have the opportunity to leverage the latest advancements in Large Language Models (LLMs) and other AI tools to explore novel data solutions, enhance existing processes, and accelerate overall development cycles. This includes experimenting with AI-assisted coding and data analysis techniques.
• • Our tech stack is modern and forward-thinking, including tools like Kafka, Benthos for data pipelining, Python and SQL for data manipulation and analysis, and GoLang for our backend services. For Machine Learning, we utilize Python, PyTorch, and LLMs. Our general development environment includes Azure for cloud services, GitHub for version control, and Linear and Notion for project management and documentation.
• • This role offers a unique opportunity to be part of a profitable green-tech startup experiencing hypergrowth. You will contribute directly to our mission of driving the energy transition through innovative software solutions, making a tangible impact on decarbonization efforts. We believe in fostering an environment where individuals can take ownership, drive lasting impact, and grow both personally and professionally.