Staff Data Engineer

📋 Description

• • At Hauler Hero, we are revolutionizing the waste management industry with our cutting-edge SaaS platform. We empower haulers and waste management companies by providing modern technology solutions that streamline operations, from sophisticated route optimization to seamless billing and comprehensive customer management. Having achieved significant product-market fit, we are now in a rapid growth phase, and a critical component of this expansion is scaling our data infrastructure, particularly our customer data migration platform. This is where you, our Staff Data Engineer, will play a pivotal role.
• • As the Staff Data Engineer, you will be entrusted with the ownership and scaling of our customer data migration platform. This is not just about building pipelines; it's about architecting the very systems, frameworks, and tooling that underpin all our customer data migrations. Your primary mission will be to transform complex, often manual, and bespoke processes into robust, reliable, and repeatable workflows. You will be instrumental in progressively shifting migrations from being engineer-led, custom efforts to standardized, self-service capabilities, thereby significantly increasing efficiency and reducing the need for ongoing, hands-on engineering intervention for each migration.
• • This staff-level role is fundamentally about leverage. The impact of your work will be amplified across the entire organization, making the engineering team faster, more efficient, and enabling them to focus on higher-level strategic initiatives. You will be the architect of efficiency, the builder of scalable solutions, and the champion of data integrity throughout the migration lifecycle.
• • **Migration Platform Architecture & Scalability:**
• • You will own the end-to-end architecture of the data migration platform, encompassing every stage from initial data ingestion, through rigorous validation, to the final production deployment. This involves designing a comprehensive and resilient system that can handle diverse data sources and complexities.
• • A key responsibility will be to design migration infrastructure that drastically reduces the per-customer engineering effort. This will be achieved through the development and implementation of reusable components, standardized patterns, and modular design principles that can be applied across multiple migration projects.
• • You will build systems designed for evolution, enabling the transition of migration processes from being primarily engineer-led to becoming standardized, automated, and eventually self-service for both internal teams and potentially customers.
• • Establishing automated data quality frameworks is paramount. This includes implementing robust data profiling to understand data characteristics, comprehensive validation rules to ensure accuracy and consistency, and sophisticated anomaly detection mechanisms to identify and flag potential issues early in the process.
• • Instrumenting the migration systems with comprehensive dashboards, key metrics, and proactive alerts is crucial for ensuring observability and enabling continuous improvement. This will provide real-time insights into migration progress, performance, and potential bottlenecks.
• • **Self-Service Tooling & Enablement:**
• • You will architect intuitive self-service import workflows designed for both customers and internal teams, empowering them to manage data imports with greater autonomy.
• • Design ingestion pipelines that are tightly aligned with Hauler Hero's Hibernate entity models and lifecycle rules, ensuring seamless integration with our core application data structures.
• • Build intelligent error handling and feedback loops that provide clear, actionable guidance to non-technical users, enabling them to easily correct data issues and resubmit without extensive engineering support.
• • Create standardized Excel-based import templates that include embedded validation, clear documentation, and formatting standards, simplifying the data preparation process for users.
• • Maintain critical alignment with evolving application changes by proactively tracking Hibernate model evolution and ensuring timely updates to ingestion pipelines and associated tooling.
• • **Technical Execution:**
• • Design, build, and maintain robust ETL pipelines specifically tailored for migrating data from legacy waste management systems, which often present unique challenges.
• • Analyze complex legacy datasets, meticulously reverse-engineer underlying business logic, and implement efficient transformation workflows to map legacy data to our modern platform structure.
• • Execute end-to-end customer migrations, working in close coordination with both technical and business stakeholders to ensure successful data transitions.
• • Develop specialized tooling to ingest data from the prevalent legacy LAN-based systems common within the waste management industry.
• • Optimize pipeline performance to efficiently handle large datasets, significantly reducing migration timelines and ensuring a positive customer experience.
• • **Technical Leadership & Collaboration:**
• • Drive critical technical decisions regarding data infrastructure, including thorough build vs. buy evaluations to ensure we are leveraging the most effective solutions.
• • Partner closely with Product Management, Engineering, and Customer Success teams to collaboratively shape scalable and impactful data solutions that align with business objectives.
• • Mentor and guide team members on best practices for data integration, migration patterns, and platform development, fostering a culture of learning and growth.
• • Create and maintain comprehensive documentation, including detailed runbooks and operational guides, to ensure knowledge transfer and operational efficiency.
• • Lead code reviews and facilitate knowledge-sharing sessions to consistently raise the technical bar across the data engineering team.
• • Your success will be defined by an ownership mindset, a systems-thinking approach to problem-solving, a strong bias toward building scalable and automated solutions, clear and effective communication, comfort navigating ambiguity, and a relentless drive for continuous improvement. You will embody engineering principles such as practicality over perfection, prioritizing context over rigid processes, and demonstrating deep customer empathy, all while building for scale.