Principal Data Engineer, Data Platform

📋 Description

• • Define and evolve the technical vision and roadmap for A Place for Mom’s enterprise Databricks Lakehouse platform, serving as the foundation for analytics, AI, machine learning, and data product development across the organization.
• • Drive the evolution of APFM’s federated data operating model, balancing domain-specific ownership with enterprise-wide governance, standards, and consistency in data definitions.
• • Partner with Analytics Engineering and business stakeholders to design, maintain, and enhance the semantic layer, ensuring business metrics, dimensions, and definitions are consistent, discoverable, and trusted.
• • Establish and enforce organization-wide standards for metadata management, data lineage, discoverability, stewardship, and data quality across all data assets.
• • Lead the development of reusable, scalable data products and shared platform capabilities that enable self-service analytics and reduce dependency on centralized engineering teams.
• • Champion the adoption and governance of core Databricks capabilities including Delta Lake, Unity Catalog, Workflows, AI/BI Genie, MLflow, and Vector Search to support advanced analytics and AI initiatives.
• • Collaborate closely with AI and Machine Learning teams to establish scalable, operationalized patterns for model development, deployment, monitoring, and governance on the Databricks platform.
• • Guide technical decision-making across multiple engineering and product teams, aligning platform investments with measurable business outcomes and strategic priorities.
• • Mentor data engineers and technical staff across the organization to elevate engineering practices, promote best practices in data platform design, and foster a culture of technical excellence.
• • Ensure the data platform remains secure, reliable, performant, and easy to use for all internal consumers, from analysts to data scientists to business users.
• • Work cross-functionally with Engineering, Product, AI/ML, and business units to translate complex data needs into scalable technical solutions that support mission-critical decision-making.
• • Maintain deep expertise in modern data platform architecture, governance frameworks, and observability tools to continuously improve the platform’s resilience and usability.
• • Serve as the primary technical authority on Databricks platform capabilities, providing guidance on architecture, optimization, and integration patterns for enterprise-scale data workflows.
• • Contribute to the strategic alignment of data governance policies with regulatory requirements and internal compliance standards to ensure data integrity and trust.
• • Promote a culture of data literacy and self-service by enabling non-technical teams to confidently access, understand, and use trusted data assets.
• • Evaluate and recommend new tools, technologies, or enhancements to the data platform that improve efficiency, scalability, or user experience while maintaining alignment with the enterprise architecture.
• • Document platform architecture, data flows, governance policies, and operational procedures to ensure knowledge retention and onboarding continuity across teams.