Senior Data Engineer

📋 Description

• • Yubo is seeking a highly skilled and experienced Senior Data Engineer to join our dynamic Infrastructure team. In this pivotal role, you will be instrumental in shaping the future of our data architecture, designing and implementing robust, scalable, and cost-efficient data pipelines that power our innovative Social Discovery app. You will operate in a fast-paced, experimental environment, collaborating closely with product squads to rapidly prototype and deploy new data models, ensuring Yubo continues to redefine social interactions for Gen Z.
• • As a Senior Data Engineer embedded within the Infrastructure team, you will have significant ownership and autonomy. Your responsibilities will span the entire lifecycle of data models, from initial design and conceptualization through to production deployment and ongoing optimization. You will work hand-in-hand with our DevOps colleagues to fine-tune databases and servers, ensuring peak performance and reliability for our high-throughput systems.
• • A key aspect of this role involves contributing to feature kick-offs as a primary stakeholder. You will actively participate in the design phase, critically evaluate proposals, and ensure that the technical solutions developed are not only innovative but also practically buildable and maintainable. This collaborative approach ensures that our data infrastructure evolves in lockstep with our product development.
• • If you are passionate about tackling complex data challenges, enjoy engaging in architectural debates with backend developers, and are driven to build data models that can scale under immense load, Yubo offers an unparalleled opportunity to make a significant impact.
• • Your core responsibilities will include building and maintaining high-throughput data pipelines capable of handling both batch and streaming workloads. This involves designing and implementing efficient data ingestion, transformation, and processing mechanisms.
• • You will play a crucial role in co-designing data models, working closely with product teams to validate their structure and functionality during the initial feature kickoff phase. This ensures alignment between data infrastructure and product requirements.
• • A significant part of your role will be to champion and ensure a sustainable architecture, upholding data integrity across all systems. You will be expected to educate and guide other stakeholders within Yubo on best practices and challenges related to data architecture and integrity.
• • You will collaborate closely with Machine Learning Engineers, providing them with clean, well-structured, and ML-ready datasets that are essential for training and deploying advanced AI models.
• • Maintaining exceptional data quality, availability, and order of entry is paramount, especially given our current volume of over 300,000 database entries per second. You will be tasked with engineering innovative solutions to further scale these capabilities and ensure seamless data flow.
• • Comprehensive documentation of pipelines, transformations, and architectural decisions is a critical requirement. Clear and concise documentation will serve as a valuable resource for future developers and your future self, facilitating easier maintenance and troubleshooting.
• • You will leverage our cutting-edge tech stack, including Kafka for data streaming and processing, a variety of databases such as MongoDB, Couchbase, KeyDB, DragonFly, Elasticsearch, and data warehousing solutions like Iceberg and Trino. Monitoring will be handled by Datadog & Grafana, with automation managed through Kubernetes operators.
• • Your impact will be immediate and far-reaching, contributing directly to the stability, scalability, and performance of Yubo's core data infrastructure. You will be a key player in enabling data-driven decision-making and the continuous improvement of our user experience.
• • This role offers a unique opportunity to work on a globally recognized platform with millions of active users, contributing to a mission focused on fostering genuine connections and ensuring a safe digital space for our community. Your work will directly influence how millions of young people socialize online.