Lead Engineer

📋 Description

• • As a Lead Engineer at Weekday 1's client, you will be at the forefront of designing, developing, and deploying sophisticated real-time data processing solutions. This is a critical role where you will not only architect complex systems but also mentor a team of talented engineers, ensuring the delivery of high-quality, scalable, and performant applications. You will be instrumental in translating intricate business needs into robust technical realities, driving innovation in stream processing and distributed systems.
• • Your primary responsibility will involve leading the end-to-end architecture, design, and implementation of cutting-edge real-time data processing pipelines. A significant focus will be placed on leveraging Apache Flink, a powerful open-source stream-processing framework, to build solutions that can handle massive volumes of data with low latency.
• • You will be responsible for developing and maintaining high-performance backend services and distributed systems, with a strong emphasis on using Java. This includes crafting efficient code, managing concurrency, and optimizing for performance in a distributed environment.
• • A key aspect of this role is designing scalable event-driven architectures. These architectures must be capable of seamlessly handling high-throughput data streams and maintaining low-latency responses, crucial for real-time decision-making and operations.
• • You will be tasked with optimizing streaming jobs to ensure peak performance, robust fault tolerance, and efficient resource utilization. This involves deep dives into system tuning, checkpointing strategies, and state management within Flink.
• • Upholding best practices in code quality, comprehensive testing, effective observability, and streamlined CI/CD processes will be paramount. You will champion these practices within the team, ensuring maintainable and reliable software.
• • Collaboration is central to this role. You will work closely with data engineering, DevOps, and product teams to define technical roadmaps, gather system requirements, and ensure alignment between technical capabilities and business objectives.
• • You will play a vital role in conducting thorough design reviews, proactively troubleshooting production issues, and implementing strategic, long-term reliability improvements to our systems.
• • Mentoring and guiding other engineers is a core component of the Lead Engineer position. You will foster a culture of technical excellence, knowledge sharing, and continuous improvement within the engineering team.
• • You will also contribute significantly to infrastructure decisions, particularly concerning distributed processing frameworks, cloud deployment strategies, and the effective use of containerized environments.
• • This role demands a deep understanding of distributed systems, data structures, algorithms, and system design principles. You will apply this knowledge to solve complex challenges and build resilient systems.
• • Experience with messaging systems like Kafka is essential for building robust data pipelines. You will leverage these tools to ensure reliable data flow between different components of the system.
• • Deployment and management of applications in major cloud environments such as AWS, Azure, or GCP will be a regular part of your responsibilities. You will ensure our solutions are deployed efficiently and scale effectively.
• • Familiarity with containerization technologies like Docker and orchestration tools such as Kubernetes is expected, enabling efficient deployment and management of microservices and applications.
• • A solid understanding of CI/CD pipelines, automated testing frameworks, and monitoring tools is required to ensure continuous delivery and operational excellence.
• • Experience with both SQL and NoSQL databases in high-scale environments will be leveraged to design and manage data storage solutions that meet demanding performance and availability requirements.
• • Beyond technical skills, you will demonstrate proven experience in leading engineering teams or taking ownership of major technical initiatives. Your architectural decision-making abilities will be crucial, with a constant focus on scalability, maintainability, and long-term success.
• • You will possess excellent problem-solving and analytical skills, with the ability to dissect complex issues and devise effective solutions.
• • Communicating complex technical concepts clearly and concisely to both technical and non-technical stakeholders is a vital part of this role.
• • Finally, a strong ownership mindset and an unwavering commitment to delivering high-quality, impactful solutions will define your success in this position.