Software Engineer - AI Performance

📋 Description

• • Join DevRev Inc. as a Software Engineer specializing in AI Performance, a pivotal role in shaping the future of enterprise SaaS. You will be instrumental in defining and building the benchmarks that measure the success and impact of AI-native applications, ushering in the next era of software intelligence.
• • At DevRev, we are pioneering the SaaS 2.0 revolution with our AI teammate, 'Computer'. This platform is designed to unify disparate data sources, tools, and workflows, empowering employees with real-time insights, proactive recommendations, and sophisticated agentic actions. 'Computer' extends existing software with AI-native applications and agents that collaborate with teams and customers, streamlining workflows, enhancing cross-functional coordination, and automating repetitive tasks. This approach, termed Team Intelligence, fosters human-AI collaboration, breaks down organizational silos, and liberates teams to focus on more complex problem-solving.
• • As a Software Engineer on the AI Performance team, you will play a critical role in developing and scaling the infrastructure necessary for evaluating AI systems within enterprise environments. This involves a deep dive into performance tuning, profiling, and benchmarking of AI models and their serving systems.
• • You will participate in a comprehensive, structured training program designed to cultivate expertise in AI benchmarking, profiling, and performance optimization. This training will equip you with the knowledge and skills to tackle the unique challenges of measuring AI performance in production settings.
• • A core responsibility will be to build and scale robust benchmarking infrastructure. This infrastructure will be used to rigorously evaluate the performance of AI systems, ensuring they meet enterprise-grade standards for reliability, speed, and accuracy.
• • You will design and implement sophisticated agent evaluation pipelines. These pipelines will meticulously measure key aspects of AI agent performance, including their reasoning capabilities, accuracy in task completion, alignment with user intent, and overall impact on user outcomes.
• • Collaborate closely with AI and Infrastructure teams to instrument model-serving systems. This includes working with technologies such as vLLM, Triton, Ray, or Kubernetes-based deployments, ensuring seamless integration and data capture for performance analysis.
• • Develop and maintain data and telemetry pipelines. These pipelines are crucial for capturing comprehensive metrics on model performance, latency, throughput, and quality, providing the data needed for continuous improvement.
• • Contribute significantly to DevRev’s open benchmarking frameworks. Your work will help define the global standards for how intelligent SaaS software is measured, setting a precedent for the industry.
• • Engage with cross-functional teams, including Product, AI Research, and Engineering, to ensure that benchmarks accurately reflect real-world customer value and the overall system impact. This collaborative approach guarantees that our performance metrics are relevant and meaningful.
• • This role offers a unique opportunity to move beyond traditional software engineering into a cutting-edge field. While prior AI experience is not a prerequisite, a strong foundation in software engineering, a keen analytical mind, and an insatiable curiosity for learning are essential. DevRev is committed to providing the necessary training and mentorship to help you excel in this dynamic and evolving domain.
• • You will be at the forefront of defining the Key Performance Indicators (KPIs) for the AI-native enterprise. This is your chance to bring scientific rigor to systems that exhibit complex reasoning and to impose structure on software that demonstrates intelligent behavior.
• • By joining DevRev, you will directly contribute to shaping how intelligence is measured and valued in the SaaS 2.0 landscape, making a tangible impact on the future of enterprise software.
• • The role requires a proactive approach to identifying performance bottlenecks, optimizing resource utilization, and ensuring the scalability and reliability of AI-powered features.
• • You will be involved in the entire lifecycle of performance engineering, from initial design and implementation to ongoing monitoring and refinement.
• • This position is ideal for engineers who are passionate about performance, eager to learn about AI, and motivated to build foundational systems for a groundbreaking technology.
• • You will work in a remote setting based in the Philippines, collaborating with a global team of talented engineers and researchers.
• • DevRev is backed by prominent venture capital firms like Khosla Ventures and Mayfield, signifying a strong financial foundation and a clear vision for growth.
• • The company culture emphasizes innovation, collaboration, and a commitment to pushing the boundaries of what's possible with AI in the enterprise.