Job Overview
Location
Washington, USA
Job Type
Full-time
Category
Data Engineer
Date Posted
March 4, 2026
Full Job Description
📋 Description
- • ProRata Inc. is seeking a highly experienced and visionary Principal Engineer to spearhead the development and architecture of our cutting-edge retrieval systems and Retrieval-Augmented Generation (RAG) construction.
- • This pivotal role is designed for an individual who thrives at the intersection of massive, licensed datasets and the dynamic demands of real-time generative inference, acting as the primary architect for our sophisticated RAG pipeline.
- • Your core mission will involve the intricate processing, intelligent chunking, and highly optimized indexing of millions of documents, ensuring seamless and powerful support for both semantic and full-text discovery mechanisms.
- • You will be instrumental in designing and implementing a robust, end-to-end RAG construction pipeline, prioritizing high-performance data ingestion and transformation capabilities for a wide array of diverse datasets, all while operating in near real-time.
- • A key responsibility will be the development and meticulous optimization of hybrid retrieval strategies. This involves expertly combining the precision and accuracy of traditional full-text search with the nuanced contextual understanding offered by modern semantic (vector) search techniques.
- • You will take complete ownership of the critical 'document-to-chunk' lifecycle. This includes implementing advanced, state-of-the-art strategies for chunking documents, enriching them with relevant metadata, and applying sophisticated quality filtering to guarantee that only the most pertinent and valuable context is fed into our generative models.
- • Architecting systems capable of efficiently managing and processing jobs across millions of documents is paramount. This requires a deep focus on optimizing indices for ultra-low latency (sub-second) retrieval and high-throughput serving capabilities.
- • A significant aspect of this role involves recommending and implementing strategic optimizations for GPU and CPU performance, concurrency, and memory management. The goal is to drive down serving costs significantly while maximizing the return on investment (ROI) for our infrastructure.
- • As a technical leader and influencer, you will guide the engineering team in adopting best practices for software design, architectural patterns, and system scalability.
- • You will also be expected to provide superior diagnostic skills, tackling and resolving complex issues within our distributed systems with efficiency and expertise.
- • This role offers a unique opportunity to work at the absolute forefront of Artificial Intelligence technology, contributing directly to the innovation and advancement of our core AI products.
- • You will be part of a collaborative and dynamic work environment, fostering a culture of innovation, continuous learning, and mutual support.
- • The position is based on-site at our Bellevue, WA office, providing a stable and dedicated workspace for focused development.
- • We are committed to offering a competitive salary and a comprehensive benefits package, reflecting the value and expertise you bring to our team.
- • Professional development and growth opportunities will be readily available, encouraging you to expand your skill set and advance your career within the company.
- • This is a chance to make a tangible and significant impact on the company's success, shaping the future of our AI retrieval capabilities.
- • We are an Equal Opportunity Employer, celebrating diversity and committed to creating an inclusive environment for all employees, ensuring all employment decisions are based on qualifications, merit, and business needs.
🎯 Requirements
- • 15+ years of overall engineering experience, with a minimum of 8+ years specifically focused on architecting and scaling large-scale distributed systems.
- • Proven expertise in Python and Golang or Rust, demonstrated through the successful development and deployment of commercial-grade software solutions.
- • Expert-level mastery in architecting and scaling high-throughput indexing pipelines using technologies such as ElasticSearch, OpenSearch, or distributed vector databases. This includes a proven ability to design sophisticated query and indexing strategies that effectively balance semantic richness with sub-second retrieval latency.
- • Deep hands-on experience optimizing large-scale search architectures designed to handle millions of documents, leveraging both traditional inverted indices and modern vector stores. Demonstrated success in implementing advanced caching and indexing optimizations to minimize generative costs while maximizing retrieval relevance.
- • Proficiency in both SQL and NoSQL databases (e.g., MongoDB, Clickhouse, Postgres) and experience with big data processing tools.
- • Excellent understanding of fundamental algorithms, data structures, graph theory, and modern distributed application principles, including REST API design, scaling strategies, and capacity sizing.
- • Master's degree in Computer Science or Engineering, or equivalent practical experience.
🏖️ Benefits
- • Opportunity to work at the forefront of AI technology, shaping the future of retrieval and generative AI.
- • Collaborative and innovative work environment with a team of highly skilled engineers and researchers.
- • Competitive salary and comprehensive benefits package, including health, dental, and vision insurance.
- • Professional development and growth opportunities, including access to training, conferences, and advanced learning resources.
- • Chance to make a significant impact on the company's success and the broader AI landscape.
Skills & Technologies
Python
Go
Rust
PostgreSQL
MongoDB
Senior
Hybrid
Degree Required
About ProRata Inc.
ProRata is an AI-driven licensing and compensation platform that indexes content from media and publishing partners, identifies when generative AI models use that content, and automatically apportions and remits royalty payments to rightsholders based on actual usage. The company provides rights-cleared datasets to AI developers and transparent revenue dashboards to publishers, aiming to align incentives and ensure creators are paid proportionally.
