Italian-Speaking AI Evaluation Specialist

📋 Description

• • The Italian-Speaking AI Evaluation Specialist will assess and improve the quality of AI-generated responses in Italian, ensuring outputs are accurate, relevant, and culturally appropriate for real user queries. This role is critical to refining AI systems’ understanding and communication in Italian, directly impacting the performance and reliability of AI products used by Italian-speaking audiences.
• • Day to day, the specialist will evaluate and compare AI-generated responses in Italian, perform side-by-side analysis of outputs from different AI systems, assess responses for accuracy, relevance, clarity, and instruction-following, identify nuances in meaning, tone, and cultural context, apply structured annotation guidelines consistently, and work with datasets of real user queries.
• • Blueprint is a technology solutions firm headquartered in Bellevue, Washington, with a strong presence across the United States. The company helps organizations unlock value from existing assets by leveraging cutting-edge technology to create additional revenue streams and new lines of business, connecting strategy, business solutions, products, and services to transform and grow companies. The team is unified by a shared passion for solving complicated problems, and values boldness, intelligence, agility, and a fun, collaborative culture.
• • In this role, the specialist will develop deep expertise in AI evaluation, linguistic analysis, and cultural nuance assessment in Italian. They will gain hands-on experience working with advanced AI systems, structured data annotation processes, and cross-functional teams including data science and product development, while building skills in analytical thinking, attention to detail, and quality assurance for language technologies.