Data Annotation Specialist, Safety

Cohere Inc.

Job Overview

Location

Canada

Job Type

Contract

Full Job Description

📋 Description

• Evaluate and improve the safety of Large Language Model outputs by labeling, ranking, auditing, and refining human- and model-generated text containing explicit, toxic, violent, sexual, or psychologically disturbing content.
• Apply nuanced safety guidelines and rubrics to assess model responses across ambiguous, sensitive, and context-dependent scenarios with consistent, well-reasoned judgment.
• Create realistic prompts, user scenarios, and adversarial test cases to uncover unsafe, evasive, over-refusing, or policy-inconsistent model behaviors.
• Identify inconsistencies in annotations and unclear safety guidelines, providing actionable feedback to improve data quality and model performance.
• Work independently in a globally distributed, asynchronous team environment with high precision and attention to detail across high-volume, repetitive tasks.
• Maintain strict adherence to safety protocols while handling explicit content that may include NSFW material, ensuring accurate classification to prevent harmful model generation.
• Contribute to the calibration of safety standards by flagging recurring edge cases, model failures, and opportunities to refine annotation criteria.
• Operate under a BYOD (Bring Your Own Device) model, using your personal laptop to complete all annotation tasks remotely.
• Commit to a minimum of 16 hours per week as an independent contractor, with compensation structured at 45 CAD/hour or 45 USD/hour (30/hour base + 15/hour hazard pay).
• Complete a virtual annotation test involving written tasks, chatbot interactions, and content evaluation exercises as part of the selection process.
• Maintain confidentiality of Cohere’s proprietary data and intellectual property, and disclose any concurrent work with direct competitors.
• Accept that this is a 12-month contract position with no guaranteed continuous workload, and that compensation is based on invoiced hours with no employee benefits.
• Engage with content of a sensitive nature, including but not limited to sexually explicit, violent, or psychologically disturbing material, requiring emotional resilience and professional detachment.
• Communicate clearly in written English to justify evaluations of content safety, quality, and policy alignment with detailed reasoning.

🎯 Requirements

• 1+ years of experience in Content Moderation, Trust & Safety, AI data annotation, LLM evaluation, or a related analytical role
• Experience applying detailed safety guidelines to complex, sensitive, and ambiguous content with strong contextual and sociocultural judgment
• Emotional resilience to work with explicit, toxic, violent, sexual, or psychologically disturbing content
• Excellent command of written English and ability to clearly justify content evaluations
• Strong attention to detail and ability to maintain consistency across high-volume, monotonous tasks
• Proven ability to work independently in a remote, asynchronous, global team environment

🏖️ Benefits

• Contract rate of 45 CAD/hour or 45 USD/hour (30/hour base + 15/hour hazard pay)
• 12-month contract with potential for extension
• Flexible remote work within Canada or the United States
• Opportunity to contribute directly to the safety and reliability of frontier AI models

Skills & Technologies

Remote

Ready to Apply?

Apply Externally

You will be redirected to an external site to apply.

Cohere Inc.

Visit Website

About Cohere Inc.

Cohere provides large language models and retrieval-augmented generation APIs for enterprise developers to embed conversational AI, search, summarization, and content generation into applications. Founded in 2021 by former Google Brain researchers, the company offers cloud and on-premise deployment, fine-tuning tools, and multilingual support to help organizations automate workflows, improve customer support, and analyze unstructured data while maintaining data privacy and security controls.

View Company Profile

Get more remote jobs like this

Subscribe to the weekly newsletter for similar remote roles and curated hiring updates.

Weekly remote jobs and featured talent.

No spam. Only curated remote roles and product updates. You can unsubscribe anytime.