Multilingual AI Response And Rubric Evaluator

Evaluate multilingual AI reference responses and scoring rubrics for Project Meteor. You’ll use detailed guidelines to identify quality issues, correct critical errors, and help improve how advanced AI systems are assessed.

About this project

Meteor focuses on the reference answers and evaluation rubrics used to train and assess multilingual AI models. The work includes both open-ended and closed-ended tasks and requires careful, consistent judgment.

What you’ll do

• Review AI-generated reference responses for quality and accuracy.
• Evaluate and improve scoring rubrics.
• Identify and correct critical P0 issues.
• Follow detailed annotation guidelines and meet project quality requirements.

Requirements

• Native-level fluency in one of the available project languages.
• Strong reading comprehension, written communication, and attention to detail.
• Basic familiarity with medical, financial, and legal topics.
• Ability to follow detailed instructions carefully.
• A computer with a reliable internet connection.

Project details

• Location: Remote within the seven eligible language and location options.
• Schedule: Flexible; task availability may vary.

Compensation

Compensation is calculated at a fixed hourly rate. The rate shown on the OneForma platform depends on the language and location you select.

Where you can work

This role is 100% remote, for applicants located in these 7 countries: Brazil, United Kingdom, Indonesia, Mexico, Malaysia, Thailand and Vietnam.