Skip to content

Research report ยท September 2026 edition

The Market for AI Evaluation & Rating (RLHF) Work

On 30 Sep 2026, 224 roles in AI Evaluation & Rating (RLHF) were open on 13 platforms (micro1 listed 43.3%); their median advertised rate was $60/hr, 3% above the $58/hr of all AI training work; each open role drew 1.91x the apply clicks of the average role.

Read the month's story: The AI Training Job Market in September 2026

Data window
1 Sep 2026 to 30 Sep 2026
30 days, UTC
Frozen
10 Oct 2026, 15:00 UTC
These figures will not change
Source
The public job boards of the AI training platforms this site tracks, read every day. Apply clicks are made on AITraining.jobs; full descriptions come from the platforms whose boards publish them.
Method
Version 1. Definitions
CC BY 4.0, how to cite

Every figure on this page as CSV and JSON, sent to your work email.

open roles on 30 Sep 2026
224
median advertised rate (all AI training work: $58/hr)
$60/hr
apply clicks per open role, against the average role
1.91x
of open roles at micro1, the largest of 13 platforms
43%

Key findings

  1. 1

    Between the week of 7 Sep 2026 and the week of 21 Sep 2026, open roles in AI Evaluation & Rating (RLHF) on the platforms tracked since 1 Sep 2026 went down 17.1%, from about 213 to 177, against -2.1% for all AI training work on the same platforms.

    Figure 1
  2. 2

    13 platforms listed the 224 open roles in AI Evaluation & Rating (RLHF); micro1 held 43.3% and the three largest 83.9% (HHI 2,754, highly concentrated, against 1,680 for all AI training work).

    Figure 3
  3. 3

    193 of the 224 open roles in AI Evaluation & Rating (RLHF) (86.2%) stated an hourly rate, with a median of $60/hr (middle half $35 to $85), 3% above the $58/hr of all AI training work.

    Figure 4
  4. 4

    Each open role in AI Evaluation & Rating (RLHF) drew 0.40 apply clicks a week on AITraining.jobs, 1.91x the average AI training role: the 4th highest of 10 kinds of work with enough clicks to rank.

    Figure 6
  5. 5

    70.1% of the 224 open roles in AI Evaluation & Rating (RLHF) accepted applicants from any country, against 37.7% of all AI training roles; of the 20 countries compared, residents of the United States (202), India (176) and Germany (174) could apply to the most, and residents of South Korea to the fewest (157).

    Figure 9
  6. 6

    The words used most in the titles of open roles in AI Evaluation & Rating (RLHF) were Evaluator or reviewer (67), Expert or SME (46) and Specialist (35), of 224 open roles.

    Figure 10
  7. 7

    Of the 148 open roles in AI Evaluation & Rating (RLHF) with a full description, 18.2% mention a PhD or master's degree, 4.7% a professional credential and 16.9% an assessment or test; the 46 stating years of experience ask for a median of 4 years.

    Figure 11
  8. 8

    Over the 30 days from 1 Sep 2026 to 30 Sep 2026, open roles in AI Evaluation & Rating (RLHF) on the platforms tracked at its start went down 24.7% (219 to 165): platforms listed 222 new roles and removed 311, and a removed role had been listed for a median of 31 days (32 across all AI training work).

    Figure 2

Figure 1

Open roles, week by week

Is the number of open roles in AI Evaluation & Rating (RLHF) rising or falling?

WeekOpen roles (fixed panel)Index
1 to 6 SepPartial week
215
100.5
7 to 13 Sep
213
100.0
14 to 20 Sep
184
86.1
21 to 27 Sep
177
82.9
28 to 30 SepPartial week
174
81.7

Sample. Average open roles in AI Evaluation & Rating (RLHF) over each week's days, on the 15 platforms tracked since 1 Sep 2026 (the fixed panel; 11 of them list this work), 1 Sep 2026 to 30 Sep 2026.

The index is the panel's open roles against the week of 7 Sep 2026 (100), beside the same index for all AI training work on the same platforms.

OpenTrain AI (first tracked 5 Sep 2026) and Welo Data (first tracked 8 Sep 2026) list this work but were added to the dataset later, so they are left out of the panel and counted only in the last column: a platform's arrival never reads as growth.

Weeks with fewer than 10 open roles on the panel, or cut by the span, are marked.

Source. AITraining.jobs Research, The Market for AI Evaluation & Rating (RLHF) Work, September 2026 edition.

Figure 2

Listings added and removed

How fast do roles in AI Evaluation & Rating (RLHF) turn over, and is the stock growing on the same platforms?

WorkOpen on 1 Sep 2026Open on 30 Sep 2026, same platformsChangeNew listings
AI Evaluation & Rating (RLHF)219165
-24.7%
222
All AI training work1,5911,656
+4.1%
1,585

Sample. Roles listed from 1 Sep 2026 to 30 Sep 2026: 535 in AI Evaluation & Rating (RLHF), 4,472 across all AI training work.

Change compares the roles open on the first and the last day of the window on the platforms tracked when it opened, so a platform added during the window does not read as growth.

A new listing first appeared in the window on a platform already tracked before that week; a platform's first week (its back catalogue) is not counted.

Median days listed runs from a removed role's listing date to its last sighting, for removed roles with a known listing date (at least 10; 304 here).

Source. AITraining.jobs Research, The Market for AI Evaluation & Rating (RLHF) Work, September 2026 edition.

Figure 3

Who lists it

Which platforms list roles in AI Evaluation & Rating (RLHF), and what does each advertise?

PlatformOpen rolesShare of open rolesMedian per hour
micro197
43.3%
$65
OpenTrain AI50
22.3%
$30
Mercor41
18.3%
$70
Welo Data9
4.0%
$20
Braintrust8
3.6%
n/a
Invisible8
3.6%
$53
Handshake AI3
1.3%
n/a
AfterQuery2
0.9%
n/a
Outlier (Scale AI)2
0.9%
n/a
Alignerr1
0.4%
n/a
Appen1
0.4%
n/a
TransPerfect DataForce1
0.4%
n/a
xAI (SpaceXAI)1
0.4%
n/a

Sample. 224 roles in AI Evaluation & Rating (RLHF) open at the end of 30 Sep 2026, by the platform that lists them: HHI 2,754 (highly concentrated), against 1,680 for all AI training work.

HHI: the sum of every platform's squared percentage share of the open roles, 0 to 10,000; under 1,500 is unconcentrated, 1,500 to 2,500 moderately and above 2,500 highly concentrated (US merger-guideline thresholds).

Shares are of listings, not of people hired or money paid: a platform posting one role per language or country weighs more here than its hiring may.

A median needs at least 8 roles stating an hourly rate.

Source. AITraining.jobs Research, The Market for AI Evaluation & Rating (RLHF) Work, September 2026 edition.

Figure 4

What it pays

What do roles in AI Evaluation & Rating (RLHF) advertise per hour, against AI training work as a whole?

WorkMedian per hour25th percentile75th percentileStating an hourly rate
AI Evaluation & Rating (RLHF)
$60
$35$8586.2%
All AI training work
$58
$30$7572.2%

Sample. Roles open on 30 Sep 2026: 224 in AI Evaluation & Rating (RLHF), 193 of them stating an hourly rate; 2,726 across all AI training work.

Pay is the hourly figure the posting advertises (the middle of a stated range). The middle half runs from the 25th to the 75th percentile.

Roles paid per task or per project, or that state no figure, do not state an hourly rate. A median needs at least 8 roles stating one.

Source. AITraining.jobs Research, The Market for AI Evaluation & Rating (RLHF) Work, September 2026 edition.

Figure 5

How its rates are distributed

What share of roles in AI Evaluation & Rating (RLHF) pay $50 an hour or more?

Advertised rateOpen rolesShare of priced rolesAll AI training work
Under $15/hr8
4.1%
7.3%
$15 to $25/hr23
11.9%
10.1%
$25 to $35/hr15
7.8%
10.8%
$35 to $50/hr27
14.0%
17.8%
$50 to $75/hr52
26.9%
28.2%
$75 to $100/hr33
17.1%
10.8%
$100 to $150/hr22
11.4%
8.0%
$150/hr or more13
6.7%
6.9%

Sample. 193 roles in AI Evaluation & Rating (RLHF) open on 30 Sep 2026 that state an hourly rate, against 1,967 across all AI training work.

Each role counts once, at the middle of its advertised range. The bands are those of the pay-transparency report.

Source. AITraining.jobs Research, The Market for AI Evaluation & Rating (RLHF) Work, September 2026 edition.

Figure 6

Applicant demand per open role

Do roles in AI Evaluation & Rating (RLHF) draw more or fewer applicants per opening than the average AI training role?

RolesClicks per role a weekIndex
AI Evaluation & Rating (RLHF)
0.40
1.91x
All AI training work
0.21
1.00x
Its roles stating an hourly rate
0.40
1.92x
Its roles stating no hourly rate
0.38
1.81x
Its roles open worldwide
0.36
1.73x
Its roles limited by country
0.56
2.65x

Sample. Apply clicks on a specific role on AITraining.jobs, from 1 Sep 2026 to 30 Sep 2026: 420 on roles in AI Evaluation & Rating (RLHF), against the days those roles were open.

Clicks per role a week = apply clicks / (open role-days / 7). Index 1.00 is the rate of every role on the site.

Rows with fewer than 30 clicks or 3 roles open on an average day are marked as a small sample.

Apply clicks measure interest among visitors to AITraining.jobs, who see roles open in their country and recent, priced roles first. Clicks are never split by platform.

Source. AITraining.jobs Research, The Market for AI Evaluation & Rating (RLHF) Work, September 2026 edition.

Figure 7

Applicant demand by listing age

Do newer roles in AI Evaluation & Rating (RLHF) draw more applicants than older ones?

Listing ageClicks per role a weekIndex (this work = 1)
Listed during the window
1.16
2.88x
Listed up to 4 weeks before itSmall sample
0.09
0.22x
Listed 4 to 12 weeks before itSmall sample
0.12
0.29x
Listed more than 12 weeks before itSmall sample
0.10
0.25x
Listing date unknownSmall sample
0.11
0.28x

Sample. Apply clicks on roles in AI Evaluation & Rating (RLHF), from 1 Sep 2026 to 30 Sep 2026, by when each role was listed relative to the window's first day (1 Sep 2026).

Listing date: the date the job board gives for the posting, or else the first day the role appeared in the feed. A role that arrived with a platform's back catalogue and gives no date has none.

The index divides each row's rate by the rate of all roles in this kind of work, so it reads within the kind.

Rows with fewer than 30 clicks or 3 roles open on an average day are marked as a small sample.

Source. AITraining.jobs Research, The Market for AI Evaluation & Rating (RLHF) Work, September 2026 edition.

Figure 8

Who may apply: location rules

Can people apply to roles in AI Evaluation & Rating (RLHF) from anywhere?

Location ruleOpen rolesShare of its open rolesAll AI training work
Open worldwide157
70.1%
37.7%
Worldwide except listed countries2
0.9%
7.3%
Only listed countries65
29.0%
55.0%
No rule stated0
0.0%
0.0%

Sample. 224 roles in AI Evaluation & Rating (RLHF) open at the end of 30 Sep 2026, against 2,726 across all AI training work.

The rule is read from each posting's location text and, where the posting is silent, from what the platform states about where it hires.

Source. AITraining.jobs Research, The Market for AI Evaluation & Rating (RLHF) Work, September 2026 edition.

Figure 9

Who may apply: roles open to each country

How many roles in AI Evaluation & Rating (RLHF) can a resident of each country apply to?

CountryOpen rolesShare of its open roles
Any country (worldwide roles only)
157
70.1%
United States
202
90.2%
India
176
78.6%
Germany
174
77.7%
United Kingdom
171
76.3%
Bangladesh
170
75.9%
Pakistan
170
75.9%
Egypt
168
75.0%
Brazil
167
74.6%
Ghana
167
74.6%
Indonesia
167
74.6%
Nigeria
167
74.6%
Philippines
167
74.6%
Argentina
166
74.1%
Colombia
166
74.1%
Japan
166
74.1%
Chile
165
73.7%
Palestine
164
73.2%
Saudi Arabia
164
73.2%
Cambodia
158
70.5%
South Korea
157
70.1%

Sample. 224 roles in AI Evaluation & Rating (RLHF) open at the end of 30 Sep 2026, for residents of a fixed set of 20 countries.

A country's count is the roles open worldwide, plus roles whose list names it, plus roles whose exclusion list does not. Roles with no rule stated count for no country.

The 20 countries are the same in every report of this series: those named most often by the location rules of open AI training roles on 9 Oct 2026, either as the only countries accepted or as excluded.

Median pay is over the roles open to that country that state an hourly rate (at least 8).

Source. AITraining.jobs Research, The Market for AI Evaluation & Rating (RLHF) Work, September 2026 edition.

Figure 10

What the roles are called

What are roles in AI Evaluation & Rating (RLHF) called: annotator, evaluator, expert or trainer?

Title wordOpen rolesShare of its open roles
Evaluator or reviewer
67
29.9%
Expert or SME
46
20.5%
Specialist
35
15.6%
Analyst or researcher
18
8.0%
Quality assurance
17
7.6%
Trainer
12
5.4%
Lead or manager
11
4.9%
Engineer or developer
10
4.5%
Rater
7
3.1%
Annotator or labeler
2
0.9%
Contributor or participant
2
0.9%
Tutor
2
0.9%
Writer or editor
2
0.9%

Sample. 179 of the 224 roles in AI Evaluation & Rating (RLHF) open on 30 Sep 2026 name one of these words in their title (79.9%); 13 title words are named.

A title can use several of these words ('Expert AI Trainer') and counts for each. The vocabularies are those of the job-titles, languages and expert-fields reports.

Median pay needs at least 8 roles stating an hourly rate.

Source. AITraining.jobs Research, The Market for AI Evaluation & Rating (RLHF) Work, September 2026 edition.

Figure 11

What the descriptions ask for

Do roles in AI Evaluation & Rating (RLHF) ask for degrees, credentials, experience or tests?

RequirementRolesShare of described roles
Mentions a PhD20
13.5%
Mentions a PhD or master's27
18.2%
Mentions any degree35
23.6%
Mentions a professional credential7
4.7%
States years of experience46
31.1%
Asks for 5 years or more20
13.5%
States weekly hours26
17.6%
Mentions an assessment or test25
16.9%
Mentions an AI-led or video interview10
6.8%
Mentions a background check1
0.7%

Sample. 148 roles in AI Evaluation & Rating (RLHF) open on 30 Sep 2026 whose job board publishes a full description (AfterQuery, Braintrust, Mercor and micro1), 66.1% of its open roles; 850 described roles across all AI training work.

A mention can be a requirement or a preference ('a PhD is a plus'): the figure counts both. The patterns are those of the requirements report.

Only some platforms publish full descriptions, so this figure describes their roles, not every platform's.

Source. AITraining.jobs Research, The Market for AI Evaluation & Rating (RLHF) Work, September 2026 edition.

For labs, data vendors and researchers

Buy the data behind this report

The figures above are a summary. Behind them are records we collect ourselves, every day:

  • 9,572 roles tracked, each with its history

    Every role on 29 job boards, read daily since 13 Jul 2026: its pay and place, and the first and last day it was open.

  • 6,200 apply clicks in 28 days

    Which roles people actually apply to, by field, pay and kind of work. It happens on this site, so nobody else can count it.

  • 39 lab and vendor hiring boards

    The job boards of 25 AI labs and 14 human-data vendors, read every day: who is hiring for human data, and who is hiring to buy it.

What you can buy

  • The records

    The rows behind any figure, with their history, not only the totals.

  • A regular feed

    The same figures every week or month, as files.

  • A custom cut or benchmark

    Your countries, languages or fields, or your own platform against the market.

Method and definitions

The public job boards of the AI training platforms this site tracks, read every day. Apply clicks are made on AITraining.jobs; full descriptions come from the platforms whose boards publish them. Every figure is computed by one database query over the window shown, the same query for the live report and for every edition. Each figure states its sample beneath it.

Kind of work
Each role is classified from its title and description into one of the twelve kinds of AI training work this site tracks; this report counts the roles classified as AI Evaluation & Rating (RLHF).
Fixed panel
The platforms already tracked when the weekly series opens. The index follows their open roles, so a platform added later (such as OpenTrain AI and Welo Data in September 2026) never reads as growth.
Index
The panel's average open roles in a week, divided by the same in the base week, times 100; shown beside the same index for all AI training work on the same platforms.
Open role
A role listed by a given day and still in its platform's feed that day.
New listing
A role that first appeared during the window on a platform already tracked before that week. A platform's first week in this dataset is its back catalogue and is not counted as new.
Removed listing
A role that left its platform's feed during the window.
Days listed
From a removed role's listing date to the last day it was seen in its feed. A median needs at least 10 removed roles.
HHI
Herfindahl-Hirschman Index: the sum of every platform's squared percentage share of open roles, 0 to 10,000. Under 1,500 unconcentrated, 1,500 to 2,500 moderately, above 2,500 highly concentrated.
Advertised pay
The hourly figure a posting states (the middle of a stated range). Medians need at least 8 roles stating a rate; the middle half runs from the 25th to the 75th percentile.
Clicks per open role a week
Apply clicks on AITraining.jobs on a specific role, divided by the days the roles were open, times 7.
Demand index
Clicks per open role a week divided by the same rate across every role on the site. Above 1, the roles draw more applicants than the average role.
Listing age
When a role was listed relative to the window's first day: during the window, up to 4 weeks before it, 4 to 12 weeks before or longer. The listing date is the job board's posting date, or else the first day the role appeared in the feed.
Location rule
Read from each posting's location text and, where that names no country, from what the posting's own description states: open worldwide, worldwide except listed countries, only listed countries, or no rule stated.
Open to a country
A role is open to residents of a country if it is open worldwide, its list names the country, or its exclusion list does not. The 20 countries compared are the same in every report of the series.
Window
Live: the last 28 complete days, in UTC, and the last twelve complete ISO weeks (from the week of 3 Aug 2026 at the earliest) for the weekly series. Edition: one calendar month, in UTC, and its weeks.

What this data cannot tell you

  • Platforms that recruit without a public job board are not counted. Several large ones invite workers to projects directly.
  • A listing is not a hire. Some platforms keep evergreen roles open for months, and some post one listing per language or country.
  • Apply clicks measure interest among the people who use AITraining.jobs, and what is shown first gets clicked more. Clicks are never published by platform.
  • Full descriptions are published by a few platforms only, so the requirements figure describes their roles and appears only when at least 10 roles of this kind are described.
  • A kind of work with few open roles gives figures that move with a handful of postings: rows on too little data are marked as a small sample.

How the job feeds are read and roles are classified: site methodology. How fresh, complete and consistent the data is, checked every day: data quality and coverage.

Editions

The live report covers the last 28 complete days and changes every day. Each complete month is also published as an edition, frozen two days after the month ends, so a figure you cite from an edition never moves.

Cite and reuse

Free to quote, chart and republish under CC BY 4.0 with a link to this page. This edition is frozen, so its figures will read the same later.

Citation

AITraining.jobs (2026). The Market for AI Evaluation & Rating (RLHF) Work, September 2026 edition. https://www.aitraining.team/research/ai-evaluation-rlhf-market/2026-09

BibTeX

@misc{aitrainingjobs_ai_evaluation_rlhf_market_2026_09,
  author = {{AITraining.jobs}},
  title = {The Market for AI Evaluation & Rating (RLHF) Work, September 2026 edition},
  year = {2026},
  howpublished = {\url{https://www.aitraining.team/research/ai-evaluation-rlhf-market/2026-09}},
}
Data: every figure as CSV (one line per value) and JSON, sent to your work email.Pay and open-roles data

More research

The same figures for every other kind of work:

All 78 research reports