Skip to content

AI Evaluation

Machine Learning & NLP Expert

Weekday logoWeekday$80–$110/hrRemote in United StatesPart-timePosted 2mo ago

About the role

Compensation: $80-$110 per hour

Join a cutting-edge AI research initiative and help shape the next generation of frontier AI models. We are seeking experienced Machine Learning & NLP Experts to contribute their technical expertise toward training and evaluating advanced AI systems. In this role, you will design challenging, real-world machine learning and natural language processing tasks, create high-quality reference solutions, and assess AI model performance to identify reasoning gaps and improve overall capabilities.

This is a part-time, fully remote opportunity requiring approximately 20 hours per week.

Key Responsibilities

  • Design challenging, real-world machine learning and natural language processing tasks covering areas such as:
  • Machine Learning Model Development and Evaluation
  • Natural Language Understanding (NLU)
  • Natural Language Generation (NLG)
  • Information Retrieval and Search
  • Applied Machine Learning Pipelines
  • Transformer Models and Large Language Models (LLMs)
  • Develop accurate reference solutions and integrate tasks into agentic development environments using Python .
  • Build executable evaluation frameworks and testing components where appropriate.
  • Evaluate AI model outputs for technical correctness, reasoning quality, and overall performance.
  • Identify capability gaps, classify model failure modes, and provide detailed written analyses.
  • Create and refine evaluation guidelines, scoring rubrics, and quality standards for ML and NLP tasks.
  • Collaborate with fellow subject matter experts to ensure consistency, accuracy, and high-quality training data.

Required Qualifications

  • Deep hands-on experience in Machine Learning and/or Natural Language Processing through industry, research, or graduate/PhD-level work.
  • Strong proficiency in Python with practical experience developing ML or NLP applications.
  • Strong understanding of modern machine learning techniques, including:
  • Model Training and Evaluation
  • Transformer Architectures
  • Large Language Models (LLMs)
  • NLP Pipelines
  • Feature Engineering and Model Optimization
  • Experience with industry-standard frameworks such as PyTorch , TensorFlow , Hugging Face Transformers , or equivalent.
  • Ability to commit approximately 20 hours per week .
  • Excellent written communication skills and the ability to work independently in a remote environment.

Preferred Qualifications

  • Experience in AI model evaluation, AI training data creation, or human-in-the-loop model assessment.
  • Familiarity with Retrieval-Augmented Generation (RAG), vector databases, embedding models, or multimodal AI systems.
  • Experience building benchmarking frameworks, automated evaluation pipelines, or testing infrastructure.
  • Contributions to open-source ML/NLP projects or published research are a plus.
  • Experience working with production-scale machine learning systems.

Role Details

  • Employment Type: Independent Contractor
  • Work Arrangement: Fully Remote
  • Schedule: Approximately 20 hours per week
  • Project Duration: Based on project requirements and performance, with opportunities for extension

Equal Opportunity

We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.

Contract & Payment Terms

  • You will be engaged as an independent contractor .
  • This is a fully remote role that can be completed on your own schedule.
  • Projects may be extended, shortened, or concluded early depending on business needs and performance.
  • Your work will not involve access to confidential or proprietary information from any employer, client, or institution.
  • Payments are made weekly via Stripe or Wise based on services rendered.
  • Please note: We are unable to support H1-B or STEM OPT candidates at this time.

Where you can work

This role is 100% remote, for applicants located in United States.

From Weekday's job posting, checked Oct 18, 2026.

The application

Just your details and CV.

Have readyCV

From Weekday's apply page, checked Oct 18, 2026.

About Weekday

A recruiting company that staffs AI labs' expert training projects, most of them US contracts paying $60 to $160 an hour.

How it pays: Not stated on its postings, which describe independent contract work for the client; its other contract postings say pay goes out weekly through Stripe or Wise.

Similar open roles

Some links on AITraining.jobs are referral links. If you sign up through one, AITraining.jobs may earn a commission at no extra cost to you. It never changes what a platform pays you.

Want companies to ask for you?

One profile for every AI training platform. Companies ask us for people with a specific language, country or credential: add yours once, and when a role fits, we introduce you, only with your yes. Free.

Create my profile

Get new Weekday roles by email

We check the Weekday job board daily and email you its newest roles each week, plus similar paid roles elsewhere. Free, unsubscribe anytime.