Subject Matter Experts: ML Model Evaluation (PhD)

46 minutes ago
Contract
Senior
Artificial Intelligence and Machine Learning
Weekday

Weekday

Weekday helps companies hire engineers who are vouched by other software engineers, enabling passive income for engineers. They offer services like drafting outreach messages, shortlisting candidates, and conducting reference checks. Backed by Y Combin...

Construction & Engineering
11-50
Founded 2020

Description

  • Review machine learning model outputs for research-focused and domain-specific prompts.
  • Evaluate responses for factual accuracy, relevance, completeness, technical validity, and overall quality.
  • Use a structured rating system to score model responses consistently.
  • Write concise, evidence-based justifications for evaluation scores.
  • Identify factual errors, reasoning gaps, inconsistencies, irrelevant content, and areas for improvement.
  • Complete approximately five hours of independent evaluation work asynchronously.
  • Maintain consistent, detail-oriented evaluation standards throughout the project.

Requirements

  • PhD or equivalent advanced subject matter expertise in a relevant academic, technical, or professional discipline.
  • A finished or ongoing paper in NeurIPS, ICML, ICLR, COLM, or RLC.
  • Strong ability to critically assess complex technical, research, and domain-specific information.
  • Excellent analytical, research, critical-thinking, and written communication skills.
  • Ability to identify subtle inaccuracies, inconsistencies, and gaps in AI-generated content.
  • Ability to work independently in an asynchronous environment.
  • Must be located in the US, UK, Canada, EU, Norway, Switzerland, or Iceland.
  • Must be able to start by Friday end of day.
  • H-1B and STEM OPT candidates are not eligible.

Benefits

  • Pay of $150 per hour.
  • Weekly payments based on services rendered.
  • Fully remote work arrangement.
  • Flexible, asynchronous schedule.
  • Independent contractor engagement.
  • Potential for the project to be extended based on project needs and performance.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

#7 - Senior Data Scientist & AI - Databricks

MUTT DATA 51-200 Internet Software & Services

Muttdata is hiring a Data Scientist to build AI and machine learning solutions for a multinational beverage company’s large-scale B2B digital customer journey across nine countries.

Apache Spark AWS Azure Databricks dbt Deep Learning Feature Engineering GCP Generative AI LLM Machine Learning MLOps Python SQL Statistics
46 minutes ago

Senior AI/ML Platform Engineer

Lyra Health 1001-5000 Health Care Providers & Services

Lyra Health is seeking a Senior AI/ML Platform Engineer to build the production platforms, infrastructure, and services that power machine learning and generative AI products for mental health care.

AWS Celery Docker FastAPI Java Kotlin Kubernetes Neo4j Python WebSockets
3 hours, 16 minutes ago

Senior AI/ML Backend Engineer

Lyra Health 1001-5000 Health Care Providers & Services

Lyra Health is seeking a Senior AI/ML Backend Engineer to build and operate scalable backend services powering AI and machine-learning features for its mental health care platform.

AWS Celery Docker FastAPI Generative AI Java Kotlin Kubernetes Python WebSockets
3 hours, 16 minutes ago

AI Operations Associate

Raydar Professional Services

Join Raydar’s client, a software company, as a remote Quality and Operations Reviewer to oversee automated external communications and resolve cases requiring human judgment during overnight India shifts.

3 hours, 46 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers