RemoteLeaf Logo
Log in Sign up

Researcher, Benchmark Reviews

1 month, 1 week ago
Anywhere
Full-time
Mid Level
AI (Artificial Intelligence)
Data Science and Analytics
Machine Learning
Apply Now
Epoch AI

Epoch AI

Epoch is a research institute focused on exploring critical trends and questions that influence the development and governance of artificial intelligence, providing valuable insights into its societal implications.

Professional Services
1-10
Founded 2022
View All Jobs 8

Description

  • Review a new AI benchmark at least every two weeks and assess its methodology.
  • Evaluate what strong performance on a benchmark implies about AI capabilities.
  • Write public-facing research reports on new benchmarks.
  • Update published reports as benchmark versions change and models improve.
  • Examine individual tasks within benchmarks in detail.
  • Use coding agents to assist with analysis while maintaining independent judgment.
  • Identify issues, strengths, and weaknesses in benchmark design and evaluation methods.
  • Contribute to a steady stream of benchmark critiques and reviews.

Requirements

  • Strong written communication with publication-quality first drafts and minimal revision needed.
  • Strong critical thinking about research methodologies and benchmark evaluation.
  • Familiarity with existing AI benchmarks, including their methodologies, strengths, and weaknesses.
  • Ability to break down and analyze dense data and complex benchmark task structures.
  • Comfort using coding agents as assistants without delegating judgment.
  • Professional-level English proficiency.
  • Ability to submit application materials in English.
  • Preferred: experience writing about AI/ML for a public or research audience.
  • Preferred: hands-on experience running or building AI evaluations.
  • Preferred: ability to overlap with US and UK time zones.

Benefits

  • Annual salary of $100,000 to $200,000 USD, depending on location, seniority, and experience.
  • Salaries may be paid in local currencies rather than USD, with conversions based on one-year average exchange rates.
  • Fully remote work environment with flexible work hours.
  • Comprehensive global benefits, including health insurance, supplemental local benefits where available, life insurance, and a pension plan where applicable.
  • Generous PTO, including 30 protected PTO days per year, unlimited personal and sick leave, and 4 months of paid parental leave for eligible permanent staff.
  • Flexible expense policy for equipment, productivity tools, and learning/development opportunities, including unlimited spending on AI tools subject to approval.
  • Paid work trips, including three staff retreats per year and relevant conferences.
  • Access to well-equipped Berkeley offices with paid meals, snacks, gym access, and at least 20 office days per year for all staff.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Mindrift.ai: Be the “I” in AI

AI Evaluation Engineer (Python, QA or Security)

Mindrift.ai: Be the “I” in AI Internet Software & Services

Mindrift is seeking experienced software developers for project-based work creating realistic coding evaluation tasks and tests for AI agents.

United States Part-time Senior AI (Artificial Intelligence) QA Engineer
$104k-$104k
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
13 hours, 12 minutes ago
Natera

Lead Bioinformatician (cfDNA Algorithms and Pipelines)

Natera 1K-5K Pharmaceuticals

Natera is hiring a Lead Bioinformatician to develop and support the computational methods behind diagnostic sequencing assays for women’s and organ health.

United States Full-time Senior Data Scientist Research Scientist
$131k-$164k
Prototyping Python
1 day, 12 hours ago
Natera

Associate Director of Bioinformatics (Women's Health and Organ Health)

Natera 1K-5K Pharmaceuticals

Natera is hiring an Associate Director to lead bioinformatics research and assay-development work for women’s health and organ health diagnostics, with responsibility for guiding a team and transitioning new methods into production.

United States Full-time Lead Data Scientist Research Scientist
$167k-$202k
AWS CI/CD Docker GCP Git Java Kubernetes Machine Learning Python R Shell Scripting
1 day, 12 hours ago
RWS Group

Social Media Content Evaluator (Hindi)

RWS Group 5K-10K Internet Software & Services

RWS is hiring a freelance part-time media analyst in India to review short-form videos and produce language classification and locale-validation data for AI training.

India Contract Entry Level AI (Artificial Intelligence) Data Annotator
$12k-$12k
Machine Learning
1 day, 12 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers