RemoteLeaf Logo
Log in Sign up

Software Engineer, Benchmarking

2 months ago
Anywhere
Full-time
Junior
Software Engineer
Software Development
Cybersecurity Machine Learning
Apply Now
Epoch AI

Epoch AI

Epoch is a research institute focused on exploring critical trends and questions that influence the development and governance of artificial intelligence, providing valuable insights into its societal implications.

Professional Services
1-10
Founded 2022
View All Jobs 6

Description

  • Implement AI benchmarks within the evaluation infrastructure, primarily using the Inspect library.
  • Develop and maintain the existing benchmark suite to support fast evaluation of new model releases.
  • Integrate benchmarks with AI providers and run them on Epoch AI’s infrastructure.
  • Design and develop brand new benchmarks for evaluating AI capabilities.
  • Pitch and prototype new benchmark ideas and related experiments.
  • Facilitate internal experiments and evaluation workflows.
  • Collaborate with researchers, analysts, and engineers to ensure accurate, insightful evaluation outputs are integrated into research products and publications.

Requirements

  • More than 2 years of professional software engineering experience building and maintaining complex systems.
  • Strong engineering skills with the ability to write high-quality, robust, and maintainable code.
  • Comfort working deeply within existing codebases and infrastructure.
  • Ability to generate ideas for new benchmarks, experiments, and other projects.
  • Motivation to support Epoch AI’s mission of delivering rigorous, independent AI evaluations.
  • AI domain expertise or cybersecurity experience is a strong plus but not required.
  • Hands-on experience running LLM evaluations is preferred.
  • Familiarity with evaluation frameworks like Inspect is preferred.
  • Solid grasp of current AI trends is preferred.
  • Professional-level English proficiency is required, and all application materials must be submitted in English.

Benefits

  • Annual salary of $125,000 to $200,000 USD.
  • Compensation can be paid in local currencies rather than only USD.
  • Fully remote work with flexible hours and schedules for most roles.
  • Comprehensive global health insurance, including supplemental local benefits where available and mandated.
  • Life insurance and a pension plan, if applicable in your country.
  • Generous paid time off, including 30 protected days per year, unlimited personal and sick leave, and up to 6 months of parental leave for permanent staff.
  • Flexible expense policy for equipment, productivity tools, and learning or development opportunities.
  • Paid work trips, including 3 staff retreats per year and relevant conferences.
  • Access to Berkeley offices with paid meals, snacks, a gym, and at least 20 office-access days per year for all staff.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Torc

Software Engineer, II - Data Engineering

Torc 251-1K Road & Rail

Software Engineer at Torc, a Daimler company developing automated-truck software, building AWS-native data ingestion, ETL, and storage systems that support enterprise analytics, simulation, and machine-learning training.

United States Full-time Mid Level Software Engineer
$139k-$167k
Apache Airflow AWS Bash CI/CD Databricks Datadog Docker GitHub Actions JavaScript Jenkins Linux NumPy Pandas Python React Terraform
15 hours, 46 minutes ago
Webflow

Senior Software Engineer, Build Loop

Webflow 251-1K Internet Software & Services

Webflow is hiring a Senior Software Engineer, Developer Productivity, to build platform tooling and migration infrastructure that helps product teams develop, deploy, and operate backend services in its next-generation monorepo.

Argentina Full-time Senior Software Engineer
Argo CD AWS Cloudflare Kubernetes NestJS Node.js Pulumi TypeScript
16 hours, 1 minute ago
TreviPay

Software Engineer II

TreviPay 251-1K Diversified Financial Services

TreviPay is seeking a Software Engineer to build and maintain secure, high-performing applications and fintech services powering its global B2B payments and invoicing platform.

Costa Rica Full-time Mid Level Software Engineer
Agile Angular AWS CI/CD Docker DynamoDB GitLab CI Heroku Java Kubernetes NestJS Node.js OpenAPI PostgreSQL REST API Spring Boot TypeScript Vue.js
16 hours, 31 minutes ago
Gynger

Senior Software Engineer (Remote)

Gynger 11-50 Diversified Financial Services

Gynger is hiring a Senior Software Engineer in NYC to build AI-native infrastructure for tech financing, including agentic systems that manage underwriting, risk monitoring, collections, and payments.

United States Full-time Senior Software Engineer
$175k-$215k
AWS Node.js React TypeScript
16 hours, 46 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers