Software Engineer, Benchmarking

3 weeks, 1 day ago
Full-time
Junior
Software Development
Epoch AI

Epoch AI

Epoch is a research institute focused on exploring critical trends and questions that influence the development and governance of artificial intelligence, providing valuable insights into its societal implications.

Professional Services
1-10
Founded 2022

Description

  • Implement AI benchmarks within the evaluation infrastructure, primarily using the Inspect library.
  • Develop and maintain the existing benchmark suite to support fast evaluation of new model releases.
  • Integrate benchmarks with AI providers and run them on Epoch AI’s infrastructure.
  • Design and develop brand new benchmarks for evaluating AI capabilities.
  • Pitch and prototype new benchmark ideas and related experiments.
  • Facilitate internal experiments and evaluation workflows.
  • Collaborate with researchers, analysts, and engineers to ensure accurate, insightful evaluation outputs are integrated into research products and publications.

Requirements

  • More than 2 years of professional software engineering experience building and maintaining complex systems.
  • Strong engineering skills with the ability to write high-quality, robust, and maintainable code.
  • Comfort working deeply within existing codebases and infrastructure.
  • Ability to generate ideas for new benchmarks, experiments, and other projects.
  • Motivation to support Epoch AI’s mission of delivering rigorous, independent AI evaluations.
  • AI domain expertise or cybersecurity experience is a strong plus but not required.
  • Hands-on experience running LLM evaluations is preferred.
  • Familiarity with evaluation frameworks like Inspect is preferred.
  • Solid grasp of current AI trends is preferred.
  • Professional-level English proficiency is required, and all application materials must be submitted in English.

Benefits

  • Annual salary of $125,000 to $200,000 USD.
  • Compensation can be paid in local currencies rather than only USD.
  • Fully remote work with flexible hours and schedules for most roles.
  • Comprehensive global health insurance, including supplemental local benefits where available and mandated.
  • Life insurance and a pension plan, if applicable in your country.
  • Generous paid time off, including 30 protected days per year, unlimited personal and sick leave, and up to 6 months of parental leave for permanent staff.
  • Flexible expense policy for equipment, productivity tools, and learning or development opportunities.
  • Paid work trips, including 3 staff retreats per year and relevant conferences.
  • Access to Berkeley offices with paid meals, snacks, a gym, and at least 20 office-access days per year for all staff.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Software Engineer, Test Infrastructure

Skyline Eco-Adventures 11-50 Hotels, Restaurants & Leisure

Zipline is seeking a Senior Software Systems Validation Engineer to lead end-to-end validation of autonomous drone delivery software, from simulation through full-system flight testing, for safe deployment and operating-domain expansion.

C++ Python Rust SQL
1 day, 21 hours ago

Senior Software Engineer - UI/UX

Dragos 251-1K Professional Services

Dragos is hiring a Software Engineer to build the user-facing experiences that help security professionals detect, investigate, and respond to cyber threats against critical infrastructure.

Cybersecurity Design Systems React TypeScript
1 day, 21 hours ago

Senior Software Engineer

Xebia 1K-5K Internet Software & Services

Xebia is hiring Senior Software Engineers for an international digital payments and fintech project focused on building and evolving a scalable, reliable payment platform.

Agile Apache AWS CI/CD CSS Generative AI Git Hibernate HTML Java JavaScript Jenkins JSON JUnit Maven Microservices REST API Selenium Shell Scripting Spring Boot SQL XML
1 day, 21 hours ago

Software Engineer, Edge

Kargo 251-1K Media

Kargo is hiring a software engineer to develop edge-device machine vision and AI systems that improve freight dock operations through real-time perception and fleet management.

C++ Computer Vision Linux Machine Learning Python
1 day, 21 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers