Software Engineer, Benchmarking

3 months, 1 week ago
Full-time
Junior
Software Development
Epoch AI

Epoch AI

Epoch is a research institute focused on exploring critical trends and questions that influence the development and governance of artificial intelligence, providing valuable insights into its societal implications.

Professional Services
1-10
Founded 2022

Description

  • Implement AI benchmarks within the evaluation infrastructure, primarily using the Inspect library.
  • Develop and maintain the existing benchmark suite to support fast evaluation of new model releases.
  • Integrate benchmarks with AI providers and run them on Epoch AI’s infrastructure.
  • Design and develop brand new benchmarks for evaluating AI capabilities.
  • Pitch and prototype new benchmark ideas and related experiments.
  • Facilitate internal experiments and evaluation workflows.
  • Collaborate with researchers, analysts, and engineers to ensure accurate, insightful evaluation outputs are integrated into research products and publications.

Requirements

  • More than 2 years of professional software engineering experience building and maintaining complex systems.
  • Strong engineering skills with the ability to write high-quality, robust, and maintainable code.
  • Comfort working deeply within existing codebases and infrastructure.
  • Ability to generate ideas for new benchmarks, experiments, and other projects.
  • Motivation to support Epoch AI’s mission of delivering rigorous, independent AI evaluations.
  • AI domain expertise or cybersecurity experience is a strong plus but not required.
  • Hands-on experience running LLM evaluations is preferred.
  • Familiarity with evaluation frameworks like Inspect is preferred.
  • Solid grasp of current AI trends is preferred.
  • Professional-level English proficiency is required, and all application materials must be submitted in English.

Benefits

  • Annual salary of $125,000 to $200,000 USD.
  • Compensation can be paid in local currencies rather than only USD.
  • Fully remote work with flexible hours and schedules for most roles.
  • Comprehensive global health insurance, including supplemental local benefits where available and mandated.
  • Life insurance and a pension plan, if applicable in your country.
  • Generous paid time off, including 30 protected days per year, unlimited personal and sick leave, and up to 6 months of parental leave for permanent staff.
  • Flexible expense policy for equipment, productivity tools, and learning or development opportunities.
  • Paid work trips, including 3 staff retreats per year and relevant conferences.
  • Access to Berkeley offices with paid meals, snacks, a gym, and at least 20 office-access days per year for all staff.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Senior Software Developer

Aspire Software 251-1K Internet Software & Services

Vocantas, part of Fluent Software Group, is seeking a Senior Software Developer to build secure, reliable workforce communication and scheduling products through hands-on full-stack development with a backend focus.

Angular C# JavaScript Microservices .NET REST API TypeScript Vue.js
7 hours, 38 minutes ago

Software Engineer (Mid-Senior)

Aether Aerospace 501-1000 aviation & aerospace

Aether Aerospace is hiring a full-time Mid-Senior Software Engineer to develop C/C++ real-time software and Angular interfaces for U.S. Department of Defense programs, working remotely with 50% travel to Hollywood, Maryland.

Agile Angular C C++
7 hours, 38 minutes ago

Intelligent Automation Engineer

Coforge 10K-50K IT Services

Coforge is hiring a remote Intelligent Automation Engineer in Costa Rica to design, develop, and optimize scalable automation solutions using UiPath, Power Automate, and AI technologies.

Agile CI/CD Machine Learning Microservices PowerShell Python Scrum
1 day, 6 hours ago

Intelligent Automation Engineer

Coforge 10K-50K IT Services

Coforge is hiring a remote Intelligent Automation Engineer in Colombia to design, develop, and optimize scalable automation solutions using UiPath, Power Automate, and AI technologies.

Agile CI/CD Machine Learning Microservices PowerShell Python Scrum
1 day, 6 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers