RemoteLeaf Logo
Log in Sign up

Software Engineer, Benchmarking

1 month, 1 week ago
Anywhere
Full-time
Junior
Software Engineer
Software Development
Cybersecurity Machine Learning
Apply Now
Epoch AI

Epoch AI

Epoch is a research institute focused on exploring critical trends and questions that influence the development and governance of artificial intelligence, providing valuable insights into its societal implications.

Professional Services
1-10
Founded 2022
View All Jobs 8

Description

  • Implement AI benchmarks within the evaluation infrastructure, primarily using the Inspect library.
  • Develop and maintain the existing benchmark suite to support fast evaluation of new model releases.
  • Integrate benchmarks with AI providers and run them on Epoch AI’s infrastructure.
  • Design and develop brand new benchmarks for evaluating AI capabilities.
  • Pitch and prototype new benchmark ideas and related experiments.
  • Facilitate internal experiments and evaluation workflows.
  • Collaborate with researchers, analysts, and engineers to ensure accurate, insightful evaluation outputs are integrated into research products and publications.

Requirements

  • More than 2 years of professional software engineering experience building and maintaining complex systems.
  • Strong engineering skills with the ability to write high-quality, robust, and maintainable code.
  • Comfort working deeply within existing codebases and infrastructure.
  • Ability to generate ideas for new benchmarks, experiments, and other projects.
  • Motivation to support Epoch AI’s mission of delivering rigorous, independent AI evaluations.
  • AI domain expertise or cybersecurity experience is a strong plus but not required.
  • Hands-on experience running LLM evaluations is preferred.
  • Familiarity with evaluation frameworks like Inspect is preferred.
  • Solid grasp of current AI trends is preferred.
  • Professional-level English proficiency is required, and all application materials must be submitted in English.

Benefits

  • Annual salary of $125,000 to $200,000 USD.
  • Compensation can be paid in local currencies rather than only USD.
  • Fully remote work with flexible hours and schedules for most roles.
  • Comprehensive global health insurance, including supplemental local benefits where available and mandated.
  • Life insurance and a pension plan, if applicable in your country.
  • Generous paid time off, including 30 protected days per year, unlimited personal and sick leave, and up to 6 months of parental leave for permanent staff.
  • Flexible expense policy for equipment, productivity tools, and learning or development opportunities.
  • Paid work trips, including 3 staff retreats per year and relevant conferences.
  • Access to Berkeley offices with paid meals, snacks, a gym, and at least 20 office-access days per year for all staff.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Atria Institute

Staff Software Engineer, Foundations

Atria Institute 51-250 Health Care Providers & Services

Atria Health Institute is hiring a Staff Software Engineer to lead platform, DevOps, and data engineering work for its new Los Angeles health care institute focused on preventive, membership-based care.

Brazil Full-time Lead Software Engineer
$200k-$280k
Express.js GCP GitHub HIPAA Mixpanel MySQL Node.js PostgreSQL React Redis System Design Terraform TypeScript
13 hours, 11 minutes ago
MongoDB

Senior Software Engineer

MongoDB 1K-5K Internet Software & Services

MongoDB is hiring a Senior Software Engineer for its fully remote U.S.-based Developer Tools team to build customer-facing tools that help users interact with data and monitor the health and performance of MongoDB deployments at global scale.

United States Full-time Senior Software Engineer
$126k-$248k
AWS Azure GCP Kubernetes Microservices MongoDB Node.js React TypeScript WebSockets
1 day, 12 hours ago
Aegis Ventures

Principal Software Engineer

Aegis Ventures 51-250 Professional Services

Avandra Imaging, backed by Aegis Ventures, is hiring a Principal Software Engineer to lead the design and delivery of secure, scalable healthcare systems that support its medical imaging data platform.

United States Full-time Lead Software Engineer
AWS CI/CD Docker HIPAA Kubernetes
1 day, 12 hours ago
Aegis Ventures

Senior Software Engineer

Aegis Ventures 51-250 Professional Services

Aegis Ventures’ Avandra Imaging is hiring a Senior Software Engineer to help build and scale a healthcare data platform that powers imaging data access and innovation.

United States Full-time Senior Software Engineer
AWS Azure Docker Elixir GCP Kubernetes PostgreSQL Python
1 day, 12 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers