Software Engineer, Benchmarking

2 months, 2 weeks ago
Full-time
Junior
Software Development
Epoch AI

Epoch AI

Epoch is a research institute focused on exploring critical trends and questions that influence the development and governance of artificial intelligence, providing valuable insights into its societal implications.

Professional Services
1-10
Founded 2022

Description

  • Implement AI benchmarks within the evaluation infrastructure, primarily using the Inspect library.
  • Develop and maintain the existing benchmark suite to support fast evaluation of new model releases.
  • Integrate benchmarks with AI providers and run them on Epoch AI’s infrastructure.
  • Design and develop brand new benchmarks for evaluating AI capabilities.
  • Pitch and prototype new benchmark ideas and related experiments.
  • Facilitate internal experiments and evaluation workflows.
  • Collaborate with researchers, analysts, and engineers to ensure accurate, insightful evaluation outputs are integrated into research products and publications.

Requirements

  • More than 2 years of professional software engineering experience building and maintaining complex systems.
  • Strong engineering skills with the ability to write high-quality, robust, and maintainable code.
  • Comfort working deeply within existing codebases and infrastructure.
  • Ability to generate ideas for new benchmarks, experiments, and other projects.
  • Motivation to support Epoch AI’s mission of delivering rigorous, independent AI evaluations.
  • AI domain expertise or cybersecurity experience is a strong plus but not required.
  • Hands-on experience running LLM evaluations is preferred.
  • Familiarity with evaluation frameworks like Inspect is preferred.
  • Solid grasp of current AI trends is preferred.
  • Professional-level English proficiency is required, and all application materials must be submitted in English.

Benefits

  • Annual salary of $125,000 to $200,000 USD.
  • Compensation can be paid in local currencies rather than only USD.
  • Fully remote work with flexible hours and schedules for most roles.
  • Comprehensive global health insurance, including supplemental local benefits where available and mandated.
  • Life insurance and a pension plan, if applicable in your country.
  • Generous paid time off, including 30 protected days per year, unlimited personal and sick leave, and up to 6 months of parental leave for permanent staff.
  • Flexible expense policy for equipment, productivity tools, and learning or development opportunities.
  • Paid work trips, including 3 staff retreats per year and relevant conferences.
  • Access to Berkeley offices with paid meals, snacks, a gym, and at least 20 office-access days per year for all staff.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Staff Software Engineer (Malware Detection)

Chainguard 51-250 Internet Software & Services

Chainguard is hiring a Staff Software Engineer to lead the architecture, scalability, reliability, and customer-facing capabilities of its production malware-scanning platform for open source artifacts.

Go Maven Terraform
2 hours ago

Senior Software Engineer (Athena)

Chainguard 51-250 Internet Software & Services

Chainguard is hiring a remote Senior Software Engineer to build Athena, a Go- and GCP-based clearing house that validates enterprise vulnerability data and routes reliable signals to remediation.

Go Terraform
2 hours, 15 minutes ago

[Job-31586] Senior Software Developer - SDLC, Brazil

CI&T 5K-10K Internet Software & Services

CI&T is seeking a Senior Software Developer – AI to support Flow initiatives embedded in strategic client projects, advancing software development lifecycles, technology modernization, and the practical adoption of AI in software engineering.

AWS Azure CI/CD Generative AI Java Kubernetes .NET Node.js Python
2 hours, 15 minutes ago

Staff Software Engineer (Malware Detection)

Chainguard 51-250 Internet Software & Services

Chainguard is hiring a Staff Software Engineer to lead the backend and production infrastructure platform that scans open source artifacts, converts security research into reliable detections, and delivers actionable verdicts to customers.

Go Maven Terraform
2 hours, 30 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers