RemoteLeaf Logo
Log in Sign up

Freelance Agent Evaluation Engineer

16 hours, 25 minutes ago
Australia
Part-time
Senior
AI (Artificial Intelligence)
Software Development
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 192

Description

  • Build realistic developer environments with a virtual company codebase, infrastructure, and development context.
  • Design tasks from intermediate states of those environments, including the prompt and definition of a solved outcome.
  • Write tests that verify agent solutions and correctly distinguish valid approaches from incorrect ones.
  • Review agent solutions, analyze failures, and refine tasks and tests based on QA feedback.
  • Iterate on evaluation criteria to ensure the benchmarking process is fair and robust.
  • Ensure tasks are solvable by an AI agent while still being sufficiently challenging for frontier models.
  • Submit completed tasks that meet the listed acceptance criteria by the deadline.

Requirements

  • 5+ years of experience in software development.
  • Experience with the core stack: Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • CV must be submitted in English.
  • Ability to work in a project-based, non-permanent engagement.
  • Comfort working on tasks estimated at around 20 hours each, with self-managed scheduling.

Benefits

  • Up to $50/hr equivalent compensation, depending on level and pace.
  • Flexible schedule with the ability to choose when and how to work.
  • Project-based paid work rather than permanent employment.
  • Tasks estimated at around 20 hours each, depending on complexity.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Helprise

Transformation Coach (AI & Agile)

Helprise

Helprise is seeking an experienced Transformation Coach (AI & Agile) to help a product-based organization evolve toward AI-driven ways of working by guiding teams, leaders, and Agile Release Trains through transformation and continuous improvement.

Poland Full-time Senior AI (Artificial Intelligence) Product Manager
Agile CI/CD Confluence JIRA
15 hours, 55 minutes ago
ChainGPT

Senior AI Engineer - APAC + Africa

ChainGPT 11-50 Internet Software & Services

ChainGPT is hiring a Senior AI Engineer to help build Brain by AIVM, an early-stage platform for verifiable and governed enterprise AI at the intersection of AI, blockchain, and agentic systems.

Singapore South Africa Vietnam Australia Kenya Nigeria Contract Lead AI Engineer Software Engineer
Blockchain System Design
15 hours, 55 minutes ago
TWO95 International

Software Engineer | Remote | Contract to hire

TWO95 International 51-250 Internet Software & Services

Software Engineer for a 100% remote 6+ month contract-to-hire role supporting enterprise data and system integration work for healthcare and financial services contexts.

United States Contract Senior Software Engineer
Angular AWS Ionic Microservices NestJS Oracle Serverless SQL SQL Server TypeScript
16 hours, 10 minutes ago
JoVE

Software Engineer, Backend

JoVE 51-250 Media

JoVE is seeking an experienced Software Engineer to help modernize its web applications and application stack while building performant, well-tested, reusable software in a collaborative product and engineering environment.

India Full-time Senior Software Engineer
Agile AWS CI/CD CloudFormation GitHub JIRA Kafka Microservices MongoDB Node.js PHP PostgreSQL RabbitMQ Redis REST API Scrum Serverless SQL System Design
16 hours, 25 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers