RemoteLeaf Logo
Log in Sign up

Freelance Agent Evaluation Engineer

8 hours, 51 minutes ago
Austria, Czech Republic, Greece, Hungary, Romania
Contract
Senior
AI (Artificial Intelligence)
Software Development
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 183

Description

  • Build realistic developer environments that include a virtual company, codebase, infrastructure, and supporting context such as tickets, documentation, and conversations.
  • Design tasks from intermediate states of those environments, including prompts, success criteria, and solvable problem definitions for an AI agent.
  • Write tests that validate agent solutions by accepting all correct approaches and rejecting incorrect ones.
  • Review agent outputs, analyze failures, and iterate on tasks and tests based on QA feedback.
  • Refine evaluation criteria until tasks are fair, robust, and representative of real-world developer work.
  • Assess whether tasks are sufficiently challenging for frontier AI coding models.
  • Ensure submitted tasks meet acceptance criteria and can be completed within the project framework.

Requirements

  • 5+ years of experience in software development.
  • Experience with a core stack including Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • Ability to understand where AI coding models fail and design scenarios that expose those weaknesses.
  • Ability to work on a project-based basis rather than as permanent employment.
  • Willingness to submit a CV in English and indicate English proficiency.
  • Ability to complete tasks estimated at around 20 hours each and submit them by deadline.

Benefits

  • Up to $50/hr equivalent compensation, depending on level and pace.
  • Flexible project-based schedule with self-directed working hours.
  • Estimated 20-hour tasks that you can complete at your own pace.
  • Opportunity to work on leading-edge AI evaluation projects for tech companies.
  • Paid project work after passing qualification(s) and joining a project.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Toloka

Record Your Daily Routine & Get Paid - AI Training (Remote)

Toloka 251-1K Internet Software & Services

Project-based freelance opportunity with an AI training platform recording first-person videos of everyday household activities to help train AI systems and robots.

Guatemala Freelance Entry Level AI (Artificial Intelligence)
$12k-$12k
8 hours, 20 minutes ago
Toloka

AI Trainer - Freelance Annotator (English)

Toloka 251-1K Internet Software & Services

Mindrift is seeking a project-based AI Trainer - Annotator to evaluate and improve AI responses for e-commerce and online shopping tasks on a non-permanent, part-time basis.

United States Part-time Entry Level AI (Artificial Intelligence) Manual QA Tester
Up to $37k
E-commerce Machine Learning
8 hours, 20 minutes ago
Weekday

Biology & Biophysics Researchers (India, Part-time)

Weekday 11-50 Construction & Engineering

An AI lab client is hiring part-time life science researchers to help train and evaluate frontier AI systems on advanced biological and biophysical reasoning.

India Part-time Lead AI (Artificial Intelligence) Research Scientist
$0k-$0k
Machine Learning
8 hours, 20 minutes ago
Toloka

Creador(a) de Contenido POV para IA - Remoto

Toloka 251-1K Internet Software & Services

Proyecto remoto y flexible con una empresa de entrenamiento de IA para grabar actividades cotidianas desde casa y ayudar a entrenar sistemas de IA y robótica.

Honduras Part-time Entry Level AI (Artificial Intelligence)
$12k-$12k
8 hours, 36 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers