RemoteLeaf Logo
Log in Sign up

Freelance Agent Evaluation Engineer

10 hours, 38 minutes ago
Germany
Part-time
Senior
AI (Artificial Intelligence)
Software Development
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 183

Description

  • Build realistic developer environments with codebases, infrastructure, and supporting context such as tickets, documentation, and conversations.
  • Design tasks from intermediate states of those environments, including prompts and clear definitions of what counts as solved.
  • Write tests that validate agent solutions, accepting valid approaches and rejecting incorrect ones.
  • Review agent solutions, analyze failure cases, and refine tasks and tests based on QA feedback.
  • Iterate on evaluation criteria until tasks are fair, robust, and solvable by an AI agent.
  • Help create a dataset used to assess how well AI coding agents handle real-world developer work.

Requirements

  • 5+ years of experience in software development.
  • Experience with a core stack including Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • Submit a CV in English.
  • Availability to complete project tasks estimated at about 20 hours each, with work submitted by deadline.
  • Comfort working in a project-based, non-permanent engagement model.

Benefits

  • Up to $50/hr equivalent compensation, depending on level and pace.
  • Flexible schedule with the ability to choose when and how to work.
  • Project-based paid work with task-based submissions.
  • Estimated 20-hour tasks, allowing focused work per assignment.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Toloka

Record Your Daily Routine & Get Paid - AI Training (Remote)

Toloka 251-1K Internet Software & Services

Project-based freelance opportunity with an AI training platform recording first-person videos of everyday household activities to help train AI systems and robots.

Guatemala Freelance Entry Level AI (Artificial Intelligence)
$12k-$12k
9 hours, 53 minutes ago
Toloka

AI Trainer - Freelance Annotator (English)

Toloka 251-1K Internet Software & Services

Mindrift is seeking a project-based AI Trainer - Annotator to evaluate and improve AI responses for e-commerce and online shopping tasks on a non-permanent, part-time basis.

United States Part-time Entry Level AI (Artificial Intelligence) Manual QA Tester
Up to $37k
E-commerce Machine Learning
9 hours, 53 minutes ago
Weekday

Biology & Biophysics Researchers (India, Part-time)

Weekday 11-50 Construction & Engineering

An AI lab client is hiring part-time life science researchers to help train and evaluate frontier AI systems on advanced biological and biophysical reasoning.

India Part-time Lead AI (Artificial Intelligence) Research Scientist
$0k-$0k
Machine Learning
9 hours, 53 minutes ago
Toloka

Creador(a) de Contenido POV para IA - Remoto

Toloka 251-1K Internet Software & Services

Proyecto remoto y flexible con una empresa de entrenamiento de IA para grabar actividades cotidianas desde casa y ayudar a entrenar sistemas de IA y robótica.

Honduras Part-time Entry Level AI (Artificial Intelligence)
$12k-$12k
10 hours, 8 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers