Freelance Agent Evaluation Engineer

5 hours, 37 minutes ago
Part-time
Senior
Software Development
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services

Description

  • Build realistic developer environments with codebase, infrastructure, tickets, docs, and conversation context.
  • Design tasks from intermediate states of those environments, including prompts and clear success criteria.
  • Define what counts as a solved task and ensure each task is solvable by an AI agent.
  • Write functional and integration tests to verify agent solutions.
  • Review agent outputs, analyze failures, and refine tasks and tests based on QA feedback.
  • Iterate on evaluation criteria to ensure tests are fair, robust, and neither too strict nor too lenient.
  • Evaluate solutions across multiple valid approaches rather than writing code from scratch.
  • Work within project-based AI opportunities focused on testing and improving AI systems.

Requirements

  • 5+ years of experience in software development.
  • Experience with Python and FastAPI.
  • Experience with JavaScript/TypeScript and React.
  • Experience with Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2+ level.
  • Submit a CV in English.
  • Ability to work on a project-based, non-permanent engagement.
  • Experience understanding failure modes in AI coding systems is preferred.

Benefits

  • Up to $40/hr equivalent compensation, depending on level and pace.
  • Estimated 20 hours of work per task, with flexible scheduling.
  • Choose when and how to work, subject to submission deadlines.
  • Project-based paid opportunities through Mindrift.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Protocol Validation Engineer (Exp 10+ Years)

Ethernovia 11-50 Automotive

Ethernovia is seeking a Protocol Validation Engineer to validate Ethernet switch and NIC silicon across the full product lifecycle, from bring-up through pre-production sign-off.

Confluence JIRA Linux Python REST API Wireshark
5 hours, 7 minutes ago

Record Your Daily Routine & Get Paid - AI Training (Remote)

Toloka 251-1K Internet Software & Services

Project-based freelance work for an AI training platform where contributors record first-person videos of everyday household activities to help train AI systems and robots.

5 hours, 37 minutes ago

Freelance: grabación de tareas cotidianas para proyecto de IA

Toloka 251-1K Internet Software & Services

The company is looking for people to record everyday activities from a first-person perspective using a mobile app to help train AI and robotics systems.

5 hours, 37 minutes ago

Record Your Daily Routine & Get Paid - AI Training (Remote)

Toloka 251-1K Internet Software & Services

Project-based freelance work on an AI training platform to record first-person videos of everyday household activities for training AI systems and robots.

5 hours, 37 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers