Freelance Agent Evaluation Engineer

11 hours, 58 minutes ago
Part-time
Senior
Software Development
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services

Description

  • Build realistic virtual developer environments with codebases, infrastructure, tickets, documentation, and conversation context.
  • Design tasks from intermediate states of these environments and define clear success criteria.
  • Write tests that validate AI agent solutions and distinguish correct from incorrect approaches.
  • Review agent outputs, analyze failures, and refine tasks and tests based on QA feedback.
  • Ensure tasks are solvable by an AI agent while still being challenging for frontier models.
  • Iterate on evaluation criteria to make assessments fair and robust.
  • Submit completed tasks by the required deadline and meet acceptance criteria.

Requirements

  • 5+ years of software development experience.
  • Experience with Python (FastAPI).
  • Experience with JavaScript/TypeScript (React).
  • Experience with Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2+ level.
  • CV must be submitted in English.
  • Participation is project-based, not permanent employment.

Benefits

  • Up to $30/hr equivalent compensation, depending on level and pace.
  • Flexible schedule with self-directed working hours.
  • Estimated 20 hours per task, depending on complexity.
  • Project-based paid work with a clear application-to-payment workflow.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Software Engineer, Test Infrastructure

Skyline Eco-Adventures 11-50 Hotels, Restaurants & Leisure

Zipline is seeking a Senior Software Systems Validation Engineer to lead end-to-end validation of autonomous drone delivery software, from simulation through full-system flight testing, for safe deployment and operating-domain expansion.

C++ Python Rust SQL
11 hours, 13 minutes ago

Senior Software Engineer - UI/UX

Dragos 251-1K Professional Services

Dragos is hiring a Software Engineer to build the user-facing experiences that help security professionals detect, investigate, and respond to cyber threats against critical infrastructure.

Cybersecurity Design Systems React TypeScript
11 hours, 28 minutes ago

Senior Software Engineer

Xebia 1K-5K Internet Software & Services

Xebia is hiring Senior Software Engineers for an international digital payments and fintech project focused on building and evolving a scalable, reliable payment platform.

Agile Apache AWS CI/CD CSS Generative AI Git Hibernate HTML Java JavaScript Jenkins JSON JUnit Maven Microservices REST API Selenium Shell Scripting Spring Boot SQL XML
11 hours, 28 minutes ago

Software Engineer, Edge

Kargo 251-1K Media

Kargo is hiring a software engineer to develop edge-device machine vision and AI systems that improve freight dock operations through real-time perception and fleet management.

C++ Computer Vision Linux Machine Learning Python
11 hours, 28 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers