RemoteLeaf Logo
Log in Sign up

Freelance Agent Evaluation Engineer

11 hours, 1 minute ago
Israel, Jordan, Kuwait, Qatar, Saudi Arabia, Turkey
Contract
Senior
AI (Artificial Intelligence)
Software Development
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 313

Description

  • Build realistic developer environments that include a codebase, infrastructure, and supporting context such as tickets, docs, and conversations.
  • Design tasks from intermediate states of those environments and define what counts as a solved outcome.
  • Write tests that validate agent solutions, accepting all valid approaches and rejecting incorrect ones.
  • Review agent solutions, analyze failures, and refine tasks and tests based on QA feedback.
  • Iterate on evaluation criteria to make tasks fair, robust, and challenging for AI coding agents.
  • Ensure each task is solvable by an AI agent and grounded in realistic developer workflows.

Requirements

  • 5+ years of software development experience.
  • Experience with the core stack: Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • Submit a CV in English.
  • Participation is project-based, not permanent employment.
  • Ability to complete tasks within an estimated 20 hours per task and meet acceptance criteria by the deadline.
  • Preferred ability to understand where AI models fail in coding tasks and design scenarios that expose those weaknesses.

Benefits

  • Up to $50/hr equivalent compensation, depending on level and pace.
  • Flexible schedule with self-directed work hours.
  • Project-based paid work opportunities.
  • Estimated 20 hours of work per task, depending on complexity.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Skyline Eco-Adventures

Software Engineer, Test Infrastructure

Skyline Eco-Adventures 11-50 Hotels, Restaurants & Leisure

Zipline is seeking a Senior Software Systems Validation Engineer to lead end-to-end validation of autonomous drone delivery software, from simulation through full-system flight testing, for safe deployment and operating-domain expansion.

United States Full-time Senior Software Engineer
$150k-$250k
C++ Python Rust SQL
10 hours, 1 minute ago
Dragos

Senior Software Engineer - UI/UX

Dragos 251-1K Professional Services

Dragos is hiring a Software Engineer to build the user-facing experiences that help security professionals detect, investigate, and respond to cyber threats against critical infrastructure.

United States Full-time Senior Frontend Engineer / Web Developer Software Engineer
$165k-$165k
Cybersecurity Design Systems React TypeScript
10 hours, 16 minutes ago
Xebia

Senior Software Engineer

Xebia 1K-5K Internet Software & Services

Xebia is hiring Senior Software Engineers for an international digital payments and fintech project focused on building and evolving a scalable, reliable payment platform.

Moldova Europe Poland Romania Bulgaria Senior Software Engineer
Agile Apache AWS CI/CD CSS Generative AI Git Hibernate HTML Java JavaScript Jenkins JSON JUnit Maven Microservices REST API Selenium Shell Scripting Spring Boot SQL XML
10 hours, 16 minutes ago
Kargo

Software Engineer, Edge

Kargo 251-1K Media

Kargo is hiring a software engineer to develop edge-device machine vision and AI systems that improve freight dock operations through real-time perception and fleet management.

United States Junior Software Engineer
C++ Computer Vision Linux Machine Learning Python
10 hours, 16 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers