RemoteLeaf Logo
Log in Sign up

Freelance Agent Evaluation Engineer

2 hours, 57 minutes ago
India
Contract
Senior
AI (Artificial Intelligence)
Software Development
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 295

Description

  • Build realistic developer environments with codebases, infrastructure, and supporting context such as tickets, docs, and conversations.
  • Design tasks from intermediate states of the environments and define clear completion criteria.
  • Write tests that validate AI agent solutions and distinguish correct from incorrect approaches.
  • Review agent outputs, analyze failures, and refine tasks and tests based on QA feedback.
  • Ensure evaluation tasks are solvable by an AI agent while still being challenging for frontier models.
  • Iterate on task design until the evaluation is fair, robust, and accurate.

Requirements

  • 5+ years of experience in software development.
  • Experience with Python (FastAPI).
  • Experience with JavaScript/TypeScript (React).
  • Experience with Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2+ level.
  • Submit CV in English and indicate English proficiency level.
  • Ability to work on a project-based, non-permanent basis.

Benefits

  • Up to $30/hr equivalent compensation, depending on level and pace.
  • Project-based flexible schedule; choose when and how to work.
  • Estimated 20 hours per task, depending on complexity.
  • Paid tasks after completing qualification(s) and joining a project.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Skyline Eco-Adventures

Engineering Test Technician – Ground Systems

Skyline Eco-Adventures 11-50 Hotels, Restaurants & Leisure

Zipline is hiring an Engineering Test Technician for its Ground Systems team to build, test, troubleshoot, and improve the electrical hardware and test infrastructure behind its autonomous delivery operations.

United States Full-time Mid Level Hardware Engineer Test Engineer
$85k-$108k
Linux Python
2 hours, 27 minutes ago
Nimble Robotics

Hardware Test Engineer II

Nimble Robotics 51-250 Industrial Conglomerates

Nimble is hiring a hardware test owner to validate the V6 robot and its subsystems for autonomous warehouse deployment.

United States Full-time Mid Level Hardware Engineer Test Engineer
$120k-$155k
Bash Git Python
2 hours, 42 minutes ago
Gramian Consultancy Group

COBOL Developer (Legacy Systems & AI Training)

Gramian Consultancy Group Professional Services

Gramian Consultancy is seeking an experienced COBOL Developer to support AI training by analyzing legacy enterprise applications, documenting business logic, and creating realistic technical scenarios.

Turkey Poland Romania India Argentina Brazil Contract Senior AI (Artificial Intelligence) Software Engineer
COBOL
2 hours, 57 minutes ago
Intetics

AI / ML Engineer.

Intetics 1K-5K Internet Software & Services

Intetics is seeking an experienced AI/ML Engineer to build and maintain production machine learning solutions for enterprise analytics, forecasting, and AI-driven decision-making.

Poland Romania Bulgaria Czech Republic Ukraine Full-time Senior AI (Artificial Intelligence)
Feature Engineering Generative AI Machine Learning MLflow MLOps NLP Python
2 hours, 57 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers