RemoteLeaf Logo
Log in Sign up

Freelance Agent Evaluation Engineer

44 minutes ago
Argentina
Part-time
Senior
AI (Artificial Intelligence)
Software Development
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 313

Description

  • Build realistic developer environments with codebases, infrastructure, and supporting context such as tickets, documentation, and conversations.
  • Design tasks from intermediate states of these environments and define what counts as a solved solution.
  • Write tests that validate agent solutions and distinguish correct approaches from incorrect ones.
  • Review agent outputs, analyze failures, and refine tasks and tests based on QA feedback.
  • Iterate on evaluation criteria to make assessments fair, robust, and challenging for advanced models.
  • Ensure tasks remain solvable by an AI agent while reflecting realistic developer work.
  • Collaborate through the project workflow from qualification to task completion and submission.

Requirements

  • 5+ years of experience in software development.
  • Experience with Python (FastAPI).
  • Experience with JavaScript/TypeScript (React).
  • Experience with Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • CV must be submitted in English.
  • Ability to complete tasks on a project-based basis and meet submission deadlines.
  • Ability to work through the qualification process before joining a project.

Benefits

  • Up to $30/hr equivalent compensation, depending on level and pace.
  • Project-based, flexible schedule with no fixed work hours.
  • Estimated 20 hours per task, allowing self-paced completion.
  • Opportunity to work on AI evaluation projects for leading tech companies.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

On.Energy

Software & Controls Engineer - Energy Management Systems

On.Energy 51-250 Electric Utilities

ON.energy is hiring a Software & Controls Engineer to develop and commission control software and automation systems for energy storage and power infrastructure projects supporting AI-era power demand.

United States Full-time Mid Level Embedded Systems Engineer Software Engineer
C# C++ CI/CD Cybersecurity Git Python
14 minutes ago
FuseGlobal Partners

MES Application Developer (remote)

FuseGlobal Partners 11-50 Internet Software & Services

FuseGlobal is hiring a remote MES Application Developer for a Fortune 100 food and beverage manufacturer to translate business needs into MES solutions and support factory operations.

Anywhere Contract Senior Software Engineer
$150k-$150k
Agile Angular Azure C# C++ CSS DB2 Entity Framework Hibernate HTML Java JavaScript Scrum SQL Server
29 minutes ago
Toloka

Record Your Daily Routine & Get Paid - AI Training (Remote)

Toloka 251-1K Internet Software & Services

Project-based freelance work for an AI training platform where contributors record first-person videos of everyday household activities to help train AI systems and robots.

Costa Rica Dominican Republic Bolivia Uruguay Freelance Entry Level AI (Artificial Intelligence)
29 minutes ago
Toloka

Freelance: grabación de tareas cotidianas para proyecto de IA

Toloka 251-1K Internet Software & Services

The company is looking for people to record everyday activities from a first-person perspective using a mobile app to help train AI and robotics systems.

Costa Rica Dominican Republic Bolivia Uruguay Part-time Entry Level AI (Artificial Intelligence)
$12k-$12k
29 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers