Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services

Information Technology

166 open positions

Links

View All Jobs

Freelance Agent Evaluation Engineer

12 hours, 38 minutes ago

Mexico, Chile, Peru, Colombia, Costa Rica, Uruguay

Part-time

Senior

AI (Artificial Intelligence)

Software Development

Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript

Apply Now

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services

View All Jobs 166

Description

Build realistic virtual developer environments with codebases, infrastructure, and supporting context such as tickets, documentation, and conversations.
Design evaluation tasks from intermediate states of those environments, including crafting prompts and defining what counts as a solved task.
Create tests that verify agent solutions by accepting valid approaches and rejecting incorrect ones.
Review AI agent solutions, analyze failures, and refine tasks and tests based on QA feedback.
Iterate on evaluation criteria to ensure tasks are fair, robust, and solvable by an AI agent.
Assess whether tasks genuinely challenge frontier coding models and reveal meaningful differences in solution quality.

Requirements

5+ years of experience in software development.
Experience with the core stack: Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, and Redis.
Experience writing functional and integration tests.
English proficiency at B2 level or higher.
Submit your CV in English.
Participation is project-based rather than permanent employment.
Ability to complete tasks by the deadline and meet the listed acceptance criteria.
Understanding of how to evaluate real-world developer tasks and identify model failure modes.
Experience with creating tasks that allow multiple valid solutions is preferred.

Benefits

Up to $40/hr equivalent compensation, depending on level and pace.
Flexible schedule with the ability to choose when and how you work.
Estimated 20 hours per task, with work completed on your own timeline.
Project-based opportunity with paid task completion.
Opportunity to work on AI evaluation projects for leading tech companies.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

MBA Internship - GTM Analytics & AI Strategy

Ottimate IT Services

Ottimate is seeking a part-time MBA or graduate intern in Mexico to work with GTM leadership on AI- and data-driven initiatives that improve business performance tracking, analysis, and forecasting.

Mexico Part-time Entry Level AI (Artificial Intelligence) Data Analyst

$42k-$42k

CRM Generative AI LLM Pandas Power BI Python SQL Tableau

12 hours, 8 minutes ago

Apply

12 hours, 8 minutes ago

Part Time Instructor - Fullstack AI

Fullstack Academy 11-50 Diversified Consumer Services

Fullstack Academy is hiring a part-time online instructor for its Fullstack AI program to teach adult learners, support their progress, and connect modern full stack and AI concepts to real-world application development.

United States Part-time Senior AI (Artificial Intelligence) Teacher

$104k-$125k

Agile Angular AWS Azure CI/CD CSS Django Express.js Flask GCP Generative AI Git HTML JavaScript MongoDB Node.js Python React REST API

12 hours, 8 minutes ago

Apply

12 hours, 8 minutes ago

Creador(a) de Contenido POV para IA - Remoto

Toloka 251-1K Internet Software & Services

Proyecto remoto y flexible con una empresa de entrenamiento de IA que paga a personas por grabar actividades cotidianas desde casa para ayudar a entrenar sistemas de IA y robótica.