Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services

Information Technology

297 open positions

Links

View All Jobs

Freelance Agent Evaluation Engineer

2 days ago

Philippines

Part-time

Senior

AI (Artificial Intelligence)

Software Development

Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript

Apply Now

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services

View All Jobs 297

Description

Build realistic virtual developer environments with codebases, infrastructure, and supporting context such as tickets, docs, and conversations.
Design tasks from intermediate environment states, including prompts and clear definitions of what counts as a solved task.
Write tests that verify agent solutions and accurately accept valid approaches while rejecting incorrect ones.
Review agent outputs, analyze failures, and refine tasks and tests based on QA feedback.
Iterate on evaluation criteria to make assessments fair, robust, and challenging for AI coding agents.
Help create realistic scenarios that reveal where coding models succeed or fail.

Requirements

5+ years of experience in software development.
Experience with Python, including FastAPI.
Experience with JavaScript/TypeScript, including React.
Experience with Docker, Postgres, Kafka, and Redis.
Experience writing functional and integration tests.
English proficiency at B2+ level.
CV must be submitted in English.
Ability to complete project-based tasks independently within a deadline.
Not a permanent employment role; participation is project-based.

Benefits

Up to $30/hr equivalent compensation.
Estimated ~20 hours of work per task/project.
Flexible schedule with self-directed work hours.
Project-based work with payment upon completion of tasks.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Freelance Annotator (English) - AI Trainer

Toloka 251-1K Internet Software & Services

Toloka is hiring freelance annotators to support generative AI projects by reviewing and labeling online content for factual accuracy, safety, and clarity when projects are available.

India Freelance Junior AI (Artificial Intelligence) Data Annotator

Up to $19k

Generative AI

0 minutes ago

Apply

0 minutes ago

Freelance Annotator (English) - AI Trainer

Toloka 251-1K Internet Software & Services

Toloka is hiring freelance annotators for project-based work on generative AI tasks, helping evaluate and improve AI outputs through remote online assignments.

Portugal Part-time Junior AI (Artificial Intelligence) Data Annotator

Up to $0k

Generative AI

0 minutes ago

Apply

0 minutes ago

Triage Associate I (Contractor)

Latitude AI 501-1000 information technology & services

Latitude AI, a Ford Motor Company subsidiary, is hiring a Triage Operations team member to analyze autonomous vehicle test data and help identify, classify, and escalate performance issues across its ADAS development program.