Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services

Information Technology

297 open positions

Links

View All Jobs

Freelance Agent Evaluation Engineer

2 days ago

New Zealand, Japan, Malaysia, Singapore, South Korea

Contract

Senior

AI (Artificial Intelligence)

Software Development

Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript

Apply Now

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services

View All Jobs 297

Description

Build realistic virtual developer environments with codebases, infrastructure, tickets, documentation, and conversation history.
Design tasks from intermediate states of the environment and define clear solved criteria.
Write tests that validate AI agent solutions and distinguish correct from incorrect approaches.
Review agent solutions, analyze failures, and refine tasks and tests based on QA feedback.
Ensure tasks are solvable by an AI agent while remaining challenging for frontier models.
Iterate on evaluation criteria until the assessment is fair and robust.

Requirements

5+ years of experience in software development.
Experience with the core stack: Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, and Redis.
Experience writing functional and integration tests.
English proficiency at B2 level or higher.
Submit your CV in English.
Ability to complete project-based tasks by the deadline and meet acceptance criteria.
Experience understanding where models fail in coding tasks is highly relevant.

Benefits

Up to $50/hr equivalent compensation, depending on level and pace.
Project-based work with a flexible schedule.
Estimated 20 hours per task, with timing depending on complexity.
Opportunity to work on AI evaluation projects for leading tech companies.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Freelance Annotator (English) - AI Trainer

Toloka 251-1K Internet Software & Services

Toloka is hiring freelance annotators to support generative AI projects by reviewing and labeling online content for factual accuracy, safety, and clarity when projects are available.

India Freelance Junior AI (Artificial Intelligence) Data Annotator

Up to $19k

Generative AI

1 minute ago

Apply

1 minute ago

Freelance Annotator (English) - AI Trainer

Toloka 251-1K Internet Software & Services

Toloka is hiring freelance annotators for project-based work on generative AI tasks, helping evaluate and improve AI outputs through remote online assignments.

Portugal Part-time Junior AI (Artificial Intelligence) Data Annotator

Up to $0k

Generative AI

1 minute ago

Apply

1 minute ago

Triage Associate I (Contractor)

Latitude AI 501-1000 information technology & services

Latitude AI, a Ford Motor Company subsidiary, is hiring a Triage Operations team member to analyze autonomous vehicle test data and help identify, classify, and escalate performance issues across its ADAS development program.