Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services

Information Technology

324 open positions

Links

View All Jobs

Senior Software Engineer - AI Agent Evaluation

1 hour, 35 minutes ago

Canada

Contract

Senior

QA Engineer

Software Development

Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript

Apply Now

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services

View All Jobs 324

Description

Build realistic virtual developer environments with codebase, infrastructure, and supporting context such as tickets, documentation, and conversations.
Design tasks from intermediate states of those environments and define clear success criteria for what counts as solved.
Write functional and integration tests that validate agent solutions and distinguish correct approaches from incorrect ones.
Review agent solutions, analyze failures, and refine tasks and tests based on QA feedback.
Iterate on evaluation materials to make the dataset fair, robust, and challenging for advanced AI models.
Work within the project-based workflow by completing assigned tasks and submitting them by the deadline.
Guide and evaluate the agent's work rather than writing most of the code from scratch.

Requirements

5+ years of experience in software development.
Experience with the core stack: Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, and Redis.
Experience writing tests, including functional and integration tests.
English proficiency at B2 level or higher.
CV must be submitted in English.
Ability to work on a project-based engagement rather than permanent employment.
Understanding of how to assess real-world developer tasks and identify failure modes in AI-generated solutions.
Ability to complete tasks estimated at around 20 hours each, subject to complexity and deadline requirements.

Benefits

Up to $50/hr equivalent compensation, depending on level and pace.
Flexible schedule with tasks completed on your own time.
Project-based paid work with estimated 20-hour task assignments.
Opportunity to work on cutting-edge AI evaluation projects for leading tech companies.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Senior ServiceNow Developer - Public Trust Clearance

LightFeather 51-250 Internet Software & Services

LightFeather is hiring a Senior ServiceNow Developer to design, build, and deploy secure, scalable ServiceNow solutions for federal government projects in a fully remote, full-time role.

United States Full-time Lead Software Engineer

$90k-$115k

Agile Angular AWS Azure CI/CD CSS GCP HTML JavaScript JIRA Kanban REST API Scrum

50 minutes ago

Apply

50 minutes ago

MediaRadar is hiring a Data Quality Engineer to build and operate the automated tooling, monitoring, and dashboards that keep governed data accurate, measurable, and continuously validated across the company’s data pipelines and platforms.

India Full-time Mid Level Data Engineer QA Engineer

Python SQL

1 hour, 5 minutes ago

Apply

1 hour, 5 minutes ago