Freelance Agent Evaluation Engineer

2 hours, 8 minutes ago
Contract
Senior
Software Development
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services

Description

  • Build realistic virtual developer environments with codebases, infrastructure, and supporting context such as tickets, docs, and conversations.
  • Design tasks from intermediate states of these environments, including prompts and clear success criteria.
  • Define what counts as a solved task and ensure each task is solvable by an AI agent.
  • Write tests that verify agent solutions and correctly accept valid approaches while rejecting incorrect ones.
  • Review agent outputs, analyze failures, and refine tasks and tests based on QA feedback.
  • Iterate on evaluation materials until they are fair, robust, and effective for assessing AI coding agents.
  • Collaborate through the project workflow from qualification to task completion and submission.

Requirements

  • 5+ years of experience in software development.
  • Experience with Python (FastAPI).
  • Experience with JavaScript/TypeScript (React).
  • Experience with Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • Ability to complete project tasks within an estimated 20 hours and by the required deadline.
  • CV must be submitted in English.

Benefits

  • Up to $30/hr equivalent compensation, depending on level and pace.
  • Project-based work with flexibility to choose when and how to work.
  • Estimated 20 hours per task, not a fixed schedule requirement.
  • Payment for completed tasks after qualification and acceptance.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

TCAC Test Analyst (Job 1405)

DLH 1K-5K Construction & Engineering

DLH is seeking a TCAC Test Analyst to support SIGINT/EW systems testing and integration for U.S. Marine Corps and MARSOC operations.

Active Directory JIRA Linux
1 hour, 38 minutes ago

Sr Quality Assurance Engineer

ELEVI Associates 11-50 Internet Software & Services

ELEVI is hiring a Senior Quality Assurance tester to support federal system upgrades and modernization efforts across web applications, databases, reporting interfaces, and secure enterprise environments.

Cybersecurity GitLab Java Windows Server
1 hour, 53 minutes ago

Staff Supplier Industrialization Engineer, Interiors Coverstock

Lucid Motors 1K-5K Automotive

Lucid Motors is hiring a Staff Supplier Industrialization Engineer, Interior Trim and Coverstock to industrialize suspension and interior component suppliers and ensure product quality across development through mass production.

1 hour, 53 minutes ago

Record Your Daily Routine & Get Paid - AI Training (Remote)

Toloka 251-1K Internet Software & Services

Project-based freelance work for an AI training platform where contributors record first-person videos of everyday household activities to help train AI systems and robots.

2 hours, 8 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers