RemoteLeaf Logo
Log in Sign up

Freelance Agent Evaluation Engineer

10 hours, 38 minutes ago
Czech Republic, Estonia, Latvia, Lithuania
Freelance
Senior
AI (Artificial Intelligence)
Software Development
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 183

Description

  • Build realistic developer environments with codebases, infrastructure, tickets, documentation, and conversation history.
  • Design tasks from intermediate states of these environments and define what counts as a solved solution.
  • Write tests that validate agent outputs, accepting all correct approaches and rejecting incorrect ones.
  • Review agent solutions, analyze failures, and refine tasks and tests based on QA feedback.
  • Iterate on evaluation criteria until tasks are fair, robust, and solvable by an AI agent.
  • Assess how well AI coding agents handle real-world developer tasks.

Requirements

  • 5+ years of experience in software development.
  • Experience with the core stack: Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • CV must be submitted in English.
  • Ability to complete project tasks by the deadline and meet acceptance criteria.
  • Prior experience working in project-based, task-oriented environments is implied.
  • Understanding of where models fail in coding tasks is preferred.

Benefits

  • Up to $40/hr equivalent compensation, depending on level and pace.
  • Project-based flexibility with self-directed scheduling.
  • Estimated 20 hours of work per task, depending on complexity.
  • Opportunity to work on AI evaluation projects for leading tech companies.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Workato

Intern, Forward Deployed Engineering

Workato 251-1K IT Services

Workato is hiring a Forward Deployed Engineering intern to support AI-driven automation initiatives by helping build intelligent agents and enterprise workflow integrations on its Agentic AI platform.

India Internship Entry Level AI Engineer Software Engineer
JavaScript JSON LLM Python REST API Salesforce
9 hours, 38 minutes ago
Black Duck Inn

Software Engineer 3

Black Duck Inn 1K-5K Internet Software & Services

Black Duck Software is seeking a License Developer to evolve legacy licensing systems and build reliable, production-ready services for secure 24/7 customer use.

India Full-time Senior Software Engineer
CI/CD DevSecOps Java Kubernetes Linux REST API Ruby on Rails
9 hours, 38 minutes ago
eClinical Solutions

Statistical Programmer Sr

eClinical Solutions 251-1K Professional Services

Experienced Statistical Programmer role at a clinical research organization focused on delivering compliant statistical programming outputs for multiple clinical studies and regulatory submissions.

India Senior Software Engineer
Git GitHub GitLab R SAP Shell Scripting
9 hours, 38 minutes ago
Career TEAM

Data Conversion Software Engineer

Career TEAM 251-1K Professional Services

Career Team is hiring a Data Conversion Software Engineer to build data transformation and integration software for government-funded workforce development programs across the United States.

United States Full-time Mid Level Data Engineer Software Engineer
$60k-$100k
Agile Angular CI/CD Docker Express.js JavaScript JSON MongoDB NestJS Next.js Node.js React Scrum TypeScript XML
9 hours, 53 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers