RemoteLeaf Logo
Log in Sign up

Freelance Agent Evaluation Engineer

10 hours, 39 minutes ago
South Korea
Part-time
Senior
AI (Artificial Intelligence)
Software Development
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 183

Description

  • Build realistic virtual developer environments with codebases, infrastructure, and supporting context such as tickets, documentation, and conversations.
  • Design tasks from intermediate states of those environments, including prompts, success criteria, and solvable problem definitions for AI agents.
  • Write tests that validate agent solutions by accepting all correct approaches and rejecting incorrect ones.
  • Review agent outputs, analyze failures, and refine tasks and tests based on QA feedback.
  • Iterate on evaluation artifacts until the task design and test coverage are fair and robust.
  • Ensure the created tasks genuinely challenge frontier coding models within realistic development scenarios.

Requirements

  • 5+ years of experience in software development.
  • Strong experience with Python, including FastAPI.
  • Strong experience with JavaScript or TypeScript, including React.
  • Experience with Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or above.
  • CV must be submitted in English.
  • Ability to complete project tasks by the deadline and meet acceptance criteria.
  • Experience understanding where AI coding models fail is important for this work.

Benefits

  • Up to $50/hr equivalent compensation, depending on level and pace.
  • Project-based work with flexible scheduling; you choose when and how to work.
  • Estimated 20 hours per task, depending on complexity.
  • Payment upon completion of accepted tasks.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Workato

Intern, Forward Deployed Engineering

Workato 251-1K IT Services

Workato is hiring a Forward Deployed Engineering intern to support AI-driven automation initiatives by helping build intelligent agents and enterprise workflow integrations on its Agentic AI platform.

India Internship Entry Level AI Engineer Software Engineer
JavaScript JSON LLM Python REST API Salesforce
9 hours, 39 minutes ago
Black Duck Inn

Software Engineer 3

Black Duck Inn 1K-5K Internet Software & Services

Black Duck Software is seeking a License Developer to evolve legacy licensing systems and build reliable, production-ready services for secure 24/7 customer use.

India Full-time Senior Software Engineer
CI/CD DevSecOps Java Kubernetes Linux REST API Ruby on Rails
9 hours, 39 minutes ago
eClinical Solutions

Statistical Programmer Sr

eClinical Solutions 251-1K Professional Services

Experienced Statistical Programmer role at a clinical research organization focused on delivering compliant statistical programming outputs for multiple clinical studies and regulatory submissions.

India Senior Software Engineer
Git GitHub GitLab R SAP Shell Scripting
9 hours, 39 minutes ago
Career TEAM

Data Conversion Software Engineer

Career TEAM 251-1K Professional Services

Career Team is hiring a Data Conversion Software Engineer to build data transformation and integration software for government-funded workforce development programs across the United States.

United States Full-time Mid Level Data Engineer Software Engineer
$60k-$100k
Agile Angular CI/CD Docker Express.js JavaScript JSON MongoDB NestJS Next.js Node.js React Scrum TypeScript XML
9 hours, 54 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers