RemoteLeaf Logo
Log in Sign up

Freelance Agent Evaluation Engineer

1 hour, 12 minutes ago
South Korea
Contract
Senior
AI (Artificial Intelligence)
Software Development
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 295

Description

  • Build realistic developer environments with codebases, infrastructure, and supporting context such as tickets, docs, and conversations.
  • Design tasks from intermediate states of those environments and define what counts as a solved solution.
  • Write tests to verify AI agent solutions and distinguish valid approaches from incorrect ones.
  • Review agent solutions, analyze failures, and refine tasks and tests based on QA feedback.
  • Ensure evaluation criteria are fair, robust, and neither too strict nor too lenient.
  • Collaborate within the project workflow to complete and submit tasks by the deadline.

Requirements

  • 5+ years of experience in software development.
  • Experience with the core stack: Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • CV must be submitted in English.
  • Ability to work on a project-based, non-permanent engagement.
  • Estimated task completion time of about 20 hours per task, with flexibility to choose when and how to work.

Benefits

  • Up to $50/hr equivalent compensation, depending on level and pace.
  • Flexible schedule with self-directed work timing.
  • Project-based paid work after qualification.
  • Estimated 20 hours per task, allowing you to control your workload.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Coveo

Senior Software Developer, Search

Coveo 251-1K Internet Software & Services

Coveo is hiring a Senior Software Developer for its Search team to advance scalable search technology across its product ecosystem.

Canada Senior Software Engineer
C++ Elasticsearch Java OpenSearch
12 minutes ago
Donorbox

Software Engineer (Full Stack)

Donorbox 51-250 Capital Markets

Donorbox is hiring an Engineer II to build and maintain its client-facing Omnigive donation form and related Rails-powered product features for nonprofits.

Latin America Full-time Mid Level Software Engineer
$52k-$58k
ActiveRecord CSS JavaScript Minitest PostgreSQL Ruby Ruby on Rails SQL
12 minutes ago
Accenture

Software Engineer

Accenture 100K+ Professional Services

Accenture Federal Services is hiring a Software Engineer to develop and enhance mission-critical software systems supporting federal operations.

United States Full-time Senior Software Engineer
$96k-$192k
Python
12 minutes ago
YipitData

Staff Software Engineer, AI

YipitData 251-1K Professional Services

YipitData is hiring a Staff Software Engineer to build AI-enabled corporate analytics products and production agent systems used by major U.S. retailers and brands.

United States Full-time Lead Software Engineer
$180k-$200k
AWS LLM Microservices Python React TypeScript
12 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers