RemoteLeaf Logo
Log in Sign up

Freelance Agent Evaluation Engineer

49 minutes ago
Japan
Part-time
Senior
AI (Artificial Intelligence)
Software Development
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 295

Description

  • Build realistic developer environments with codebases, infrastructure, and supporting context such as tickets, docs, and conversations.
  • Design tasks from intermediate environment states and define clear success criteria for what counts as solved.
  • Write tests that validate agent solutions, accepting all correct approaches and rejecting incorrect ones.
  • Review agent outputs, analyze failures, and refine tasks and tests based on QA feedback.
  • Iterate on evaluations to ensure the benchmark is fair, robust, and challenging for advanced AI models.
  • Work within project-based assignments rather than permanent employment.
  • Submit completed tasks by the deadline and according to the stated acceptance criteria.

Requirements

  • 5+ years of experience in software development.
  • Experience with Python (FastAPI).
  • Experience with JavaScript/TypeScript (React).
  • Experience with Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • Ability to work on project-based tasks and complete roughly 20-hour assignments.
  • Ability to review model solutions and distinguish valid from invalid implementations.

Benefits

  • Up to $50/hr equivalent compensation, depending on level and pace.
  • Flexible schedule with self-directed work timing.
  • Project-based work with tasks estimated at around 20 hours each.
  • Payment upon completion of accepted tasks.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Nimble Robotics

HIL Test / Automation Engineer

Nimble Robotics 51-250 Industrial Conglomerates

Nimble is hiring an engineer to own a core HIL regression pillar for its autonomous robotics supply chain platform, building automated validation infrastructure for the V6 push.

United States Full-time Junior SDET (Software Development Engineer in Test)
$130k-$170k
C++ CI/CD Embedded Systems Jenkins MATLAB Python Rust
19 minutes ago
Gramian Consultancy Group

COBOL Developer (Legacy Systems & AI Training)

Gramian Consultancy Group Professional Services

Gramian Consultancy is seeking an experienced COBOL Developer to support AI training by analyzing legacy enterprise applications, documenting business logic, and creating realistic technical scenarios.

Turkey Poland Romania India Argentina Brazil Contract Senior AI (Artificial Intelligence) Software Engineer
COBOL
34 minutes ago
Intetics

AI / ML Engineer.

Intetics 1K-5K Internet Software & Services

Intetics is seeking an experienced AI/ML Engineer to build and maintain production machine learning solutions for enterprise analytics, forecasting, and AI-driven decision-making.

Poland Romania Bulgaria Czech Republic Ukraine Full-time Senior AI (Artificial Intelligence)
Feature Engineering Generative AI Machine Learning MLflow MLOps NLP Python
34 minutes ago
Weekday

Personal Financial Advisor (Private Wealth)

Weekday 11-50 Construction & Engineering

Remote contractor role for a Personal Financial Advisor (Private Wealth) supporting the training and evaluation of AI-powered financial advisory tools for a U.S. client.

United States Contract Mid Level AI (Artificial Intelligence) Financial Analyst
$166k-$229k
34 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers