RemoteLeaf Logo
Log in Sign up

Freelance Agent Evaluation Engineer

53 minutes ago
Japan
Part-time
Senior
AI (Artificial Intelligence)
Software Development
Docker FastAPI GitHub Actions JavaScript Kafka Linux PostgreSQL Pytest Python React Redis REST API TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 60

Description

  • Build virtual company environments with codebases, infrastructure, documentation, tickets, and development history.
  • Assemble and calibrate tasks from intermediate states of the virtual company, including prompts and evaluation criteria.
  • Design isolated workstation-based tasks using Linux, developer tools, MCP servers, and a real web application codebase.
  • Write tests that correctly accept valid solutions and reject incorrect ones.
  • Iterate on tests with an AI agent to verify edge cases, failure modes, and solution validity.
  • Review agent-written code and analyze why an agent failed or succeeded.
  • Design adversarial scenarios and edge cases that meaningfully challenge coding agents.
  • Incorporate feedback from expert QA reviewers and refine work to meet quality criteria.

Requirements

  • English proficiency at B2 level; CV must be submitted in English.
  • Degree in Computer Science, Software Engineering, or a related field.
  • 5+ years of software development experience, primarily in Python.
  • Experience with Python tooling such as FastAPI, pytest, async/await, subprocess, and file operations.
  • Background in full-stack development, including React-based interfaces and robust back-end systems.
  • Experience writing functional and integration tests, not just executing them.
  • Experience working with Docker containers and infrastructure tools such as Postgres, Kafka, and Redis.
  • Understanding of CI/CD workflows, especially GitHub Actions triggers, labels, and result checking.
  • Comfort reading and reasoning about code across the full stack.
  • Open to part-time, non-permanent, project-based work.

Benefits

  • Up to $40 per hour equivalent compensation, depending on contribution level and pace.
  • Project-based flexibility with tasks chosen on your own schedule.
  • Estimated 20 hours of work per project, depending on complexity.
  • Opportunity to work with leading tech companies on AI evaluation projects.
  • Possibility to join multiple projects with different earning levels on the platform.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Escape Velocity Entertainment

AI Designer | North America | Canada | Europe | Fully Remote

Escape Velocity Entertainment Internet Software & Services

Escape Velocity is hiring a remote AI Designer to help create enemy and NPC AI for a high-mobility multiplayer action game.

Anywhere Canada Europe Full-time Junior AI (Artificial Intelligence)
Game Design Unreal Engine
2 hours, 20 minutes ago
Wing Assistant

Language Data Contributor

Wing Assistant 51-250 Professional Services

Wing Data is hiring freelance voice data contributors in West Bengal for remote, project-based AI training work using short speech tasks to help improve client speech systems.

India Freelance Entry Level AI (Artificial Intelligence) Data Annotator
$4k-$8k
2 hours, 28 minutes ago
Provectus

Senior Solution Owner | AI & Data Solutions

Provectus 251-1K Professional Services

Provectus is seeking a Senior Solution Owner to lead client-facing AI and data solution strategy and execution for GenAI, Agentic AI, and large-scale data transformation initiatives.

United States Full-time Senior AI (Artificial Intelligence) Product Owner
Agile AWS Azure Databricks GCP Generative AI LLM Machine Learning Scrum Snowflake
2 hours, 47 minutes ago
ISTA Solutions

PH Nurse | Clinical Auditor (WFH)

ISTA Solutions 251-1K Professional Services

ISTA Solutions is hiring an experienced PHRN Clinical Auditor to support clinical review and documentation compliance work for a rapidly growing outsourcing/offshoring team.

Philippines Full-time Entry Level QA Engineer
3 hours, 21 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers