RemoteLeaf Logo
Log in Sign up

Freelance Agent Evaluation Engineer

1 hour, 42 minutes ago
South Korea
Part-time
Senior
AI (Artificial Intelligence)
Software Development
Docker FastAPI GitHub Actions JavaScript Kafka Linux PostgreSQL Pytest Python React Redis REST API TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 60

Description

  • Build virtual company environments with codebases, infrastructure, and supporting context such as conversations, documentation, and tickets.
  • Assemble tasks from intermediate states of a virtual company by crafting prompts and defining fair, solvable evaluation criteria.
  • Design tasks in isolated workstation-style environments with Linux tools, MCP servers, and a real web application codebase.
  • Write tests that accept all correct solutions and reject incorrect ones without being overly strict or lenient.
  • Iterate on tests with an AI agent to verify they catch real issues and do not break valid solutions.
  • Review agent-written code to analyze successes and failures and design edge cases and adversarial scenarios.
  • Incorporate feedback from expert QA reviewers and revise tasks to meet quality criteria.
  • Submit completed tasks by the deadline and ensure they meet acceptance requirements.

Requirements

  • Please submit your CV in English and indicate your level of English proficiency.
  • This is project-based, non-permanent work rather than full-time employment.
  • 5+ years of software development experience, primarily in Python.
  • Experience with FastAPI, pytest, async/await, subprocess, and file operations.
  • Background in full-stack development, including React-based interfaces with JavaScript or TypeScript and robust back-end systems.
  • Experience writing functional and integration tests, not only executing tests.
  • Experience working with Docker containers.
  • Familiarity with infrastructure tools such as Postgres, Kafka, and Redis.
  • Understanding of CI/CD workflows, especially GitHub Actions triggers, labels, and reading results.
  • Degree in Computer Science, Software Engineering, or a related field.
  • English proficiency at B2 level.
  • Comfort reading and reasoning about code across the stack.
  • Experience with test automation or testing complex systems is preferred.

Benefits

  • Earn up to $40 per hour equivalent, depending on contribution level and pace.
  • Flexible part-time project work with tasks you choose when and how to complete.
  • Estimated 20 hours of work per project, depending on complexity.
  • Work on leading tech company AI opportunities through Mindrift.
  • Project-based engagements with different earning levels across projects based on scope and expertise.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Escape Velocity Entertainment

AI Designer | North America | Canada | Europe | Fully Remote

Escape Velocity Entertainment Internet Software & Services

Escape Velocity is hiring a remote AI Designer to help create enemy and NPC AI for a high-mobility multiplayer action game.

Anywhere Canada Europe Full-time Junior AI (Artificial Intelligence)
Game Design Unreal Engine
2 hours, 22 minutes ago
Wing Assistant

Language Data Contributor

Wing Assistant 51-250 Professional Services

Wing Data is hiring freelance voice data contributors in West Bengal for remote, project-based AI training work using short speech tasks to help improve client speech systems.

India Freelance Entry Level AI (Artificial Intelligence) Data Annotator
$4k-$8k
2 hours, 30 minutes ago
Provectus

Senior Solution Owner | AI & Data Solutions

Provectus 251-1K Professional Services

Provectus is seeking a Senior Solution Owner to lead client-facing AI and data solution strategy and execution for GenAI, Agentic AI, and large-scale data transformation initiatives.

United States Full-time Senior AI (Artificial Intelligence) Product Owner
Agile AWS Azure Databricks GCP Generative AI LLM Machine Learning Scrum Snowflake
2 hours, 49 minutes ago
ISTA Solutions

PH Nurse | Clinical Auditor (WFH)

ISTA Solutions 251-1K Professional Services

ISTA Solutions is hiring an experienced PHRN Clinical Auditor to support clinical review and documentation compliance work for a rapidly growing outsourcing/offshoring team.

Philippines Full-time Entry Level QA Engineer
3 hours, 23 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers