RemoteLeaf Logo
Log in Sign up

Freelance Agent Evaluation Engineer

9 hours, 1 minute ago
South Korea
Freelance
Senior
AI (Artificial Intelligence)
Artificial Intelligence and Machine Learning
Cybersecurity Docker FastAPI JavaScript Kafka Machine Learning PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 15

Description

  • Build realistic simulated developer environments with codebases, infrastructure, tickets, documentation, and conversation history.
  • Design challenging coding tasks from intermediate environment states, including prompts and solution criteria.
  • Write functional and integration tests that accept valid agent solutions and reject incorrect ones.
  • Review AI-generated solutions and analyze failures using QA feedback.
  • Iterate on tasks and tests to make evaluations fair, solvable, and robust.
  • Guide and evaluate AI coding agents rather than primarily writing production code from scratch.

Requirements

  • 5+ years of software development experience.
  • Proficiency with Python and FastAPI.
  • Experience with JavaScript or TypeScript and React.
  • Experience with Docker, PostgreSQL, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • Master’s degree in Computer Science, Software Engineering, Data Science, Artificial Intelligence, NLP, Information Systems, or a related field.
  • A bachelor’s degree is accepted with 5 years of relevant experience.
  • At least 3 years of professional experience in related QA automation/testing or cybersecurity roles.
  • Submit a CV in English and indicate English proficiency.

Benefits

  • Remote, part-time freelance work that can fit around other professional or academic commitments.
  • Payment per accepted task, with rates based on qualification tier and efficiency.
  • Earning potential up to the equivalent of $50 per hour.
  • Experience working on advanced AI projects to strengthen your portfolio.
  • Opportunity to influence how future AI models understand and communicate in software development.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Fresh Prints

AI Automation & Ops Lead

Fresh Prints 251-1K Textiles, Apparel & Luxury Goods

Nucleus Network is seeking a remote AI Automation & Operations Lead to identify, build, and implement practical AI-powered workflows that improve operations, relationship data, and scalability across its Advisory, Ventures, and Community businesses.

Philippines Full-time Mid Level AI (Artificial Intelligence) Operations Specialist
ClickUp CRM Cybersecurity Generative AI HubSpot JavaScript LLM Notion Python SQL
8 hours, 46 minutes ago
Acclaro

English to Turkish MTPE & AI Translation Evaluator (Software Localization)

Acclaro 251-1K Professional Services

Acclaro is seeking a freelance English-to-Turkish MT/LLM evaluator to assess and improve AI-generated software, cloud, SaaS, and IT localization for global technology projects.

United States Canada Latin America Freelance Senior AI (Artificial Intelligence) Linguist
9 hours, 1 minute ago
Weekday

Music & Lyrics Expert - Norwegian

Weekday 11-50 Construction & Engineering

A client is seeking a Norwegian Music & Lyrics Expert to evaluate AI-generated lyrics for quality, originality, cultural authenticity, and alignment with intended prompts in a fully remote research project.

United States Contract Mid Level AI (Artificial Intelligence)
$87k-$162k
2 days, 9 hours ago
Toloka

Creador(a) de Contenido POV para IA - Remoto

Toloka 251-1K Internet Software & Services

Participa en un proyecto remoto de entrenamiento de IA grabando actividades cotidianas desde casa con tu propio dispositivo para ayudar a desarrollar sistemas de IA y robótica.

Costa Rica Dominican Republic Bolivia Uruguay Part-time Entry Level AI (Artificial Intelligence)
$12k-$12k
2 days, 9 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers