RemoteLeaf Logo
Log in Sign up

Freelance Agent Evaluation Engineer

44 minutes ago
Germany
Contract
Senior
AI (Artificial Intelligence)
Software Development
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 313

Description

  • Build realistic virtual developer environments with codebase, infrastructure, and supporting context.
  • Design tasks from intermediate states of the environment and define what counts as a solved solution.
  • Write tests that validate agent solutions and accurately accept all correct approaches.
  • Review agent outputs, analyze failures, and refine tasks and tests based on QA feedback.
  • Iterate until the evaluation is fair, robust, and challenging for advanced AI models.
  • Ensure tasks are solvable by an AI agent and grounded in realistic development scenarios.
  • Complete project tasks within the acceptance criteria and submission deadline.

Requirements

  • 5+ years of experience in software development.
  • Experience with Python (FastAPI).
  • Experience with JavaScript/TypeScript (React).
  • Experience with Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2+ level.
  • CV must be submitted in English.
  • Participation is project-based, not permanent employment.

Benefits

  • Up to $50/hr equivalent compensation, depending on level and pace.
  • Flexible schedule with tasks completed on your own time.
  • Estimated 20 hours per task, depending on complexity.
  • Project-based paid work after passing qualification(s).

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

On.Energy

Software & Controls Engineer - Energy Management Systems

On.Energy 51-250 Electric Utilities

ON.energy is hiring a Software & Controls Engineer to develop and commission control software and automation systems for energy storage and power infrastructure projects supporting AI-era power demand.

United States Full-time Mid Level Embedded Systems Engineer Software Engineer
C# C++ CI/CD Cybersecurity Git Python
14 minutes ago
FuseGlobal Partners

MES Application Developer (remote)

FuseGlobal Partners 11-50 Internet Software & Services

FuseGlobal is hiring a remote MES Application Developer for a Fortune 100 food and beverage manufacturer to translate business needs into MES solutions and support factory operations.

Anywhere Contract Senior Software Engineer
$150k-$150k
Agile Angular Azure C# C++ CSS DB2 Entity Framework Hibernate HTML Java JavaScript Scrum SQL Server
29 minutes ago
Toloka

Record Your Daily Routine & Get Paid - AI Training (Remote)

Toloka 251-1K Internet Software & Services

Project-based freelance work for an AI training platform where contributors record first-person videos of everyday household activities to help train AI systems and robots.

Costa Rica Dominican Republic Bolivia Uruguay Freelance Entry Level AI (Artificial Intelligence)
29 minutes ago
Toloka

Freelance: grabación de tareas cotidianas para proyecto de IA

Toloka 251-1K Internet Software & Services

The company is looking for people to record everyday activities from a first-person perspective using a mobile app to help train AI and robotics systems.

Costa Rica Dominican Republic Bolivia Uruguay Part-time Entry Level AI (Artificial Intelligence)
$12k-$12k
29 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers