RemoteLeaf Logo
Log in Sign up

Freelance Agent Evaluation Engineer

37 minutes ago
Mexico, Chile, Peru, Colombia, Costa Rica, Uruguay
Freelance
Senior
AI (Artificial Intelligence)
Software Development
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 295

Description

  • Build realistic virtual developer environments with codebases, infrastructure, and supporting context.
  • Design challenging tasks from intermediate environment states and define clear success criteria.
  • Write tests that verify AI agent solutions and distinguish valid approaches from incorrect ones.
  • Review agent outputs, analyze failures, and refine tasks and tests based on QA feedback.
  • Ensure tasks are solvable by an AI agent while remaining sufficiently challenging.
  • Iterate on evaluation criteria to make assessments fair and robust.
  • Complete project tasks within deadline requirements and submit accepted work for payment.

Requirements

  • 5+ years of experience in software development.
  • Experience with Python, including FastAPI.
  • Experience with JavaScript/TypeScript, including React.
  • Experience with Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • Ability to work on a project-based, non-permanent engagement.
  • Willingness to estimate and complete tasks that may take about 20 hours each.

Benefits

  • Up to $40/hr equivalent compensation, depending on level and pace.
  • Flexible schedule with self-directed working hours.
  • Project-based work with tasks estimated at around 20 hours each.
  • Opportunity to work on AI evaluation projects for leading tech companies.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

DSA

Software Developer

DSA 251-1K Internet Software & Services

Data Systems Analysts, Inc. (DSA) is hiring a Software Developer to support the Organizational Messaging Service National Gateway Centers’ command-and-control messaging interoperability mission.

United States Full-time Junior Software Engineer
$95k-$100k
Angular CI/CD Git Java JavaScript Linux UI Design
7 minutes ago
Gramian Consultancy Group

COBOL Developer (Legacy Systems & AI Training)

Gramian Consultancy Group Professional Services

Gramian Consultancy is seeking an experienced COBOL Developer to support AI training by analyzing legacy enterprise applications, documenting business logic, and creating realistic technical scenarios.

Turkey Poland Romania India Argentina Brazil Contract Senior AI (Artificial Intelligence) Software Engineer
COBOL
37 minutes ago
Intetics

AI / ML Engineer.

Intetics 1K-5K Internet Software & Services

Intetics is seeking an experienced AI/ML Engineer to build and maintain production machine learning solutions for enterprise analytics, forecasting, and AI-driven decision-making.

Poland Romania Bulgaria Czech Republic Ukraine Full-time Senior AI (Artificial Intelligence)
Feature Engineering Generative AI Machine Learning MLflow MLOps NLP Python
37 minutes ago
Topstep

Staff Software Engineer

Topstep 51-250 Capital Markets

Topstep is hiring a Staff Software Engineer to lead the design and operation of the trading platform systems that power real-time trading, market data delivery, and brokerage integrations.

United States Full-time Lead Software Engineer
$200k-$250k
Argo CD AWS C# Docker GitOps Helm Kubernetes Microservices .NET PostgreSQL React SQL Server TypeScript WebSockets
37 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers