RemoteLeaf Logo
Log in Sign up

Freelance Agent Evaluation Engineer

49 minutes ago
Argentina
Part-time
Senior
AI (Artificial Intelligence)
Software Development
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 295

Description

  • Build realistic developer environments with codebases, infrastructure, and supporting context such as tickets, docs, and conversations.
  • Design tasks from intermediate states of those environments, including prompts and clear definitions of what counts as a solved solution.
  • Write tests that validate agent solutions, accepting valid approaches and rejecting incorrect ones.
  • Review agent solutions, analyze failures, and refine tasks and tests based on QA feedback.
  • Iterate on evaluation criteria until the dataset and assessments are fair and robust.
  • Assess whether tasks are solvable by an AI agent within the simulated environment.

Requirements

  • 5+ years of experience in software development.
  • Experience with Python, including FastAPI.
  • Experience with JavaScript/TypeScript, including React.
  • Experience with Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • Ability to work on project-based tasks rather than permanent employment.

Benefits

  • Up to $30/hr equivalent compensation, depending on level and pace.
  • Flexible schedule with self-paced work.
  • Tasks estimated at about 20 hours each.
  • Project-based paid work after qualification.
  • Opportunity to work on AI evaluation for leading tech companies.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Nimble Robotics

HIL Test / Automation Engineer

Nimble Robotics 51-250 Industrial Conglomerates

Nimble is hiring an engineer to own a core HIL regression pillar for its autonomous robotics supply chain platform, building automated validation infrastructure for the V6 push.

United States Full-time Junior SDET (Software Development Engineer in Test)
$130k-$170k
C++ CI/CD Embedded Systems Jenkins MATLAB Python Rust
19 minutes ago
Gramian Consultancy Group

COBOL Developer (Legacy Systems & AI Training)

Gramian Consultancy Group Professional Services

Gramian Consultancy is seeking an experienced COBOL Developer to support AI training by analyzing legacy enterprise applications, documenting business logic, and creating realistic technical scenarios.

Turkey Poland Romania India Argentina Brazil Contract Senior AI (Artificial Intelligence) Software Engineer
COBOL
34 minutes ago
Intetics

AI / ML Engineer.

Intetics 1K-5K Internet Software & Services

Intetics is seeking an experienced AI/ML Engineer to build and maintain production machine learning solutions for enterprise analytics, forecasting, and AI-driven decision-making.

Poland Romania Bulgaria Czech Republic Ukraine Full-time Senior AI (Artificial Intelligence)
Feature Engineering Generative AI Machine Learning MLflow MLOps NLP Python
34 minutes ago
Weekday

Personal Financial Advisor (Private Wealth)

Weekday 11-50 Construction & Engineering

Remote contractor role for a Personal Financial Advisor (Private Wealth) supporting the training and evaluation of AI-powered financial advisory tools for a U.S. client.

United States Contract Mid Level AI (Artificial Intelligence) Financial Analyst
$166k-$229k
34 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers