RemoteLeaf Logo
Log in Sign up

Freelance Agent Evaluation Engineer

6 hours, 43 minutes ago
United Kingdom
Part-time
Senior
AI (Artificial Intelligence)
Software Development
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 97

Description

  • Build realistic developer environments that include a virtual company codebase, infrastructure, and supporting context such as tickets, documentation, and conversations.
  • Design tasks from intermediate states of these environments, including writing the prompt and defining what counts as a solved solution.
  • Ensure each task is solvable by an AI agent while still being challenging and realistic.
  • Write tests that validate agent solutions, accepting all correct approaches and rejecting incorrect ones.
  • Review agent outputs and solution attempts to identify failures and refine tasks and tests based on QA feedback.
  • Iterate on evaluation criteria to make the assessment fair, robust, and reliable.
  • Analyze where models fail and use those insights to create tasks that better distinguish strong solutions from weak ones.

Requirements

  • 5+ years of experience in software development.
  • Experience with the core stack: Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • Ability to work on a project-based, non-permanent engagement.
  • Willingness to submit a CV in English and indicate English proficiency.
  • Experience with building or evaluating software tasks for AI systems is a plus.

Benefits

  • Up to $50/hr equivalent compensation, depending on level and pace.
  • Flexible schedule with the ability to choose when and how to work.
  • Project-based work with an estimated 20 hours per task.
  • Payment upon task completion after meeting acceptance criteria.
  • Opportunity to work on cutting-edge AI evaluation projects for leading tech companies.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

HelloFresh

Senior Analyst

HelloFresh 10K-50K Hotels, Restaurants & Leisure

Senior Analyst at HelloFresh will shape global pricing and discount strategy across multiple meal kit brands, using AI-driven optimization to modernize decision-making and improve customer lifetime value.

United States Full-time Senior AI (Artificial Intelligence) Business Analyst
$119k-$134k
Generative AI Python R SQL Tableau
5 hours, 43 minutes ago
Mercier Consultancy

Portuguese-Speaking GenAI Content Trust and Safety Experts - Work In Greece

Mercier Consultancy Professional Services

Mercier Consultancy MD is hiring Portuguese-Speaking GenAI Content Trust and Safety Experts in Greece to review and safeguard AI-generated content while supporting responsible and compliant use of generative AI.

Spain Bulgaria Greece Portugal Full-time Mid Level AI (Artificial Intelligence) Customer Support
Generative AI
6 hours, 13 minutes ago
Orion Innovation

Senior AI & Data Engineer

Orion Innovation 1K-5K IT Services

Orion Innovation is seeking a Web Delivery Lead to drive end-to-end delivery of AEM-based global websites for its digital engagement platforms, with a focus on reliable launches, governance, and AI-enabled web operations.

United States Full-time Lead AI (Artificial Intelligence) Data Engineer
Agile AWS Azure CDN CI/CD Computer Vision DNS Docker GCP GPT GraphQL Hugging Face Java JavaScript Kubernetes LLM NLP Python PyTorch SEO TensorFlow
6 hours, 13 minutes ago
Pure Storage

AI/HPC/NeoCloud Pre-Sales System Engineer UAE

Pure Storage 1K-5K IT Services

Everpure is seeking a hunting Systems Engineer to support enterprise AI, sovereign cloud, and HPC sales efforts across the METCA region by helping land new infrastructure projects and displace legacy architectures.

United Arab Emirates Saudi Arabia Full-time Senior AI (Artificial Intelligence)
Cisco Deep Learning Kubernetes
6 hours, 28 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers