RemoteLeaf Logo
Log in Sign up

Freelance Agent Evaluation Engineer

1 hour, 27 minutes ago
Czech Republic, Estonia, Latvia, Lithuania
Freelance
Senior
AI (Artificial Intelligence)
Artificial Intelligence and Machine Learning
Cybersecurity Docker FastAPI JavaScript Kafka Machine Learning NLP PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 16

Description

  • Build realistic simulated developer environments with codebases, infrastructure, tickets, documentation, and conversations.
  • Design challenging tasks from intermediate environment states, including prompts and success criteria.
  • Write functional and integration tests that accept valid solutions and reject incorrect ones.
  • Review AI agent solutions, analyze failures, and refine tasks and tests based on QA feedback.
  • Ensure evaluation tasks are solvable, fair, robust, and representative of real-world development work.

Requirements

  • 5+ years of software development experience.
  • Proficiency with Python and FastAPI.
  • Experience with JavaScript/TypeScript and React.
  • Experience with Docker, PostgreSQL, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • Master’s degree in computer science, software engineering, data science, AI/ML, NLP, information systems, or a related field.
  • Bachelor’s degree accepted with 5 years of relevant experience.
  • At least 3 years of professional experience in related QA automation/testing or cybersecurity roles, where applicable.

Benefits

  • Remote, part-time freelance work that can fit around other commitments.
  • Payment per accepted task, with rates up to the equivalent of $40 per hour.
  • Opportunities to work on advanced AI projects and strengthen your portfolio.
  • Opportunity to influence how future AI models perform and communicate in software development.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

x.ai

AI Tutor - Video (Weekend)

x.ai 51-250 Internet Software & Services

SpaceXAI is hiring a Video Specialist to curate and annotate multimedia data that improves Grok’s ability to understand and generate technically precise video content in a weekend-focused role.

Anywhere Mid Level AI (Artificial Intelligence)
$83k-$156k
Python
27 minutes ago
Prolific

AI Trainer - Armenian - Greece

Prolific 51-250 Professional Services

Prolific is seeking fluent Armenian speakers to evaluate AI-generated written text for naturalness, authenticity, cultural fit, and linguistic nuance.

Greece Freelance Entry Level AI (Artificial Intelligence)
$42k-$54k
27 minutes ago
Prolific

AI Trainer - Armenian - Poland

Prolific 51-250 Professional Services

Prolific is seeking fluent Armenian speakers to evaluate AI-generated written content for naturalness, authenticity, tone, and cultural accuracy.

Poland Freelance Entry Level AI (Artificial Intelligence)
$42k-$54k
42 minutes ago
Acclaro

Freelance Software Localization and AI Language Evaluator | English to Czech

Acclaro 251-1K Professional Services

Acclaro is seeking remote freelance English-to-Czech Software Localization and AI Language Evaluators to assess and improve localized software and technical content for Czech users across technology projects.

Czech Republic Freelance Senior AI (Artificial Intelligence) Linguist
1 hour, 27 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers