RemoteLeaf Logo
Log in Sign up

AI Evaluation Engineer (Python, QA or Security)

13 hours, 1 minute ago
United States
Part-time
Senior
AI (Artificial Intelligence)
Artificial Intelligence and Machine Learning
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 33

Description

  • Build realistic developer environments with codebases, infrastructure, and contextual artifacts such as tickets, docs, and conversations.
  • Design tasks from intermediate states of those environments, including prompts and clear success criteria.
  • Define what it means for a task to be solved and ensure it is solvable by an AI agent.
  • Write tests that validate agent solutions, accepting all correct approaches and rejecting incorrect ones.
  • Review agent solutions and analyze failures to improve task quality.
  • Iterate on tasks and tests based on QA feedback to make evaluations fair and robust.
  • Collaborate in a workflow that includes qualification, project assignment, task completion, and review.

Requirements

  • 5+ years of experience in software development.
  • Experience with the core stack: Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • Submit your CV in English.
  • Ability to work in a project-based arrangement rather than permanent employment.
  • Comfort with evaluating complex coding tasks and identifying where models fail.

Benefits

  • Up to $50/hr equivalent compensation, depending on level and pace.
  • Set your own schedule.
  • Tasks are estimated at about 20 hours each.
  • Project-based paid work with leading tech companies.
  • Opportunity to work on cutting-edge AI evaluation projects.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Robusta Technology Group

Senior QA Engineer - Salesforce | 3-Month Contract | Remote | Immediate Hiring

Robusta Technology Group 51-250 Professional Services

Octopus, part of Robusta Technology Group, is hiring a Senior QA Engineer for a 3-month remote contract to ensure the quality and reliability of Salesforce-based solutions across the development lifecycle.

Egypt Contract Senior QA Engineer Salesforce Developer
Agile CI/CD Salesforce
13 hours, 1 minute ago
RWS Group

Social Media Content Evaluator (Hindi)

RWS Group 5K-10K Internet Software & Services

RWS is hiring a freelance part-time media analyst in India to review short-form videos and produce language classification and locale-validation data for AI training.

India Contract Entry Level AI (Artificial Intelligence) Data Annotator
$12k-$12k
Machine Learning
1 day, 12 hours ago
Kyivstar

QA Engineer (project TutTam)

Kyivstar 1K-5K Wireless Telecommunication Services

Kyivstar.Tech is hiring a QA Engineer for the TutTam FoodTech marketplace to ensure quality across its mobile app and web product throughout the development lifecycle.

Ukraine Full-time Junior QA Engineer
Agile Android CI/CD Confluence Firebase iOS JIRA Kanban Postman REST API Scrum SQL
1 day, 12 hours ago
Warner Music Group

Forward Deployed Engineer, Data & AI

Warner Music Group 5K-10K Media

Warner Music Group is hiring a Forward Deployed Engineer to partner with business stakeholders across labels and global functions to rapidly build data- and AI-driven solutions within its Global Tech Data and Insights organization.

United Kingdom Full-time Mid Level AI (Artificial Intelligence)
Databricks Python React Snowflake
1 day, 12 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers