AI Evaluation Engineer (Python, QA or Security)

14 hours, 1 minute ago
Part-time
Senior
Artificial Intelligence and Machine Learning
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services

Description

  • Build realistic developer environments with codebases, infrastructure, and supporting context such as tickets, docs, and conversations.
  • Design tasks from intermediate environment states, including prompts and clear success criteria.
  • Define what counts as a solved task and ensure each task is solvable by an AI agent.
  • Write tests that validate agent solutions, accepting correct approaches and rejecting incorrect ones.
  • Review agent solutions, analyze failures, and refine tasks and tests based on QA feedback.
  • Iterate on evaluation materials until the benchmark is fair, robust, and effective.
  • Assess model outputs in scenarios that reveal differences between strong and weak solutions.

Requirements

  • 5+ years of experience in software development.
  • Experience with Python, including FastAPI.
  • Experience with JavaScript or TypeScript, including React.
  • Experience with Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • Ability to work in a project-based, non-permanent engagement.
  • Willingness to submit a CV in English and indicate English proficiency.

Benefits

  • Up to $30/hr equivalent compensation, depending on level and pace.
  • Tasks are estimated at approximately 20 hours each.
  • Set your own schedule.
  • Project-based work with leading tech companies.
  • Opportunity to work on AI evaluation projects.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Founding BDR — AI Consulting (Outbound + Pipeline)

PHIZENIX 11-50 information technology & services

Phizenix is hiring its first outbound sales rep to generate and qualify leads for AI agent services and keep a strong pipeline moving to founder-led closing.

CRM Salesforce
13 hours, 1 minute ago

GenAI Motion Graphics Designer

OLIVER 1K-5K Media

OLIVER is hiring a remote GenAI Motion Designer in Toronto to create and localize high-volume motion and static marketing assets for a leading vacuum and floorcare brand across the U.S. and other markets.

Copywriting Design Systems E-commerce Generative AI Illustrator Photoshop
14 hours, 1 minute ago

Materials & Technical Compliance Engineer

rePurpose Global 51-250 Professional Services

rePurpose Global is seeking a Materials & Technical Compliance Engineer to analyze product and packaging materials, determine regulatory classifications, and support defensible compliance filings for packaging sustainability and EPR obligations.

ERP Python SQL
14 hours, 1 minute ago

AI Trainer - Electrical Engineers - CAD/Python Expertise - (Remote Advisory - Norway)

Prolific 51-250 Professional Services

Prolific is recruiting Electrical Engineers for a remote advisory expert network project that helps train AI models using real-world CAD design and validation workflows.

Linux Python
1 day, 12 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers