AI Evaluation Engineer (Python, QA or Security)

1 hour, 53 minutes ago
Part-time
Senior
Artificial Intelligence and Machine Learning
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services

Description

  • Build realistic developer environments with codebases, infrastructure, and supporting context such as tickets, docs, and conversations.
  • Design tasks from intermediate environment states, including prompts and clear definitions of what counts as solved.
  • Write tests that verify agent solutions and correctly accept valid approaches while rejecting incorrect ones.
  • Review agent solutions, analyze failures, and refine tasks and tests based on QA feedback.
  • Iterate until evaluation criteria are fair, robust, and solvable by an AI agent.

Requirements

  • 5+ years of experience in software development.
  • Experience with Python (FastAPI).
  • Experience with JavaScript/TypeScript (React).
  • Experience with Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • Ability to indicate English proficiency and submit a CV in English.
  • Experience working with AI coding agents or evaluating software tasks is implied as relevant, though not explicitly required.

Benefits

  • Up to $30/hr equivalent compensation, depending on level and pace.
  • Project-based work with payment for completed tasks.
  • Self-set schedule with approximately 20 hours per task.
  • Opportunity to work on leading tech companies’ AI evaluation projects.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

AI Trainer - Advanced Mandarin Fluency - France

Prolific 51-250 Professional Services

Prolific is hiring remote AI Trainer - Mandarin participants to join live video conversations that help train AI models using real human speech and interaction.

53 minutes ago

AI Training - Executive Assistant - Memphis, US

Prolific 51-250 Professional Services

Prolific is seeking experienced Executive Assistants to join its expert network and help train and evaluate AI models by documenting real administrative workflows.

53 minutes ago

AI Trainer - Vietnamese - France

Prolific 51-250 Professional Services

Prolific is hiring fluent Vietnamese speakers for a remote, task-based AI training role centered on live conversation data collection to help improve how AI models understand natural human speech.

Machine Learning
53 minutes ago

AI Training - Personal Assistant - Seattle, US

Prolific 51-250 Professional Services

Prolific is recruiting experienced personal and executive assistants to join its participant network and help train and evaluate AI models using real administrative workflows.

53 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers