AI Evaluation Engineer (Python, QA or Security)

1 hour, 30 minutes ago
Part-time
Senior
Artificial Intelligence and Machine Learning
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services

Description

  • Build realistic virtual developer environments with codebases, infrastructure, and supporting context such as tickets, documentation, and conversations.
  • Design tasks from intermediate states of these environments and define what counts as a solved outcome.
  • Write tests that validate agent solutions and distinguish correct approaches from incorrect ones.
  • Review agent outputs, analyze failures, and refine tasks and tests based on QA feedback.
  • Iterate on evaluation criteria to ensure tasks are fair, robust, and solvable by AI agents.
  • Assess model behavior to identify scenarios that reveal differences between strong and weak solutions.

Requirements

  • 5+ years of software development experience.
  • Experience with Python (FastAPI).
  • Experience with JavaScript/TypeScript (React).
  • Experience with Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • Ability to work on a project-based basis rather than permanent employment.
  • Willingness to submit a CV in English.

Benefits

  • Up to $50/hr equivalent compensation, depending on level and pace.
  • Tasks estimated at approximately 20 hours each.
  • Flexible schedule with self-directed work hours.
  • Project-based paid work opportunities with leading tech companies.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

AI Trainer - Advanced Mandarin Fluency - France

Prolific 51-250 Professional Services

Prolific is hiring remote AI Trainer - Mandarin participants to join live video conversations that help train AI models using real human speech and interaction.

45 minutes ago

AI Training - Executive Assistant - Memphis, US

Prolific 51-250 Professional Services

Prolific is seeking experienced Executive Assistants to join its expert network and help train and evaluate AI models by documenting real administrative workflows.

45 minutes ago

AI Trainer - Vietnamese - France

Prolific 51-250 Professional Services

Prolific is hiring fluent Vietnamese speakers for a remote, task-based AI training role centered on live conversation data collection to help improve how AI models understand natural human speech.

Machine Learning
45 minutes ago

AI Training - Personal Assistant - Seattle, US

Prolific 51-250 Professional Services

Prolific is recruiting experienced personal and executive assistants to join its participant network and help train and evaluate AI models using real administrative workflows.

45 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers