AI Evaluation Engineer (Python, QA or Security)

8 hours, 20 minutes ago
Part-time
Senior
Artificial Intelligence and Machine Learning
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services

Description

  • Build realistic developer environments with codebase, infrastructure, and supporting context such as tickets, documentation, and conversations.
  • Design tasks from intermediate states of those environments, including the prompt and clear completion criteria.
  • Define what a solved task means and ensure each task is solvable by an AI agent.
  • Write tests that validate correct agent solutions while rejecting incorrect ones.
  • Review agent solutions, analyze failures, and iterate on tasks and tests based on QA feedback.
  • Refine evaluation materials until they are fair, robust, and challenging for strong models.

Requirements

  • 5+ years of experience in software development.
  • Experience with Python and FastAPI.
  • Experience with JavaScript or TypeScript and React.
  • Experience with Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • Ability to work on a project-based, non-permanent basis.
  • Strong understanding of where coding models fail and how to design challenging evaluation scenarios.

Benefits

  • Up to $50/hr equivalent compensation, depending on level and pace.
  • Tasks are estimated at about 20 hours each.
  • Flexible schedule with self-directed timing.
  • Project-based AI opportunity with leading tech companies.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

AI Trainer - Mechanical Engineers - CAD Expertise - (Remote Advisory - France)

Prolific 51-250 Professional Services

Prolific is seeking Mechanical Engineers to join its remote expert network for CAD-focused AI research projects that capture real-world design and validation workflows.

Linux Python
6 hours, 50 minutes ago

Senior QA Engineer – API Testing

Medium 51-250 Internet Software & Services

Nuvolar is hiring a Senior QA Engineer to shape quality for a digital health platform supporting complex healthcare workflows and sensitive data within a growing product team.

Agile AWS CI/CD Docker DynamoDB GitHub GitHub Actions Java Maven OpenAPI Playwright PostgreSQL Postman REST API Scrum Spring Boot SQL
7 hours, 5 minutes ago

AI Trainer - Mechanical Engineers - CAD Expertise - (Remote Advisory - Germany)

Prolific 51-250 Professional Services

Prolific is seeking mechanical engineers to join a remote advisory talent pool supporting AI research by documenting real-world CAD design and validation workflows.

Linux Python
7 hours, 5 minutes ago

AI Trainer - Mechanical Engineers - CAD Expertise - (Remote Advisory - Italy)

Prolific 51-250 Professional Services

Prolific is seeking mechanical engineers to join a remote advisory talent pool supporting AI model development through CAD design challenges and validation workflows.

Linux Python
7 hours, 35 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers