AI Evaluation Engineer (Python, QA or Security)

1 hour, 14 minutes ago
Part-time
Senior
Artificial Intelligence and Machine Learning
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services

Description

  • Build realistic developer environments with codebase, infrastructure, and project context such as tickets, docs, and conversations.
  • Design tasks from intermediate environment states, including prompts and clear success criteria.
  • Define what it means for a task to be solved and ensure each task is feasible for an AI agent.
  • Write tests that validate correct solutions while rejecting incorrect ones.
  • Review agent solutions, analyze failures, and iterate on tasks and tests based on QA feedback.
  • Refine evaluation materials until they are fair, robust, and challenging for strong models.
  • Help assess how well AI coding agents handle real-world developer tasks.

Requirements

  • 5+ years of experience in software development.
  • Experience with Python (FastAPI).
  • Experience with JavaScript/TypeScript (React).
  • Experience with Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • Ability to work project-based rather than as permanent employment.

Benefits

  • Up to $40/hr equivalent compensation, depending on level and pace.
  • Tasks estimated at about 20 hours each.
  • Set your own schedule.
  • Project-based AI opportunities with leading tech companies.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

QA Analyst (AI Systems)

CodeRoad 51-250 Internet Software & Services

CodeRoad is hiring a Junior AI Agent Quality Engineer to validate autonomous AI systems, with a focus on testing agent reliability across agentic workflows, RAG, and Python-based APIs.

Docker JMeter JSON Locust OpenTelemetry Playwright Pytest Python REST API Selenium
14 minutes ago

Senior GenAI Transformation Lead

OLIVER 1K-5K Media

OLIVER is hiring a Senior GenAI Transformation Lead in Singapore to drive the identification, implementation, and scaling of AI-enabled transformation across client accounts and internal agency operations.

Generative AI LLM
29 minutes ago

Director- Applied AI & Machine Learning (SME)

3Cloud 251-1K Internet Software & Services

3Cloud is hiring a Director, Applied AI & Machine Learning to lead Microsoft-based AI and data solution strategy for enterprise clients across pre-sales, architecture, and executive engagement.

Azure Databricks Machine Learning
59 minutes ago

Manager, AI Operations

Xsolis 251-1K IT Services

Xsolis is hiring an AI Operations leader to oversee the reliability, observability, and secure production support of its AI portfolio across traditional ML, Generative AI, and agentic AI systems in healthcare.

AWS Datadog Generative AI Machine Learning MLOps
59 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers