Freelance Agent Evaluation Engineer

10 hours, 9 minutes ago
Freelance
Senior
Artificial Intelligence and Machine Learning
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services

Description

  • Build realistic simulated developer environments containing codebases, infrastructure, tickets, documentation, and conversations.
  • Design challenging tasks from intermediate environment states, including prompts and solution criteria.
  • Write functional and integration tests that accept valid agent solutions and reject incorrect ones.
  • Review agent-generated solutions, analyze failures, and refine tasks and tests based on QA feedback.
  • Ensure tasks are solvable, realistic, fair, and robust for evaluating AI coding agents.

Requirements

  • 5+ years of software development experience.
  • Proficiency with Python and FastAPI.
  • Experience with JavaScript/TypeScript and React.
  • Experience with Docker, PostgreSQL, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • Master’s degree in Computer Science, Software Engineering, Data Science, AI/ML, NLP, Information Systems, or a related field.
  • A bachelor’s degree is accepted with 5 years of relevant experience.
  • At least 3 years of professional experience in related roles or domains, specifically for QA automation/testing or cybersecurity roles.
  • Submit a CV in English and indicate English proficiency.

Benefits

  • Remote, part-time freelance work that can fit around other commitments.
  • Payment per accepted task, with rates up to the equivalent of $50 per hour.
  • Access to advanced AI projects and portfolio-building experience.
  • Opportunity to influence how future AI models perform in software development.
  • Project-based workflow: qualify, join a project, complete tasks, and get paid.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

QA Engineer II

Tactacam 51-250 Household Durables

Tactacam is hiring a QA Engineer to ensure the quality of its outdoor and action camera products by building thorough test coverage and supporting quality throughout the software development lifecycle.

JIRA
8 hours, 54 minutes ago

Director, Advisory - Anthropic & FinOps

phData 251-1K IT Services

phData is hiring a remote-first Consulting Director to lead its Anthropic-focused advisory practice, helping enterprise customers develop AI strategy, governance, organizational change, and FinOps approaches while growing the company’s related services business.

8 hours, 54 minutes ago

Principal AI/ML Engineering Lead

Vida Health 251-1K Health Care Providers & Services

Vida is hiring a senior AI/ML platform engineer to help rebuild its secure healthcare data platform and enable reliable, privacy-preserving machine learning capabilities.

AWS Azure CI/CD dbt Docker Feature Engineering GCP GraphQL HIPAA Kubernetes LLM Machine Learning MLOps Python REST API Secrets Management SQL Terraform Vertex AI
9 hours, 9 minutes ago

Strategic Operations Manager, AI-Enabled Systems (AI-Fluent) Consultant

Right Side Up 51-250 Professional Services

Right Side Up is seeking freelance operations-minded builders for its talent collective to help client teams replace manual workflows and ad hoc tools with scalable processes, AI-enabled systems, and internal applications.

Supabase Vercel
9 hours, 24 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers