AI Evaluation Engineer (Python, QA or Security)

1 hour, 49 minutes ago
Part-time
Senior
Artificial Intelligence and Machine Learning
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services

Description

  • Build realistic simulated developer environments with codebases, infrastructure, tickets, documentation, and conversation history.
  • Design coding tasks from intermediate environment states, including prompts and clear success criteria.
  • Write functional and integration tests that accept valid agent solutions and reject incorrect ones.
  • Review agent-generated solutions and analyze evaluation failures.
  • Iterate on tasks and tests based on QA feedback to make evaluations fair and robust.

Requirements

  • 5+ years of software development experience.
  • Experience with Python and FastAPI.
  • Experience with JavaScript or TypeScript and React.
  • Experience with Docker, PostgreSQL, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • Ability to submit a CV in English and indicate English proficiency.

Benefits

  • Compensation of up to $40 per hour equivalent, depending on level and pace.
  • Flexible self-managed schedule.
  • Project-based participation with opportunities to join projects after qualification.
  • Tasks estimated at approximately 20 hours each.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Filipino Quality Control Specialist - Project Lyra

Welo Global Professional Services

Welo Data is seeking Filipino-speaking linguistics experts in the Philippines for a four-week remote freelance project evaluating and improving AI-generated language outputs for accuracy, grammar, and cultural relevance.

LLM
49 minutes ago

Arabic (Egyptian) Quality Control Specialist - Project Lyra

Welo Global Professional Services

Welo Data is seeking Arabic (Egyptian)-speaking linguistics experts in Egypt or Saudi Arabia to evaluate and improve AI-generated language outputs for a four-week conversational AI project.

LLM
49 minutes ago

Technology Trainer – Anthropic Enablement-UK/EMEA

NewRocket 251-1K Internet Software & Services

NewRocket is seeking a Technology Trainer to design and deliver Anthropic Claude enablement programs that build technical proficiency and accelerate adoption among employees, partners, and clients.

Confluence Notion
1 day, 1 hour ago

Senior Director, Enterprise Customer Care - AI Strategy

Toast 1K-5K Hotels, Restaurants & Leisure

Toast is seeking a Senior Director of AI Strategy to lead the AI and analytics roadmap for Enterprise Care, turning support data and intelligent tools into measurable improvements for agents, leaders, and customers.

Salesforce
2 days ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers