RemoteLeaf Logo
Log in Sign up

Senior Python Engineer - AI Coding Agent Evaluation (Freelance)

1 week ago
United States
Part-time
Lead
AI (Artificial Intelligence)
Software Development
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 117

Description

  • Build realistic virtual developer environments with codebases, infrastructure, and supporting context such as tickets, docs, and conversations.
  • Design tasks from intermediate environment states, including prompts and clear definitions of what counts as solved.
  • Write tests that validate agent solutions, accepting all correct approaches and rejecting incorrect ones.
  • Review agent solutions, analyze failures, and refine tasks and tests based on QA feedback.
  • Iterate on evaluation criteria to ensure tasks are fair, robust, and appropriately challenging.
  • Ensure each task is solvable by an AI agent and reflects real-world developer work.
  • Submit completed tasks by the deadline and meet the listed acceptance criteria.
  • Collaborate with the project workflow from qualification through task completion.

Requirements

  • 8+ years of experience in software development.
  • Experience with Python (FastAPI).
  • Experience with JavaScript/TypeScript (React).
  • Experience with Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • Submit your CV in English.
  • Complete qualification(s) before joining a project.
  • Availability to work on a project-based basis, with tasks estimated at about 30 hours each.
  • Ability to choose your own schedule while meeting deadlines.

Benefits

  • Up to $200/hr equivalent compensation, depending on level and pace.
  • Project-based work with flexible scheduling.
  • Estimated 30 hours per task, with no fixed schedule requirement.
  • Opportunity to work with leading tech companies on AI evaluation projects.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Prolific

AI Trainer - Mechanical Engineers - CAD Expertise - (Remote Advisory - France)

Prolific 51-250 Professional Services

Prolific is seeking Mechanical Engineers to join its remote expert network for CAD-focused AI research projects that capture real-world design and validation workflows.

France Contract Senior AI (Artificial Intelligence)
$25k-$250k
Linux Python
3 hours, 55 minutes ago
Prolific

AI Trainer - Mechanical Engineers - CAD Expertise - (Remote Advisory - Germany)

Prolific 51-250 Professional Services

Prolific is seeking mechanical engineers to join a remote advisory talent pool supporting AI research by documenting real-world CAD design and validation workflows.

Germany Contract Senior AI (Artificial Intelligence)
$25k-$250k
Linux Python
4 hours, 11 minutes ago
Scopely

Senior Software Engineer (Gaming Backend) - WWE Champions

Scopely 1K-5K Internet Software & Services

Scopely is hiring a Senior Software Engineer for the WWE Champions live mobile game to own backend systems and support real-time gameplay and live-service operations across North America.

Mexico United States Canada Full-time Senior Backend Engineer Game Developer
$145k-$220k
AWS C# CI/CD Microservices Unity
4 hours, 25 minutes ago
Prolific

AI Trainer - Mechanical Engineers - CAD Expertise - (Remote Advisory - Italy)

Prolific 51-250 Professional Services

Prolific is seeking mechanical engineers to join a remote advisory talent pool supporting AI model development through CAD design challenges and validation workflows.

Italy Freelance Senior AI (Artificial Intelligence)
$0k-$0k
Linux Python
4 hours, 40 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers