RemoteLeaf Logo
Log in Sign up

Senior Python Engineer - AI Coding Agent Evaluation (Freelance)

8 hours, 32 minutes ago
United Kingdom
Freelance
Lead
AI (Artificial Intelligence)
Software Development
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 327

Description

  • Build realistic virtual developer environments with codebases, infrastructure, and supporting context such as tickets, documentation, and conversations.
  • Design tasks from intermediate environment states, including prompts and clear definitions of what counts as a solved task.
  • Write tests that validate agent solutions by accepting correct approaches and rejecting incorrect ones.
  • Review agent solutions, analyze failures, and iterate on tasks and tests based on QA feedback.
  • Refine evaluation criteria to ensure tasks are fair, solvable, and robust for AI agent assessment.
  • Assess model performance against real-world developer tasks within project-based AI workflows.

Requirements

  • 8+ years of experience in software development.
  • Strong experience with Python, including FastAPI.
  • Experience with JavaScript or TypeScript, including React.
  • Experience with Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2+ level.
  • Submit CV in English.
  • Ability to complete tasks by deadline and meet acceptance criteria.
  • Preferred/implicit fit for evaluating complex coding agent outputs and failure modes.

Benefits

  • Compensation up to $200/hr equivalent, depending on level and pace.
  • Project-based work with no permanent employment commitment.
  • Flexible schedule: choose when and how to work.
  • Estimated ~30 hours of work per task, depending on complexity.
  • Payment for completed tasks that meet acceptance criteria.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

TWO95 International

Sr .Net Developer | Contract | King of Prussia, PA

TWO95 International 51-250 Internet Software & Services

Sr. .NET Developer at a company in King of Prussia, PA, supporting and enhancing database-driven applications on a 6+ month contract basis with remote work during COVID.

United States Contract Senior Backend Engineer
Azure C# jQuery
8 hours, 17 minutes ago
lago

AI Specialist

lago Professional Services

AI Specialist at a remote Philippines-based team, focused on building and maintaining internal automation workflows for client projects using no-code and low-code tools.

Philippines Full-time Mid Level AI (Artificial Intelligence)
$17k-$29k
JavaScript Python
8 hours, 32 minutes ago
ThisWay® Global

Solution Architect - AI / HPC Infrastructure (Pre-Sales)

ThisWay® Global 11-50 Professional Services

TWG is hiring a remote U.S.-based Solution Architect for AI/HPC infrastructure to partner with customers, sales, and engineering on designing deployable distributed systems solutions.

United States Full-time Senior AI (Artificial Intelligence) Sales Engineer
8 hours, 32 minutes ago
Better Impact

Senior .NET Developer

Better Impact 11-50 Internet Software & Services

Better Impact is hiring a senior C#/.NET engineer to lead a live migration of its Volunteer Impact platform from VB.NET/.NET Framework to C#/.NET 10 while modernizing the backend and keeping the production system running.

United States Full-time Lead Backend Engineer
C# CI/CD .NET
8 hours, 32 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers