RemoteLeaf Logo
Log in Sign up

Senior Python Engineer - AI Coding Agent Evaluation (Freelance)

4 hours, 9 minutes ago
Jamaica, Costa Rica, Uruguay, Ecuador
Freelance
Lead
AI (Artificial Intelligence)
Software Development
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 327

Description

  • Build realistic virtual developer environments with codebases, infrastructure, and project context.
  • Design tasks from intermediate environment states and define clear success criteria.
  • Write tests that verify agent solutions and correctly distinguish valid from invalid approaches.
  • Review agent outputs, analyze failures, and refine tasks and tests based on QA feedback.
  • Ensure evaluation tasks are solvable by an AI agent and fairly measured.
  • Iterate on prompts, task definitions, and test coverage until the evaluation is robust.
  • Submit tasks by the deadline and meet the stated acceptance criteria.

Requirements

  • 8+ years of experience in software development.
  • Experience with Python (FastAPI).
  • Experience with JavaScript/TypeScript (React).
  • Experience with Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2+ level.
  • Submit CV in English and indicate English proficiency level.
  • Availability to work on a project-based, non-permanent basis.
  • Ability to complete tasks estimated at around 30 hours each, with self-managed scheduling.

Benefits

  • Up to $150/hr equivalent compensation, depending on level and pace.
  • Flexible schedule with self-directed work timing.
  • Project-based paid work on AI evaluation tasks.
  • Tasks are estimated at ~30 hours each, allowing manageable assignments.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

TWO95 International

Sr .Net Developer | Contract | King of Prussia, PA

TWO95 International 51-250 Internet Software & Services

Sr. .NET Developer at a company in King of Prussia, PA, supporting and enhancing database-driven applications on a 6+ month contract basis with remote work during COVID.

United States Contract Senior Backend Engineer
Azure C# jQuery
3 hours, 54 minutes ago
lago

AI Specialist

lago Professional Services

AI Specialist at a remote Philippines-based team, focused on building and maintaining internal automation workflows for client projects using no-code and low-code tools.

Philippines Full-time Mid Level AI (Artificial Intelligence)
$17k-$29k
JavaScript Python
4 hours, 9 minutes ago
ThisWay® Global

Solution Architect - AI / HPC Infrastructure (Pre-Sales)

ThisWay® Global 11-50 Professional Services

TWG is hiring a remote U.S.-based Solution Architect for AI/HPC infrastructure to partner with customers, sales, and engineering on designing deployable distributed systems solutions.

United States Full-time Senior AI (Artificial Intelligence) Sales Engineer
4 hours, 9 minutes ago
Better Impact

Senior .NET Developer

Better Impact 11-50 Internet Software & Services

Better Impact is hiring a senior C#/.NET engineer to lead a live migration of its Volunteer Impact platform from VB.NET/.NET Framework to C#/.NET 10 while modernizing the backend and keeping the production system running.

United States Full-time Lead Backend Engineer
C# CI/CD .NET
4 hours, 9 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers