RemoteLeaf Logo
Log in Sign up

Senior Python Engineer - AI Coding Agent Evaluation (Freelance)

23 hours, 18 minutes ago
Indonesia, Malaysia, Philippines, South Africa
Part-time
Lead
AI (Artificial Intelligence)
Software Development
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 276

Description

  • Build realistic developer environments that include a virtual company codebase, infrastructure, tickets, docs, and conversations.
  • Design coding tasks from intermediate states of those environments and define what counts as a solved outcome.
  • Write tests that accurately verify agent solutions, accepting valid approaches and rejecting incorrect ones.
  • Review agent solutions, analyze failures, and refine tasks and tests based on QA feedback.
  • Iterate on evaluation criteria to make tasks fair, robust, and solvable by AI agents.
  • Help evaluate how well AI coding agents handle real-world developer tasks.

Requirements

  • 8+ years of experience in software development.
  • Experience with Python, including FastAPI.
  • Experience with JavaScript/TypeScript, including React.
  • Experience with Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2+ level.
  • Submit a CV in English.
  • Availability for project-based work with tasks submitted by deadline.
  • Ability to work on tasks estimated at around 30 hours each.
  • Understanding of how to evaluate coding solutions and identify valid versus invalid approaches.

Benefits

  • Up to $150/hr equivalent compensation, depending on level and pace.
  • Flexible schedule: choose when and how to work.
  • Project-based work rather than permanent employment.
  • Tasks estimated at ~30 hours each, depending on complexity.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Titan OS

Senior Ruby on Rails Developer

Titan OS 51-200 information technology & services

Titan OS is hiring a Ruby on Rails Developer in Barcelona to build and maintain the backend systems and APIs that power its entertainment and advertising products at scale.

Spain Full-time Senior Backend Engineer
AWS Azure Docker GCP Kubernetes REST API Ruby Ruby on Rails SQL
2 hours, 48 minutes ago
Kaseya

Senior Staff Network Software Engineer

Kaseya 1K-5K IT Services

Kaseya is seeking a Sr. Staff Network Software Engineer to design and deliver scalable, multi-tenant cloud-native networking and security systems for public, private, and hybrid cloud environments.

Canada Full-time Lead Backend Engineer Network Engineer
Cassandra Cybersecurity ELK Stack Git Go Kafka Kubernetes Linux PHP Redis
23 hours, 3 minutes ago
3Cloud

Managing Principal - AI Strategy

3Cloud 251-1K Internet Software & Services

3Cloud is hiring a Managing Principal, AI Strategy to lead its enterprise AI center of excellence and advise clients on complex AI transformations across strategy, architecture, delivery, and adoption.

United States Full-time Lead AI (Artificial Intelligence) Product Strategist
Azure Databricks Generative AI Machine Learning
23 hours, 3 minutes ago
Volga Partners

AI Evaluation & Annotation Reviewer (L3 - Advanced Level) - Italian (Canada)

Volga Partners 51-250 Internet Software & Services

AI Evaluation & Annotation Reviewer role for a long-term artificial intelligence project, focused on reviewing Italian language content and improving the quality of AI responses.

Canada Contract Entry Level AI (Artificial Intelligence) Data Annotator
$0k-$0k
Machine Learning
23 hours, 3 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers