RemoteLeaf Logo
Log in Sign up

Senior Python Engineer - AI Coding Agent Evaluation (Freelance)

23 hours, 3 minutes ago
Spain
Part-time
Lead
AI (Artificial Intelligence)
Software Development
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 276

Description

  • Build realistic developer environments with codebases, infrastructure, and supporting context such as tickets, docs, and conversations.
  • Design tasks from intermediate states of those environments and define what counts as a solved outcome.
  • Write tests that verify agent solutions and correctly accept valid approaches while rejecting incorrect ones.
  • Review agent solutions, analyze failures, and refine tasks and tests based on QA feedback.
  • Iterate until the evaluation is fair, robust, and challenging for advanced AI coding models.
  • Assess whether tasks are solvable by an AI agent and grounded in realistic development scenarios.

Requirements

  • 8+ years of experience in software development.
  • Experience with the core stack: Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • Submit a CV in English.
  • Ability to work on project-based tasks and meet submission deadlines.
  • Strong understanding of how to evaluate code quality and solution correctness.
  • Experience with AI or coding-agent evaluation is implied by the role context (preferred).

Benefits

  • Up to $150/hr equivalent compensation, depending on level and pace.
  • Project-based work with flexibility to choose when and how to work.
  • Tasks are estimated at around 30 hours each.
  • Paid work after passing qualification(s) and joining a project.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Titan OS

Senior Ruby on Rails Developer

Titan OS 51-200 information technology & services

Titan OS is hiring a Ruby on Rails Developer in Barcelona to build and maintain the backend systems and APIs that power its entertainment and advertising products at scale.

Spain Full-time Senior Backend Engineer
AWS Azure Docker GCP Kubernetes REST API Ruby Ruby on Rails SQL
2 hours, 48 minutes ago
Kaseya

Senior Staff Network Software Engineer

Kaseya 1K-5K IT Services

Kaseya is seeking a Sr. Staff Network Software Engineer to design and deliver scalable, multi-tenant cloud-native networking and security systems for public, private, and hybrid cloud environments.

Canada Full-time Lead Backend Engineer Network Engineer
Cassandra Cybersecurity ELK Stack Git Go Kafka Kubernetes Linux PHP Redis
23 hours, 3 minutes ago
3Cloud

Managing Principal - AI Strategy

3Cloud 251-1K Internet Software & Services

3Cloud is hiring a Managing Principal, AI Strategy to lead its enterprise AI center of excellence and advise clients on complex AI transformations across strategy, architecture, delivery, and adoption.

United States Full-time Lead AI (Artificial Intelligence) Product Strategist
Azure Databricks Generative AI Machine Learning
23 hours, 3 minutes ago
Volga Partners

AI Evaluation & Annotation Reviewer (L3 - Advanced Level) - Italian (Canada)

Volga Partners 51-250 Internet Software & Services

AI Evaluation & Annotation Reviewer role for a long-term artificial intelligence project, focused on reviewing Italian language content and improving the quality of AI responses.

Canada Contract Entry Level AI (Artificial Intelligence) Data Annotator
$0k-$0k
Machine Learning
23 hours, 3 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers