RemoteLeaf Logo
Log in Sign up

Senior Python Engineer - AI Coding Agent Evaluation (Freelance)

12 hours, 15 minutes ago
Indonesia, South Africa, Malaysia, Philippines
Contract
Lead
AI (Artificial Intelligence)
Software Development
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 313

Description

  • Build realistic virtual developer environments with codebases, infrastructure, tickets, docs, and conversation history.
  • Design tasks from intermediate environment states, including prompts and clear definitions of what counts as solved.
  • Write tests that validate agent solutions, accepting correct approaches and rejecting incorrect ones.
  • Review agent solutions, analyze failures, and refine tasks and tests based on QA feedback.
  • Iterate until evaluation criteria are fair, robust, and representative of real-world development work.
  • Assess whether tasks are solvable by an AI agent and appropriately challenging for frontier models.

Requirements

  • 8+ years of software development experience.
  • Experience with Python, including FastAPI.
  • Experience with JavaScript or TypeScript, including React.
  • Experience with Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • Submit CV in English and indicate English proficiency.
  • Ability to complete project-based work and meet task deadlines.
  • Experience working across multiple valid solution approaches (preferred).

Benefits

  • Up to $150/hr equivalent compensation, depending on level and pace.
  • Project-based work with flexible scheduling.
  • Estimated ~30 hours per task, with work completed on your own schedule.
  • Opportunity to work on leading-edge AI evaluation projects for tech companies.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Epic Games

Senior Backend Engineer, Developer Platform (Horde)

Epic Games 5K-10K Internet Software & Services

Epic Games is hiring a senior programmer for its Unreal Engine developer platform team to work on Horde, the internal platform that supports CI/CD, remote execution, build health, storage, and developer workflows.

United States Canada Full-time Senior Backend Engineer
$173k-$279k
AWS Azure C# CI/CD gRPC MongoDB .NET Redis
11 hours, 15 minutes ago
Coupang

Senior Staff Back-end Engineer (eCommerce Engineering)

Coupang 1K-5K Internet Software & Services

Coupang is hiring a Senior Staff Backend Engineer to lead the architecture and technical strategy for its core e-commerce platform services across global markets.

South Korea Full-time Lead Backend Engineer
Apache Airflow Apache Spark AWS Cassandra Grafana Java Kafka Kotlin Kubernetes Microservices OpenTelemetry Prometheus Spring
11 hours, 30 minutes ago
Toloka

Record Your Daily Routine & Get Paid - AI Training (Remote)

Toloka 251-1K Internet Software & Services

Project-based freelance work for an AI training platform where contributors record first-person videos of everyday household activities to help train AI systems and robots.

Costa Rica Dominican Republic Bolivia Uruguay Freelance Entry Level AI (Artificial Intelligence)
12 hours ago
Toloka

Freelance: grabación de tareas cotidianas para proyecto de IA

Toloka 251-1K Internet Software & Services

The company is looking for people to record everyday activities from a first-person perspective using a mobile app to help train AI and robotics systems.

Costa Rica Dominican Republic Bolivia Uruguay Part-time Entry Level AI (Artificial Intelligence)
$12k-$12k
12 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers