RemoteLeaf Logo
Log in Sign up

Senior Python Engineer - AI Coding Agent Evaluation (Freelance)

1 day, 19 hours ago
New Zealand, Japan, Singapore, South Korea
Freelance
Lead
AI (Artificial Intelligence)
Software Development
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 257

Description

  • Build realistic developer environments with a virtual company codebase, infrastructure, tickets, docs, and conversations.
  • Design tasks from intermediate environment states, including the prompt and the definition of a solved outcome.
  • Write tests that verify agent solutions and accurately distinguish correct from incorrect approaches.
  • Review agent solutions, analyze failures, and refine tasks and tests based on QA feedback.
  • Iterate on evaluation criteria to make assessments fair, robust, and solvable by an AI agent.
  • Evaluate how well AI coding agents handle real-world developer tasks.

Requirements

  • 8+ years of experience in software development.
  • Experience with Python (FastAPI).
  • Experience with JavaScript/TypeScript (React).
  • Experience with Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2+ level.
  • Submit a CV in English.
  • Availability to work on a project-based basis and meet submission deadlines.
  • Ability to choose your own working schedule within an estimated 30-hour task scope.

Benefits

  • Up to $200/hr equivalent compensation, depending on level and pace.
  • Project-based work with flexible scheduling.
  • Estimated 30 hours per task, with no fixed work schedule.
  • Opportunity to work on AI evaluation projects for leading tech companies.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Epic Games

Senior Backend Engineer, Developer Platform (Horde)

Epic Games 5K-10K Internet Software & Services

Epic Games is hiring a senior programmer for its Unreal Engine developer platform team to work on Horde, the internal platform that supports CI/CD, remote execution, build health, storage, and developer workflows.

United States Canada Full-time Senior Backend Engineer
$173k-$279k
AWS Azure C# CI/CD gRPC MongoDB .NET Redis
1 day, 18 hours ago
Coupang

Senior Staff Back-end Engineer (eCommerce Engineering)

Coupang 1K-5K Internet Software & Services

Coupang is hiring a Senior Staff Backend Engineer to lead the architecture and technical strategy for its core e-commerce platform services across global markets.

South Korea Full-time Lead Backend Engineer
Apache Airflow Apache Spark AWS Cassandra Grafana Java Kafka Kotlin Kubernetes Microservices OpenTelemetry Prometheus Spring
1 day, 18 hours ago
Toloka

Record Your Daily Routine & Get Paid - AI Training (Remote)

Toloka 251-1K Internet Software & Services

Project-based freelance work for an AI training platform where contributors record first-person videos of everyday household activities to help train AI systems and robots.

Costa Rica Dominican Republic Bolivia Uruguay Freelance Entry Level AI (Artificial Intelligence)
1 day, 19 hours ago
Toloka

Freelance: grabación de tareas cotidianas para proyecto de IA

Toloka 251-1K Internet Software & Services

The company is looking for people to record everyday activities from a first-person perspective using a mobile app to help train AI and robotics systems.

Costa Rica Dominican Republic Bolivia Uruguay Part-time Entry Level AI (Artificial Intelligence)
$12k-$12k
1 day, 19 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers