RemoteLeaf Logo
Log in Sign up

Senior Python Engineer - AI Coding Agent Evaluation (Freelance)

12 hours, 3 minutes ago
Bahrain, Kuwait, Oman, Qatar, Saudi Arabia, United Arab Emirates
Part-time
Lead
AI (Artificial Intelligence)
Software Development
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 313

Description

  • Build realistic developer environments with codebases, infrastructure, and supporting context such as tickets, docs, and conversations.
  • Design tasks from intermediate environment states, including the prompt and clear criteria for what counts as solved.
  • Write tests that validate agent solutions, accepting valid approaches and rejecting incorrect ones.
  • Review agent solutions, analyze failures, and iterate on tasks and tests based on QA feedback.
  • Refine evaluations until they are fair, robust, and suitable for assessing coding agents.
  • Ensure tasks are solvable by an AI agent while still being challenging for frontier models.

Requirements

  • 8+ years of experience in software development.
  • Strong experience with Python (FastAPI).
  • Strong experience with JavaScript/TypeScript (React).
  • Experience with Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • CV must be submitted in English.
  • Availability to complete project tasks estimated at about 30 hours each, by the required deadline.
  • Understanding of real-world developer workflows and failure modes in AI coding systems (implied by the role).

Benefits

  • Up to $200/hr equivalent compensation, depending on level and pace.
  • Project-based work with flexible scheduling; you choose when and how to work.
  • Estimated ~30 hours per task/project, depending on complexity.
  • Opportunity to work on leading tech companies' AI evaluation projects.
  • Paid work after passing qualification(s) and joining a project.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Epic Games

Senior Backend Engineer, Developer Platform (Horde)

Epic Games 5K-10K Internet Software & Services

Epic Games is hiring a senior programmer for its Unreal Engine developer platform team to work on Horde, the internal platform that supports CI/CD, remote execution, build health, storage, and developer workflows.

United States Canada Full-time Senior Backend Engineer
$173k-$279k
AWS Azure C# CI/CD gRPC MongoDB .NET Redis
11 hours, 18 minutes ago
Coupang

Senior Staff Back-end Engineer (eCommerce Engineering)

Coupang 1K-5K Internet Software & Services

Coupang is hiring a Senior Staff Backend Engineer to lead the architecture and technical strategy for its core e-commerce platform services across global markets.

South Korea Full-time Lead Backend Engineer
Apache Airflow Apache Spark AWS Cassandra Grafana Java Kafka Kotlin Kubernetes Microservices OpenTelemetry Prometheus Spring
11 hours, 33 minutes ago
Toloka

Record Your Daily Routine & Get Paid - AI Training (Remote)

Toloka 251-1K Internet Software & Services

Project-based freelance work for an AI training platform where contributors record first-person videos of everyday household activities to help train AI systems and robots.

Costa Rica Dominican Republic Bolivia Uruguay Freelance Entry Level AI (Artificial Intelligence)
12 hours, 3 minutes ago
Toloka

Freelance: grabación de tareas cotidianas para proyecto de IA

Toloka 251-1K Internet Software & Services

The company is looking for people to record everyday activities from a first-person perspective using a mobile app to help train AI and robotics systems.

Costa Rica Dominican Republic Bolivia Uruguay Part-time Entry Level AI (Artificial Intelligence)
$12k-$12k
12 hours, 3 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers