RemoteLeaf Logo
Log in Sign up

Senior Python Engineer - AI Coding Agent Evaluation (Freelance)

22 minutes ago
Bahrain, Kuwait, Oman, Qatar, Saudi Arabia, United Arab Emirates
Freelance
Lead
AI (Artificial Intelligence)
Software Development
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 346

Description

  • Build realistic virtual developer environments with codebases, infrastructure, and supporting context such as tickets, docs, and conversations.
  • Design tasks from intermediate environment states and define clear success criteria for what counts as a solved task.
  • Write tests that validate agent solutions, accepting valid approaches and rejecting incorrect ones.
  • Review agent solutions, analyze failure cases, and refine tasks and tests based on QA feedback.
  • Iterate on task design to ensure evaluations are fair, robust, and challenging for advanced AI models.

Requirements

  • 8+ years of experience in software development.
  • Experience with the core stack: Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • CV must be submitted in English.
  • Ability to complete qualification steps before joining a project.
  • Ability to work project-based and submit tasks by the deadline.
  • Experience understanding where coding models fail and how to design tasks that expose those weaknesses (preferred).

Benefits

  • Up to $200/hr equivalent compensation, depending on level and pace.
  • Flexible project-based schedule with self-directed work hours.
  • Estimated 30 hours of work per task, depending on complexity.
  • Opportunity to work with leading tech companies on AI evaluation projects.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

MUTT DATA

Senior Backend Python Developer

MUTT DATA 51-250 Internet Software & Services

Muttdata is hiring a Senior Backend Python Developer to build and maintain core microservices for a banking client’s AI platform in a remote-first startup environment.

Argentina Full-time Senior Backend Engineer
AWS Azure CI/CD Databricks dbt Docker DynamoDB FastAPI GCP JWT Kafka Kubernetes LLM Microservices OAuth PostgreSQL Python Terraform
7 minutes ago
Devsu

Senior BPM Developer (AuraQuantic Expert)

Devsu 51-250 Internet Software & Services

Senior AuraQuantic Developer for a remote LATAM role at Devsu supporting strategic financial-automotive projects that automate and optimize critical business processes.

Mexico Peru Latin America Full-time Senior Backend Engineer Software Engineer
Angular C# .NET REST API SQL Server
7 minutes ago
Bluelight Consulting

Software Engineer (Elixir/Phoenix) - Latin America (REMOTE)

Bluelight Consulting 11-50 Internet Software & Services

Bluelight is seeking a Backend Engineer to build and maintain Elixir-based systems and APIs for client projects in a fast-growing software consultancy.

Uruguay Full-time Mid Level Backend Engineer
Agile Ecto Elixir GraphQL Phoenix PostgreSQL REST API
7 minutes ago
Bluelight Consulting

Software Engineer (Elixir/Phoenix) - Latin America (REMOTE)

Bluelight Consulting 11-50 Internet Software & Services

Bluelight is hiring a backend software consultant to develop high-performance applications and APIs for client projects within a fast-growing, collaborative consultancy.

Chile Full-time Mid Level Backend Engineer
Agile Ecto Elixir GraphQL Phoenix PostgreSQL REST API
7 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers