RemoteLeaf Logo
Log in Sign up

Technical AI Evaluator - Python, Automation or Security

3 weeks ago
United States
Part-time
Senior
AI (Artificial Intelligence)
Software Development
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 21

Description

  • Build realistic developer environments with codebases, infrastructure, and supporting context such as tickets, documentation, and conversations.
  • Design coding tasks from intermediate environment states, including prompts and clear definitions of what counts as solved.
  • Write functional and integration tests that validate correct solutions and reject incorrect ones.
  • Review AI agent solutions, analyze failure cases, and refine tasks and tests based on QA feedback.
  • Iterate on evaluation criteria to ensure tasks are fair, robust, and solvable by AI agents.
  • Assess how well AI coding agents handle real-world developer tasks.

Requirements

  • 5+ years of software development experience.
  • Experience with Python, including FastAPI.
  • Experience with JavaScript/TypeScript, including React.
  • Experience with Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • Submit your CV in English and indicate your English proficiency level.
  • Able to work in a project-based, non-permanent role.
  • Preferred candidates are comfortable evaluating complex coding tasks and solution validity.

Benefits

  • Up to $50/hr equivalent compensation, depending on level and pace.
  • Tasks are estimated at approximately 20 hours each.
  • Flexible schedule with self-directed working hours.
  • Project-based opportunities with leading tech companies.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Margo Bank

Senior C# / .NET Developer

Margo Bank Professional Services

Senior developer role in a six-person technology unit supporting a global Retail Media company, focused on designing and building a Core User Platform and partner APIs from the ground up.

Poland Contract Senior Backend Engineer
$61k-$72k
C# .NET
9 hours, 28 minutes ago
Megaport

Senior Backend Engineer

Megaport 251-1K Diversified Telecommunication Services

Megaport is hiring a Senior Software Engineer to evolve its global Network as a Service platform by developing backend services, modernizing core systems, and guiding technical improvements.

Australia Full-time Senior Backend Engineer
CI/CD Hibernate Java Kotlin REST API SQL
9 hours, 28 minutes ago
MUTT DATA

Java Back End Developer Senior

MUTT DATA 51-250 Internet Software & Services

Mutt Data is seeking a Senior Java Backend Developer to build scalable data, reporting, and analytics systems for machine learning and data-driven client solutions in a remote startup environment.

Argentina Full-time Senior Backend Engineer
Apache Airflow AWS Azure CI/CD Databricks dbt Deep Learning Docker GCP Generative AI Git GraphQL Java Kubernetes Machine Learning Python React Terraform TypeScript
9 hours, 28 minutes ago
Lingaro

Backend Developer - Contract

Lingaro 5K-10K IT Services

Backend Developer for a four-month remote contract in India, responsible for building and operating scalable Python services, APIs, and cloud-based microservices.

India Contract Senior Backend Engineer
Agile Azure CI/CD Databricks Django Docker FastAPI Git Go Kafka Kubernetes Microservices Python React REST API Scrum SQL Terraform
9 hours, 28 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers