RemoteLeaf Logo
Log in Sign up

Senior Python Engineer - AI Coding Agent Evaluation (Freelance)

6 hours, 38 minutes ago
United States
Part-time
Lead
AI (Artificial Intelligence)
Software Development
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 53

Description

  • Build realistic virtual developer environments with codebases, infrastructure, tickets, documentation, and conversation history.
  • Design coding tasks from intermediate states of those environments and define what counts as a solved task.
  • Write tests and evaluation criteria that accept valid solutions and reject incorrect ones.
  • Review agent solutions, analyze failures, and refine tasks and tests based on QA feedback.
  • Iterate on task design until the evaluation is fair, robust, and solvable by an AI agent.
  • Assess how well models handle real-world developer tasks in challenging scenarios.

Requirements

  • 8+ years of software development experience.
  • Experience with Python, including FastAPI.
  • Experience with JavaScript/TypeScript, including React.
  • Experience with Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2+ level.
  • Submit a CV in English.
  • Ability to complete qualification(s) before joining a project.
  • Ability to complete tasks within the listed deadline; tasks are estimated to take about 30 hours in total.

Benefits

  • Paid per accepted task.
  • Compensation can reach the equivalent of up to $200/hr depending on qualification tier and efficiency.
  • Flexible project-based work with no permanent employment commitment.
  • Choose when and how to work.
  • Payment is based on completed tasks, which can increase effective hourly rate with faster pace.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Dragos

Backend Engineering Manager

Dragos 251-1K Professional Services

Dragos is hiring a Backend Software Engineering Manager to lead the backend team that develops and sustains core platform services for xOT cybersecurity in support of critical infrastructure.

United States Full-time Lead Backend Engineer Engineering Manager
$220k-$220k
Agile Cybersecurity HashiCorp Vault Microservices RabbitMQ Rust
6 hours, 8 minutes ago
Weekday

Voice Actor: CX Agent Voice Cloning (Standard French)

Weekday 11-50 Construction & Engineering

One of our clients is hiring a remote French voice talent to record and evaluate speech samples for an internal CX AI Agent and next-generation text-to-speech technology.

Anywhere United States Contract Mid Level AI (Artificial Intelligence) Customer Support
$0k-$0k
6 hours, 38 minutes ago
Zyte

Python Developer - Remote

Zyte 251-1K Professional Services

Zyte is hiring a Software Engineer for its Delivery team to build and support Scrapy-based web crawlers and distributed data extraction systems for web-scale crawling.

Pakistan Full-time Mid Level Backend Engineer
HTML HTTP JavaScript Linux MapReduce NLTK Pandas Python Scikit-learn Unix
6 hours, 38 minutes ago
FutureSight

Co-Founder & CEO - AI WealthTech

FutureSight 11-50 Internet Software & Services

FutureSight is looking for a Co-Founder & CEO to lead Weli AI, a B2B WealthTech venture that brings embedded AI intelligence into advisors’ existing tools to drive proactive outreach and AUM growth.

United States Full-time Executive AI (Artificial Intelligence) Business Development
CRM LLM
6 hours, 38 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers