RemoteLeaf Logo
Log in Sign up

Freelance Agent Evaluation Engineer

1 hour, 41 minutes ago
Canada
Contract
Senior
AI (Artificial Intelligence)
Software Development
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 295

Description

  • Build realistic developer environments with codebase, infrastructure, and supporting context such as tickets, documentation, and conversations.
  • Design tasks from intermediate states of the environment, including prompts and clear definitions of what counts as solved.
  • Write tests that validate agent solutions and correctly distinguish valid approaches from incorrect ones.
  • Review agent solutions, analyze failures, and refine tasks and tests based on QA feedback.
  • Iterate on evaluations until they are fair, robust, and challenging for advanced AI models.
  • Ensure tasks are solvable by an AI agent without writing the full code from scratch.
  • Submit completed tasks by the deadline and meet the required acceptance criteria.

Requirements

  • 5+ years of experience in software development.
  • Experience with Python (FastAPI).
  • Experience with JavaScript/TypeScript (React).
  • Experience with Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • CV must be submitted in English.
  • Ability to work on a project-based, non-permanent basis.
  • Estimated availability for tasks of around 20 hours each, depending on complexity.

Benefits

  • Up to $50/hr equivalent compensation, depending on level and pace.
  • Flexible schedule with self-directed working hours.
  • Project-based work with the ability to choose when to work.
  • Paid tasks with an estimated effort of about 20 hours each.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Accenture

QA Automation Engineer

Accenture 100K+ Professional Services

Accenture Federal Services is hiring a QA Automation Engineer to build and maintain automated testing for federal software applications in a remote role with occasional DMV-area travel.

United States Full-time Mid Level SDET (Software Development Engineer in Test)
$111k-$140k
Agile AWS Azure Cypress Docker Gatling GCP Git GitHub Actions GitLab CI Jenkins Jest JMeter JUnit K6 Kubernetes Playwright Postman Pytest REST API Selenium SQL TestNG
41 minutes ago
Nimble Robotics

HIL Test / Automation Engineer

Nimble Robotics 51-250 Industrial Conglomerates

Nimble is hiring an engineer to own a core HIL regression pillar for its autonomous robotics supply chain platform, building automated validation infrastructure for the V6 push.

United States Full-time Junior SDET (Software Development Engineer in Test)
$130k-$170k
C++ CI/CD Embedded Systems Jenkins MATLAB Python Rust
1 hour, 26 minutes ago
Gramian Consultancy Group

COBOL Developer (Legacy Systems & AI Training)

Gramian Consultancy Group Professional Services

Gramian Consultancy is seeking an experienced COBOL Developer to support AI training by analyzing legacy enterprise applications, documenting business logic, and creating realistic technical scenarios.

Turkey Poland Romania India Argentina Brazil Contract Senior AI (Artificial Intelligence) Software Engineer
COBOL
1 hour, 41 minutes ago
Intetics

AI / ML Engineer.

Intetics 1K-5K Internet Software & Services

Intetics is seeking an experienced AI/ML Engineer to build and maintain production machine learning solutions for enterprise analytics, forecasting, and AI-driven decision-making.

Poland Romania Bulgaria Czech Republic Ukraine Full-time Senior AI (Artificial Intelligence)
Feature Engineering Generative AI Machine Learning MLflow MLOps NLP Python
1 hour, 41 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers