RemoteLeaf Logo
Log in Sign up

Senior Software Engineer - Testing AI Coding Agents

11 hours, 35 minutes ago
Bulgaria, Czech Republic, Poland, Romania, Hungary, Ireland
Part-time
Senior
SDET (Software Development Engineer in Test)
Software Development
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 310

Description

  • Build realistic developer environments with codebase, infrastructure, and supporting context such as tickets, docs, and conversations.
  • Design tasks from intermediate states of those environments, including prompts and clear success criteria.
  • Define what it means for a task to be solved and ensure each task is feasible for an AI agent.
  • Write tests that validate agent solutions, accepting all correct approaches and rejecting incorrect ones.
  • Review agent solutions, analyze failures, and refine tasks and tests based on QA feedback.
  • Iterate on evaluation criteria to make assessments fair, robust, and challenging for strong models.
  • Work within a project-based workflow to complete and submit tasks by the required deadlines.

Requirements

  • 5+ years of experience in software development.
  • Experience with Python and FastAPI.
  • Experience with JavaScript or TypeScript and React.
  • Experience with Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • Ability to work on project-based assignments and submit work by deadline.
  • CV must be submitted in English.

Benefits

  • Up to $50/hr equivalent compensation, depending on level and pace.
  • Flexible schedule with the ability to choose when and how to work.
  • Project-based work with tasks estimated at around 20 hours each.
  • Payment for completed tasks upon acceptance.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Esri

Software Development Engineer

Esri 1K-5K Internet Software & Services

Esri is hiring a software testing professional to help build, automate, and maintain integration and regression test frameworks for AI, GIS, and web-based software solutions.

United States Full-time Junior Software Engineer
$137k-$140k
CI/CD Java Jenkins JUnit Python Selenium
10 hours, 20 minutes ago
Cordial

Senior Software Engineer

Cordial 51-250 Media

Cordial is hiring a Senior Software Engineer to lead development of scalable, high-performance systems for its marketing platform and guide engineering direction across the product.

United States Full-time Senior Software Engineer
$160k-$175k
System Design
10 hours, 20 minutes ago
Esri

Software Development Engineer

Esri 1K-5K Internet Software & Services

Esri is hiring a software engineer to help build and enhance ArcGIS Online and Enterprise, delivering scalable geospatial and analytics products across cloud and on-premises environments.

United States Full-time Entry Level Software Engineer
$137k-$140k
Java Microservices Python
10 hours, 35 minutes ago
Omilia

Conversational AI Application Developer (English and Spanish speaker)

Omilia 251-1K IT Services

Omilia is hiring a fully remote Conversational AI application developer to join its Solution Delivery team and build production-grade voice and chat solutions for enterprise customers across Europe.

Malta Europe Poland Portugal Romania Spain Italy Full-time Entry Level AI Engineer Software Engineer
CI/CD Generative AI Git Java JavaScript Machine Learning NLP REST API
10 hours, 50 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers