RemoteLeaf Logo
Log in Sign up

Software Engineering Evaluation Specialist

1 day, 11 hours ago
Germany
Part-time
Mid Level
AI (Artificial Intelligence)
Software Development
Bash Docker Go Hypothesis Java Linux Nginx Node.js NumPy Pytest Python PyTorch Rust SciPy
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 25

Description

  • Create realistic software-development scenarios involving bugs, broken ETL pipelines, or missing features.
  • Build reproducible Docker environments with pinned dependencies.
  • Write deterministic pytest tests that verify outcomes without revealing the solution.
  • Document tasks in Jira-style instruction files.
  • Create reference solve.sh scripts demonstrating that tasks are solvable.
  • Calibrate task difficulty so advanced AI agents succeed approximately 20–60% of the time.
  • Iterate on tasks using expert QA feedback.
  • Review other authors’ tasks as a QA reviewer when needed.

Requirements

  • 3+ years of production software development experience in Python, Go, Node.js, Java, or Rust, with depth in one backend stack.
  • Fluency with Python and pytest, including fixtures, parametrization, monkeypatching, timeouts, and conftest.py.
  • Experience authoring reproducible Dockerfiles, pinned dependencies, multi-stage builds, and non-root containers.
  • Comfort debugging Linux environments and containers using Bash and tools such as strace, lsof, and journalctl.
  • Hands-on experience using AI coding agents such as Claude Code, Cursor, or Roo Code for non-trivial work.
  • B2+ written English proficiency.
  • Preferred: expertise in security, system administration, scientific computing, DevOps, or Git internals.
  • Preferred: experience with modern Python tooling, coverage tools, fuzzing, property-based testing, or agent-evaluation benchmarks.
  • Ability to complete a 90-minute sample-task qualification and short behavioral interview.

Benefits

  • Project-based, flexible contribution schedule with approximately 8–20 hours per week.
  • Paid contributions at rates up to $35/hour, depending on expertise, performance, location, and project needs.
  • Task-based compensation with potential incentive payments on some projects.
  • Higher contribution volume may be available for top performers.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Weekday

Senior Technical Lead - Agentic AI / Generative AI

Weekday 11-50 Construction & Engineering

A mid-senior-level software development role with one of our clients, focused on contributing to software projects remotely from India.

India Full-time Lead AI (Artificial Intelligence)
$36k-$60k
Generative AI
10 hours, 29 minutes ago
Welo Global

Generative AI Analyst | Arabic (Morocco)

Welo Global Professional Services

Welo Data is seeking a remote freelance Generative AI Analyst fluent in Moroccan Arabic to evaluate, annotate, and quality-review AI-generated content across text, audio, images, video, and related data projects.

Morocco Freelance Mid Level AI (Artificial Intelligence)
$9k-$9k
Generative AI
10 hours, 44 minutes ago
TTEC Digital

Conversational AI Engineer - Google/GECX & Code Agents

TTEC Digital 1K-5K Professional Services

TTEC Digital is seeking a fully remote Conversational AI Engineer in the Philippines to build, deploy, and optimize customer-facing multi-agent workflows on Gemini Enterprise for Customer Experience.

Philippines Full-time Mid Level AI (Artificial Intelligence)
CI/CD GCP Git JSON Node.js OpenAPI Python REST API TypeScript
11 hours ago
Welo Global

French Quality Control Specialist - Project Lyra

Welo Global Professional Services

Welo Data is seeking French-speaking linguistics experts for a four-week remote freelance project evaluating and improving AI-generated language outputs for accuracy, grammar, and cultural relevance.

France Freelance Mid Level AI (Artificial Intelligence) Linguist
$83k-$83k
LLM NLP
11 hours, 14 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers