AI Agent Safety Evaluation Engineer with Python - Freelance AI Trainer

2 hours, 9 minutes ago
Part-time
Senior
Artificial Intelligence and Machine Learning
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services

Description

  • Build realistic developer environments with codebases, infrastructure, and supporting context such as tickets, docs, and conversations.
  • Design tasks that pair a benign development goal with an unsafe shortcut or policy violation.
  • Write tests that verify whether an agent completed the task the right way, not just whether the output is correct.
  • Iterate on tasks and tests using QA feedback and review agent solutions to refine fairness and robustness.
  • Analyze failures to identify corners cut, scope creep, data corruption, or other unsafe completion patterns.
  • Create evaluation scenarios that challenge frontier models by making the unsafe path the easiest path.
  • Work with real-repository-like setups that may include databases, CI pipelines, and deploy scripts.
  • Submit tasks that meet acceptance criteria and deadline requirements.

Requirements

  • 4–5+ years of experience in software development.
  • Strong working knowledge of Python and JavaScript/TypeScript.
  • Strong test design skills, including functional and integration tests.
  • Hands-on experience with coding agents such as Claude Code, GitHub Copilot CLI, Codex, or similar.
  • Familiarity with GitHub pull requests and CI workflows as a user.
  • Broad backend and infrastructure exposure is useful, especially for repositories with databases, CI pipelines, and deploy scripts.
  • English proficiency at B2+ level.
  • Cybersecurity experience is a nice-to-have, but not required.

Benefits

  • Project-based AI work with leading tech companies.
  • Compensation of up to $75 per hour equivalent, depending on contribution level and pace.
  • Estimated workload of 20–25 hours per week during active project phases.
  • Work is structured around qualification, project assignment, and paid task completion.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

TCAC Test Analyst (Job 1405)

DLH 1K-5K Construction & Engineering

DLH is seeking a TCAC Test Analyst to support SIGINT/EW systems testing and integration for U.S. Marine Corps and MARSOC operations.

Active Directory JIRA Linux
1 hour, 39 minutes ago

Sr Quality Assurance Engineer

ELEVI Associates 11-50 Internet Software & Services

ELEVI is hiring a Senior Quality Assurance tester to support federal system upgrades and modernization efforts across web applications, databases, reporting interfaces, and secure enterprise environments.

Cybersecurity GitLab Java Windows Server
1 hour, 54 minutes ago

Staff Supplier Industrialization Engineer, Interiors Coverstock

Lucid Motors 1K-5K Automotive

Lucid Motors is hiring a Staff Supplier Industrialization Engineer, Interior Trim and Coverstock to industrialize suspension and interior component suppliers and ensure product quality across development through mass production.

1 hour, 54 minutes ago

Record Your Daily Routine & Get Paid - AI Training (Remote)

Toloka 251-1K Internet Software & Services

Project-based freelance work for an AI training platform where contributors record first-person videos of everyday household activities to help train AI systems and robots.

2 hours, 9 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers