šŸ–ļøAgent Evals Specialist - Job Code: 1778280366787

11Ā hours, 13Ā minutes ago
Full-time
Junior
Artificial Intelligence and Machine Learning
Coconut VA

Coconut VA

Coconut VA: Premier virtual assistant service providing top talent for admin, operations, marketing, and sales. Streamline your business operations with our rigorous selection process and expert VAs.

Professional Services
11-50
Founded 2021

Description

  • Read the source content and the AI agent’s output side by side to verify accuracy.
  • Review what the AI agent created, changed, or left out.
  • Score a short rubric covering accuracy, coverage, organization, and rule adherence.
  • Write detailed, specific feedback about mistakes and missed information.
  • Submit completed evaluations and move on to the next task.
  • Follow the required review process and complete all application items before proceeding.

Requirements

  • Strong written English skills.
  • Ability to read dense technical content for hours without losing focus.
  • Consistency in scoring over time.
  • Ability to provide clear, specific feedback.
  • Prior experience as an AI trainer, tutor, or evaluator is preferred.
  • Background in technical writing, editing, QA, translation, paralegal work, or research assistance is preferred.
  • Familiarity with Markdown is preferred.
  • Experience using Slack and an internal platform.
  • Must provide a CEFR English proficiency rating.
  • Must submit a Coconut VA profile in the required English template.
  • Must record and provide a 1-minute introduction video link.
  • Availability for 40 hours per week.
  • Preferred overlap with 10 AM to 10 PM PST.

Benefits

  • Competitive salary.
  • 13th month pay.
  • 12 days of paid time off.
  • 12 paid US holidays.
  • Maternity and paternity leave.
  • Comprehensive healthcare benefits.
  • Life insurance through the full-time HMO plan.
  • Mental health support and wellness resources.
  • Milestone gifts and birthday treats.
  • Team experiences such as island getaways, virtual town halls, and regional meetups.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Quantum Research Scientist with Python - Freelance AI Trainer

Mindrift.ai: Be the ā€œIā€ in AI Internet Software & Services

Mindrift is seeking physics specialists for part-time, project-based AI work focused on creating and validating computational physics problems for leading tech companies.

C MATLAB NumPy Pandas Python R SciPy SQL
23Ā minutes ago

AI Shapeshifter - Transform Any Role Through AI

CloudWalk 51-250 Diversified Financial Services

CloudWalk is hiring a remote, full-time AI-focused professional in SĆ£o Paulo to help shape AI-driven operations and products across teams in fintech.

Generative AI LLM
35Ā minutes ago

Quantum Research Scientist with Python - Freelance AI Trainer

Mindrift.ai: Be the ā€œIā€ in AI Internet Software & Services

Mindrift is seeking part-time physics specialists for project-based AI tasks that involve creating and validating computational physics problems for leading tech companies.

C MATLAB NumPy Pandas Python R SciPy SQL
52Ā minutes ago

AI Training Specialist (Global Egocentric Video)

Toloka 251-1K Internet Software & Services

Toloka Annotators is hiring freelance contributors to record point-of-view videos of everyday household tasks that help train generative AI systems.

53Ā minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers