Coconut VA

Coconut VA: Premier virtual assistant service providing top talent for admin, operations, marketing, and sales. Streamline your business operations with our rigorous selection process and expert VAs.

Professional Services

Industrials

11-50 (30)

Founded 2021

10 open positions

Links

View All Jobs

🏖️Agent Evals Specialist - Job Code: 1778280366787

11 hours, 13 minutes ago

Anywhere

Full-time

Junior

AI (Artificial Intelligence)

Artificial Intelligence and Machine Learning

Apply Now

Coconut VA

Coconut VA: Premier virtual assistant service providing top talent for admin, operations, marketing, and sales. Streamline your business operations with our rigorous selection process and expert VAs.

Professional Services

11-50

Founded 2021

View All Jobs 10

Description

Read the source content and the AI agent’s output side by side to verify accuracy.
Review what the AI agent created, changed, or left out.
Score a short rubric covering accuracy, coverage, organization, and rule adherence.
Write detailed, specific feedback about mistakes and missed information.
Submit completed evaluations and move on to the next task.
Follow the required review process and complete all application items before proceeding.

Requirements

Strong written English skills.
Ability to read dense technical content for hours without losing focus.
Consistency in scoring over time.
Ability to provide clear, specific feedback.
Prior experience as an AI trainer, tutor, or evaluator is preferred.
Background in technical writing, editing, QA, translation, paralegal work, or research assistance is preferred.
Familiarity with Markdown is preferred.
Experience using Slack and an internal platform.
Must provide a CEFR English proficiency rating.
Must submit a Coconut VA profile in the required English template.
Must record and provide a 1-minute introduction video link.
Availability for 40 hours per week.
Preferred overlap with 10 AM to 10 PM PST.

Benefits

Competitive salary.
13th month pay.
12 days of paid time off.
12 paid US holidays.
Maternity and paternity leave.
Comprehensive healthcare benefits.
Life insurance through the full-time HMO plan.
Mental health support and wellness resources.
Milestone gifts and birthday treats.
Team experiences such as island getaways, virtual town halls, and regional meetups.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Quantum Research Scientist with Python - Freelance AI Trainer

Mindrift.ai: Be the “I” in AI Internet Software & Services

Mindrift is seeking physics specialists for part-time, project-based AI work focused on creating and validating computational physics problems for leading tech companies.

Mexico Chile Colombia Costa Rica Peru Uruguay Part-time Junior AI (Artificial Intelligence) Research Scientist

$35k-$35k

C MATLAB NumPy Pandas Python R SciPy SQL

23 minutes ago

Apply

23 minutes ago

AI Shapeshifter - Transform Any Role Through AI

CloudWalk 51-250 Diversified Financial Services

CloudWalk is hiring a remote, full-time AI-focused professional in São Paulo to help shape AI-driven operations and products across teams in fintech.

Brazil Full-time Senior AI (Artificial Intelligence)

Generative AI LLM

35 minutes ago

Apply

35 minutes ago

Quantum Research Scientist with Python - Freelance AI Trainer

Mindrift.ai: Be the “I” in AI Internet Software & Services

Mindrift is seeking part-time physics specialists for project-based AI tasks that involve creating and validating computational physics problems for leading tech companies.