ML Annotation QA Engineer

18 hours, 5 minutes ago
Full-time
Mid Level
Artificial Intelligence and Machine Learning
Gather AI

Gather AI

Gather AI specializes in providing autonomous drone technology for inventory management in warehouses, enabling businesses to enhance inventory accuracy, reduce costs, and significantly improve operational productivity.

Air Freight & Logistics
11-50
$17M raised

Description

  • Own daily review queues, verdicts, and root-cause analysis for judgment-heavy annotation quality issues.
  • Define, version, and refine verdict taxonomies, decision rules, quality guidelines, and failure-pattern documentation.
  • Build performance trackers showing error rates by facility, site, equipment, and data format over time.
  • Detect anomalies against baselines and distinguish annotation, model, tooling, and field-degradation causes.
  • Report reproducible findings and confidence levels to Machine Learning and Engineering teams.
  • Query and analyze annotation data directly using Python and SQL.
  • Translate labeling findings into vendor SOP and instruction changes.
  • Specify and validate annotation-tool improvements that reduce manual analysis.
  • Establish quality analysis and reporting for new annotation programs.
  • Track work in Jira and support pre-release validation.

Requirements

  • Bachelor’s degree in Computer Science, Engineering, Electrical Engineering, or equivalent experience.
  • 2–5 years of experience in ML QA, annotation quality, data quality, or analytics at an AI/ML company.
  • Hands-on experience with annotated ML or computer vision datasets and label-quality assessment.
  • Experience diagnosing and reporting data anomalies to technical audiences.
  • Strong statistical, analytical, and root-cause-analysis skills.
  • Ability to independently reach defensible conclusions from unfamiliar data.
  • Proficiency with Python and SQL or an equivalent query language.
  • Experience creating quality guidelines, decision rules, labeling taxonomies, or SOPs.
  • Understanding of data privacy and confidentiality for customer operational data.
  • Preferred: experience with Jira, annotation platforms, gold-set validation, inter-annotator agreement, dashboarding, warehouse automation, robotics, or computer vision.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

AI Training - Fluent Japanese Speakers - Kitakyushu, Japan

Prolific 51-250 Professional Services

Prolific is seeking fluent Japanese speakers to participate in flexible, remote video conversations that provide natural language data for training AI models.

NLP
16 hours, 35 minutes ago

AI Training - Fluent Japanese Speakers - Saitama, Japan

Prolific 51-250 Professional Services

Prolific is seeking fluent Japanese speakers for flexible, remote, task-based AI training through natural video conversations that help improve AI models’ understanding of Japanese speech and culture.

16 hours, 35 minutes ago

AI Training - Fluent Japanese Speakers - Matsudo, Japan

Prolific 51-250 Professional Services

Prolific is seeking fluent Japanese speakers for flexible, task-based remote video conversations that provide natural language data for training AI models.

16 hours, 35 minutes ago

AI Training - Fluent Japanese Speakers - Shizuoka, Japan

Prolific 51-250 Professional Services

Prolific is seeking fluent Japanese speakers to participate remotely in live video conversations that help AI models learn natural Japanese speech, tone, and cultural nuance.

16 hours, 50 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers