RemoteLeaf Logo
Log in Sign up

ML Annotation QA Engineer

1 month, 1 week ago
India
Full-time
Mid Level
AI (Artificial Intelligence)
Artificial Intelligence and Machine Learning
Computer Vision Git JIRA Machine Learning Python SQL Statistics
Apply Now
Gather AI

Gather AI

Gather AI specializes in providing autonomous drone technology for inventory management in warehouses, enabling businesses to enhance inventory accuracy, reduce costs, and significantly improve operational productivity.

Air Freight & Logistics
11-50
$17M raised
View All Jobs 2

Description

  • Own daily review queues, verdicts, and root-cause analysis for judgment-heavy annotation quality issues.
  • Define, version, and refine verdict taxonomies, decision rules, quality guidelines, and failure-pattern documentation.
  • Build performance trackers showing error rates by facility, site, equipment, and data format over time.
  • Detect anomalies against baselines and distinguish annotation, model, tooling, and field-degradation causes.
  • Report reproducible findings and confidence levels to Machine Learning and Engineering teams.
  • Query and analyze annotation data directly using Python and SQL.
  • Translate labeling findings into vendor SOP and instruction changes.
  • Specify and validate annotation-tool improvements that reduce manual analysis.
  • Establish quality analysis and reporting for new annotation programs.
  • Track work in Jira and support pre-release validation.

Requirements

  • Bachelor’s degree in Computer Science, Engineering, Electrical Engineering, or equivalent experience.
  • 2–5 years of experience in ML QA, annotation quality, data quality, or analytics at an AI/ML company.
  • Hands-on experience with annotated ML or computer vision datasets and label-quality assessment.
  • Experience diagnosing and reporting data anomalies to technical audiences.
  • Strong statistical, analytical, and root-cause-analysis skills.
  • Ability to independently reach defensible conclusions from unfamiliar data.
  • Proficiency with Python and SQL or an equivalent query language.
  • Experience creating quality guidelines, decision rules, labeling taxonomies, or SOPs.
  • Understanding of data privacy and confidentiality for customer operational data.
  • Preferred: experience with Jira, annotation platforms, gold-set validation, inter-annotator agreement, dashboarding, warehouse automation, robotics, or computer vision.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

RWS Group

Search Engine Evaluator - English (Ireland)

RWS Group 5K-10K Internet Software & Services

RWS Group is seeking Ireland-based Search Quality Evaluators to assess everyday search results and multimedia content, helping improve AI data and online search experiences through flexible, part-time remote work.

Ireland Contract Entry Level AI (Artificial Intelligence)
Up to $37k
5 hours, 25 minutes ago
RWS Group

Search Engine Evaluator - Telugu (India)

RWS Group 5K-10K Internet Software & Services

RWS seeks a part-time, work-from-home Search Quality Evaluator in India to assess everyday search results and multimedia content, helping improve AI data and online search experiences.

India Contract Entry Level AI (Artificial Intelligence)
Up to $10k
5 hours, 25 minutes ago
RWS Group

Search Engine Evaluator - English (Singapore)

RWS Group 5K-10K Internet Software & Services

RWS Group is seeking Singapore-based Search Quality Evaluators to assess everyday search results and provide feedback that improves AI data and online search experiences.

Singapore Contract Entry Level AI (Artificial Intelligence)
Up to $19k
5 hours, 25 minutes ago
RWS Group

Search Engine Evaluator - Turkish (Turkiye)

RWS Group 5K-10K Internet Software & Services

RWS is seeking a Turkey-based, part-time Search Quality Evaluator to assess search results and multimedia content, providing feedback that improves clients’ AI and online search experiences.

Turkey Contract Entry Level AI (Artificial Intelligence)
Up to $19k
5 hours, 25 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers