RemoteLeaf Logo
Log in Sign up

Senior Product Quality Analyst - AI Voice

2 weeks, 6 days ago
United Kingdom
Senior
AI (Artificial Intelligence)
Artificial Intelligence and Machine Learning
Generative AI
Apply Now
Spotify

Spotify

Spotify is a leading global audio streaming service empowering artists and inspiring fans worldwide with over 70 million tracks. It has 365 million users, including 165 million subscribers, across 178 markets.

Media
Founded 2006
View All Jobs 17

Description

  • Own the quality bar for Spotify’s AI voice models across different products, audiences, languages and use cases.
  • Design evaluation rubrics and build evaluation methods to assess naturalness, tone, pacing, pronunciation, accent, audio quality and overall experience.
  • Plan and run evaluation campaigns and turn findings into recommendations for Voice Production, Speak and product teams.
  • Establish an ongoing evaluation cadence to reassess model quality as products and technology evolve.
  • Benchmark Spotify’s AI voices against third-party models and competitors.
  • Help determine quality trade-offs as Spotify expands into new languages and regions.
  • Partner with voice experts, machine learning teams, product partners and vendors to maintain a consistent quality bar.
  • Identify the most meaningful model gaps and use evaluation insights to influence research and technical priorities.

Requirements

  • Experience in product quality, evaluation, data curation or a closely related field using human judgment to assess model or product performance.
  • Ability to independently define evaluation criteria and convert subjective qualities into clear, repeatable rubrics.
  • Experience designing and running qualitative and MOS-style evaluations, with the ability to synthesize results into actionable insights.
  • Understanding of voice or audio quality, including naturalness, pacing, tone, accent and pronunciation, or the ability to develop that expertise quickly.
  • Strong judgment and the ability to balance different product requirements, markets and stakeholder perspectives.
  • Comfort working autonomously and establishing processes and cadence for ongoing evaluation.
  • Strong communication skills across creative, technical and product teams, with experience serving as a reliable point of contact for internal partners and external vendors.
  • Curiosity about generative AI, speech technology and the evolving AI voice landscape.
  • Role is based in London.
  • Flexibility to work from home with some in-person meetings.

Benefits

  • Flexible work setup with the ability to work where you work best, including from home.
  • Some in-person meetings while still maintaining work-from-home flexibility.
  • Reasonable accommodations available during the interview and application process.
  • Inclusive and accessible recruitment process.
  • Equal opportunity workplace.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Meridial Marketplace, by Invisible

AWS Trainium/NKI (TRN) AI Task Auditor - Freelance AI Trainer Project

Meridial Marketplace, by Invisible 501-1000 information technology & services

Remote freelance contract for an AWS Trainium and Neuron Kernel Interface (NKI) technical specialist to audit AI training and evaluation tasks for accuracy, practicality, reproducibility, and technical rigor.

Anywhere Contract Senior AI (Artificial Intelligence)
$146k-$208k
18 hours, 59 minutes ago
Prolific

AI Training - Fluent Japanese Speakers - Kōbe, Japan

Prolific 51-250 Professional Services

Prolific is seeking fluent Japanese speakers for flexible, task-based remote video conversations that provide natural language data to train and improve AI models.

Japan Contract Entry Level AI (Artificial Intelligence)
Up to $37k
19 hours, 29 minutes ago
Wikimedia Foundation

Software Quality Manager

Wikimedia Foundation 251-1K Internet Software & Services

The Wikimedia Foundation seeks a Software Quality Manager to lead distributed testing and technical documentation teams supporting engineering teams that build and maintain Wikipedia and other Wikimedia projects.

Singapore United States Canada United Kingdom Germany Italy Netherlands Poland Ghana Kenya Spain Switzerland Mexico Morocco Brazil Colombia South Africa India Indonesia Full-time Lead QA Engineer Technical Writer
$129k-$203k
Go PHP Python
19 hours, 29 minutes ago
x.ai

AI Tutor - Design Specialist

x.ai 51-250 Internet Software & Services

SpaceXAI is seeking an AI Design Specialist to evaluate, annotate, refine, and create design examples that improve Grok’s ability to distinguish clear, functional, and high-quality design across visual, product, web, and interactive contexts.

Remote
Mid Level AI (Artificial Intelligence)
$73k-$156k
Figma Game Design Notion UI Design UX Design
19 hours, 29 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers