RemoteLeaf Logo
Log in Sign up

Senior Product Quality Analyst - AI Voice

2 weeks, 6 days ago
United States
Full-time
Senior
AI (Artificial Intelligence)
Artificial Intelligence and Machine Learning
Generative AI Machine Learning
Apply Now
Spotify

Spotify

Spotify is a leading global audio streaming service empowering artists and inspiring fans worldwide with over 70 million tracks. It has 365 million users, including 165 million subscribers, across 178 markets.

Media
Founded 2006
View All Jobs 17

Description

  • Own the quality bar for Spotify’s AI voice models across products, audiences, languages, and use cases.
  • Design evaluation rubrics and build evaluation approaches that assess naturalness, tone, pacing, pronunciation, accent, audio quality, and overall experience.
  • Plan and run evaluation campaigns and translate findings into recommendations for Voice Production, Speak, and product teams.
  • Establish an ongoing evaluation cadence for reassessing model quality as products and technology evolve.
  • Benchmark Spotify’s AI voices against third-party models and competitors.
  • Help determine quality trade-offs as Spotify expands into additional languages and regions.
  • Partner with voice experts, machine learning teams, product partners, and vendors to maintain a consistent quality bar.
  • Surface meaningful gaps in model performance and use evaluation insights to influence research and technical priorities.

Requirements

  • Experience in product quality, evaluation, data curation, or a closely related field using human judgment to assess model or product performance.
  • Ability to independently define evaluation criteria and turn subjective qualities into clear, repeatable rubrics.
  • Experience designing and running qualitative and MOS-style evaluations, with the ability to synthesize results into actionable insights.
  • Understanding of voice or audio quality, including naturalness, pacing, tone, accent, and pronunciation, or the ability to develop that expertise quickly.
  • Strong judgment and the ability to balance competing product requirements, markets, and stakeholder perspectives.
  • Comfort working autonomously and establishing evaluation processes and cadence over time.
  • Ability to communicate clearly across creative, technical, and product teams and serve as a reliable point of contact for partners and vendors.
  • Curiosity about generative AI, speech technology, and the evolving AI voice landscape.
  • Based in New York or Los Angeles.
  • Equal opportunity employer; accommodations available during the interview process.

Benefits

  • Base salary range of $114,486–$163,552 USD plus equity.
  • Health insurance.
  • Six-month paid parental leave.
  • 401(k) retirement plan.
  • Monthly meal allowance.
  • 23 paid days off.
  • Paid flexible holidays.
  • Paid sick leave.
  • Flexible work-from-home arrangement with some in-person meetings.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Meridial Marketplace, by Invisible

AWS Trainium/NKI (TRN) AI Task Auditor - Freelance AI Trainer Project

Meridial Marketplace, by Invisible 501-1000 information technology & services

Remote freelance contract for an AWS Trainium and Neuron Kernel Interface (NKI) technical specialist to audit AI training and evaluation tasks for accuracy, practicality, reproducibility, and technical rigor.

Anywhere Contract Senior AI (Artificial Intelligence)
$146k-$208k
19 hours ago
Prolific

AI Training - Fluent Japanese Speakers - Kōbe, Japan

Prolific 51-250 Professional Services

Prolific is seeking fluent Japanese speakers for flexible, task-based remote video conversations that provide natural language data to train and improve AI models.

Japan Contract Entry Level AI (Artificial Intelligence)
Up to $37k
19 hours, 30 minutes ago
Wikimedia Foundation

Software Quality Manager

Wikimedia Foundation 251-1K Internet Software & Services

The Wikimedia Foundation seeks a Software Quality Manager to lead distributed testing and technical documentation teams supporting engineering teams that build and maintain Wikipedia and other Wikimedia projects.

Singapore United States Canada United Kingdom Germany Italy Netherlands Poland Ghana Kenya Spain Switzerland Mexico Morocco Brazil Colombia South Africa India Indonesia Full-time Lead QA Engineer Technical Writer
$129k-$203k
Go PHP Python
19 hours, 30 minutes ago
x.ai

AI Tutor - Design Specialist

x.ai 51-250 Internet Software & Services

SpaceXAI is seeking an AI Design Specialist to evaluate, annotate, refine, and create design examples that improve Grok’s ability to distinguish clear, functional, and high-quality design across visual, product, web, and interactive contexts.

Remote
Mid Level AI (Artificial Intelligence)
$73k-$156k
Figma Game Design Notion UI Design UX Design
19 hours, 30 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers