RemoteLeaf Logo
Log in Sign up

Senior Product Quality Analyst - AI Voice

1 month, 1 week ago
United States
Full-time
Senior
AI (Artificial Intelligence)
Artificial Intelligence and Machine Learning
Generative AI Machine Learning
Apply Now
Spotify

Spotify

Spotify is a leading global audio streaming service empowering artists and inspiring fans worldwide with over 70 million tracks. It has 365 million users, including 165 million subscribers, across 178 markets.

Media
Founded 2006
View All Jobs 19

Description

  • Own the quality bar for Spotify’s AI voice models across products, audiences, languages, and use cases.
  • Design evaluation rubrics and build evaluation approaches that assess naturalness, tone, pacing, pronunciation, accent, audio quality, and overall experience.
  • Plan and run evaluation campaigns and translate findings into recommendations for Voice Production, Speak, and product teams.
  • Establish an ongoing evaluation cadence for reassessing model quality as products and technology evolve.
  • Benchmark Spotify’s AI voices against third-party models and competitors.
  • Help determine quality trade-offs as Spotify expands into additional languages and regions.
  • Partner with voice experts, machine learning teams, product partners, and vendors to maintain a consistent quality bar.
  • Surface meaningful gaps in model performance and use evaluation insights to influence research and technical priorities.

Requirements

  • Experience in product quality, evaluation, data curation, or a closely related field using human judgment to assess model or product performance.
  • Ability to independently define evaluation criteria and turn subjective qualities into clear, repeatable rubrics.
  • Experience designing and running qualitative and MOS-style evaluations, with the ability to synthesize results into actionable insights.
  • Understanding of voice or audio quality, including naturalness, pacing, tone, accent, and pronunciation, or the ability to develop that expertise quickly.
  • Strong judgment and the ability to balance competing product requirements, markets, and stakeholder perspectives.
  • Comfort working autonomously and establishing evaluation processes and cadence over time.
  • Ability to communicate clearly across creative, technical, and product teams and serve as a reliable point of contact for partners and vendors.
  • Curiosity about generative AI, speech technology, and the evolving AI voice landscape.
  • Based in New York or Los Angeles.
  • Equal opportunity employer; accommodations available during the interview process.

Benefits

  • Base salary range of $114,486–$163,552 USD plus equity.
  • Health insurance.
  • Six-month paid parental leave.
  • 401(k) retirement plan.
  • Monthly meal allowance.
  • 23 paid days off.
  • Paid flexible holidays.
  • Paid sick leave.
  • Flexible work-from-home arrangement with some in-person meetings.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Welo Global

Andromeda Sadiradra - Audio Rater - Generalist - English (Ireland)

Welo Global Professional Services

Welo Data is seeking freelance Audio Raters in Ireland to evaluate recorded conversations between users and AI voice agents for response quality and naturalness.

Ireland Freelance Entry Level AI (Artificial Intelligence)
$57k-$57k
4 hours, 9 minutes ago
Welo Global

Andromeda Sadiradra - Audio Rater - Generalist - English (United Kingdom)

Welo Global Professional Services

Welo Data is seeking UK-based freelance Audio Raters to evaluate the content and vocal quality of recorded conversations between users and AI voice agents.

United Kingdom Freelance Entry Level AI (Artificial Intelligence)
$48k-$48k
4 hours, 9 minutes ago
Bitly

Engineering Technician 4, Quality Assurance (R5977)

Bitly 51-250 Internet Software & Services

Shield AI is seeking an Engineering Technician to support XBAT prototype ground and flight testing by inspecting hardware, documenting issues, and verifying test articles meet engineering requirements.

Anywhere Full-time Lead QA Engineer
$73k-$110k
4 hours, 24 minutes ago
CourtAvenue

QA Engineer (Contract)

CourtAvenue 51-250 Media

CourtAvenue is hiring a junior-to-mid-level contract QA Engineer to support its MarTech practice by validating DAM migrations, content platforms, and integrations before client release.

United States Contract Junior QA Engineer
$114k-$156k
Agile JIRA Scrum
4 hours, 24 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers