Senior Product Quality Analyst - AI Voice

Join Spotify's Personalization team as a Senior Product Quality Analyst focused on evaluating AI voice models, ensuring quality and performance across various platforms.

• Own the quality bar for Spotify’s AI voice models, defining what “good enough” means for different products, audiences, languages and use cases.
• Independently design evaluation rubrics and build an evaluation approach that measures qualities such as naturalness, tone, pacing, pronunciation, accent, audio quality and overall experience.
• Plan and carry out evaluation campaigns, turning qualitative and quantitative findings into clear recommendations for Voice Production, Speak and product teams.
• Establish an ongoing evaluation rhythm, including when and how we reassess model quality as our technology and products evolve.
• Benchmark Spotify’s AI voices against third-party models and competitors to understand where our experiences lead, where they differ and where there are opportunities to improve.
• Help determine appropriate quality trade-offs as Spotify expands into additional languages and regions, considering the product use case and needs of each market.
• Partner closely with voice experts, machine learning teams, product partners and vendors to maintain a consistent quality bar across demanding and sometimes competing priorities.
• Surface the most meaningful gaps in model performance and use evaluation insights to help influence future research and technical priorities.

• You have experience in product quality, evaluation, data curation or a closely related field where human judgment is used to assess model or product performance.
• You know how to independently define evaluation criteria and turn subjective qualities into clear, repeatable rubrics.
• You are comfortable designing and running both qualitative and MOS-style evaluations and synthesizing the results into actionable insights.
• You have an understanding of voice or audio quality, including qualities such as naturalness, pacing, tone, accent and pronunciation, or the ability to develop that expertise quickly.
• You have strong judgment and can maintain a consistent quality bar while balancing different product requirements, markets and stakeholder perspectives.
• You are comfortable working autonomously, identifying what needs to be evaluated and establishing the processes and cadence to make that evaluation useful over time.
• You communicate clearly across creative, technical and product teams and can act as a reliable point of contact for internal partners and external vendors.
• You are curious about generative AI, speech technology and the evolving AI voice landscape and can translate changes in the industry into meaningful benchmarks for Spotify.
• This role is based in London
• We offer you the flexibility to work where you work best! There will be some in person meetings, but still allows for flexibility to work from home.