Authors:
Jan Malte Lichtenberg、Giuseppe Di Benedetto、Matteo Ruffini
Paper:
https://arxiv.org/abs/2408.09168
Introduction
In the evolving landscape of media streaming services, platforms are increasingly offering a diverse array of content types. For instance, audio streaming services that initially focused solely on music now also provide podcasts, videos, and merchandise. This diversification presents a significant challenge for traditional learning-to-rank (LTR) algorithms, which struggle to rank items from different content types due to varying user engagement patterns. This paper explores a novel method called multinomial blending (MB) to address this challenge, enhancing ranking quality while maintaining interpretability, ease-of-use, and stability in dynamic environments.
Related Work
Challenges in Cross-Content-Type Ranking
Traditional LTR algorithms face several challenges when applied to multi-content-type settings:
1. Disjoint Item Feature Sets: Different content types have distinct feature sets.
2. Cold-Start Problem: New content types introduced later face initial ranking difficulties.
3. Differing Engagement Patterns: Users engage with different content types at varying frequencies, leading to unbalanced exposure.
Existing Approaches
Several methods have been proposed to tackle these challenges:
– Diversity-Optimizing Policies: These modify the LTR algorithm’s loss function to promote diversity but lack content-type exposure guarantees and require complex parameter tuning.
– Reward Shaping: Assigns different weights to clicks from various content types, but estimating the relative importance of reward signals is complex.
– Post-Processing Approaches: Manual overrides and re-ranking methods ensure exposure but are labor-intensive and can bias propensity estimates.
Research Methodology
Multinomial Blending (MB)
Multinomial blending is a straightforward yet effective method defined by a multinomial probability distribution with a vector of sampling probabilities. The process involves:
1. Scoring all candidate items using a scoring function.
2. Sampling a content type based on predefined probabilities.
3. Selecting the highest-scoring remaining candidate from the sampled content type.
4. Repeating the process until the desired slate size is achieved.
Advantages of MB
- Interpretability: Each parameter maps to the expected content-type average exposure.
- Stability: Exposure guarantees are independent of the scoring function, ensuring stability even after model retraining.
- Personalization: Preserves personalized ranking within each content type.
Experimental Design
A/B Testing
The effectiveness of MB was tested through an A/B experiment on Amazon Music, where podcasts and music containers were ranked together. The experiment compared three approaches:
1. Control Treatment: Manual ranking overrides to boost podcast exposure.
2. MMR Diversification: Using the existing ranker with maximal marginal relevance (MMR) diversification.
3. MB Diversification: Using the existing ranker with MB diversification.
Evaluation Metrics
The treatments were evaluated based on:
– Podcast Listening Time: Increase in the time users spent listening to podcasts.
– Overall Engagement: Improvement in overall user engagement metrics.
Results and Analysis
Performance Comparison
Both MMR and MB treatments achieved a Pareto improvement over the control treatment. However, MB demonstrated several operational advantages:
– Ease of Setup and Maintenance: MB is easier to set up and update over time compared to MMR, which requires extensive parameter tuning.
– Computational Efficiency: MB is more computationally efficient as it does not require sequential re-computation of scores.
– User Acquisition Metrics: MB outperformed MMR in podcast user acquisition metrics, ensuring consistent podcast exposure at the user level.
Overall Conclusion
Multinomial blending (MB) offers a robust solution for cross-content-type ranking in media streaming services. It balances personalized diversification with interpretability and ease-of-use, making it suitable for dynamic, industry-scale environments. The A/B test results on Amazon Music highlight MB’s operational advantages and its potential to enhance user engagement and satisfaction. Future work could explore methods to achieve full personalization while maintaining MB’s stability and interpretability benefits.
By addressing the challenges of multi-content-type ranking, MB paves the way for more effective and user-friendly content recommendation systems in the ever-evolving media streaming landscape.
