Exploring the Limits of Predicting User Watching Behavior with Short-Form Videos on TikTok

Short-form video platforms such as TikTok rely on highly adaptive algorithms to curate personalized content streams. While these platforms are widely perceived as effective, one might expect that improvements in personalization would change user-watching behavior, for example, by increasing the proportion of videos watched until the end. However, prior work shows that the fraction of videos watched until the end rarely exceeds 60% and remains largely stable over time. In this paper, we investigate the limits of predicting user-watching behavior - operationalized as whether a video is watched until the end - and examine the extent to which it can be inferred from observable features. We conducted a controlled experiment in which participants interacted with a curated TikTok playlist, allowing us to isolate content-related effects from personalization, and compared these results with real-world data. Across both controlled and real-world settings, simple video metadata, particularly video duration, are the strongest predictors of whether a video will be watched until the end. When incorporating user demographic information, predictive performance improves only marginally, suggesting fundamental limits to modeling user-watching behavior in short-form video contexts. These findings challenge common assumptions about the effectiveness of fine-grained personalization and point to a potential disconnect between perceived vs. actual adaptivity and actual user-watching behavior.