Towards Extracting Highlights From Recorded Live Videos: An Implicit Crowdsourcing Approach

Live streaming platforms need to store a lot of recorded live videos on a daily basis. An important problem is how to automatically extract highlights (i.e., attractive short video clips) from these massive, long recorded live videos. One approach is to directly apply a highlight detection algorithm to video content. While various algorithms have been proposed, it is still hard to generalize them well to different types of videos without large training data or high computing resources. In this paper, we propose to tackle this problem with a novel implicit crowdsourcing approach, called Lightor. The key insight is to collect users’ natural interactions with a live streaming platform, and then leverage them to detect highlights. Lightor consists of two major components. Highlight Initializer collects time-stamped chat messages from a live video and then uses them to predict approximate highlight positions. Highlight Extractor keeps track of how users interact with these approximate highlight positions and then refines these positions iteratively. We find that the collected user chat and interaction data are very noisy, and propose effective techniques to deal with noise. Lightor can be easily deployed into existing live streaming platforms, or be implemented as a web browser extension. We recruit hundreds of users from Amazon Mechanical Turk, and evaluate the performance of Lightor on real live video data. The results show that Lightor can achieve high extraction precision with a small set of training data and low computing resources.