Introducing LikeLines
LikeLines unlocks user-sourced video and serves as a building block for rich news story-telling. Interesting bits of a video emerge naturally through community interaction with the video using an intelligent video player, enabling video navigation, browsing, retrieval and linking at the fragment level.
Design and prototype
LikeLines transforms existing video players into intelligent ones by adding heatmap navigation below the player. The heatmap shows which parts of a video are found to be interesting by the viewer community and allows viewers to jump to these interesting bits right away. A prototype showcasing the heatmap can be viewed below. |
| Click here to try the prototype in action! |
The hotspots in the heatmap are generated through interaction with the video. When a user explicitly expresses "liking" at a particular time point during playback of the video, this act of expression together with the current playback position of the video is stored in the system. The system aggregates over this feedback to derive the "hottest" points in a video. In addition to explicit feedback, implicit feedback in the form of playback and seeking behavior is also used. Information on users playing, pausing, re-playing and seeking in the video can be used to refine existing or discover new hot points.
The LikeLines system consists of two components: a client-side script that extends existing video players and a LikeLines repository server that is responsible for aggregating user feedback and deriving the hotspots in the video. The LikeLines API allows the web application developer to pick any source of videos and any LikeLines repository. The LikeLines repository stores and allows applications to retrieve heatmap metadata (i.e., time-code specific popularity information about specific videos). The key, innovative contribution that LikeLines makes to unlocking video is the collection and management of heatmap metadata, which can be applied in wide variety of use cases.
Integration into existing newsroom infrastructure
There are many examples where LikeLines can be used:Tips bin: If news organizations open up their tips bin such that visitors can see submitted videos, visitors can already begin interacting with these videos (liking/seeking) and thereby annotating them. This eases the task of the news staff of sorting through the submitted videos as they can focus on the highlights.
Archive: When LikeLines is deployed in archives, users can find the most popular past segments, which will fuel ideas for new story subjects. It can be used in both private archives and public archives (e.g., Dutch Footage). Related videos can be linked at the fragment level, allowing discovery of new and interesting patterns.
Web monitoring tools: When LikeLines is adopted externally, e.g., on YouTube, monitoring of these video sites can be improved. Instead of indiscriminately showing everything, snippets based on the hottest parts of a video can be generated and displayed instead.When building LikeLines and these tools, it is important to work closely with both journalists and end-users. End-users need to be able to understand and use the LikeLines interface if we want to generate heatmaps effectively. On the other hand, the metadata coming from LikeLines needs to be sufficiently suitable for the purposes of journalists.
Collaborative power
LikeLines combines eyewitnesses, Internet viewers and news reporters into a strong collaborative workforce. Eyewitnesses can capture news on the street using their cellphones and upload their raw videos onto the web. No editing is needed. Instead, viewers watching these videos are annotating which parts are hot through their interaction with the LikeLines player. News reporters can then process these enriched videos by extracting the interesting bits and weaving a story out of it.Challenges and unknowns
- How to interpret user clicks on the like button and their implicit playback behavior? How to amalgamate and denoise user input?
When a user clicks the like button, it is not certain if the "like" should apply to this position or a position several seconds earlier. A user study involving an early working prototype is needed to address this aspect of the concept and also refine the user interface and determine the optimal algorithm for aggregation of the heatmaps of multiple users. - How to deal with the cold start problem, i.e., unwatched videos?
For new videos, the user-feedback process can be jump-started by generating an initial heatmap, for example using multimedia content analysis (MCA). We need to address the issue of finding platforms with sufficient computational capacity for MCA and motivating them to make the necessary investment to generate initial heatmaps. Further, platforms using LikeLines need to make sure fresh content is highlighted so that the process of aggregating user feedback starts as soon as possible. Attention should be devoted to the development of mechanisms for incentivizing users (e.g., via awards such as access to premium content) to contribute user feedback for fresh video. - How to ensure a large user-base?
The success of LikeLines will requires that the system be used by a critical mass of viewers in order to generate useful heatmap. To ensure a sufficiently large user-base, LikeLines is designed as an open and versatile building block such that it can easily be integrated in existing web applications.
Executive summary
- Gets to the core of news quickly and effectively — as stories are breaking.
- Supports creation of news stories attuned to current viewer concerns by exploiting the compelling story-telling power of first-hand accounts and user-sourced video.
- Solves the problem of time-consuming sifting through user-sourced video, which can be critical under deadline pressure.
- Competes effectively in current user-sourced footage landscape where coverage is low because individuals must filter raw footage.
- Makes it affordable for news organizations to be present along more steps of the user-driven production-through-consumption chain.
Great concept. This would be very useful for Youtube, Vimeo etc. so users can get right to the point! It can be used internally at organizations for flagging material as well.
ReplyDeleteHi Nicole, thanks for your comment. :)
ReplyDeleteI agree and I really hope that this idea gets adopted. The old boring time-based seek bar needs to go! :p
Well done. Love the idea.
ReplyDeleteAre likes a start-end interval or a point?
ReplyDeleteAgain, thanks for the comments. :)
ReplyDelete> Are likes a start-end interval or a point?
Since likes are expressed by clicking the Like button, they correspond to single points in time. However, it is unlikely that the user only liked a single frame in the whole video. As such, this needs to be taken into account when interpreting these clicks. How exactly these clicks are to be interpreted needs further investigation (and an improved prototype that actually stores the clicks — unfortunately I didn't manage to implement that in time).
Now, an alternative would be to allow users to mark areas in a video to be liked, but I fear not many would engage in such an activity. It's like Wikipedia. Most of the users consume rather than contribute. Liking/disliking/flaggin a video, that's okay and doesn't cost much effort. But annotating segments in a video? I don't think most people would do that.
Raynor, I think people would be just as keen to mark a liked segment as a liked point, but it'd have to be on a second pass, because, as you've written in your proposal, on the first it would start and end too late.
ReplyDeletePerhaps each "point like" can pop-up a clickable snapshot that makes it easy for the user to return to that point, where they're asked to use the slider to mark the start and end of the good section.
Yes, few will do this, but enough to give you the data you need. People are willing to do these things if the collective action also benefits them.
Mark, thanks for keeping the discussion going. :)
ReplyDeleteIn the current prototype, liked points already pop-up under the heatmap and are clickable. I was thinking of having them at least draggable, such that users can move them to the left or to the right.
I like your idea of the slider, which would then be a small extension to my idea of draggable like-points. I'm just wondering at the moment how you'd present it in the UI, since there would be a lot of possible actions to a single like-point (click to seek in video, drag to move, to mark start/end points).
> People are willing to do these things if the collective action also benefits them.
While typing this comment, I came up with a possible incentive for users to mark areas. Now, I'm not sure how to exactly present this in the UI, but there could be something like a dialog or notification box inviting the user to share parts of the video:
"Did you like ? Share this fragment with your friends!"
This of course requires a way of linking to parts of a video. YouTube already allows you to link to a particular point in the video, but AFAIK not to segments. Hmm, wasn't there a MozNewsLab participant that was working on some linking scheme to make this possible?
Ah, I think I remember... I saw a demo of Hyperdisken somewhere that allowed you to link to segments in an audio file.
ReplyDeleteBtw, for anyone interested, this project by Julien is also related: MetaFragments.
ReplyDeleteKnows what exactly? :p
ReplyDelete