The Video Passage Retrieval Task
Video search benchmark evaluations typically focus on queries/topics that are constructed with reference to visual objects such as a specific person, thing, activity or location. Consequently, for solving these types of topics, approaches that make use of combination of modalities (visual, audio, metadata) are not really required. However, there are many use scenarios and information needs thinkable in the context of the exploitation of video repositories that could benefit from a broader interpretation of the notion of topic, i.e. more focused on factoid information, or targeting downstream applications such as cross-media linking, content-based video summarization, question-answering and multimedia storytelling. For such types of topics the exploitation of information coming from the analysis of multiple modalities becomes crucial. Note also that in this context the segmentation of the video becomes a significant part of the search process.
Task
The Video Passage Retrieval Task (new in 2010), involves the identification of relevant jump-in points in video given a set of queries based on the combination of modalities (audio, speech, visual, metadata). Manually generated metadata, speech transcripts (including speech/non-speech and speaker segmentation will be provided) and video concept labels will be made available with the data. Queries will be provided both in Dutch and English.
Target group
The task is of interest to researchers in the area of multimodal retrieval and spoken content retrieval.
Data
The task uses a Dutch-language television collection from the Netherlands Institute for Sound & Vision (used in TRECVid 2007-2009) and consists of several hundreds of hours of Dutch news magazine, science news, news reports, documentaries, educational programmes and archival video.
Groundtruth and Evaluation
Queries are created by making use of passage descriptions that were created by professional archivist of Sound and Vision. The time-markers of the descriptions will serve as groundtruth for evaluation.
Example query
Conversation with weatherman John Bernard about how the weather could be forecasted on the basis of animal behaviour
(Dutch: Gesprek met weerman John Bernard over de manier waarop het weer te voorspellen is aan de hand van het gedrag van dieren)
Task coordinator: Roeland Ordelman, University of Twente and Netherlands Institute for Sound & Vision (a.k.a. Beeld & Geluid)
(rordelman at beeldengeluid dot nl)
