What is MediaEval?
MediaEval is a benchmarking initiative dedicated to evaluating new algorithms for multimedia access and retrieval. It emphasizes the 'multi' in multimedia and focuses on human and social aspects of multimedia tasks. MediaEval attracts participants who are interested in multimodal approaches to multimedia involving, e.g., speech recognition, multimedia content analysis, user-contributed information (tags, tweets), viewer affective response, social networks, temporal and geo-coordinates.
How can I get involved?
MediaEval is an open initiative, meaning that any interested research group is free to signup and participate. Groups sign up for one or more tasks, they then receive task definitions, data sets and supporting resources, which they use to develop their algorithms. At the very end of the summer, groups submit their results and in the fall they attend the MediaEval workshop. More information on MediaEval is available on the FAQ page. See also Why Participate? and if you have some time to watch video MediaEval video trailer (2.5 min) or Interviews at the MediaEval 2010 Workshop (5 min.)
Who runs MediaEval?
MediaEval is a community-drive benchmark that is run by the MediaEval organizing committee consisting of the task organizers of all the individual task in a given year. The overall coordination is carried out by Martha Larson and Gareth Jones, who founded MediaEval in 2008 as VideoCLEF, a track in the CLEF Campaign. MediaEval became an independent benchmarking initiative in 2010 under the auspices of the PetaMedia Network of Excellence. In 2012, it also received support from ICT Labs of EIT. In 2012, MediaEval became associated with the Cubrik project, but enjoys complete autonomy. Cubrik keeps us in Rubik's cubes that we distribute at the end of the workshop to participants who have made particularly outstanding contributions.
Can I get MediaEval data from past years? MediaEval data is generally available from past years. However, the release of the past year's data is delayed until after the next year's tasks have been decided (mid February 2012 in the case of MediaEval 2011). Data that requires a license (for example, Tagging Task Professional 2010, Affect Task 2010 and LInking Task 2009) is difficult for us to release, but please contact us (m.a.larson at tudelft.nl) if you are intent on using this data and we can put you in contact with the licensers.
Is MediaEval the same as TRECVid?
No, not at all. It's true that both benchmarks work with video. But in fact, MediaEval takes great pains to complement rather than duplicate tasks running in the TRECVid video retrieval evaluation. MediaEval runs tasks that emphasize the human and social aspects of multimedia and focuses on exploiting speech, context and user-contributed information.
How are the MediaEval tasks chosen?
We chose tasks by first formulating a list of proposed tasks and then ascertaining which tasks are popular enough to run using a yearly survey. If you or your project would like to propose or organize a task in future years, please contact Martha Larson (m.a.larson at tudelft.nl).
Are there intellectual property rights (IPR) issues with the data?
MediaEval uses Creative Commons data wherever possible. When we need to make an exception we take pains to ensure that participants can license data through the appropriate channels.
For more information contact Martha Larson m.a.larson (at) tudelft.nl.