The 2019 Multimedia for Recommender System Task

Task description
For this task, participants analyse items and derive feature sets combining modalities, for instance, audio, images, and text. Subsequently, they implement predictors that estimate which items will be relevant to users.

Participants can target two subtasks that cover different domains. First, the movie recommendation task asks participants to predict the average rating of movies given by users, the variance as a measure of raters’ agreement along with popularity of movies. The provided data set includes links to movie trailers, precomputed state-of-the-art audio-visual features, and metadata from MovieLens. Second, the news recommendation task challenges participants to predict the reading frequencies of news articles. The provided data set comes from a set of German publishers and spans multiple months. It features text snippets, image URLs, and some pre-extracted neural image representations.

Recommender Systems (RecSys) permeate our digital landscape. Whenever users face an overwhelming amount of information, system operators introduce recommender functionalities to preselect a subset of expectedly relevant information. Multimedia RecSys investigates which role multimodal data can play to improve recommendations.

Task motivation
Traditionally recommender systems and multimedia data processing are studied by separate groups of researchers. These researchers have a lot to learn from each other, and this task offers in interdisciplinary forum that promotes exactly such exchange.

An often-cited motivation exploiting features derived from multimedia is cold start. However, in this task, we also relate the importance of using multimedia in recommender systems to the drawbacks for personalization. Personalized information access comes with some caveats. Predictions become successful for some users whereas they fail for others. Understanding how multimedia affects users’ perception of items facilitates creating fair and unbiased information access systems. Recommender systems have been found to induce “filter bubbles” preventing access to some information. The high complexity of content data promises to overcome this issue as content similarities can be defined among all items. Further, the use of multimedia has potential to promote the development of recommender systems that need less user-specific interaction data in order to make recommendations, thus promoting privacy.

Target group
Researchers will find this task interesting if they work in the research areas of multimedia processing, personalization and recommender system, machine learning and information retrieval.

The movie dataset includes links to the videos (Youtube URLs), precomputed state of the art audio-visual features, and metadata from MovieLens. The news dataset is collected from a set of German publishers and spans multiple months. It includes text snippets, image URLs (and some pre-extracted neural image features).

Ground truth and evaluation
to be added

Recommended reading
Yashar Deldjoo, Mihai Gabriel Constantin, Markus Schedl, Bogdan Ionescu, Paolo Cremonesi. MMTF-14K: A Multifaceted Movie Trailer Feature Dataset for Recommendation and Retrieval, Proceedings of the 9th ACM Multimedia Systems Conference, 2018.

Yashar Deldjoo, Mehdi Elahi, Paolo Cremonesi, Franca Garzotto, Pietro Piazzolla, Massimo Quadrana. Content-based Video Recommendation System based on Stylistic Visual Features, Journal on Data Semantics, 5(2), pp. 99-113, 2016.

Yimin Hou, Ting Xiao, Shu Zhang, Xi Jiang, Xiang Li, Xintao Hu, Junwei Han, Lei Guo, L. Stephen Miller, Richard Neupert, Tianming Liu. Predicting Movie Trailer Viewer's “like/dislike” via Learned Shot Editing Patterns, IEEE Transactions on Affective Computing, 7(1), pp. 29-44, 2016.

Lommatzsch, Andreas, et al. “CLEF 2017 NewsREEL overview: A stream-based recommender task for evaluation and education.” International Conference of the Cross-Language Evaluation Forum for European Languages. Springer, Cham, 2017.

Corsini, Francesco, and M. A. Larson. “CLEF newsreel 2016: Image-based recommendation.” (2016).

Task organizers
Yashar Deldjoo, Politecnico di Milano, Italy, first.last
Markus Schedl, Johannes Kepler University Linz, Austria
Andreas Lommatzsch, TU Berlin, Germany, first.last at
Benjamin Kille, TU Berlin, Germany, first.last at

Task schedule
Data release: 15 May
Runs due: 20 September
Results returned: 25 September
Working Notes paper due: 30 September
MediaEval 2019 Workshop (in France, near Nice): 27-29 October 2019