C@merata: Querying Musical Scores
Texts about classical music frequently make reference to specific features of works under discussion. These references can be specific (e.g., C sharp in the treble clef) or general (fugal passage, Thor’s Hammer). Two kinds of user might be interested in finding such passages in music. Firstly, musicologists or musicians who often wish to find detailed features of a work in order to carry out further analysis or to prepare for a performance. These users know exactly what they are looking for and can express it in sophisticated natural language but may need help finding it. Secondly, people learning about music who would like to see examples of concepts they are not sure about (e.g., a perfect cadence) so that they can learn more about music itself.
The C@merata task is aimed at both classes of user. Task participants are asked to develop a system capable of answering a series of short questions. The two inputs for each question are: (1) a short noun phrase in English and (2) a short classical music score in MusicXML. The required response is a list of one or more passages in the score which contain the feature.
Target group
This task will suit computer scientists who have a good knowledge of Western classical music or musicians or musicologists who have a good knowledge of programming.
Data
The C@merata 2015 task builds on the first edition of this task in 2015 for which there were 200 questions to be matched against twenty scores in a carefully defined distribution of question types. Some questions were simple (e.g., quarter note rest) while others were slightly harder (e.g., D4 in the right hand). A few were more complex musically (e.g., tonic triad). Half the questions used English terminology (e.g., quaver) and half American (e.g., eighth note). The 2015 task will be carried out on similar lines with a wider range of more challenging questions. However, there will still be a distribution of easier and more difficult types. Last year’s test questions will be available for training.
Ground truth and evaluation
To evaluate systems we define an answer to be Beat Correct if it starts and ends in the correct bars and at the correct beat in each bar. On the other hand an answer is Measure Correct if it starts and ends in the correct bars, not necessarily at the correct beats. We then define Beat Precision and Beat Recall in terms of Beat Correct passages, and similarly Measure Precision and Measure Recall. Evaluation is therefore automatic, based on a Gold Standard produced by the organisers.
Recommended reading
[1] Sutcliffe, R., Peñas, A., Hovy, E., Forner, P., Rodrigo, A., Forascu, C., Benajiba, Y., Osenova, P. (2013). Overview of QA4MRE Main Task at CLEF 2013. Proceedings of QA4MRE-2013.
[2] Sutcliffe, R. F. E., Crawford, T., Fox, C., Root, D. L., & Hovy, E. (2014). The C@merata Task at MediaEval 2014: Natural language queries on classical music scores. In Proceedings of the MediaEval 2014 Workshop, Barcelona, Spain, October 16-17 2014.
Task organizers
Richard Sutcliffe, University of Essex, UK rsutcl(at)essex.ac.uk
Eduard Hovy, Carnegie-Mellon University, US
Deane L. Root, University of Pittsburgh, US
Chris Fox, University of Essex, UK
Task schedule
1 May Training data release
15-19 June Run submission
28 August: Working notes paper deadline
14-15 September MediaEval 2015 Workshop