Mid-term report

CompMusic has reached the midpoint and our funding agency, ERC, asked us to write a brief report on our progress. Here it is.

--------------

In CompMusic we want to advance in the field of Music Information Research by approaching a number of current research challenges from a culture specific perspective. We want to develop information modelling techniques of relevance to several non-Western music cultures and in the process we want to contribute to the overall field of MIR.

The work carried out during the first half of the project, from July 2011 until now, can be grouped under six headings: (1) choosing the music repertoires and building the research team, (2) identifying and defining the research problems and methodologies, (3) building the research and test corpora, (4) characterizing melody and rhythm, (5) developing an overall evaluation strategy, and (6) disseminating the project goals and results.

As an initial disclaimer we want to mention that it is impossible to make justice to all the work done in a big project like CompMusic with such a short report; the website reflects better the progress of the project.

1. Choosing the music repertoires and building the research group

When the project started we had decided to work on five music traditions but it was not clear the exact repertories that we would be working on and who was going to carry out the research. The two issues are very much related since not every researcher can work on any repertoire. In terms of music repertoires we focus on: Hindustani (North India), Carnatic (South India), Turkish-makam (Turkey), Arab-Andalusian (Maghreb), and Beijing Opera (China). In terms of the research team, at the UPF we have researchers working on each of the repertoires and we also have teams in Mumbai (India), Chennai (India) and Istanbul (Turkey) that specialize on their respective cultures. We have not established groups in China and Maghreb, but we have experts in those countries with which we collaborate.

The CompMusic team and project organization reflects the multicultural and interdisciplinary focus of the project. The repertoires selected are well suited for our research goals.

2. Identifying and defining the research problems and methodologies

Within the field of MIR there are many research problems that can benefit from a culture specific perspective. We decided to focus on the extraction of features from audio recordings related to melody and rhythm, and on the semantic analysis of the contextual information of those recordings. The goal is to characterize culture specific musical facets of each repertoire and to develop musically meaningful similarity measures with them. The computational approaches are based on signal processing, machine learning and semantic technologies, which are applied both to the audio recordings and to their contextual information.

The characterization of melody and rhythm has been chosen as the core of our research and we have defined the methodologies that we will be using for this characterization. This is a closed task.

3.  Building the research and test corpora

The research in CompMusic is data driven, thus it revolves around corpora. We had no relevant data collections when the project started and their construction has been a fundamental and demanding task. We have been constructing a research corpus for each music tradition. The types of data gathered are mainly audio recordings and editorial metadata, which are then complemented with descriptive information about the items we have, and in some cases with music scores and/or lyrics. The basic unit in a corpus is an audio recording and the set of information items that accompany it.

We defined the criteria with which to create and evaluate the goodness of the research corpora [Serra, 2014]. From the developed criteria we can say that the goodness of each one is currently at different levels. For example the Carnatic corpus is the best right now while the Arab-Andalusian one is still the smallest and less complete.

Together with the research corpora, we are putting together test corpora for testing, evaluating performance, and calibrating the tools used in our experiments. The test corpora are task specific and we have been developing them as they are required for particular experiments.

The developed research and test corpora are an important contribution of the project. They are all available for the research community and relevant research collaborations are emerging through them.

4. Characterizing melody and rhythm

Melody and rhythm are fundamental elements in the music traditions we are studying and at the same time they are very much culture specific. We are focusing on their study to develop methodologies with which to characterize and explore each music repertoire.

In the Indian traditions melody revolves around the concept of raga. In the Arab-Andalusian music the tab’ is the core melodic concept. In Turkish-makam music makam is the concept that is used to talk about melody. Finally in Beijing Opera shengqiang is the corresponding melodic concept. Each melodic concept has its peculiarities and needs to be addressed specifically.

About rhythm, tala is the rhythmic framework in the Indian traditions, with differences between Hindustani and Carnatic music. In the case of Arab-Andalusian music each mizán is characterized by a specific rhythmic pattern. In Turkish-makam music the rhythmic patterns are called usuls and in Beijing Opera its particular rhythmic patterns are referred as banshi. Here also each rhythm concept is different and specific computational methods have to be developed.

Most of our research contributions have resulted from developing signal processing approaches to characterize the basic melodic and rhythmic features of each tradition, using audio and scores. We have also started to apply pattern matching and semantic analysis techniques to characterize and discover relationships between the different musical concepts that we have identified, using both metadata accompanying to the recordings and community generated information. Some initial results on this have been published but the major contributions are expected in the second half of the project.

It is impossible in this short report to summarize the research done so far in the analysis of the different music repertoires; please consult our fastly growing publications repository.

5. Developing an overall evaluation strategy

Since not everything can be evaluated using quantitative and objective methods, one of the major difficulties in MIR is the evaluation of the research results. Our approach to evaluation has been to define a specific user scenario and to develop a complete system-level application with which users can interact and with which we can evaluate most of the research results of the project from a user perspective. This approach is combined with task specific evaluations that can be done using ground truths and quantitative approaches.

The system we are developing is called Dunya (originally an Arabic word which means the temporal world that was passed to many other languages) and is a web-based application to explore the gathered music collections. It is aimed at music connoisseurs of the particular music traditions. Using the technologies developed for melodic and rhythmic description the user can navigate through the audio recordings and through the other information items available in a particular collection database. This navigation promotes the discovery of relationships between the different information entities.

Currently we have a prototype for the Carnatic music collection and it includes some initial research results. We have not yet started any formal evaluation of research results using Dunya.

6. Disseminating the project goals and results

A major goal of CompMusic is to promote culture specific approaches in music information research and in the field of information technologies in general. To that aim the dissemination activities have been important, both within the MIR community and outside [project news].

The outcomes of the research have been published as articles in journals and in conference proceedings. We have organized three dedicated workshops and many special events in which much more has been presented and discussed. We have also been present in conferences of other research fields and in general we have pushed our ideas in a wide variety of academic and non-academic forums.

CompMusic has already had a relevant impact within the MIR community. Most of the community knows about the project and quite a few researchers have been influenced by it.

The major dissemination strategy of the project is to be open about everything we do; sharing our ideas, goals and results as openly and widely as we can. All our publications are available as soon as they are written, all our code is open source, and all the data we generate is available under open licenses. This has triggered research collaborations and its effect should be more noticeable in the near future. 

Important goals of the project are to develop technologies that can be used by the general public and to promote technology transfer. Dunya is a good demonstrator of our technologies with which we want to attract industrial collaborations and promote the transfer our technologies and its use in practical applications.

To access the research outputs of the project check the results page.

Conclusion

As a conclusion we want to say that the project is developing well and that we are in the right path to be able to accomplish by the end of the project the ambitious goals that the project has.

 

...xavier