Inside a String Quartet

Researchers at the Music and Science Lab have enjoyed a unique collaboration with BBC Radio 3 and musicians from the Royal Northern Sinfonia.  The BBC’s Free Thinking festival, broadcast live from the Sage Gateshead, features the fruits of this collaboration, as we discuss analysis of the performances of two string quartet movements by Sophie Appleton (1^st violin), Sarah Roberts (2^nd violin), James Slater (viola) and Daniel Hammersley (cello), who performed the first movement of Haydn’s Op. 76 No. 5, and the third of Beethoven’s Op.59 No.2. Our aim was to explore the relationship between the four musicians in a number of modalities – studying not only the sound they produced but also their movement, breathing and heart rate. Such a volume and variety of performance data can rarely have been analysed with such a short deadline and under such pressure – a curiously liberating alternative to what is usually a painstaking journey from recording to the first results. In this blog we briefly describe the process of recording and analysis, and what it might tell us about ensemble performance…

What did we record?

Our recording studio was set up in advance of the session with an array of recording equipment. Sound was captured with microphones clipped onto each instrument to separate their sounds as much as possible (while we also recorded a stereo track to capture the combined sound). Seven video cameras provided a focused shot of each musician’s upper body, the group from two angles, as well as an overhead view. The musicians wore wristbands with coloured markers to facilitate the visual tracking of their arm movements.  Physiological recording devices captured the respiration and heart rate (ECG) of each musician. BBC Producer Matthew Dover contributed his own sound recording equipment to capture the ambience of the session. Accommodating all of the above without – we hope – inducing too much discomfort in the players was a challenge in itself.

 

We decided to record the two pieces in a number of stages: a run through of the piece was followed by a period of rehearsal to iron out any problems, after which two complete takes were captured. With some time left in the session, and thanks to the good humour of Sophie and colleagues, we proceeded to try a number of informal experiments. The musicians gamely played along with a challenge to see how fast, and how slowly, they could coordinate a passage of the Beethoven. Sophie and Sarah swapped violin parts and played each other’s for the first time. Pushing the envelope further, they agreed to try to play the Beethoven blindfold, and finally with the directions of their chairs reversed so that they could see the music but not the other musicians. At the end of the day, good will miraculously intact, they left us with an impressive collection of research data: enough to keep us busy for months, in truth, but we had less than two weeks to discover something of interest to the Free Thinking festival…

 

What did we look for?

There are several ways in which we can study ensemble coordination. For instance, in many kinds of music different musicians must perform in such a way that they produce sounds simultaneously. How they do this depends on a process of entrainment through which they come to share a sense of time – where the beat is. This ‘sensorimotor synchronisation’ can be measured, for instance by calculating the time differences between the sounds produced by different musicians.

This is not the only way in which the group is coordinated, though. Musicians must manage their performance so that transitions can be completed effectively, and steady states maintained: this is equally true whether or not the music is written down. In the string quartet, a combination of following the written music and listening intently to each other is enough to achieve an appropriate level of coordination, but we are also interested to know what role the visual cues picked up from each other’s movement play: visual information is in fact known to be crucial to coordination in many contexts.

Since we are talking about instrumental rather than vocal music there is no reason to assume that the musicians’ breathing should necessarily be coordinated. On the other hand, musicians often consciously use breathing as a cue, for instance inhaling conspicuously just before an important entry. Might we find, after all, some form of coordination between their breathing?

And finally, what could we learn about the performance of music by measuring the musicians’ heart rates? Two people’s heart beats will only synchronise – literally, rather than metaphorically – in specific circumstances: for instance, this has been reported for high performance rowers where the combination of strenuous effort and synchronised movement and breathing can lead to this result. The idea that musicians’ heart rates synchronise in a comparable manner is highly implausible – at least in most musical contexts we might think of. However, other things can be determined from ECG measurements, for example heart rate variability, which can be interpreted as a stress marker (that is, under stress our heart rates become less variable). Might we observe this effect in musical performance?

Alongside all of this, we also noted the timing of each bar of music in the final takes, and with that the timings of important structural points in the music. Using this information we could relate our findings to specific places in the music.

What did we find?

A difficulty in measuring synchronisation in a string group is that the onset times of notes produced by bowing are notoriously difficult to calculate, because of their inherently slow attack. However, armed with Tuomas Eerola’s latest computer scripts we were able to obtain data for a good proportion of the final takes of each movement. These data show that the timing differences (asynchronies) between parts are very small (generally below about 5 hundredths of a second – the variability is measured by a standard deviation of about  3 hundredths of a second). This is roughly what we would have predicted, in line with the (relatively few) comparable measurements of chamber music groups in the scientific literature. We now have different comparators, however: examples of Malian jembe music recorded by Rainer Polak and analysed as part of our IEMP Project, for example, show only about half as much variability. Why is that? Because the latter play drums, which have short sounds with very well-defined onsets, while bowed string sounds start more gradually and last longer – it may not be possible to produce the same level of precision in that case. Our musicians, then, show the high level of precision one would expect from expert players.

Another thing we can calculate from this data is whether any one musician plays (on average) ahead of the others. Previous research has pointed to a phenomenon called ‘melody lead’, in which the instrument carrying the main melody part plays slightly ahead of the others. The string quartet seem to show this: after marking up the score according to which instrument carries the melody line at each point, we calculated that that ‘leading’ player sounds on average about 10 ms ahead of the others. This is a tiny amount, too little to be heard as a timing difference. What this might do, nonetheless, is to allow that part to stand out a little more from the texture, attracting the listener’s attention.

What can we tell from the musicians’ movement? To discover this, we used computer vision techniques to measure their head movement from the video recordings, and then calculated the relationship between the periodic movements of each pair of players. What we have found in other kinds of music is a high correlation between the peaks of this coordination and section boundaries: in other words, people move together more when they coordinate a change in the musical texture. This also seems to be the case in our string quartet recordings, although the data are rather dense – with 6 pairs to consider, comprising all combinations of four players – and will take some time to analyse fully. Obvious examples include the Haydn movement, in which a gradual deceleration to a clear ending is followed by a brisk concluding Allegro section. How do the musicians coordinate in such a way that they re-enter together at the right speed? The movement data points to the answer: by watching each other’s movements, so that by the time we hear their instruments they are already synchronised.

What about their breathing? This relates to movement, of course, as we move when we breathe, but the data show distinct aspects of the relationship between the musicians. Clear peaks in the coordination between the breathing rates of the musicians seem to occur at specific points: certain points of emphasis, such as loud chords (the Beethoven contains great examples when, after some very lively playing in which the instruments’ rhythms clash with each other, they all play a loud sustained chord together); some joint entries (where two players need to be sure they will come in together); and when all musicians need to slow down in synchrony, as they often do approaching the end of a section. Just as interestingly, this breathing coordination disappears almost completely in the moments of relaxation immediately after a dramatic cadence.

And finally… what about the heart rate data? This is the trickiest of all to study, but the heart rate variability analysis points to certain sections of the music in which the musicians may be showing signs of stress. (For the cardiologists reading, we know this is not a simple matter!) Some of these stand out as the moments in which the musicians must stay together while playing fast, intricate passages. In some cases, in all honesty, it is not completely clear to us why the musicians are stressed, if indeed that is what the data shows. This is just one of many things for us to investigate in the coming weeks and months.

Thanks to BBC Radio 3 for setting us this challenge, and to four terrific musicians for their beautiful music and tremendous good humour. These members of our team contributed: Simone Tarsitani (recording engineer) and Petr Nuska (recording assistant); Martin Clayton, Tuomas Eerola and Kelly Jakubowski (analysis).

You can discover more about the topic here.

Please tell us about your experiences playing and listening to music. Which aspects of music would you like to learn more about?