Music, visual imagery, and emotions

We have plenty of parallels between the visual and auditory domains. Just think of how the word “brightness” is used to describe not only colours but also sounds. Music can also go “up” and it can give rise to elaborate visual narratives. Some applications draw direct parallels

with visual information and how to play the music (e.g., Guitar Hero), which are also to an extent capitalised in traditional Western scores. Some scholars have also mapped emotions and colours in musical context and produced these intriguing colour palettes of musical emotions (see Palmer et al., 2013; Lindborg & Fribeerg, 2015, image on the right).

All of these are perceptual phenomena which stress the cross-modal and metaphoric links between auditory and visual information. However, music alone can often trigger visual imagery, either colours, abstract shapes or complex visual narratives (films in your mind’s eye). How frequent are such experiences, how do they they actually work, and is there a strong link to emotions? These are some of the questions articulated in a recent international workshop on music, emotions, and visual imagery. Below is my short summary of the academic highlights of the event.

Workshop on visual imagery, music, and emotions

Almost ten years ago, Patrik Juslin and Daniel Västfjäll proposed a set of mechanisms that could be involved in inducing emotions via music (2008). Several of these mechanisms such as entrainment and episodic memory have received empirical attention over the last years, but others have remained fairly marginal. One of these, visual imagery, is only now starting to get attention. A special KOSMOS-dialogue workshop took place in early June at Humboldt University at Berlin, organised by Mats Küssner, André Weinreich, and me. Here’s a small summary of the insights that were presented at the workshop.

What is the connection between visual imagery, emotions and music?

We do not know whether visual imagery actually precedes emotions. At the workshop, Robina Day and Bill Thompson (Macquarie University, Australia) provided some early evidence that the emotions actually precede visual imagery. If true, it does pose a problem for the idea that visual imagery is a fundamental emotion induction mechanism. Their probed the question simply by asking listeners to indicate with a key press in three experiments when they (1) recognised an emotion in music, (2) experienced an emotion induced by the music, or (3) had an visual imagery related to music. Although the visual imagery often appeared after the emotions in their data, this could also be relate to the types of emotions they were interested in (low level affects such as valence, energy, and tension) whereas the construction of visual imagery probably takes more time. However, I wonder whether similar slow process might be in question if more complex, aesthetic emotions such as wonder or transcendence are being reported.

Fabian Moss and Fernando Bravo from Technical University Dresden also demonstrated how we have strong analogies between the visual and auditory domains. They allowed participants to adjust visual elements such as colour saturation, brightness, and contrast of Kandinsky paintings to best match music that had been composed with a continuum ranging from dissonant to consonant intervals. As one can expect, consonant sounds led to brighter colours and sharper contrasts.

There were several other highly engaging presentations about visual imagery in different situations – explained by Renee Vulto from Utrecht University – and how imagery and emotions possibly alternate in time, and the contents of visual imagery during music listening as presented by Graziana Presicce from the University of Hull.

Visual imagery is part of larger phenomenon of mind wandering and meta-awareness?

Liila Taruffi from Freie Universität, Berlin, presented perhaps the neatest series of studies at the workshop. She had explored mind-wandering when listening to either happy or sad music and found that sad music enhances self-reflection and really is conducive to mind wandering and interestingly, to visual imagery. Moreover, she also presented intriguing fMRI activations under the influence of sad and happy music which offered a similar story of sad music showing areas related to the brain’s default mode network, which is active when we go on autopilot. Her final set of analyses suggested that activations during the sad music listening are also linked with empathy (or at least there are correlations between the active of the key brain areas and subscales of the interpersonal reactivity index). These observations are similar to our findings concerning the amount of being moved by sad music that seems to be directly associated with empathy. Why would capacity for social interpretations be linked with enhanced processing of such self-reflective moods remains to be explored, but the findings do open up intriguing directions for future research.

Another related and inspiring work was carried out by Thijs Vroegh (from the Max Plank Institute for Empirical Aesthetics) that looked at the topic of what happens in the mind even more in detail. He had examined the structure of consciousness whilst participants were listening to their favourite music. The experiences were captured using a self-report instrument called the Phenomenology of Consciousness Inventory (Pekala, 1991), which probes various themes relevant for conscious thinking such as the level of attention, arousal, visual imagery, negative affects, rationality, self-awareness, and volitional control. Although self-reports cannot encapsulate all the facets of mental states, Thijs had identified interesting listening profiles from a group of volunteers. These distinct mental state profiles described how people become absorbed in their favourite music, some with losing the awareness of their surroundings completely, some with a more focussed sense of their mental states during the listening. These different consciousness profiles during the listening were not related to visual imagery as such but to underlying emotions, whether they were positive or mixed. This was a great reminder of the complexities involved but opened up new directions for future research.

Applied directions of music and visual imagery

Many of the presentations at the workshop provided attempts to probe the fundamentals but there were elaborate examples of the applications of visual imagery in pedagogic and therapeutic contexts. For instance, Mary Black (University of Leeds) gave fantastic examples of how conductors use visual imagery in the choral rehearsals to accomplish desired interpretations or sound qualities. We also heard several beautiful examples of how image schemata are being used in musical works (e.g. in the opera La bohème as explained by Stamatis Gerothanasi from the Aristotle University of Thessaloniki).

Visual imagery is also being actively used in music therapy. Two presentations addressed the interaction between music and imagery in actual therapy; my former colleague Jörg Fachner (now at Anglia Ruskin University) explored the resting state networks during therapy involving the Guided Imagery Method (GIM) and Helena Dukic from the University of Graz focussed on the musical devices (interval sizes) and the types of imagery during so-called nurturing music, as described by GIM. The great advantage of the GIM method is that it is well-established, the music tracks have been carefully selected and the whole therapy programme is well-documented and seems to provide positive results under certain conditions (Burns, 2001), so bringing such existing practices within the discussion of basic research was constructive.

Priority topics in the visual imagery and music research?

At the end of the workshop, we worked in small groups to discuss the central ideas, bottlenecks and methods related to the topic. To me, this was untested territory and the freedom given to the groups could easily have led to frustrations due to wide range of disciplines represented by the group members. Instead, the groups were able to articulate rather clear sets of issues such as the necessity to carry out cross-cultural research, probe the voluntary attention involved, different triggers, and the temporal order of imagery and emotions in order to make progress in this topic. We agreed to follow up the workshop with sharing the information, planning similar events in the future, and by directing attention to the topic by editing a special issue in a good journal in our field.

For more information, see the Music, Emotion, and Visual Imagery site.