1.1

Models of creativity: what behavioral and brain-imaging studies reveal

The breath of literature on creative thinking and its subtopics—achievement, expert knowledge, imagination, insight, intelligence, problem solving, and exceptional accomplishments and/or artistic production post brain trauma as chronicled through case studies—is extensive and rich. There is no shortage of reflection or experimentation, with perspectives ranging from art history, musicology, and philosophy to cognitive science, computer science, and (neuro)psychology. ^{12, 14-17, 20-35} The systematic cognitive psychological study of creativity, however, did not receive significant attention until American psychologist J. P. Guilford made it the focus of his now classic presidential address to the American Psychological Association in 1950.²¹ This call for investigation came right as the cognitive (and computer) revolution began to respond to the dominant behavioristic approach of scientific psychology and its theories about stimulus-response associations with arguments for the existence of mental representations to fully characterize the complexity of human behavior.¹⁸ Under this creative cognition approach, which developed more extensively in the 1990s and the one adopted here, creativity is an accessible and testable behavior amenable to the methods of the empirical cognitive sciences and is defined as a mental phenomenon that engages multiple cognitive processes such as attention, emotion, memory, reasoning, and reflective decision-making judgments to generate unconventional, novel and useful solutions to problems.^36-38 The simultaneous advent and mass availability of brain-imaging techniques in the 1990s precipitated a broad range of experiments, and the effort continues to establish a link between creative (artistic) behavior and particular brain networks.^39-55

1.2

The interdisciplinary move ahead

The study of (artistic) creativity is undeniably interdisciplinary. We arrive at the crossroads of multiple disciplines and at a moment in history, and in the study of this fundamental human intellectual capacity, where bold methods of investigation are called for if we are to fully and successfully model the complex process of innovation. In essence, we have perhaps circled back to what Renaissance humanists from the fourteenth to sixteenth centuries sought for: eloquence, or a mastery of many disciplines so that lives of wisdom and consequence could be led.⁷² I suggest, therefore, to take stock of what we have done and reconsider what we could do in the near future. The studies discussed have used traditional scientific reductionist methods and controlled testing environments for empirical exploration. In other words, pose question x, pick contrasting variables y and z and a control, bring artists into the lab to perform conditions a and b, analyze the observed c behavior to determine if the expected answer can support previous claims regarding question x, and repeat in search of reproducibility and validity. The neuroscience perspective to the creative cognition approach similarly needs to be adopted with care.⁷³ As with other complex psychological categories such as feelings, memories, and the self, thinking is a collection of mental states originating from the brain. Known as the mind-brain correspondence problem, psychological categories are not necessarily instantiated in a one-to-one correspondence with brain states and the field of cognitive (neuro)psychology may need to reassess its framework of investigation.⁷⁴ Issues aside, the results as they stand have been elucidating. As a side note, the tendency towards experimenting with highly proficient and successful artists when studying creativity does not rule out individuals who are highly proficient and successful in other professions such as business, education, engineering, journalism, law, medicine, etc., that demand enormous amounts of creativity both during immediate and more long-term situations; this has been a flaw from the overall investigators’ side of the field, a loss of interdisciplinarity within and between disciplines, and, more broadly, the inevitable immaturity of a field still in its infancy.

An alternative method of investigation that has begun to shine in the spotlight, despite its initiation in the 1950s,^75,76 is known as arts-based behavioral research, or the systematic use of the artistic process in its natural environment as the primary way to explore, experiment with, challenge, and understand perception, emotion, memory, consciousness, and other cognitive functions.⁷⁷ It is of course plausible to consider this method entirely applicable to other disciplines. In this domain of inquiry, a question x is posed or a desired effect is established with an openness to in-the-moment change of conceptual direction, variables y and z are chosen for exploration, the artist tests a range of possibilities to see what will emerge or until the desired effect is achieved, and the process is repeated (with alterations), always in search of variations and new effects. A notable example is witnessed in films where Pablo Picasso’s creative painting process in his studio was twice the subject of observation: for Belgian filmmaker Paul Haesaerts’ Visit to Picasso (“Visite à Picasso”) in 1950 and French filmmaker Henri-Georges Clouzot’s The Mystery of Picasso (“Le mystère Picasso”) in 1956 in which most of the paintings Picasso created on camera were subsequently destroyed after filming, an action perfectly in-line with the objective of the film (i.e. the creative process, not the resulting product). In both documentaries we witness first-hand concept origin, brushstroke technique, idea transformation during implementation, concept reinspiration, and aesthetic evaluation and re-evaluation all unfold in real-time; the core elements of a problem space solution-searching model. I make no claims of methodological superiority, simply an open-mindedness to accept Art and Science’s individual and collective value as toolboxes of truth searching and a resurgence of arts-based inquiry into the creative process to capture spontaneity in action. Which brings me to the crux of this paper: inspired by artistic methods, but guided by empirical questions, I have created two unique projects within film and theatre at the intersection of Art and Science with the goal of witnessing and revealing how audiovisual sensory information, emotion, language, and the generation of new material all interact in a given live performance moment to produce a coherent artistic object.

2.1

Moments

The inspiration for creating a live performance of improvised music scoring to a new contemporary film came about after watching one of the most famous examples of improvised film scoring in film history, Louis Malle’s 1958 debut feature Elevator to the Gallows (“Ascenseur pour l’échafaud”) with trumpeter Miles Davis at the helm.⁸³ The interaction between image and sound is breathtaking and multilayered in many scenes throughout the movie, but one particular now iconic 3-minute long scene stands out: the heroine, played by French actress Jeanne Moreau, walks with poise through the dark rainy streets of Paris in search of her lover while Davis’ trumpet sings with modal moodiness all her sadness, loneliness, and anger in one single track. In my contemporary reenactment, I wrote, filmed, and directed a 39-minute long film titled Moments with minimal spoken dialogue where the six universal emotions are dissected as a woman and man’s relationship is tested through a series of events. The purpose of including minimal dialogue, a total of fourteen lines spoken by the woman and 7 by the man, was to provide an almost purely visual yet realistic platform for the musicians to interpret. The film is divided into five chapters with the following headings: Ch. 1: Sadness, Ch. 2: Surprise and Disgust, Ch. 3: Fear, Ch. 4: Happiness, Ch. 5: Anger. Jazz musicians improvised the film’s soundtrack live during a public screening in Baltimore, MD.⁸⁴ Prior to the public screening I informally screened it to several unbiased viewers to check for the stability of the emotional content within each scene. All viewers agreed that the emotions expressed by the characters were realistic and identifiable.

The trio who improvised the music included piano, tenor saxophone, and bass and all three musicians were familiar with each other but had not yet played together or composed music for film prior to this performance; they had a combined average of eleven years of professional working experience. The musicians and I met several weeks prior to the performance to discuss their role as soundtrack improvisers who would be tasked with creating a soundtrack that “fits” the narrative of the movie and playing continuously from the film’s beginning to the closing credits, and watch the film and review the narrative structure of the storyline. Musical ideas and a soundtrack, however, were not developed, charted out, or scored and the musicians left without a copy of the film. I specifically did not make explicit how music is used in and for film contexts to avoid constraining musical ideas. The purpose of this arrangement was to familiarize the musicians with the task and layout of the screening and performance venue, and to avoid any musical biases or composed material from being planned and arising during the performance in order to create an entirely novel live audiovisual experience. During the movie’s screening the minimal dialogue was muted and included as subtitles and the musicians stood on stage, as in any live concert figuration, each with a small laptop or iPad from which to view the film in real-time. The performance was recorded live. Afterwards, I interviewed a jazz musician with approximately seven years of professional working experience about the technical musical aspects of the improvised compositions. My experimental questions were: what effects do clearly demarcated realistic visual emotional stimuli have on improvised musical content and how will multiple musicians deal with this musical task live on stage?

2.2

Última Partida / The Final Draw

While the theatrical empirical follow-up to Moments, Última Partida (“The Final Draw”) was thematically inspired by Italian filmmaker Michelangelo Antonioni’s 1961 narrative on social deterioration and romantic alienation in La Notte (“The Night”). In Última Partida, an original one-act dramatic play I wrote, directed, and produced with La Petite Noiseuse Productions, three fictitious characters convene at a cabaret during a single night: a woman, a man, and a pianist. The woman and man’s past, present, and future relationship unfolds in six scenes as they discuss love, disaffection, solitude, and death: i. Oferta para dialogar (“Offering to talk”), ii. Esperanza de volver (“Hoping to get back together”), iii. Interludio festivo (“Comic interlude”), iv. Imposibilidad de volver (“The impossibility of getting back together”), v. Dolor de la memoria (“The pain of memory”), and vi. El último adiós (“The final goodbye”).⁸¹ The audience is introduced to the jazz pianist through his live improvisations on stage; the character does not have a speaking role. I created this particular role for the pianist to not only push the boundaries of live music within contemporary theatre, but to specifically transform the emotional narrative of both the dialogue and the scene into the language of music, either as explicitly instructed in the script during particular scenes and moments of silence between the characters’ dialogue or as the musical background as determined by the pianist. Furthermore, the language and dialogue play with word meanings and rhymes to offer a rich setting for the pianist to improvise (and in stark contrast with Moments). This creative improvisational type of presentation necessitates unique music every performance.

The pianist, who also improvised for Moments, had an average of 18 years working professionally and while had done some musical theatre containing scored orchestral work before, had not improvised live for a theatrical production of this type. I met with the pianist several weeks prior to rehearsals with the actors to discuss his role, the overall storyline, and the nuevo tango (“new tango”) style of music I wanted to be implemented as part of the melodic foundation from which to improvise. The pianist then attended the last two weeks of rehearsals to familiarize himself with the dialogue, stage and musical entrance cues explicitly noted in the text, timing between scenes, and particular elements I designated to be incorporated throughout (i.e. the use of ostinato or the repetition of a particular phrase or motif). A soundtrack, however, was not charted out or scored. The play was presented to the public eight times in the metropolitan Baltimore, MD and Washington, D.C. areas. All performances were photographed and audio and video recorded live. After the performance I interviewed a jazz musician with approximately seven years of professional working experience about the technical musical aspects of the improvised composition. The presented musical data come from one of the performances. My experimental question was: what effects do clearly established realistic linguistic and visual emotional stimuli have on improvised musical content?

3.1

Moments files

Audio 1.

Audio samplesMoments. Representative film stills from each emotion category within the film short Moments and their respective audio excerpts. http://dx.doi.Org/10.1117/12.2083880.l

3.2

Última Partida / The Final Draw (theatre)

Audio 2.

Audio samples_Ultima Partida. Representative photographs from several scenes from the staging of Última Partida and their respective audio excerpts. http://dx.doi.Org/10.1117/12.2083880.2

Combining the data from both projects, several observations can be made. First observation: visual emotional content has a bijective effect on improvised music. For example, a happy scene elicits the creation of happy music, a fearful scene fearful music, a seductive scene seductive music, a tense-filled scene tense music, and so on. Second observation: musical variables such as mode (major or minor), tempo (fast or slow in beats per minute (BPM)), rhythm, pitch range (high or low), and dynamics (loud or soft) are all used in a variety of combinations to create enhanced emotional contexts to the visual and spoken narrative via mimicry of the physiological characteristics of each emotion. In Moments, for example, we see the following as described in Table 1. In Ch. 1: Sadness where the female character receives bad news and begins crying the tempo falls to larghetto, dynamics are at mezzo-piano, and the bass, in particular, mimics the pulse of the woman’s slow, heavy breathing. In Ch. 4: Happiness where the scene follows the two characters during their pleasant and relaxed time together outside, the music starts off in a major key, at a lento tempo, with a bluesy feel, and at about an octave above middle C. As the characters become more playful, the music increases in tempo to an allegro moderato. In Ch. 5: Anger, the scene shows the same two characters in a far more estranged way. Their backs are to the camera, a large empty space sits between them, and their subsequent actions are unpredictable. Here, the music is atonal, tempo is erratic, dissonances arise, and dynamics are uneven. In Última Partida, we see similar effects as described in Table 2. In scene 1 near the beginning of the play when the cabaret setting is introduced and the two characters begin a back and forth yes-and-no game of proposals by the man and rejections by the woman, the music shifts between minor and major tonalities and the tempo liberally speeds up and slows down ever so slightly (rubato) to mimic their push and pull. Then as the couple begins to close in on and seduce each other through their words, a bluesy melody slips in, the tonality is more major, and the tempo stays at a steady andante. In scene 4 when the woman delivers an intimate, passionate monologue about self, the piano enters softly and tenderly with single notes. Then as the couple begins to clash, dissonant semitone clusters persist to jarring effect. Third observation: salient non-emotional scenographic elements within the narrative are also simultaneously musically translated. For example, in Ch. 2: Surprise and Disgust in Moments, the two characters are shown ascending a long narrow seemingly infinite staircase to the rooftop. An overall effect of continuous circular movement is created as the piano ascends up the keyboard with heavy chords and repeating bass notes (pedal tones), the bass descends, and the saxophone repeats his same dissonant melody with slight variations through various keys. In Ch. 3: Fear where a woman’s hand unrelentingly slides up the banister, the pianist mimics with a continuous legato touch. Within Ch. 4: Happiness, water falling in a fountain is present within the visual frame and the improvised music mimics its glistening sound effects with rolling ascending chords. In Última Partida, we also witness a similar effect. Most significantly, in scene 2 where the couple begins to dance flirtatiously, the music enters with a tango. As scene 2 moves forward and they discuss why their relationship failed, the man mentions a “statue of glass” and its fall. The music enters with parallel chords (a sequence of chords containing intervals that do not change as the chord moves up or down), then moves to an even higher register on the keyboard to shattering effect, and then continues with the dissonant parallel chords as the couple’s views begin to clash. In scene 3 when the couple throws flirtatious poetic stanzas back and forth, the pianist reflects the repetitiveness of their actions with the continuous repetition of a short melodic phrase (ostinato) in their right hand and the alternating call and response with a flowing left hand below and above the right hand. Fourth observation: in the case of Moments where a trio improvised the score, the musicians simultaneously engaged in interactive communication between each other that included various common musical exchanges exemplified within jazz such as melodic and rhythmic imitation and motivic development^15,79 as significantly observed in all the chapters, and with each instrument taking the melodic or rhythmic lead at various points.

Table 1.

Chapters in Moments with their respective synopsis and primary distinguishing musical observations.

Table 2.

Significant musical observations with their respective scene, synopsis, and text (translated from the original Spanish).