Motivation
Within the last couple of decades, audio production has moved from analog towards digital due to several reasons. The same applies to orchestral film scoring. Digital film scoring, though, can sound unnatural quickly (even when professionally done). On the other hand, not every film composer’s budget allows an orchestral production through and through. We first will take a look at a typical orchestra before we will discuss some of the aspects defining an orchestral sound and some key elements to achieve a maximum of realism in digital orchestral productions.
Orchestra
A classical orchestra is defined as a set of musicians consisting of multiple instrumentalists. In a typical orchestra, a musical voice is being played by multiple instruments simultaneously. It can be divided into two basic categories defined by their size: Chamber orchestra (usually 12 to 20 musicians) and a symphony orchestra (usually 150-200 musicians). (Kah n.d.)
Function & organization
A common orchestra consists of 4 different instrument families: the strings, the woodwinds, the brass and the percussion. The 4 most commonly used string instruments are the violin (the highest-pitched string instrument), the viola (tuned like a violin but the fourth string is tuned down a fifth), the cello (only an octave lower than the viola), and the double bass (an octave below the cello). Oftentimes a harp may be added to the string section of an orchestra. The first and second violins play two different parts and are being supported by the viola’s harmony. The cello plays a supportive role in the harmony, even though it can go higher into the tenor voice at other times.
There are 3 ways in which sound is generated in the woodwind instruments: “by blowing air across the edge of or into the mouthpiece (flute or piccolo), by blowing air between a single reed and a fixed surface (clarinet and bass clarinet), or by blowing air between two reeds (oboe, English horn, bassoon, and contrabassoon). Like the violin, the flute often carries the main melody due to its pitch. The piccolo acts as a short violin, even if it’s even higher pitched. The oboe is expected to carry the melody as the flutes are. Though, they have a more mellow sound the English horn is similar to the oboe but pitched a fifth below the oboe. The clarinet and its lower relative are very versatile instruments with a range of 4 octaves. Its tone quality may vary depending on the articulation, the musician, the mouthpiece and the reed. Both bassoons mark the lowest woodwind instruments in an orchestra. As a gap between the woodwinds and the brass, but still classified as a woodwind, acts the saxophone. Its versatility exceeds the other woodwinds instruments. Though it’s not very common in an orchestra, there are some orchestral pieces composed for saxophone.
The last instrumental family, and also the biggest one, is the percussion section. “Percussion instruments are an international family, representing musical styles from many different cultures” (Instruments of the Orchestra n.d.). Common percussion instruments in an orchestra are the timpani, the bass drum, the snare drum, gongs and other cymbals. The piano is widely considered a percussion instrument as well, because of the way sound is being produced: the player presses a key, which causes a damped hammer to strike on the strings.
Hall resonances & instrument localization
The localization of each instrument and the resonating hall plays an enormous influence on the overall sound of an orchestra. Instrument localization rules have changed over time, but one rule seems to persist over time: the first violas are always far left from the conductor’s view of an orchestra. Architects spend much time focusing on the acoustics when designing an orchestral hall, more than on other concert halls due to its impact on the sound experience of the audience. Samples from different sample libraries may include different reverberations. Reverbs as a plug-in may help here by modeling different halls and soundscapes, but their algorithm is no substitute to the real hall.
One reason for that is instruments have different sound radiation patterns. Omnidirectional sound radiation can only be found in the low frequencies below 500Hz. The higher frequency content has different radiation patterns depending on the instrument: brass instruments are focused at a narrow-angle around the instrument’s axis, the flute’s radiation can be described as a dipole radiating from the embouchure and the first side hole. The string instrument’s radiation pattern dependency on the frequency is more particularly significant. It depends on the phase distribution of the vibrating parts – especially the belly. In placing an instrument in the virtual orchestra, the ratio between direct sound and reflected sound needs to be considered as well. For example, the trombone displays very narrow sound radiation towards the front, and therefore the audience exhibits a greater direct sound than reflected sound. Whereas the french horn for example which directs its sound mostly towards the back of the stage the audience exhibits a greater reflected sound than direct sound in the same acoustic space.
Digital production
Since most movies do not have a budget for a whole orchestral production and music production moving into the digital realm within the last decades, there have been several attempts to simulate an orchestra digitally. Actual orchestral instruments are recorded in a specific setting and are collected into a so-called sample library. Using a sequencer, a composer can access these specific sounds using MIDI.
Libraries and notation software
Individual orchestral instruments are being sampled in various settings to simulate a real orchestra. Two of the most common available libraries I focus on here as an example are the Vienna Symphonic Library (VSL) and the EastWest Symphonic Orchestra. The EastWest Symphonic Orchestra is a high-quality sample pack, recorded in the East-West Studios in Hollywood, California. Each instrument was recorded separately using 3 microphone positions: “close, stage (conductor’s place) and hall” (East West Symphony Library, n.d.). Each instrument was recorded in a wide range of articulations and dynamics, so the composer can trigger those via MIDI easily. The VSL offers different sample packs of each section of the Vienna Orchestra. The Vienna Orchestra is one of the world’s most famous orchestras: Vienna is considered the capital of classical music. Nearly all sampled instruments are recorded at the highest input level in to get the cleanest sound, which needs to be considered in the sequencing.
Nowadays, there are plenty of notation software. With Avid’s Sibelius as an example, the composer can edit any articulation of each instrument while scoring. One of the key benefits is that he can integrate any sample library into Sibelius with instant playback. This changes the way how the composer works with his orchestral piece: he can now directly review his musical choices within the software while still composing. Sibelius is designed to “playback a score with a reasonably accurate interpretation of the notated music” (Sundstrup 2009: 14). However, there is more to the real orchestral experience than just the notation, which we will see in chapter 4. To edit the performance expressions, the written score is being exported into a sequencer.
A short review of MIDI
The core of digital music production is MIDI, which is an acronym for Musical Instrument Digital Interface. It is a communication protocol in binary between electronic instruments, computers and related audio devices for playing, editing and recording music. MIDI contains information for example about pitch, time and duration and velocity of every single note. It also widely used within most sequencing applications, recording software and notation programs.
Sequencing
When talking about MIDI, the concept of sequencing comes up a lot. Sequencing describes the programming of notes and sounds to play in melodic and rhythmic patterns and musical phrases, which are then arranged and/or performed using a machine for playback instead of real musicians playing these notes live. They can be programmed to play in a certain articulation and effects can be added as well. By using sample libraries or synthesizers, one can play almost any instrument one can imagine with MIDI. This gives digital production a huge advantage compared to real live recording. The producer can thus program any musical setting of choice. Nevertheless, you need to consider different aspects of sequencing to make the performance human and natural sounding. Some of those aspects will be discussed in the following paragraphs.
Key aspects of virtual orchestration
“Interpretation is one of the most important aspects of music performance”. (Fridberg n.d.). The human performance is characterized by the result of deviations of the mathematically perfect interpretation of notated music. Those deviations can be intentionally or unintentionally, though, not all sophisticated performance rules can be modeled by software. We further discuss some of these performance rules. These rules categorized into two classes: expressive human performance (e.g. intonation, dynamics, timing,...) and instrument technique (articulation, timbre, sample repetition, performance transitions,…).
Ensemble timing
The timing of an entire ensemble has two influences, which we need to focus on for a convincing virtual orchestral performance: the first one is based on acoustics and the speed of sound, the second one on each musician’s individual timing and musical expression. The acoustical situation of each musician varies from the listener’s. For example, the delay time of the direct sound between the instruments at the conductor’s position may rise to 35 ms. Between the outer musicians of an orchestra, it rises to 45 ms. In a study where an ensemble was recorded at the Royal Guard of Oman Music Studio Hall at different tempi, researchers found a correlation between the deviation of the ensemble timing and the tempi: The faster the tempo, the smaller the deviation and vice versa. Adjusting the timing considering the notated note length instead of the note duration (in ms), this effect of the ensemble timing can be achieved within a notation software such as Sibelius. Whereas a software performance is perfectly on time, a human performance contains some deviations in time. This deviation is crucial for the human feel and expression in the music. It also creates a random element of timing.
Intonation
One of the key elements that makes an orchestra sound huge and full of texture is the variety in pitches of each played note. Unlike in the digital realm, where everything is perfectly in tune, in an orchestral environment, each instrument plays a slightly different pitch of a given note. Detuning each section by some cents in an orchestral simulation can have a profound effect. This can be done to the instruments individually or the individual sections, depending on the way the samples were recorded. If each section was recorded as a whole unit, detuning the solo instruments will not be necessary, since it already contains the natural variability between different players. In that setting, detuning each section of an orchestra by a set parameter is recommended.
Articulation
The articulation of a note depends on the musical situation. Articulation in general describes the ratio of staccato and legato each note has. Each is defined as a ratio between the note’s duration and the time from one note’s decay to the next note’s attack. In the virtual orchestra, each articulation is being recorded in a specific setting. But before orchestras got sampled in that way as the current libraries are, even the most common articulations – like staccato, marcato, accent and tenuto - had to be simulated by changing each note’s ADSR (attack, decay, sustain, release). The benefit of recording samples in all various articulations – including the dynamic levels – and playing (triggering) each note with a MIDI controller in quality “can be astonishing” (Pejrolo and DeRosa 2017: 120). Before that was common practice, the synthesizer had to artificially re-create complex acoustic waveforms using filters and oscillators. Even though the quality of the sound libraries increased tremendously, however, musical articulations are not static. Unintentional variations within each performed articulation occur, which needs to be considered for realism.
Velocity & timbre
Velocity and Timbre are directly correlated in an orchestral realm: the louder an instrument plays, the higher its timbre is (more high-frequency content). A perfect example gives the brass section. At low dynamics the brass achieves the warmest and smoothest sound of an orchestra, however, at a high dynamic levels, it achieves the loudest and brightest sound. In an orchestral simulation, you can automate filters to get that relation between both. Nevertheless, the current libraries record at different sound levels, so they can capture that effect in the recording process. In reality, though, changes in timbre according to the sound level are happening continuously and not intermittent. Thus, it requires the use of a technique called cross-fading. In cross-fading, multiple samples are blended to achieve more realness. It’s mostly used at transitions, where the velocity changes. Without cross-fading and filters, playing louder would equal to a change in volume only, not the timbre. As in anything that is played by humans, there are micro fluctuations of velocity and timbre during an orchestral performance, which give a random character to it. They can be programmed into MIDI on a random level to simulate those little imperfections.
Dynamic pitch
As discussed above, the note’s dynamic and its pitch are directly correlated. As an addition to that, the velocity range of a single instrument is crucial for the dynamic balance and the appropriate instrumental timbre. In his research, Meyer found that “In general strings are slightly softer than the woodwinds, and these are softer by about 10dB than brass instruments.” (Meyer n.d.: 203). This gives us just an overview of the different dynamic ranges in an orchestra. The dynamic range of each instrument varies in different pitch registers. A good example to demonstrate this effect is the flute: its dynamic range in its higher register has a much greater sound intensity level than in its lower register. Also, a dynamic of fortissimo in its lower register is softer than a pianissimo in its higher register. By using only MIDI messages, one cannot achieve this effect. This is because MIDI only allocates the general level of volume between 0 and 127 leading to triggering incorrect timbre patches of a given velocity. For example, the composer wants to play pianissimo, which is programmed to playback in the higher register only, but triggers a mezzo forte or forte sample. MIDI is carrying non-pitch specific velocities.
Note repetition
When repeating the same note several times in a virtual orchestral score, an artificial “machine gun” effect can occur. This is caused by the fact that the samples are played in the same pitch and dynamic level. It does not include the naturally occurring deviations in playing. To create more realism in the virtual orchestra, these have to be programmed specifically by adjusting each note’s dynamic, level, timbre and duration by a small amount.
Conclusion
All in all, an orchestral mock-up (digital orchestra) has some huge financial benefits compared to the real one. But even for a highly skilled audio engineer, achieving a real orchestral sound digitally is a huge challenge, and not manageable without certain tradeoffs. Music in general is a language of communication: in an orchestral setting, the musicians are reacting to one another musically. An aspect that is missing in a virtual orchestra. Thus, in big productions with a budget allowing an orchestral performance, one is highly appreciated. Even then, both virtual and real orchestra may be blended into a final film score.
Bibliography
- Albrecht, G., Osorovitz, C., Fellinger, B. (2010) Musikinstrumente und wie man sie spielt: ein Führer durch das Orchester für kleine und große Leute. Nachdr. Zürich: Atlantis-Musikbuch-Verl.
- Concert Hall - Venue Hire - Sydney Opera House (n.d.) [online] Available at https://www.sydneyoperahouse.com/general/venue-hire/concert-hall-venue-hire.html (Accessed 14.08.2020)
- Davis, R. (2010) Complete Guide to Film Scoring: The Art and Business of Writing Music for Movies and TV. Hal Leonard Corporation.
- Friberg, A., Bresin, R., Sundberg, J. (2006) ‘Overview of the KTH rule system for musical performance’ In: Advances in Cognitive Psychology 2, doi: 10.2478/v10053-008-0052-x
- Fridberg, A. (n.d.) ‘A Quantitative Rule System for Musical Performance’ [online] Available at http://www.speech.kth.se/music/publications/thesisaf/sammfa2nd.htm (Accessed 13.08.2020)
- Gilreath, P. (2010) The Guide to MIDI Orchestration 4e. 1 edition. Amsterdam; Boston: Focal Press.
- How To Use a Sequencer To Make Music (2019) Andertons Blog
- https://www.vsl.co.at/en/bbo_map (n.d.) Vienna Symphony Orchestra Sound Library
- http://www.soundsonline.com/symphonic-orchestra (n.d.) East West Symphony Orchester
- Instruments of the Orchestra (n.d.) Arapahoe Philharmonic [online] Available at https://www.arapahoe-phil.org/learn/resources-instruments-orchestra/ (Accessed 13.08.2020)
- Kah, R. (n.d.) ‘Orchesteraufbau - Instrumente und Sitzordnung in einem Orchester’
- Kompanek, S. (2011) From Score to Screen: Sequencers, Scores, & Second Thoughts the New Film Scoring Process. Schirmer Trade Books.
- London Symphony Orchestra: They started recording film scores (n.d.) Classic FM [online] Available at https://www.classicfm.com/discover-music/latest/worlds-greatest-orchestras-pictures/star-wars-recording-john-williams-lso/ (Accessed 13.08.2020)
- Meyer, J. (n.d.) ‘Jürgen Meyer: “The sound of orchestras and soloists as a combination of room acoustics and acoustical properties of the musical instruments” — CIRMMT’ [online] Available at https://www.cirmmt.org/activities/distinguished-lectures/meyer (Accessed 13.08.2020)
- Orchestra (n.d.) DK Find Out! [online] Available at https://www.dkfindout.com/us/music-art-and-literature/musical-instruments/orchestra/ (Accessed 13.08.2020)
- Pejrolo, A., DeRosa, R. (2017) Acoustic and MIDI orchestration for the contemporary composer: a practical guide to writing and sequencing for the studio orchestra. Second edition. New York; London: Routledge.
- Raines, R. (2015) Composition in the Digital World: Conversations with 21st Century American Composers. Oxford University Press.
- Sundstrup, L. (2009) ‘The virtual orchestra: a systematic method of realising music composition through sample-based orchestral simulation’ In: University of Wollongong Thesis Collection 1954-2016
- Vienna’s musical heritage - Mozart, Strauss, Haydn and Schubert (n.d.) Music of Vienna [online] Available at https://www.musicofvienna.com/musical-heritage.htm (Accessed 13.08.2020)