We’re joined today for our podcast with electroacoustic expert Steve Ellison. He’s currently applications director of digital products at Meyer Sound. For many of you, that may make you think of the Constellation System. He’s going to talk a little bit about that, and also some general principals of immersive sound. It’s a term that gets bandied about a lot and we’re going to try to bring some focus to it today. Something we’re definitely seeing more of in the marketplace and we’re seeing it at all levels, budgets and applications so I want you guys to be a little more familiar with it. Just quickly before we start I’m going to turn it over to Steve. He’s going to remind you all a little bit about his background and then we’re going to get into this topic.
Thank you, Cynthia. I’m Steve and happy to be here on this podcast. I’ve been involved with multichannel sound and acoustics for a long time now. And some of my early work that I got involved with was in the mid to late 80’s with multichannel sound and developing new techniques for panning sound in multichannel spaces and systems. And that work kind of led to space map, which was an algorithm that is part of our digital audio series Dmitri, but was first commercialized with a company that I co-founded, Level Control Systems, back in 1992. So I started off based mostly in theatrical sound and theatrical sound automation, and then helped bring along the variable room acoustic system, or VRAS technology, into acoustic systems that LCS worked with and then Meyer Sound systematized as Constellation Acoustic System, which we’ve now been shipping and installing, tuning, etc. for the last 10 years. So the last decade or so I’ve been focused heavily on Constellation and adjustable acoustics, but all along I’ve been interested in surround sound and immersive sound.
What is it about the humans in the space? What do acoustics and movement of sound through a space – what difference does it make?
One of the senses, our sense of hearing, is something that we connect deeply to on a very intuitive level that can move us in so many ways. And music moves us in so many ways. And so in a sense we can think of movies and sound as being part of this emotional transportation system that can transport us from one place to another emotionally or in our mind imaginatively. And sound can help convey that. Here’s an example. Historically, opera houses have created alternate realities in the sense of this artistic musical piece with actors and lighting and costumes and music. And this goes back years and years. There’s a venue I had the pleasure of visiting at an acoustic conference a couple of years ago in Argentina’s Teatro Colón, which opened in 1908. And one of the features was – I don’t know the architectural term for it, but above the main seating at the roof, the center of the interior, there’s a circular catwalk which cannot be seen from below. And in the early productions they would put a boys’ choir up in that area above the audience so they could sing, and so they were like the sound of angels above the audience. Now that must have been really something in 1908 to experience. Now we’re able to create those experiences without putting a boys’ choir above us in the venue. We can put loudspeakers and put sound there to achieve some of those same emotional impact and effects. And really, theatrical sound designers, we have to give a big tip of the hat to them because they have been pushing the envelope for years and using sound to create these emotional experiences both by placing sounds not just in front of you, but above you, behind you. I mean, I remember one of the very first LCS or Level Control Systems control systems. The system that went out, it was George Lucas’ Super Live Adventure that toured arenas in Japan. And one of the effects that was used was the sound of Darth Vader breathing in the surrounds behind the audience. And that was a really effective moment back in 1993. And since then we have more and more techniques to play back sound and to move sound. And so the tools are there. It’s really the sound designers that have been driving that, such as Jonathan Deans who did that show and also a number of the Cirque shows in Las Vegas, pushing the envelope of what sound can do. So we as manufacturers, like Meyer Sound, we’re creating tools that they take and then apply as tools in our toolbox, but they are the sound artists, the sound designers.
Right. So I hear you saying the way that artists have really driven this on the content side and the experiences they’ve created. And then there’s another side to this. There’s the room itself and the atmosphere and emotion of the room. That sounds like it’s something we can actually get creative with and control in ways that we couldn’t before.
Absolutely. And a good example from this also comes from the world of theater and the Cirque de Soleil show O, which is in Las Vegas. And there’s a huge pool and there’s amazing artistry of the performers diving into this pool. And the music is beautiful and it’s a beautiful show. And it was going great on its opening. It was really going well except that there was some concern that the audiences weren’t responding as well as they might like for the parts of the show when the clowns would come out and interact. And this is a common element in Cirque shows where the clowns will come out and goof around with the audience and people laugh. It’s a fun way of engaging the audience in the action and in the show. But in O, early on there wasn’t quite the response that the director had hoped for and so there was talk of changing the room acoustics. Now the room acoustics were designed to be low reverberance so that the PA could be high-powered and have good impact without the architecture smearing the sound too much; which was great for the high-powered parts. But in the interaction with the audience what would happen would be small groups of people would clap and others wouldn’t clap and there didn’t seem to be this audience dynamic like they were in a room together enjoying this event together as much as they might have liked. So on the one hand the thought was to change the room acoustics physically – architecturally – by hardening up the walls, removing some of the drapes and so forth. But instead, after a demonstration was made, they added the variable room acoustic system with LCS technology that became Constellation to electronically add reverberance from the entire room so that when the clowns would come out the sound operator could press a button and the room all of a sudden was more reverberant and allowed for more interaction between the audience so they could hear each other better. So that kind of use of active acoustics for audience engagement, if you will, is a good example of how the room was able to adjust – or they were able to adjust the room’s acoustics in order to suit the needs of the show at that time. While the VRAS technology made its way into Cirque de Soleil initially for the function of audience enhancement of the O show, subsequently it was used for its creative potential. And in shows like Love and KÀ in Las Vegas, active acoustics of Constellation are used in conjunction with space map panning through all the multiple loudspeaker systems installed in the venue. So it provides a rich palette for the sound designer to both change and adapt the ambiance of the space acoustically, but also the generated immersive sound field from sounds that are played back all around you, above you, and even behind your ears and seat speakers. But where active acoustics technology has really found its home is in situations where one room wants to be a chameleon and serves all kinds of functions from maybe drama on one night to chamber music another night to jazz another night or choral music. One room and the ideal acoustic for one of those functions wouldn’t be ideal for another. So the same acoustic you want for a lecture is not the acoustic you want for a choral performance. To look at the extremes, for a choral performance you’d like it to be typically more reverberant and for spoken word typically you want it to be less reverberant so you can have better intelligibility. So in that sense active acoustics is used to adjust the room to suit the function. It’s a utilitarian function and it’s found its home in places like civic auditoriums, high school and university auditoriums such as Palo Also High School where the system provides adjustable acoustics for everything from spoken word to orchestra to choir to drama to a capella singers who are amplified. So by having this adjustable system in the room the venue can be fine-tuned to support the artists or the lecturers in the way that best suits that content. And so because of that you get a better, more engaging performance from those performing and you also, as an audience, you experience it in a whole way where the acoustic is correct and supportive of the event rather than something that you’re fighting against.
Steve, just for those people who are maybe not familiar, can you just very briefly explain what we mean when we talk about variable acoustics and electroacoustics?
When we’re out in the world we experience sound in all directions all around us. Whether we’re in a natural setting in nature in a park, in a forest, on a street, or in architecture in a room, in a performance. Immersive sound is part of what we grow up with and what our experience is as human. So it’s only natural to want to create auditory experiences with sound, with electroacoustic technology that also are immersive and reflect what we experience in the world. There’s a couple of aspects of immersive sound that I want to point out. First of all, room acoustics are one aspect of immersive sound that we experience and it changes from room to room. For instance, in a cathedral with a pipe organ or a choir we can be immersed in that rich reverberation that’s all around us or perhaps trumpets behind us up in a balcony doing antiphonal music. That’s an immersive audio experience in architecture. On the other hand, we have venues such as movie theaters, cinemas, that now are providing immersive audio experiences with surround sound and the acoustics in those rooms are very low reverberation time because that immersive experience is being delivered by the film. So in terms of these systems and how we hear sound, it’s dependent on a large part on physical architecture of the room. And what we’re hearing are both the direct sound from sound sources, but also the reflections of that acoustic environment. And so Constellation, which is an active acoustic system, is an example of an immersive technology that is used to adjust room acoustics electronically. So when I say active acoustics as opposed to passive acoustics, active acoustics are systems that use microphones, signal processing and loudspeakers distributed around and above the space in order to add acoustic energy to a room in order to create a new acoustic signature that is more appropriate for an event. That is a very important immersive technology that had its start in the – really in the 50’s and early 60’s and it’s been developed for many decades by many individuals and companies. So Constellation is one of these tools that we provide for adjusting acoustics and creating immersive acoustic experiences. Now hand-in-hand with the architectural sound of immersion and the immersion of room acoustics, there’s also that program audio that directs sound field that in many cases, if we go to a concert, it’s just a stereo experience, if that. Often it’s mono. We just have main sound coming from in front of us. Cinemas, on the other hand, will be surrounded by sound around us and some formats, like Atmos, provide sound overhead as well. So there’s a balance here between the physical architecture and then the system and what the system is providing in terms of this immersive sound field. So I’d say that the immersive sound is both acoustics as well as a sound field production of sound sources or virtual or amplified sound sources around you. And there are many different ways of distributing the sound to create this environment. One of the technologies earlier is Space Map. And Space Map is a spatial sound distribution algorithm that’s built into the Meyer Sound digital product, Dmitri. This allows the sound designer to place an arbitrary layout of loudspeakers and create a smooth panner to move sound around the space with all sorts of different properties. And where this kind of found its footing back in the early 90’s was in theatrical sound, both in theaters such as repertory theater like South Coast Repertory here in Orange County in Southern California and also Cirque de Soleil shows such as Mystère in Las Vegas was one of the early venues that used this technique. And so it provided a sound designer with a way to create a system that is architecturally driven – like where can you put loudspeakers – as well as functionally driven by what you want to achieve with sound movement and the sound placement. So it’s a system that allows the sound designer to dictate the characteristics of the panning based on the architecture and where loudspeakers can be placed. So it’s a very customizable system for that function. To give you an idea of the breadth of application for this immersive technology, we have an interesting example that was brought to us by Threshold Acoustics in Chicago for Northwestern’s auditory research lab there in Chicago. They wanted to create a room in which they could create any sonic environment and let participants, for instance, try their new assisted listening devices under different acoustic environments, sound environments, to see how they’re working or to train them on how to use them. So they created this room with Constellation which had 16 microphones and on the order of 30 loudspeakers around and above them, and signal processing to create any sort of acoustic. In addition it had our Wild Tracks playback system with Space Map so they could create immersive audio environments with sound effects such as the clang of the dishes or tinkling of forks or conversation in order to create a noisy restaurant, for example, or create a classroom environment or the inside of a car. So they could set the auditory environment to be any of these conditions and the participants could try their new devices. For instance, one of their clients was complaining about the assisted listening device and didn’t feel like it was working properly in a restaurant. And by going into this room and setting up this noisy atmosphere of a restaurant, in the clinic they were able to demonstrate the features of that device and how the client could use their cell phone as a remote microphone for their assisted listening devices and improve intelligibility across tables. Things like that. So there’s a clinical application of immersive sound as a benefit that we wouldn’t have thought of when we were conceiving of all this technology.
How do you see the venue of the future, and does it really apply across a range of applications and budgets as we go forward?
Sound Box in San Francisco is a great example of the venue of the future that is happening today. And this is a repurposed rehearsal space. The San Francisco Opera has rehearsed in this room for many years. And through the application of Constellation acoustic system as well as lighting, video screens and video production, custom craft cocktails, etc., the San Francisco Symphony has created a new sort of new music venue that enabled them to showcase music from all sorts of eras. From early music to music by composers who are still living across a wide range of styles. Adjust the acoustics and in addition use the same loudspeakers that are providing all the acoustic energy for sound playback. So they, for instance, did a piece of Steve Ritea’s where there’s a marimba player performing live in a beautiful, warm, enveloping acoustic and at the same time playing along with prerecorded marimba parts above you in rows of speakers above your heads. It was a really compelling piece that was extremely engaging. And what’s been fun to see is how some of those concepts that have been first explored by companies such as Cirque de Soleil and San Francisco Symphony, some of those elements make it to lower-cost productions. And a good example of this would be a production of Steve Ritea, again. His music Different Trains, which is performed by string quartet playing simultaneously with a prerecorded string quartet as well as some recordings of trains that are from historical recordings. So it’s a very moving piece – literally. I mean, there’s the sounds of these trains moving and so forth. You get the sense of motion and the driving of the music. It’s a beautiful piece. And so Brendan Speltz co-produced with ABC Cirque in Brooklyn at the Muse, a venue in Brooklyn, a version of this with his string quartet along with aerial artists in this venue in Brooklyn. They’re in costume and there’s multiple loudspeakers and there are two rooms in this kind of immersive theater experience that was a creative way of using the space and using the music and musicians and the choreographed aerial artists who are hanging on these ropes and tumbling in these wheels to kind of convey the sense of some of the music and the themes behind different trains. And that was an incredibly moving experience for me to appreciate. And once again it was kind of this marriage between music, architecture and sound design. Behind this was David Bullard, who on the other spectrum of budgets did amazing sound design work for War of the Worlds at Disney Hall a couple of months ago that was maybe the most immersive live performance that I’ve ever experienced. Taking some of those elements and doing it in a small-purposed space in a warehouse in Brooklyn, to me that’s amazing. That’s where the future is and seeing the principals that are applied on a big grand scale of Sound Box, Cirque de Soleil, Los Angeles Philharmonic, Disney Hall, those could be applied in smaller venues. The concept of using multiple speakers, I mean even in this piece in Brooklyn there’s maybe 10 loudspeakers total on stands in different positions. But just the positioning, the sound design, the orchestration, the way the venue was designed, lit, etc., it was a true synchronicity of all of the theatrical design elements applied to live music.
Okay. Can you just briefly give us a broad picture of how these active acoustics work, technically speaking?
Active acoustics is a technique for taking the acoustic of one room and changing it instantly with the press of a button. So taking a room that sounds like a theater with low reverberation time. You press a button and you’re instantly adding acoustic energy that’s coming from the room itself, from all around you, the way physical architecture would work. And different systems approach this in different ways. Constellation approaches this as a whole room challenge where we want sound sources wherever you are in the room to activate the acoustics the way physical architecture does. So because of that there are microphones throughout a room, not just on stage. And there’s also loudspeakers all around you and above you. So by pressing a button one moment you can be in a room that’s great for speech and is adding clarity and early reflections to reinforce the spoken word in all directions. Press another room and all of a sudden you’re in this beautiful, immersive, enveloping, reverberant environment that’s better for a symphony or a chorus. One of the very first installations of Constellation was done a little over 10 years ago; I can say exemplifies the most common use case for Constellation and for active acoustics in general. This was a gala performance celebrating Cal Performances and in this one performance there was symphony, dance, choral music, and including a finale on which the singers went out into the hall itself and were singing while the symphony was playing. It was an amazing event that had a need for quick changes on stage, changes in the acoustics for various music pieces. So we hit the ground running with Constellation. Subsequently, a permanent system was installed.
So can you talk a little bit about the challenges in bringing this to live sound?
There’s several challenges in bringing this to a live sound environment. Some of those are architectural in that if, say, a concert is on tour one venue may not be exactly like another venue. And so setting up a system in a venue in a short time frame, that’s a challenge. Productions like Cirque that have weeks or months of production time and permanent venues like the ones I mentioned – Northwestern or Sound Box or Palo Alto High School – those are permanent systems that are set up and left. Live sound typically is more temporary. So there’s one challenge is the architecture, the time to load in, set up, calibrate a system. Another is who is making those artistic decisions? Is it the artist for how a sound is spatially distributed? In live theater, the sound designer is part of that creative team that are working with composer, arranger, director to determine how sound should be placed. Where should it be placed? When should it be placed? And they’re typically planned out and it’s a sequential event. So there’s a cue list and if they’re going to script as the action changes in the theater, the sound effect is triggered and moves overhead, etc. With live sound it’s not like that. There may be some elements of that with a show that’s developed and they have sequence of songs or tunes that have a certain type of spatial movement, but often it can be more on the fly. So I’d say the improvisational nature of a live show can be a challenge. And then the direction of who’s making those artistic decisions? Is it the artist or is it the front of house mixer? Is the front of house mixer now thought of like that sound designer who has some say as to the creative decisions about moving the sound? So we have a great opportunity this May at Moogfest in Durham, North Carolina, where we’re teaming up with Moogfest and Virginia Tech to create an immersive sound venue in a venue called the Armory. And this will have a main/left/right array as well as distributed surrounds and overhead position as well. So we’re looking forward to this as an opportunity to work with artists, some of whom are engaging Virginia Tech and Meyer Sound in advance to plan out what they want to do. And some of them are going to be basically showing up as a stereo set or EDM artist or DJ doing music live in this multichannel venue. So I would say those are the challenges. Now set up time, calibration time, the amount of time to prepare, how much improvisation is part of the performance that’s kind of spatial sound mixing as an improvisational element, and then who is controlling that and who’s guiding that? Is it the artist on stage? Is it the sound mixer at the front of house position? How do they communicate? So there’s a lot of challenges to pulling it off and we’re looking forward to Moogfest as being a great opportunity to take some of the principals and concepts that we’ve developed in live theater and live sound and bring them to this very creative, experimental music venue.
Thanks very much for being here, Steve. And next time we’ll talk to the consultant who specified the Constellation system for Palo Alto High School. Join us then. Thanks.
