Shootouts With Meaning: Here's How

Everyone wants to know which loudspeaker is best for their applications. What could be more natural than to line several models up and listen to them? This is called a shootout. 10/23/2007 6:45 AM Eastern

Shootouts With Meaning: Here's How

Everyone wants to know which loudspeaker is best for their applications. What could be more natural than to line several models up and listen to them? This is called a shootout.

EVERYONE wants to know which loudspeaker is best for their applications. What could be more natural than to line several models up and listen to them? This is called a shootout.

The idea is a sound one: line them up and have a listen. I always have advocated that the only way to account for all of the variables in the way that a loudspeaker sounds is to listen to it. You can't have a shootout with spec sheets. In fact, this is the very reason that spec sheets should be approached with a bit of skepticism. It's not that manufacturers are being untruthful, but there are many ways to come up with specifications such as power rating or sensitivity rating, and marketing encourages the use of methods that produce the biggest numbers.

Logistical Problems

The problem with shootouts is not with the concept, but with the execution. Werner Heisenberg is famous for formalizing the idea that you cannot observe a system without disturbing it. Nowhere is this more true than when listening to loudspeakers. The response — and therefore the sound — of a loudspeaker is profoundly affected by its immediate surroundings. The same is true for the listener. A fair comparison would require that each loudspeaker be located at the same position (not possible without a time interval to change loudspeakers) and listened to from exactly the same position in space. This negates the idea of a listener group, because the presence of the others affects the outcome, and all the listeners can't be at the same place.

Loudspeakers interact with their surroundings in ways that change their sound. If you line up a group of loudspeakers in a row, the one in the center will benefit from the acoustic loading provided by the adjacent boxes. The ones on the ends will not. This is why some sound system designers put “wings” on their low-frequency systems. A row of loudspeakers means that each one has different wings, and this alone will affect significantly the low-frequency performance.

Human Factors

While the human auditory system is a wonderful analyzer, it is not calibrated or consistent. You know what you are hearing, but you don't know why you are hearing it. Subjective judgments such as “Brand A sounds warmer,” can be describing a higher distortion level. A conclusion such as “Brand C sounds more open,” can stem from the acoustic comb filter produced by its interaction with the floor. The conclusion “Brand D has more kick,” can come from a mistuned cabinet that produces a low-frequency response peak.

A meaningful shootout requires measured data, and I don't just mean a real-time analyzer running with a mic connected. The complete transfer function of a loudspeaker must be assessed with complex data that includes both magnitude and phase information. The immediate environment around the loudspeaker and test microphone profoundly affect such data. You can't have a shootout in a test chamber.

Humans can only remember what something sounds like for a short time. If you listen to one box, then take it down and replace it with another, the time lapse in between ruins the comparison. If you set up both loudspeakers and switch between them, you have affected the result by listening to sound coming from different positions in space with different surroundings.

You can't do it with your ears alone, no matter how good you think your ears are. It is not physically possible to listen simultaneously to a half-dozen loudspeakers located at the same point in 3D space from multiple listening positions. Yet this would be the only way to do a truly meaningful comparison.

Vision Factor

Even the way that a loudspeaker looks can affect the outcome. A techie trap box with a metal grill just looks like it should sound better than a white box with grill cloth. If you don't think that looks affect the sale, wander into the car stereo department of the local Circuit City. An objective shootout would require invisible loudspeakers, or at least a scrim that prevents the listeners from seeing them.

So to do it right, you have to be able to move each loudspeaker to the same point in space, make it invisible to the observer, and instantly switch to the next model. This has been done for small loudspeakers in a small listening room. JBL invested hundreds of thousands of dollars into such a setup to evaluate studio monitors. Imagine what it would take to do it in a full-size auditorium.

What About Coverage?

Any loudspeaker in a sound reinforcement system will be covering an area. A loudspeaker may be perceived as being superior to another when listened to on-axis, but the outcome may be the different if evaluated from several different positions.

Loudspeakers will have differences in their “working distance” based on their directivity. It can be meaningless to compare different directivities at the same distance or at a distance that is too far or too near for either device. A studio monitor will always sound better than a large format touring box in a dead environment. But, move them both into a large reverberant space and the touring box will win every time.

Which One Goes the Loudest?

If you run both loudspeakers up to 85dBA slow-reading, how do you know how hot each is getting, and how reliable it will be over time? You don't. The one that sounds “best” to you may be near failure. You had better include a laser thermometer in your demo kit to see how hot each is running, and that can only be done with a controlled program source like pink noise. The only way to determine the maximum SPL available from each is to take it to the threshold of failure, and doing so will likely permanently alter the response of the loudspeaker.

Evaluating loudspeakers is a lot like evaluating an HVAC or lighting system. The only way you can really do it is to install it and then evaluate its performance in all parts of the room. Little could be determined from a temporary setup, yet that is the only practical way to evaluate a loudspeaker.

If the design requires an array of the loudspeakers, listening to one box will reveal precious little about the performance of the array. When you cluster loudspeakers together you are forming a new loudspeaker. Its radiation pattern will be grossly modified from the single box. Some loudspeakers are more arrayable than others, but you would have to conduct a shootout between various arrays to determine this.

1 2 Next

Shootouts With Meaning: Here's How

Everyone wants to know which loudspeaker is best for their applications. What could be more natural than to line several models up and listen to them? This is called a shootout.

A Meaningful Shootout

What can you do? Here's a method that combines the measurement and listening processes to truly reveal the differences between your loudspeaker candidates. This process can be combined with a conventional shootout to identify the most appropriate loudspeaker for your application.

  • Reserve some time in your listening space when the room will be empty and quiet. It is imperative that this be the venue where the loudspeaker will actually be used because the acoustic environment will have a profound impact on the results. A high directivity loudspeaker may not sound as “musical” at a close distance or in a dead room as a lower directivity design. But take both of them into a reverberant space, and the higher Q device will produce superior speech intelligibility and music clarity.
  • Limit the attendance to yourself and a few helpers. Lots of people milling around will affect the results.
  • Select an audience area for the test, ideally away from room boundaries. It should be deep enough to evaluate the longest expected throw distance for the design.
  • Place the device-under-test onstage at ear height. Better yet, a lift can be used to elevate it. Aim it down the center line of the audience area and at the back row.
  • Move to the back row and measure the distance to the device-under-test. This is the third test position.
  • Cut this distance in half and mark it. This is the second test position.
  • Cut this distance in half and mark it. This is the first test position.
  • Select three additional test positions at these same distances, but along the left or right edge of the seating area.
  • Configure the device-under-test with the appropriate settings, sound level, etc.
  • Measure a stereo impulse response at each test position. A simple method for doing this can be found at change.htm.
  • Once you have the IRs, here is how to evaluate them.

    Listen to the IRs convolved with anechoic program material using a software convolver. The freeware Gratisvolver is superb for this purpose ( I would start with a speech track, an acoustic guitar track, and a drum track.

    Use a measurement program to determine speech intelligibility ratings such as the C50 (the ratio of the sound energy in decibels before and after 50 ms) and speech transmission index for each test position ( Forget about frequency response. For reasons I mentioned earlier, you cannot get a valid transfer function for a loudspeaker in a casual setup. Be content with the time domain data that describes the energy ratios that affect speech intelligibility and music clarity.

    This is the only practical way to determine how each loudspeaker will perform for an audience. You now have a data set that will be meaningful for either objective or subjective evaluation.

    Since you now have recorded audio system measurements in the real space, you may use a near-field monitor or headphone playback system to compare the WAV files made with Gratisvolver. You can move instantly to any position for any loudspeaker and listen to any type of program material. Try that in a typical shootout.

    The room noise will be absent if the IRs were gathered correctly. My experience has been that when evaluated in this manner, the result is often a different winner than would be selected by a group of listeners at a typical shootout.

    The performance of a loudspeaker is not independent of the listening environment, and the method that I have described will factor this into the equation.

    Pat Brown is president of Synergetic Audio Concepts (Syn-Aud-Con) Inc. and Electro-Acoustic Testing Company (ETC) Inc. Syn-Aud-Con conducts training seminars in audio and acoustics worldwide for those who operate, install, and design sound reinforcement systems. ETC performs precision loudspeaker testing for the audio industry. Pat can be reached at

    Want to read more stories like this?
    Get our Free Newsletter Here!
    Past Issues
    October 2015

    September 2015

    August 2015

    July 2015

    June 2015

    May 2015

    April 2015

    March 2015