Stereophile Editor's Note: In 1985 and 1986, an argumentative thread ran through Stereophile's pages, discussing the benefits or lack of double-blind testing methods in audio component reviewing, triggered by J. Gordon Holt's review of the ABX Comparator. As this debate is still raging nearly 15 years later, we present here the entire discussion that bounced back and forth between the magazine's "Letters" section and features articles. It was kicked off by a letter from C.J. Huss that appeared in Vol.8 No.5.—John Atkinson
ABX...
Editor: First off, thanks for the enjoyable reading your magazine provides. While I do not agree with everything printed therein, as your product reviews seem too often free of the kinds of proper controls that avoid psychological prejudice, the liveliness of the writing makes up for this. Your willingness to entertain differing points of view is refreshing, and contrasts with the editorial rigidity of some other magazines, both mainstream and underground. But I have a question about your testing approach.
In the April 1985 issue of Audio, Laurence Greenhill and David Clark reported the results of some tests performed on a McIntosh MC2 002 power amp and some Sansui top-of-the-line separates. In addition to the technical evaluations, a subjective listening test was performed for each of these products, then compared to the results when an ABX comparator unit was used to achieve double-blind testing conditions.
Mr. Greenhill was able to correctly identify the unit under test 10 out of 16 times when he Mac was compared with his I presume reference units, 16 out of 16 times when testing the Sansul preamp, and 9 out of 16 times for the Sansui power amp. Only one of these tests is statistically significant, but purely subjective evaluations of all these components were all very different when the unit under test was known.
