Hydrogenaudio vs. Stereophile

by

Ross M. Miller
Miller Risk Advisors
www.millerrisk.com
November 9, 2009

Over the years I have frequently written about music and audio equipment. This month I will move out from specifics to the big picture as represented by two seemingly opposing views--those found in Stereophile, the last mass-market audio magazine that appears in print form, and Hydrogenaudio.org, a website mainly about the compression of digital audio into MP3, FLAC, ACC, Ogg Vorbis, and various esoteric formats. Viewed simply, Stereophile is about the subjective side of audio and Hydrogenaudio is about the objective side. Things are far from that simple.

If Sterophile only had better photographs, it would qualify as a bizarre form of fetish pornography. Instead of picturing women unattainable to all but the select few, it specializes in picturing audio equipment unattainable to all but the select (and possibly deranged) few. Stereophile does contain useful information for the less privileged and more sane. It reviews regularly reviews "budget" audio components that range in price from $25 or less on eBay to a few thousand dollars from the handful of retail audio salons that have not gone broke yet.

I am a long-time Stereophile subscriber and, unlike many writers in their letters to the editor section, I have yet to find a reason to cancel my subscription in a verbal hissy fit. While some of Stereophile's writers do come off as absolute lunatics, Michael Fremer springs to mind, even analog-obsessed Sam Tellig seems like a reasonable, if obsessed, guy.

Glossy audio magazines used to be crowded field. Sterophile, which came late to the party and at first appeared only sporadically, lived for decades in the shadows of Audio, Stereo Review, and High Fidelity. But those three publications are now gone while Sterophile still limps along. Stereophile was founded by the recently departed J. Gordon Holt, an over-the-top guy if ever there was one. He started Stereophile with the idea of its being the first audio magazine to incorporate a subjective view of audio equipment, treating it more like wine than sterile sound-reproduction machinery. While the present-day, post-Holt Stereophile is biased toward tubes and vinyl, Holt actively supported both solid-state components and digital audio.

Holt's core audio belief was that the goal of audio equipment was to reproduce live musical performances with the greatest accuracy possible. Of course, most popular musical is not created live, but rather is manufactured in the studio. Holt viewed pop recordings (and the baby-boomers who listened to it) as essentially evil. Holt died believing that baby boomers had led to the ruin of serious audio.

Stereophile's audio equipment reviews follow a set pattern. The main narrative of each review tells cute stories about the equipment, the set-up process, and then wallows in its sound. An extended sidebar by editor John Atkinson goes into the numbers, measurements of the accuracy of the equipment's sound reproduction. Editor Atkinson usually notes the discrepancies between his objective measurements and the subjective opinions of the reviewer.

A good example of a Stereophile review is the aforementioned Michael Fremer's take on the dCS Scarlatti SACD/CD playback system. A mere $80,000 gets you what Stereophile claims is the best CD player that money can buy. Here is John Atkinson's picture of what a sine wave looks like when played through this $80,000 player:

Of course, neither the blue nor red line looks anything like a sine wave; a sine wave is nice and curvy. This thing looks like it was drawn during an earthquake. Actually, this is quite good for a CD player. Moreover, ever the cheapest piece of analog equipment outputs nice, pretty sine waves when fed one, like these two:

These charts provide incontrovertible objective evidence that CD players by their very digital nature butcher music, which is largely combinations of sine waves a la Fourier. Compressing the WAV files on CDs into the ubiquitous MP3s file only butchers them more. (Technically, the WAV files are merely containers for raw pulse code modulation (PCM) information.)

MP3s are where Hydrogenaudio comes is. Hydrogenaudio is not any kind of magazine; it is the host of Internet forums, wikis, etc. Many of the threads on its forums and wikis concern the best way to use software such as EAC (Exact Audio Copy), LAME, and dBpoweramp to rip compressed files from the WAV files on CDs. They are few waveforms presented on Hydrogenaudio because it is clear that they more you compress music, the worse it measures.

It does not follow, however, that measuring bad means that it sounds bad. How music "sounds" is inherently subjective. Hydrogenaudio, however, makes the subjective "scientific" by subjecting it to double-blind testing, the gold standard of science.

The most popular form of double-blind testing on Hydrogenaudio is the ABX test, and the site provides links to software that allows one to run one's own ABX tests. The idea behind ABX testing is straightforward. Suppose that A is the original CD track and B is an MP3 file that was ripped from it. In an ABX test of the two, the listener would listen to the CD track (A), then the MP3 file (B), and then an unknown track (X). The compute known whether X is A or B and the listener can listened to A, B, and X as much as he desires until he is ready to choose whether X is A or B. Because the odds of choosing correctly for a single sample is 50%, the process is repeated 10 or more times to establish statistical significance.

Over the summer I ripped my CD collection to WAV and MP3 files using dBpoweramp and LAME. Before I embarked upon this lengthy process, I conducted by own ABX test. Using Roxy Music's "Avalon" as my test sample, I compared a WAV file taking straight from the CD with two of LAME's preset compression formats known as V0 and V4. The V0 format compresses file to roughly 250kbps and the V4 format squeezes them tighter, down to around 160 kbps. Using ABX testing I determined that I could, with some effort, reliably tell the WAV file from the MP3 file with V0 compression. I could not, however, tell the V0 and V4 MP3s apart.

I ripped my CDs to all three formats, but based on these results, I try to listen to the WAV files whenever possible. I don't feel bad when I listen to MP3s, however, because I found the difference between them and the original WAV files to be rather subtle. This is not, however, an opinion that I could post on Hydrogenaudio. That is because the discourse on Hydrogenaudio is governed by a set of "Terms of Service." The most important TOS is the eighth one:

8. All members that put forth a statement concerning subjective sound quality, must -- to the best of their ability -- provide objective support for their claims. Acceptable means of support are double blind listening tests (ABX or ABC/HR) demonstrating that the member can discern a difference perceptually, together with a test sample to allow others to reproduce their findings. Graphs, non-blind listening tests, waveform difference comparisons, and so on, are not acceptable means of providing support.

Violators of TOS 8 are first warned and then booted. As you might imagine, the folks at Stereophile are reviled by Hydrogenaudio. While I find that Hydrogenaudio is a great source of info about file compression technology, I consider the ABX tests that they report to be worthless. Stereophile editor John Atkinson even suggested on Hydrogenaudio itself that all members post audiologist test results along with their ABX results and he was broadly ridiculed by the membership.

I don't care what other people can hear; I care what I can hear and clearly the Stereophile crowd cares what they can hear. The Hydrogenaudio folk, however, are convinced that whatever the Stereophile's reviewers and readers hear is imaginary. While it is easy to perform ABX tests of digital music compressed in various ways, it ranges from difficult to impossible to do any kind of double-blind testing on most audio equipment. The subjective approach taken by Stereophile does not even attempt to make double-blind comparisons; indeed, comparisons of components are frequently done based on the reviewer's memory of sound. While the fascistic tone of Hydrogenaudio more fanatical posters (and its Terms of Service) is offputting, these posters are almost certainly correct that placebo effect is rampant at Stereophile, where the more money that is spent, the more powerful the placebo is.

While I may have once had certified "golden ears," that's what the audiologist at GE told me, that does not mean that I'm about to pay $80,000 for a CD player, even if it plays SACDs to boot. As a child of the 1960s, everything now, from modest iPod on up, sounds a lot better than it did back then. Vinyl does not sound so wonderful when it's played on a $75 Garrard turntable and music on the typical AM radio of the time was beyond dreadful. I just turned on my Roku SoundBridge R1000 to Radio Paradise at 192 kbps and through the cobbled-together sound system in my office and it sure blows anything that I could afford during the sixties away, not to mention that there was no Radiohead back then. Of course, such opinions are unwelcome on Hydrogenaudio, but I do not have to do an impossible double-blind test to tell you that Internet radio coming through my office sound system sounds different from (and better than) the Bakelite AM radios of my youth.

That said, I have no great love for digital audio, I just don't have the time to mess around with vinyl anymore and I am sure that I cannot listen to it in my car. But I do listen to vintage speakers, right now Sarah McLachlan is singing to me through my Advent /1 center-channel speaker and I have a pair of original "big" Advent speakers in my family room. Further discussion of the joy of vintage speakers will have to wait until next year.