Thursday, September 30, 2010

The reviewer's deceit

At the heart of audiophile magazines rests a claim that is difficult if not impossible to sustain. Namely, all reviews and all product rankings carry with them a statement, implied or explicit, that you should not rely on the reviewer's impressions but instead, should seek out these products and hear them for yourself to determine if they are truly as good as reported. On top of this, we are reminded of system synergies, that your mileage may vary, and no two rooms are the same etc. In other words, the reviewers can report what they like, defended against objections by an appeal to subjective experience and unreproducible context.

Sadly, most reviewers must know that not only are the contextual variables impossible to replicate in any reader's situation, but worse, most readers can never anticipate being able to hear these products, even in a dealership. Consequently, even though review after review reminds people not to take the experiences of one as a guide, the reality is interested parties must rely on the advice given if they are to form a sense of what might be good and not so good in the world of audio products. Since reviewers won't submit themselves to blind tests to confirm that they actually can hear what they report hearing without recourse to sight, the reviewing literature is always questionable.

Now there are positives here that should not be overlooked. One can, over time, learn to identify the reviewing biases and preferences of some reviewers and then calibrate one's own tastes accordingly. I have done this so that I now know, if a well known reviewer raves about a certain digital front-end, I probably will not be able to hear what he hears, but if another finds no problems with a certain speaker, then I will likely enjoy that product too. But this is a less than precise or efficient process, and it still requires experience listening to products that have been reviewed. And this is the nub of the problem: contemporary audiophile product selling does not reflect the world in which reviewers reside and from which they advise.

Nowhere is this clearer than in trying to purchase speakers. Here's a component that you really have to hear before making a decision to buy but tell me, where can a typical purchaser hear most of the loudspeaker products reviewed in a typical issue of TAS or Stereophile? I put this to the test a couple of years ago when I wanted to learn what I could hear locally. In a relatively large, well-populated part of the US, within 25 miles of my home I could hear in a dealer's showroom the following $10-20k models:

Wilson Sophia
Eggleston Andra
Martin Logan Summits
B&W 802D

And to have even this many, I consider myself lucky. No Thiel, no Von Schweikert, no Magico, no Verity, no Quad, no Avalon, etc. Go through the top ranked speakers in the recommended components list of S'phile, or the increasingly frequent award winning listings in TAS and try to find some of these models. Not round here fella!
Leaving aside the fact that only one of these models was supported by a dealer who would consider a home trial, I wondered how I could even lay eyes on some of the highly touted products in this competitive price range, never mind hear them.

Contacting manufacturers directly resulted in some insightful responses. Avalon told me to contact a dealer over 1200 miles away, who subsequently told me I'd love the product and he'd work with me if I bought them. But no, I could not hear them anywhere but his dealership first. A Legacy dealer told me to fly to Illinois and he'd reimburse my flight if I bought them. The editor of a leading mag told me to attend an audio show and make a choice from there. Wow.....this is how one spends $20k on audio then. Oh, and by the way, don't pay too much attention to reviews! And heaven help you if you want to buy a turntable of any quality.

The disconnect between mainstream reviewing, the industry, and how real people get to purchase gear explains why Audiogon thrives, why there are so many forums for people to share experiences, and why so many times we end up arguing about reliability and objectivity of reviews. I don't think the reviewers in the mainstream mags are to blame, but I sure wish they would wake up to the reality of ordinary purchaser's opportunities to hear equipment, and in so doing, review in a manner that helps rather than hides behind vague endorsement and the convenient defense of 'hear it yourself to decide'. When you cannot hear it for yourself, we need reviewers who can be more than generally positive about everything while hiding behind the standard caveats, and we need an audio press that works on a realistic appreciation of how people get to know audio equipment then campaigns to have manufacturers make this process easier. A little honesty about the audio purchasing experience would go a long way.


Anonymous said...

I have gone down the old road and no longer seriously consider reviews. I did my own research and found Siegfried Linkwitz website and ultimately bought the Orion + speaker system. Now my music reproduction sounds like real music. My preamp is a passive preamp, Chapman's TAP X. I just moved to a computer based front end and find it sounds better than a great CD player but is still a bit tedious to sort out.

MikeMercer said...

As a writer/reviewer - I always tackle it from the user's point-of-view (as I am a user, and was before I started writing professionally, when I was just a kid filling journals - and am still to this day) - and I also think the wall between critics and users has been shattered, but still remains in certain mags (and I worked for TAS - so I'm just saying what I feel).

However, when you're trying to describe the product (whether it be music or hardware) I think its a must to separate yourself from the concerns you speak of; with regard to real-world abilities in terms of actually hearing certain stuff! It's a paradox I know, but don't fault the writer! That said - I also intentionally review stuff that's more affordable, and has a dealer network, I also make myself available to readers, and have helped MANY find the things I've written about.

We can argue, and talk, and pound all this stuff into the ground -

but, and I can only speak for me - I write because I LOVE it, and I love Hifi cuz I LOVE music - and so I am blessed to be able to write about both - - I only tell users to have a listen on their own as a way of saying I'm NO rockstar, or the FINAL word, nor is any writer/reviewer IMO (unless you're Lester Bangs, Hunter S. Thompson, or Harry for Hifi - though he doesn't see himself that way)

PatrickD said...


thanks for your detailed and heartfelt comments. I am not as convinced as you are that we should separate the reality of purchasing from the review, though I can see how such a position makes logical sense. What I am trying to advance is a recognition that reviews should try to be more constructive and perhaps explicit where possible, even if that risks upsetting some people. I feel that too many writers use the 'hear it yourself' argument as a convenient excuse to avoid making a real judgment call that they would stand behind. It's difficult, I realize, and I don't wish to attribute blame to writers who I recognize are underpaid and here for the love of music first and foremost. I'd love a simple solution, and not just for reviewing :)