Wine Business Wine Business Monthly Media Kit Wine Industry Publications Contact Us Wine Industry Blogs Wine Industry Classifieds Wine Industry Events Wine News Archives Wine People News Vineyard Weather Wine Jobs
Wine Business Monthly Home Subscribe to Wine Business Monthly
December 15, 2004
Industry Forum: Wine Ratings 2 of 3
Ten Reasons We All Lose When Numbers Dominate the Marketplace
by W. R. Tish

Robert M. Parker's Wine Advocate is turning 25 this year. What Robert Parker has achieved with his publication--developing a loyal readership, rewarding a hedonistic style that has influenced winemakers worldwide and never touching advertising--is amazing. His ratings struck a chord with the cork-shy American public, essentially reducing the complexity of wine to the simplicity of a school report card. The 100-point rating scale clicked and helped attract throngs of new consumers into the arena of fine wine. Parker's indefatigability and consistency, not to mention the commercial impact of his scores, helped catapult numbers into the vocabulary of American wine.

But here in 2004, even while Robert Parker has remained atop the wine heap in terms of influence, I would argue that imitation and misuse of his system have turned the wine heap into a big numbers-driven mess.

The 100-point scale is now the chosen tool of countless critics, publications and websites. Almost everyone in the wine industry has learned to play along with the Rating Game, which is played like this: (1) wine gets a 90-point rating, (2) marketers proclaim the rating in their sales materials, (3) distributors use the rating to make placements and (4) retailers use the rating to make sales.

I can understand why observers of this cycle might say, "So, what's the problem? If the numbers still sell wine, then the system is working, right?" My response is that the damage numbers have done to the wine market is still below the radar. The Rating Game has bred laziness, complacency and neglect on the part of wine traders. In turn, consumers have come to take for granted the premise that wines are "measurable" and, moreover, that critics' opinions, seemingly objectified via numbers, are more valid than all other opinions, perhaps even their own. Indeed, the most damaging effect may be that widespread reliance on numbers has put a 94-point damper on wine appreciation among a generation of new wine drinkers.

I have come to this conclusion as a former wine magazine editor who now comes in contact with the wine-drinking public on a weekly basis while involved with tastings. I believe that the potential for "John Q. Pinot" to develop a 90-point habit spells trouble for the future of wine in America.

Here are 10 ways numbers are hurting wine.

1. Numbers Do Not Work

Numerical scores imply precision, yet precision in wine tasting is humanly impossible.

It is a simple fact: no critic can replicate his or her scores on a 100-point scale. If they could, we would see blind tastings in public. If the same critic can't give the exact same score to the same wine on two different days, then accuracy of ratings is pure fallacy (or statistically unreliable at best).

Perhaps the only reason numerical wine ratings have been accepted is that America had so little wine criticism at all to start with back in the 1980s. Imagine if numbers like 83 and 96 and 79 were suddenly applied in other cultural or culinary areas, such as books, art, movies, restaurants or potato chips; conventional wisdom would reject the ratings as foolish if not fraudulent. The numbers just wouldn't stick. Unfortunately, with wine, the numbers have become ingrained. Consumer knowledge and confidence are not high enough for such a rejection...yet.

2. Numbers Get Detached and Draw Undue Attention

Everyone who defends wine ratings on a 100-point scale sooner or later argues that the ratings are just part of a review, which includes tasting notes that go beyond the number to help people decide if they really think they might like the wine. True. The problem, however, is that in most cases where numbers are used to sell wine, the ratings are already detached. Via sell sheets, shelf talkers, ads, email promotions, catalogs and searchable databases, ratings have taken on a life of their own (e.g., RP92, WS89), separate from their original context.

Further, these bare, bite-sized numbers still demand attention. To use a somewhat risqué analogy, I'd compare these thumbnail ratings to topless women: they stick out from the crowd and grab the eye. Plus, once viewed, it's hard not to register some kind of instantaneous judgment. It is numbers themselves that are impacting consumers, not the well-intentioned criticism from which the numbers came.

3. Palates Vary, Numbers Don't

This point is obvious on one level: John Critic's palate is different than Jane Critic's palate. Everyone has preferences, conscious or not, based on their personal history of living, eating and drinking. On a less obvious level, not only do critics' palates naturally differ, but so do their scales. They vary in how numerical segments are defined (e.g., 85-89, Very good: a wine with special qualities) and in whether the score applies to the wine at the time it was tasted or at its hypothetical peak.

One of the more cogent arguments in favor of ratings is that intelligent consumers understand the variability among critics and scales, and interpret scores accordingly. Unfortunately, in my experience, most people have no real clue about the origin of the scores they see in ads and at stores. The numbers are seen as absolute; the notion that one person's 89 is another's 86 and yet another's 84 does not really register.

4. Volume Creates Compression and Homogeneity

Consumers today are confronted with more wine ratings than ever. Paradoxically, however,the range of numbers seems narrower than the use of a 100-point scale would imply.

Indeed, people rarely care about or remember ratings below 85, which is also the line at which Wine Spectator and others stop automatically including tasting notes. The vast majority of ratings that get regurgitated are in the 88-93 point range. This compression helps create a fairly unnatural sense of homogeneity. At first glance, wines that are completely different appear alike if they have similar scores, whereas wines that are utterly similar appear to be worlds apart.

Another way to look at the compression phenomenon is this: The numbers encourage consumers and retailers alike to lump wines into a perverse sort of caste system. There are the "excellent" wines (93 and up); the "really good" (90, 91, 92); the "pretty good" (88, 89); and, well, everything else. A sad irony here is that once upon a time, numbers were praised as a way to help make sense of the confusing multiplicity of wines. Ratings made the daunting look manageable. Today, American shelves are lined with the most diverse array of wines in history, and yet those easy-to-grasp numbers impose an atmosphere of sameness that hardly encourages understanding, let alone experimentation.

5. Higher Ratings Equal Higher Prices

In my experience conducting tastings, I have found that most wine drinkers equate higher quality with higher prices. Why? Because it's the highest-priced wines that get the highest scores.

For a graphic example, see the recent "Over 90 Points, Under $90" issue of LA-based Wally's retail catalog; I counted 215 wines, of which 54 were under $40, 23 under $30 and eight under $20. (Incidentally, the equation does not flip; people do not equate all high prices with top quality.)

Again, this basic perception of pricier wines getting higher ratings is not so awful, you might think. But there are problems lurking within this mindset: (1) Critic-anointed quality has nothing to do with personal taste. In fact, I'd argue that there is a converse relation. More wine-drinking Americans probably prefer a soft 81-point Blackstone Merlot to a more structured 91-point St. Emilion. The price-rating correlation implies that these people simply don't know quality, which, of course, is poppycock. (2) There is a subtle but palpable sentiment among wine drinkers that the really good stuff is still accessible only to elite pocketbooks. (3) Even worse, the quality of wine overall is getting lost in the consumer's mind. (See #10.)

6. Blind Tasting is Anti-Reality

Does it surprise anyone that blind tasting has remained the unquestioned gold standard of rating wines? The anonymity of such tastings is supposed to level the playing field, to let apples compare to apples. But if our common goal as an industry is to get more people to enjoy more wine more often with food, then what is the point of the emphasis on lab-like blind-tastings?

Even the most inexperienced drinker understands that how a wine tastes will vary depending on the food and situation. Drinking a heavy red before a light white, for instance, can make the latter seem like water. On a picnic, a simple rosé wine is likely a better choice than an okay Chardonnay, even if the Chard got a 92 and the rosé an 82. Numbers simply don't account for context.

7. The Vicious Numbers Cycle

I don't have personal experience in this area, but it seems obvious--especially since the tightening of the market over recent years--that consideration of ratings reaches deep into wineries, bringing pressure on winemakers to adjust their stylistic decisions to achieve better scores.

This phenomenon has helped deify certain consultants. It has also spawned wine-country labs that offer to analyze a winery's bottlings and prescribe how to get better ratings.

When the 90-point tail starts wagging the winemaking dog with impunity, The Rating Game has gone one step too far.

8. Numbers Are Abused Through Disingenuous Selection

Consider Landmark's 2002 Overlook Chardonnay. Spectator gave it 88 points, Parker gave it 91. Which score do you think will get funneled into the Rating Game? The 91, obviously. Wineries choose to publicize their best scores, and retailers will select the highest possible ratings to sell specific wines. But isn't this disingenuous? Would people be better informed if they learned that a given wine garnered scores ranging from 82 to 91, from six different critics? Of course they would, and it might help them remember that ratings are merely numbers attached to subjective opinions. It might also help them see where critics whose scores they have trusted in the past weigh in, even if the score is less than stellar.

But multiple scores would be utterly confusing, and nobody wants that. So instead, we are mired in a very distorted system, wherein only the highest scores survive and circulate via the Rating Game. This creates a false sense of "grade inflation" that in the long run reinforces the tyranny of numbers themselves more than the acumen of specific critics.

9. Numbers Deny Style and Divide Wineries

Numbers are tailored for quick-pick sales, not long-term understanding of style, which is the basis on which everyone finds what wines they truly like.

For instance, numbers won't tell people who like an Aussie Shiraz that they may well like a Cotes-du-Rhône. Similarly, if someone likes $7.99 California Merlot but decides to splurge on a seductive 93-point,$30 Napa Valley Merlot, he or she may be turned off by the more potent structure--and then may hold that against Napa for years to come.

Consider the message being sent to consumers over and over when they see shelf talkers, ads and other media touting one wine as better than its peers. The message is that wines within a genre are more different than they are alike. In the long run, that is not helping the consumer who is mostly concerned with finding wines he or she likes, not necessarily finding the better bottle at a given moment.

If one winery's Zinfandel is a 93 and another's is an 86, logic says the 93 is better. But the truth is that a Zin lover may get just as much pleasure out of either. And that message has not been getting past all the "numbers pollution." As long as wineries continue to play the Rating Game and tout their own bottlings over their neighbors', it's going to be very difficult to become a nation at ease with wine.

10. We Are All Missing the 100-Point Point

Is wine better across the board than it was 15 or 20 years ago? Zero doubt. This is true all over the globe. But this fact--arguably the most important story in wine over the past quarter century--is being lost in the numerical sauce.

If you toss out the ratings and instead used various publications' definitions of their numbers' meanings, the lion's share of reviewed wines would rate as "good" or better. But this is masked by the popular perception that the most expensive wines are still "the best" as well as the natural inclination of people to dismiss ratings under 85 as just OK, if not below average.

We are living in the Golden Age of wine; quality and selection of bottlings available in the US are both unprecedented. If we all stop slinging numbers around so frequently, more people would be able to enjoy this Golden Age.

So, are there any easy answers to this complex numerical predicament? No, but there are ways to limit the damage wine ratings have already done. Perhaps that's fodder for another article. In the meantime, the first step is for everyone who cares about wine to be more conscious of how endemic the Rating Game has become. And personally, I will continue to apply my own personal scale: in my view, every wine rates an 88. Done. wbm

W. R. Tish  , editor of Wine Enthusiast magazine from 1988-1998, writes on wine and food for various publications and develops wine events through his NY-based firm, Wine For All. He also distributes a free “WineFlash” email newsletter via his website, www.wineforall.com.

Copyright© 1994-2010 by Wine Communications Group. All Rights Reserved. Copyright protection extends to all written material, graphics, backgrounds and layouts. None of this material may be reproduced for any reason without written permission of the Publisher. Wine Business Insider, Wine Business Monthly, Grower & Cellar News and Wine Market News are all trademarks of Wine Communications Group and will be protected to the fullest extent of the law.