« WIDWD: King Estate 2006 Domaine Pinot Gris | Main | WIDWD: Seven Hills 2005 Cabernet Sauvignon Seven Hills Vineyard »

Reducing bias in wine reviews

Andy Perdue of The Wine Knows recently posted about his approach to reviewing wines, which he conducts through "blind" tastings. Without knowing the winery or winemaker behind each wine, Andy prefers to conduct his tastings in two ways, either "single blind" (e.g., wine variety or style) or "double blind" (e.g., wine color only).

His post sheds some useful light on how some reviewers try to minimize bias, that is any subjectivity or prejudice that might skew a rating higher or lower. Reducing bias is a key focus in any market research, as you want to generate data and insights that are as objective as possible. The more bias present, the less reliable the results. For example, a wine reviewer who is sampling wine with the bottle in front of them may be far too biased to perform an objective review versus tasting the same wine in a blind tasting.

While Andy's post was interesting, I found the comments much more enlightening.

Paul Gregutt of The Seattle Times commented on Andy's post, suggesting he was being "dogmatic" in advocating blind reviews. Paul suggested that context was important toward reviewing wine, for knowing the "who" behind the wine was critical toward assessing its quality. He likened blind tastings to reviewing a movie without knowing the names of the director or actors, critiquing a book without knowing the author, or evaluating a painting without knowing the name of the artist.

I wonder if Paul grasps the impact of his argument here, for I could not disagree more with this assertion. In fact, this open approach raises even more questions beyond that addressed in my earlier post, Breaking the Unspoken Code.

Let's revisit Paul's analogies regarding the downside of reviewing movies, books and art without knowing the person directing, acting, writing or painting. Paul claims any blind review of these art forms robs the critic of important context. Beyond genre, period, or place, what is lacking in not knowing who was behind the piece?

Most movie, book and art reviewers focus their evaluations on the writing, the acting, the style, the technique, etc., with certain benchmarks to ensure consistency in their assessments. Can't the same approach be taken when evaluating wine? Does knowing the winery and/or winemaker provide the context needed to objectively assess a wine's merit?

I think not.

Rather, there is an importance to understanding the what (e.g., varietal) and where (e.g., region) when reviewing wine. However, knowing the who (e.g., winery/winemaker) introduces a ton of bias into the equation. It goes to the heart of the matter Andy describes when he opened a Turley zinfandel and shared it with a group of judges. Knowing the cult-like following behind Helen Turley, the judges were clearly biased in praising the wine while Andy felt it was average.

I side with Andy's approach here, for I would much rather be guided by ratings of wine that are done "blind" versus ones done "open". In fact, I am looking forward to reading more of Andy's reviews, for it is clear he is much better positioned to unveil new discoveries as a result of his blind tasting approach.

His discoveries of Kettle Valley and Barking Frog are well received, for I am much more interested in these finds than chasing the cult-wines that constantly rank at the top of reviewer's lists who taste open, let alone in the presence of the winemaker.


Post a comment

(If you haven't left a comment here before, you will need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting. And spammers, please note your comments will automatically be deleted, therefore don't waste your time here.)

 Subscribe in a reader


www.flickr.com
This is a Flickr badge showing items in a set called Wine. Make your own badge here.



Alltop, all the top stories
Creative Commons License
This weblog is licensed under a Creative Commons License.