I wrote an email criticizing Robb Report's method of reviewing cigars aka only smoking 1/4 of each.
They got back to me today. Here is their response:
"Dear Mr. XXXXX
Thank you for your input. Although we feature numerous cigar reviews authored by individual experts throughout the year, our most recent approach to Best of the Best has been to invite a panel of enthusiasts to make the final selections from a group of top products chosen by staff. I forwarded your comments to Brent Butterworth, who wrote the Gurkha piece you cite. I’ve attached Brent’s response below and copied him and Richard Carleton Hacker in the event that you have further questions or comments. Thank you again for taking the time to offer us your opinion.
Brett Anderson
Senior Vice President, Editor in Chief
Robb Report
CurtCo Robb Media
"Brent Butterworth’s response:
We appreciate your observations. Every cigar aficionado knows that cigars can change in flavor as they burn down. But by the time you're into the first inch, you've got perhaps 80% of the cigar's character.
In comparison tests, you have to make some compromises. We preselect for the panel what Richard Carleton Hacker and I, as experts, believe to be the best candidates based on our comprehensive reviews throughout the year. Yet even with this pared-down number of finalists, it would be difficult for every panelist to finish every cigar. Not only would the test take 10 hours, but the participants would grow fatigued. Nicotine intoxication would become a problem (and probably cause most panelists to drop out). By the time you got around to cigar #9, no one would remember what cigar #2 tasted like.
Also, cigars are inherently inconsistent, as anyone who's seen them made knows. The Gurkha one panelist smoked probably varied noticeably in flavor from that of another. So to achieve a truly accurate comparison test, you'd really have to smoke, say, 10 samples of each cigar, from maybe 5 different boxes, to get a full profile of each cigar's flavor. In that case, you'd have Mr. Hacker and me just picking our favorite cigars of the year, since we were the only ones who had that kind of complete experience. At that point, the results merely reflect the personal tastes of two reviewers, rather than the consensus of a panel, which in our opinion is far more valuable."
They got back to me today. Here is their response:
"Dear Mr. XXXXX
Thank you for your input. Although we feature numerous cigar reviews authored by individual experts throughout the year, our most recent approach to Best of the Best has been to invite a panel of enthusiasts to make the final selections from a group of top products chosen by staff. I forwarded your comments to Brent Butterworth, who wrote the Gurkha piece you cite. I’ve attached Brent’s response below and copied him and Richard Carleton Hacker in the event that you have further questions or comments. Thank you again for taking the time to offer us your opinion.
Brett Anderson
Senior Vice President, Editor in Chief
Robb Report
CurtCo Robb Media
"Brent Butterworth’s response:
We appreciate your observations. Every cigar aficionado knows that cigars can change in flavor as they burn down. But by the time you're into the first inch, you've got perhaps 80% of the cigar's character.
In comparison tests, you have to make some compromises. We preselect for the panel what Richard Carleton Hacker and I, as experts, believe to be the best candidates based on our comprehensive reviews throughout the year. Yet even with this pared-down number of finalists, it would be difficult for every panelist to finish every cigar. Not only would the test take 10 hours, but the participants would grow fatigued. Nicotine intoxication would become a problem (and probably cause most panelists to drop out). By the time you got around to cigar #9, no one would remember what cigar #2 tasted like.
Also, cigars are inherently inconsistent, as anyone who's seen them made knows. The Gurkha one panelist smoked probably varied noticeably in flavor from that of another. So to achieve a truly accurate comparison test, you'd really have to smoke, say, 10 samples of each cigar, from maybe 5 different boxes, to get a full profile of each cigar's flavor. In that case, you'd have Mr. Hacker and me just picking our favorite cigars of the year, since we were the only ones who had that kind of complete experience. At that point, the results merely reflect the personal tastes of two reviewers, rather than the consensus of a panel, which in our opinion is far more valuable."
Last edited:
