Rating Zagat

Where shall we eat tonight? If that's not the burning question of our time, it is surely one that torments those who want to win at the restaurant game. Where, then, can one turn for reliable, independent and authoritative advice? Where else, one might ask, but the nation's most popular guides, the Zagat surveys?

Between iconic wine-dark covers emblazoned with bold white type, each Zagat survey lists hundreds of restaurants, all rated on food, decor and service by a supposedly impartial and knowing public rather than by professional critics. The guides are nothing if not handy, with concise codes telling readers everything they need to know about a restaurant, except what to order. As the publishers do not reimburse their respondents for meals (although they do compensate each one with a copy of the final work), they can review more establishments than would be humanly or financially feasible for one person, or even one periodical. In last year's survey of New York City, 20,424 people rated nearly 2,000 places.

That unrivaled scope, and the democratic voting system, have earned the Zagat surveys many encomiums. According to its back-cover blurbs, the guide has been dubbed "the gastronomic bible" by the Wall Street Journal, "indispensable" by the Los Angeles Times and "the single best source of accurate dining information" by the Washington Post, which apparently doesn't mind undercutting its own critics. In this culture of celebrity, it is no surprise to find additional endorsements from Bill Cosby ("I love good food. That's why I love Zagat") and Andrew Lloyd Webber ("Obliterates the need for any other guide").

In fact, with their huge sales, the Zagat surveys have just about obliterated all other guides. Last year, true believers bought about 650,000 copies of the New York City volume alone. Tim and Nina Zagat, the founders and cochairs of Zagat Survey LLC, currently publish 45 city and regional guides covering the U.S., Toronto, Vancouver, London, Paris and Tokyo, plus separate reports on shops, hotels and nightspots. Now, backed by $31 million from assorted investors, the Zagats are preparing a major expansion of their Internet services and possibly even an initial public offering. That's a long way from the hobby that began in 1979 when the Zagats, as young lawyers living in Paris, mimeographed sheets of their friends' restaurant suggestions.

But what's missing from all these thousands of ratings is the most crucial evaluation of all: How well do the guides really work? If you took their advice, trusting that high scores for food at a given restaurant would translate into great meals, how would you fare? To find out, I recently toured eight cities, eating at the restaurant singled out by Zagat as having the best food in town. These top-rated establishments were Le Bernardin in New York; L'Espalier in Boston; the French Laundry in the Napa Valley (first place in the San Francisco guide); the French Room in Dallas; the Inn at Little Washington in Virginia (the winner in Washington, D.C., and the only restaurant at which I felt I had been recognized); Matsuhisa in Beverly Hills; André's in Las Vegas; and Norman's in Coral Gables, Florida (the Miami champion)—the last two being the only ones I had never covered before in my 35 years of restaurant reviewing. To gain perspective on the local food scenes, I also ate at many of the runners-up.

In a couple of instances, the survey's top choice was on the money, nowhere more so than at THE FRENCH LAUNDRY in Yountville, California. That's fortunate, considering the maddening obstacles to making a reservation, a feat so difficult that had this not been a work project, I would have given up, thinking no meal quite worth it. But if any were, this would be it. Although Thomas Keller, the celebrated chef and owner, was away, the kitchen turned out his inventive dishes exquisitely.

Keller's food is surprising but never unnervingly over-the-top. His soigné inventions seemed predestined: a gratin of oysters with sweet-sour Meyer lemon and caviar; toasted brioche sandwiching soft-shell crab and tomato confit; roasted saddle of lamb with a polenta cake and artichokes; and seared duck breast with a counterpoint of bittersweet endive marmalade. A plate of cheese that included strong and runny époisses with brandied prunes led to Keller dessert classics such as "coffee and doughnuts"—cinnamon-and-sugar-dusted mini-doughnuts with silky cappuccino semifreddo—and a refreshing Alsatian rhubarb tart with creamy mascarpone sorbet.

I was curious to visit the top-rated NORMAN'S in Coral Gables, where Norman Van Aken is the chef; much as I admired Van Aken when he was cooking at A Mano in South Beach a decade ago, I feared that his florid inventiveness might by now have gone too far. But magically, even improbable-sounding combinations seemed harmonious. The food was lusty but subtle, from lightly fried shrimp with mashed yucca, habanero tartar sauce and Van Aken's famous mojo verde—tomatillo mayonnaise—to Vietnamese vegetarian spring rolls with crunchy jicama and fine noodle filaments. A lemongrass soy sauce perfectly complemented rose-red tuna carpaccio, as did a cumin coating on a succulent rare lamb porterhouse. And cool, satiny café con leche panna cotta with tiny, cinnamon-scented churros was an inspired Latin-Italian fusion.

Because one can quibble endlessly about the relative excellence of New York's best restaurants (and because it is the city I know best), I used a slightly different approach here, visiting the survey's top five picks in an attempt to determine only whether each deserved to be among the elite. I would not, for instance, rank LE BERNARDIN, CHANTERELLE and NOBU first, second and third, as Zagat does, awarding each 28 points out of a possible 30. Still, all three are remarkable enough that the choices are understandable. But when I scanned Zagat's list of the runners-up, plausibility flew out the window.

The really incredible designations are the fourth-rank status of SUGIYAMA, with its esoteric Japanese cuisine, and the fifth-rank showing of PETER LUGER steak house. This places both restaurants above the consistently excellent and far more ambitious Jean Georges and Daniel, both with 28 points, and Lespinasse, La Grenouille, Union Square Cafe and Gramercy Tavern, all with 27. Sugiyama, which serves a formalized progression of courses that can include steak on a hot stone, sushi and creative small dishes, does all of them very well but excels at nothing. And at Peter Luger, I had very good lamb chops and steak, but also encountered unripe tomatoes, tasteless shrimp left too long on ice, burned German-fried potatoes, a brassy steak sauce, and creamed spinach that would make even Gerber wince. One can only surmise that the 28 rating applies to the meat alone.

The appearance of a French restaurant at the head of the New York list is repeated in Zagat guides across the country. Of the top-rated restaurants I visited, five are French, and a sixth—the Inn at Little Washington—is certainly French informed. Apparently, reports of the demise of haute cuisine are greatly exaggerated. Also surprising is that despite all we hear about Italian cuisine and its regional splendors, only four Italian places are ranked first for food in the 40 areas covered in the 2001 edition of Zagat's nationwide guide, America's Top Restaurants. Perhaps an unsure public still feels more confident in declaring for the French kitchen.

Because almost all of the restaurants with the highest ratings for food are extremely fancy looking and expensive, I had to wonder if amateur critics are capable of separating the cooking from the surroundings. Would they rate the same dishes as highly if they were served in a simpler setting? And do they perhaps feel the need to reassure themselves, as they're paying a check that tops $200, that the meal they've just eaten was superb?

A perfect case in point is THE FRENCH ROOM in Dallas, a gilded Versailles wanna-be with green marble floors, gold-leaf trim, murals of flying cherubs and trompe l'oeil clouds. The overly ambitious kitchen sends forth such misguided creations as quail filled with roasted red peppers and Parmesan cheese wrapped in bacon and served with caramelized endive polenta on a balsamic red-currant sauce. I have heard postmodernism in architecture described as the illiterate application of symbols, and so it is with this dish. Inventions require more artistry than this kitchen exhibited with a tough lobster tail that was not helped by wild mushrooms or soupy, "sorrel-infused" risotto. Nor did a metallic lemongrass sauce do much for a badly overcooked crab cake. Zagat's capsule review describing the menu as "classic" is simply bizarre, although the best offering, a delicious rare-roasted rack of lamb, hints that traditional efforts might be better rewarded.

The French Room's top ranking raises other questions. For one, how valid is local opinion for the visitor? A Dallas native might consider French food more suitable for a special occasion than the regional cuisine. But a visitor would (or should) prefer the stunning Southwestern specialties at the third-ranked Mansion on Turtle Creek. I would rather dine there four nights in a week than at either the French Room or the Riviera, whose innocuous French food is Zagat's second choice. But then, I'm a stranger here myself.

Like the kitsch-laden interior at the French Room, the dining rooms of L'ESPALIER in Boston, laid out on two floors of a Victorian-Edwardian town house, provide the kind of grandiose setting that Zagat respondents seem to crave. About 10 years ago, I considered the food here to be as excellent as it was diverting, but now the kitchen seems to be overreaching. Most dishes were disappointingly bland, including a giveaway appetizer of shrimp in a seaweed gel with overly chilled cucumbers, a technically correct game-bird pâté, overcooked day-boat halibut sautéed under a mush of crushed sunflower seeds, and a nicely juicy squab obscured by a heavy sauce and a starchy taro-and-parsnip cake. As for lavender-perfumed mashed potatoes, think of eating potpourri. Among better choices were the assiette of lamb and the black sesame-flecked fried soft-shell crabs.

As I made my rounds, I began to wonder what role sentiment plays in the Zagat ratings of restaurants that have been the scene of so many family milestone dinners over the years. Can it be that the food is highly seasoned with nostalgia? That, plus a certain local resentment toward celebrated interlopers, must be at work in Las Vegas. The level of cooking in the city has risen dramatically over the past few years, but the people's choice is still ANDRÉ'S, for 21 years a fixture in the deserted old downtown residential neighborhood. This local icon is a funky anachronism with an "old auberge" look—dark wood beams, china bric-a-brac and lace curtains and tablecloths. The more traditional French food is pleasant in a gentle way, as with a sautéed fillet of sole véronique complete with green grapes (itself something of an anachronism, albeit well prepared), a nicely garlicked rack of lamb, a honey-roasted breast of duck and a Grand Marnier soufflé. But everything else failed, including a canapé of some duckish mousse on crumbly, dry croutons and a dreadful version of the old-time banquet cliché of sweetbreads in a patty shell, here with pea-size chicken quenelles in a soggy bouchée.

One might suppose that the opinions in the guides would be reasonably up-to-date since the first instruction on the survey form is "Please rate the restaurants you've visited in the past year." But compliance with this request relies strictly on the honor system. Perhaps the weakest link in the Zagat method is that respondents are not asked to supply any proof as to when—if ever—they visited the restaurants they voted on. Zagat could request photocopies of receipts; even requiring those surveyed to write in the approximate date of their last visit would dampen the impulse to fill out the form willy-nilly, whether the responses are based on recent experience, fond memories or, perhaps, imaginings. Any such tactic, though, might discourage people from taking part in the survey. (Tim and Nina Zagat were not interviewed for this article, but they did respond in writing to several issues, including this one. They said they believe the "overwhelming majority" of those surveyed follow the questionnaire's instructions, and added, "In any event, with thousands of surveyors, we always have people visiting each restaurant up to the last minute. And our local editors, who are active food writers, have current knowledge of the restaurants surveyed.")

THE INN AT LITTLE WASHINGTON, in the Washington, D.C., area, has won the top Zagat rating for food every year since 1995; if, as I believe, the kitchen's performance has slipped a few notches in that time, the survey doesn't reflect it. I find the gussied-up Victorian-Edwardian decor stifling but had always felt it was more than made up for by the subtlety of the food. Not so on this latest visit. In his magnificently outfitted kitchen, Patrick O'Connell relies too heavily on fruit, whether in the apple coulis that overly sweetens bland boudin blanc, or the tropical fruit mix compromising lovely crabmeat, or the hot pineapple with duck that is said to be cooked "thrice," meaning overcooked to a fare-thee-well. Apple cider sauce ruined the flavor of braised rabbit, but tart pickled cranberries perfectly complemented delicious, pepper-encrusted venison. O'Connell's obvious sweet tooth serves him well with desserts, judging by the luscious liquid Valrhona chocolate cake and the crunchy marjolaine with its hints of hazelnuts and chocolate. The cookies were good too.

Another restaurant that seems to be sliding is the number one choice in Los Angeles, MATSUHISA. When it opened in 1987, it was indeed a stunner, as Nobuyuki Matsuhisa, the affable Japanese sushi chef, who has spent time in Peru, Argentina and elsewhere in South America, created his own brand of fusion cuisine. But as he now divides his attention among his seven Nobu outposts around the world (including the one in New York), Matsuhisa has suffered. Baby eggplants with miso, cold soba noodles and the marinated yellowtail with slivers of jalapeños were all delicious. But many of the fusion specialties were oversalted and obscured by heavy brown sauces, especially overpowering when pitted against delicate tuna in a sashimi salad. Others, such as the Kobe beef with vegetables and the squid "pasta," suggested stir-fried Chinese takeout. Even the usually silken, signature black cod with miso arrived dry and shriveled, and the tempura was third-rate, soggy and floury. The colorful crowd, the helpful staff and the more inviting dishes would draw me back, but the indifferent preparations hardly justify the top rating.

Other writers have said that Zagat scores are self-fulfilling prophecies, a phenomenon William Grimes of the New York Times has called "the Zagat Effect." Grimes suggests that diners who go to highly rated restaurants, "convinced that they are eating at a top-flight establishment, cannot bring themselves to believe otherwise."

Because stories abound of restaurateurs trembling when Tim and Nina Zagat appear at the door, one has to ask whether they or their editors exert undue influence on their books. What does it mean, for example, when a humble score of 19 for food at Guastavino in the 2001 New York City guide prompts the editors to point out that the restaurant complex includes a more formal dining room "whose high quality is not adequately reflected by our ratings"? Says who? The Zagats reply that the editors made this statement because it was clear from survey comments that some people "had confused Guastavino's informal downstairs brasserie with its upstairs fine-dining restaurant." Then there is the matter of which places get into the guide. Last fall, publication of the Boston survey had to be delayed after a local critic noticed that one restaurant was described in the galleys as if it were already open when, in fact, it wasn't.

All of which says nothing about two phenomena that would seem to be beyond the publisher's control. Several restaurateurs have told me about visits from rogue respondents who announce themselves as survey participants in hopes of getting special food and service, if not a free meal. (The Zagats respond, "If anyone should be so crass as to act that way, we hope the restaurant would ignore him or her, just as they would any other patron who...drops a name in hopes of getting a good table.") Some restaurateurs, for their part, have enlisted friends, relatives, staff and clientele to stuff the ballot box. Although Tim Zagat says he has a system for detecting such a ploy, doing so seems virtually impossible. Such engineered responses obviously could result in huge profits for the restaurateurs and a disappointing waste of money for the customer.

These glitches aside, the Zagat surveys stand or fall on their central premise: that thousands of separate opinions add up to something like the truth. Asked about the reliability of their guides, the Zagats answered, "We argue...that our numerical ratings and our consumer-based reviews are more reliable than any individual critic because they draw on the shared experiences of a large cross-section of savvy customers (200,000 this year alone) rather than the personal biases of one, frequently recognized, professional critic....[T]he enormous sales success of our books and the steady increase in the number of our participating surveyors year after year suggest that restaurant-goers do find our method to be a reliable basis for rating restaurants, which is a good enough measure for us."

Having always distrusted consensus, I feel the system of relying on a vast public rather than professional critics has no more validity in assessing restaurants than it would if applied to art or theater. The majority can be wrong, and one well-informed opinion is worth more than those of a thousand amateurs. Popular success is not a measure of excellence. If it were, it would mean that McDonald's serves the world's best hamburgers, KFC makes perfect fried chicken, Pizza Hut is the envy of Naples and, come to think of it, that the Zagat Survey is our best restaurant guide.

Mimi Sheraton was the New York Times restaurant critic from 1976 to 1984. She has written for Time and Vanity Fair.