The NYT Critic Watch project was designed to gives us some numbers about modern theater criticism. We are awash in opinion, in curiosity, in hurt feelings, in hopes for a hit, and in the dual notion of wanting not to care but desperate for praise. 

Reviews have great power on the life of a new play, especially ones from the New York Times. But do we agree with them? Do we enjoy reading them? Are they focusing on the right elements of a new play? Are the helping or harming us as theater artists?
The project picked out every review of a new play printed in the New York Times in the year 2010. We crafted a survey to determine what reader thought of the review itself (not of the play being reviewed). We let this loose on the internet and had generous volunteers read a review and fill out a short survey.  
Here's what we found:
1) Mostly Thumbs Up!
Frankly, we expected a lot more slams. Based on the way our colleagues discuss reviews over post-show drinks, you'd think almost every review is catty and brutal. But the collected answers to the first survey question: "Rate the Review from 1 (rave), 2 (liked almost all of it), 3 (Meh, Neutral), 4 (Didn't really like it), 5 (complete slam)" were these:
Only 3% of the reviews were judged as slams. 13% were judged as raves. The largest percentage of reviews at 40% were rated as 2s, or "liked most of it." Ratings of 3 and 4 ("Meh" and "Didn't really like it") were almost exactly the same at 23% and 21%.
So it seems that most of the reviews land in the positive. The second largest is the "neutral", which we understand would read as not a good review to many of our colleagues. "Meh" might be just as bad as "bad" in our "raves only" sensibility. But "meh" is not always bad. Half the time it's good, as you'll see later. And most of our crafty marketing colleagues can find a few good lines to splatter on posters from a "Meh" review.

Bar graph displaying reader reactions to NYT theatre reviews

2) All About Playwriting
The second survey question was: "On what element of the production was the review mainly focused?"
Hands down this one went to playwriting. No big surprise here. But what did surprised us was that the lowest categories discussed in the play is design (0%) and directing (1%). Wow. No mention of directing in a review of a new play? Now this might be mitigated by the admittedly vague category of "Everything", which received 20% for this question. It seems that a director's work is both obvious and invisible, and the play is truly the thing.

Bar graph displaying the overall focus of each review by production element

3) To Go or Not To Go
Our third survey question was simple: "Does this review encourage you to go see the play or not?" The numbers landed like this:
"Go See This Play" = 65%
"Do Not Go See This Play" = 35%
In our rough math we found that the percentage of "raves" + "liked most of it" + half of the "meh"s from the first survey question add up to 64.5%, or almost exactly the "Go See This Play" percentage noted above. So this helps our theory that a "Meh" review doesn't completely suck. It only half sucks. 

Pie chart displaying levels of reader encouragement after reading a NYT review

4) Dead Even
The answers we got to the final question made us laugh. 
"If you saw this production of this play, did you agree with the reviewer?"
Dead tie. Not a bit more or less of either answer. Precisely even. Of the people who actually saw the play in question exactly half said they agreed, and half said they did not agree. I think most bar conversations after plays would follow this math. A play is a gesture of art, we think. Art does not affect us evenly, we think. Now we know that in this case it affects exactly evenly—evenly uneven.
So yes, everyone is a critic even of critics. 

Pie chart displaying reader/audience agreement with NYT reviews.

Our next post will unpack the tone of reviews, and what our surveyors said they want from a fair review.