Are Criticism Rating Systems Serving Anybody?
“Are you going to let somebody else grade your work?” Victory Gardens Theater Artistic Director Chay Yew responded to my latest snarky comment about the number of stars my most recent show had received in the Chicago Tribune.
I paused. “No, of course not!” Until that moment, I hadn’t ever thought of it as a “grade.”
But I certainly don’t ignore the star rating. Words have sometimes hurt when I’ve read them about my work, but when I’ve reread a review later it usually has meant something different and has perhaps even been helpful to me with the cool eye of distance. But unlike words, the number of stars is so conclusive and final; reducing the worth of the work to a numeric appraisal, “this production is worth this many stars.” Like most artists, I teeter tentatively between caring what others think and not, and I sometimes can’t help but feel like I wear those stars, like a scarlet letter, around with me during the days after the review goes to print.
As a director, I put my most intimate thoughts and feelings into my theatrical work. I spend hours of mental space researching, analyzing, dreaming, collaborating, rehearsing, teching, and fine-tuning every play I direct, sometimes for more than a year in the making. And then there it is, after opening, the long anticipated review. The printed response. The permanent proof of this ephemeral piece of art that I have spearheaded.
And lately, it comes complete with a rating in bold. The Chicago Sun-Times has a “not recommended” to “highly recommended” review system with some in between; TimeOut Chicago has a range of 5 stars. In the Chicago Tribune, the rating is between 1 to 4 stars, 1 being terrible, 4 being the highest accolade. And I’m not going to lie, over the years I have gotten almost every rating in that spectrum.
Like most artists, I teeter between caring what others think and not, and I sometimes can’t help but feel like I wear those stars, like a scarlet letter, around with me during the days after the review goes to print.
This past fall I found myself in conversations with artists and producers everywhere I went, from backstage to the bar, about review rating systems. I hadn’t questioned them before, since they seemed like a permanent fixture, but it wasn’t that long ago that the Tribune didn’t use them at all. In the past fifteen years, systems of rating plays, from stars, to recommendations, to the Little Man of the San Francisco Chronicle, have become a shorthand, distilling the critic’s thoughts on the play into something the public can glance at, ascertain whether they should see it or skip it, and move on. But I wonder: is this serving any of us? The artists, the critics, the audience? Are we cheapening the discourse on theatre and the possibility of active engagement from readers and possible audience members by using these systems?
First, to dispel some mystery around the rating systems, I spoke with three major theatre critics from three major US cities: Chris Jones from the Chicago Tribune, Robert Hurwitt from The San Francisco Chronicle, and Peter Marks from The Washington Post (which does not use a rating system). The Chronicle’s Little Man, whose five poses indicate his opinion of a play or film, has famously been part of the paper since the 1940s, mostly reviewing movies and “larger” theatre productions, and was expanded around 2000 to cover all theatre productions. Chris Jones at the Tribune explained to me that stars were historically only used for restaurants and movies and then there was discussion around using them for theatre as well, “…it was not my idea but I did not object to it.” Peter Marks is relieved that it hasn’t come up at the Washington Post, “If I were asked I’d resist. I didn’t get into this because I want to grade productions.” The idea of putting a star-value on a show makes him nervous, “I don’t think I’d even know how. There are so many variables. My hat’s off to reviewers who find it helpful to them in some way.”
I’ve spoken with many artists who believe that the critic does not chose their own ratings, that the editor does, but Jones and Hurwitt both assured me that they do. They have slightly different methods, Jones told me he writes the review first and then chooses the star rating, and Hurwitt said that he chooses the Little Man rating and then sometimes while writing changes his mind and adjusts the rating. Both agreed that the easiest ratings are the highest and lowest ratings, although both seldom give their lowest (which for Jones is one star, and for Hurwitt is “LM5”, the Little Man’s empty chair). The middle range seems to be more complicated for both of them.
For a show to appear in “Chris Jones Recommends,” on The Theatre Loop (the Tribune’s online theatre page) it needs 3 stars or above. So the threshold between 2.5 stars and 3 stars is a big division. A rating of 2.5 stars signifies that for Jones the show was, “almost there, there were significant things I didn’t like, but usually there was some really good stuff,” and a rating of 3 stars means that is overall good and that “people will enjoy it.”
This threshold seems similar in the Little Men between what Hurwitt calls “LM2” and “LM3” (“LM1” is the highest rating, where the little man is leaping out of his chair and applauding wildly and they each get slightly less excited until “LM5” as described above). He describes the two middle categories as both being broad, and LM2 as, “ranging from strongly recommended to well above average,” and LM3 as “for the full range of pretty good shows but ones I’d recommend only with reservations or only to people with specific interests; good plays that could be better done or excellent productions of not very interesting plays.”
None of these critics championed the review rating system. Marks doesn’t have to use it, and while Hurwitt doesn’t love it, he has found for himself an upside, “There are times when I feel having to justify a rating has sharpened my thoughts about a piece.” Jones likes that he can really push a play he truly loves, even if he thinks it might be challenging for some audience members, “when you can give 4 stars, you can make an impact.”
It’s true, the power of these ratings can make or break the sales of a show. Hurwitt explained, “One commercial producer, whose long-running show had an LM1 rating, told me years ago that during a two-week period when the rating had mistakenly slipped to an LM2, his box office dropped by almost 30 percent.”
And this is where producers seem to sit on the fence about how they feel about rating systems, at least the ones I spoke to, whose companies ranged from a small storefront to a major LORT theatre. The smaller companies in particular, who have little-to-no marketing budget feel the impact of the ratings. Peter Moore, artistic director of Steep Theatre in Chicago told me, “if we get a 4-star review [in the Tribune] the show has sold out sometimes in the matter of days, and gone beyond original run.” Nick Sandys, Artistic Director of Remy Bumppo Theatre in Chicago similarly told me that after the review came out for the four-star rated Our Class last season they had the biggest single-ticket sales day in their eighteen-year history. Would this have been possible without the graphic of the four stars?
Hurwitt noted that, “although Bay Area theatre artists have complained—loudly and often—about the Chronicle’s rating system, many change their minds when they have a show running with a positive Little Man rating in the calendar.” I asked Carey Perloff, artistic director of ACT in San Francisco what she thought about this marketing push. “In the short-term it’s a good advertising campaign, but in the long run it won’t sustain the field. The work is worth encountering in all its complexity. Not just what is vetted and is a hit. What makes a relationship with theatre satisfying is a long-term belonging to a conversation, that’s what everybody wants.”
Which brings us back to the number one reason that both artists and critics dislike review ratings: they are reductive. They reduce the content of the critic’s opinion down to something a reader can glance at and know immediately if they want to read the review and/or see the play. Playwright Lauren Gunderson, (who also spoke very articulately about the artist/critic relationship at the American Theatre Critics Association conference last year) asserted that rating systems simplify the complexity that is the theatre-going experience, by reducing it to a good/bad paradigm, “…bad and good are meaningless, that’s not a way to talk about art.” Similarly, Marks told me that it conjures for him the image of someone “standing in front of a Van Gogh and thinking, ‘2.5 stars.’” Hurwitt wishes he didn’t have to rate plays with the Little Man, “I think any rating system is a disservice to the art form being reviewed, to the writer, and to the reader. It encourages a simplistic approach to the work under consideration.”
And while some papers have only come to use rating systems recently, and in some places, such as London, every single publication now uses a rating system, playwright Samuel D. Hunter told me that he feels review ratings are a relic of a time when theatre was mainstream entertainment. Understood this way, the rating of a play like the ones he writes on the same scale of the rating of a big, commercial musical just seems wrong, because they are two very different things. “It’s fine for someone looking for a great night on the town that picks the 5-star show,” he told me. Can one rate Jersey Boys and The Whale using the same rubric? Even if a critic likes a show for two very different reasons, doesn’t the stamp of a star rating then equate the two things?
Perloff said something similar about what gets rated in the Chronicle,
…it’s always been a real mystery and a great shame because the so-called ‘high arts’ have no rating system, as to say these are complex and serious art forms that require a complexity and ambiguity of approach whereas theatre and movies have a little man clapping. I have never understood why theatre is considered amenable to that reductive, while other art forms are considered more complex.
This made me consider something that had come up in many of my interviews: review ratings are effective when rating forms of entertainment, but not art. One can rate that on a scale how much they enjoyed their experience, but an art show is not about being “entertained,” so while the work can be reviewed, it cannot be simply rated. The experience of a piece of art is not necessarily about how much you enjoy it. Theatre exists somewhere in the middle of art and entertainment: some play-going experiences are entertainments that could be rated, but many are complex art works that the experience of cannot be reduced to a numerical value. Using a rating system generalizes the critic’s response instead of highlighting exactly what it is—an individual response to an individual artwork.
So given the issues that many have with review rating systems, why do they still exist in so many major publications? As Hurwitt mentioned, Perloff also spoke of the Bay Area theatre community asking the San Francisco Chronicle to stop using the Little Man for theatre, to which the paper argued, “it’s iconic, it’s what the readers look for, it’s what’s unique.” If the Little Man left the theatre section, would San Francisco residents mourn? Do they need that Little Man jumping up and down in order to know they want to see a show? Or, would they read the review? Chris Jones speculates that some people only read the extreme reviews, and when they see a 2.5-star review they don’t even read it, “I don’t have evidence, but I know there’s a danger.” Hurwitt is also concerned that even if they read the review, the rating “may discourage the reader from seeing something she or he might find more interesting than the critic did.” So the rating system simplifies not only the critical engagement with the piece of theatre, but the manner in which a reader engages with the review. So, if it comes down to serving the readers, is this what audience members are truly looking for?
I only spoke to artists, producers, and critics. I didn’t have any conversations with editors or theatre patrons. But this is the question that I’m left with, because everybody I spoke with found review rating systems problematic: do readers need them? If we took them out of our major publications for even, say, three months, as an experiment, what would happen? Would readers notice? Would they miss it? Would they adjust to reading reviews without them? Would they still come to the theatre?
I’d like to think that they would. That they would adjust to reading the content and deciding for themselves, without the guide of the rating, whether that show is something they’d want to see. That they would engage in the criticism, thinking about the critic as an individual and their experience of the production. And that maybe, this fuller engagement with theatergoers, might even encourage what Perloff called “belonging to a conversation.”
I want my productions to make deep connections with people on every level, whether they are viewing it or reading about it. I don’t want the response to my art to be a consumer guide, because audience members aren’t just consumers, they are a vital part of the conversation that is theatre. Review rating systems are dangerous because they not only cheapen the discourse on theatre between artist and critic, but they skew the perspective of the public, creating (or perpetuating) an expectation of the transaction between theatre-creator and audience, in which audiences sit back to be entertained, not sit forward and engage. Theatre-going is at best a complicated and beautiful experience that is created to be as individual as it is communal. The critic’s review is a vital voice that reports their individual experience to the public.
But the rating? It is a grade. And we all know from school that the feedback is far more vital than the grade itself, but the grade is what goes down on your permanent record. And the very fact that the least valuable piece of the review has the most value, needs to change.