Are Criticism Rating Systems Serving Anybody?

“Are you going to let somebody else grade your work?” Victory Gardens Theater Artistic Director Chay Yew responded to my latest snarky comment about the number of stars my most recent show had received in the Chicago Tribune.

I paused. “No, of course not!” Until that moment, I hadn’t ever thought of it as a “grade.”

But I certainly don’t ignore the star rating. Words have sometimes hurt when I’ve read them about my work, but when I’ve reread a review later it usually has meant something different and has perhaps even been helpful to me with the cool eye of distance. But unlike words, the number of stars is so conclusive and final; reducing the worth of the work to a numeric appraisal, “this production is worth this many stars.” Like most artists, I teeter tentatively between caring what others think and not, and I sometimes can’t help but feel like I wear those stars, like a scarlet letter, around with me during the days after the review goes to print.

As a director, I put my most intimate thoughts and feelings into my theatrical work. I spend hours of mental space researching, analyzing, dreaming, collaborating, rehearsing, teching, and fine-tuning every play I direct, sometimes for more than a year in the making. And then there it is, after opening, the long anticipated review. The printed response. The permanent proof of this ephemeral piece of art that I have spearheaded.

And lately, it comes complete with a rating in bold. The Chicago Sun-Times has a “not recommended” to “highly recommended” review system with some in between; TimeOut Chicago has a range of 5 stars. In the Chicago Tribune, the rating is between 1 to 4 stars, 1 being terrible, 4 being the highest accolade. And I’m not going to lie, over the years I have gotten almost every rating in that spectrum.

Like most artists, I teeter between caring what others think and not, and I sometimes can’t help but feel like I wear those stars, like a scarlet letter, around with me during the days after the review goes to print.

This past fall I found myself in conversations with artists and producers everywhere I went, from backstage to the bar, about review rating systems. I hadn’t questioned them before, since they seemed like a permanent fixture, but it wasn’t that long ago that the Tribune didn’t use them at all. In the past fifteen years, systems of rating plays, from stars, to recommendations, to the Little Man of the San Francisco Chronicle, have become a shorthand, distilling the critic’s thoughts on the play into something the public can glance at, ascertain whether they should see it or skip it, and move on. But I wonder: is this serving any of us? The artists, the critics, the audience? Are we cheapening the discourse on theatre and the possibility of active engagement from readers and possible audience members by using these systems?

First, to dispel some mystery around the rating systems, I spoke with three major theatre critics from three major US cities: Chris Jones from the Chicago Tribune, Robert Hurwitt from The San Francisco Chronicle, and Peter Marks from The Washington Post (which does not use a rating system). The Chronicle’s Little Man, whose five poses indicate his opinion of a play or film, has famously been part of the paper since the 1940s, mostly reviewing movies and “larger” theatre productions, and was expanded around 2000 to cover all theatre productions. Chris Jones at the Tribune explained to me that stars were historically only used for restaurants and movies and then there was discussion around using them for theatre as well, “…it was not my idea but I did not object to it.” Peter Marks is relieved that it hasn’t come up at the Washington Post, “If I were asked I’d resist. I didn’t get into this because I want to grade productions.” The idea of putting a star-value on a show makes him nervous, “I don’t think I’d even know how. There are so many variables. My hat’s off to reviewers who find it helpful to them in some way.”

a cartoon
Little Man poses, devised in 1942 by cartoonist Warren Goodrich for movie ratings in the San Francisco Chronicle. Photo by the San Francisco Chronicle. 

I’ve spoken with many artists who believe that the critic does not chose their own ratings, that the editor does, but Jones and Hurwitt both assured me that they do. They have slightly different methods, Jones told me he writes the review first and then chooses the star rating, and Hurwitt said that he chooses the Little Man rating and then sometimes while writing changes his mind and adjusts the rating. Both agreed that the easiest ratings are the highest and lowest ratings, although both seldom give their lowest (which for Jones is one star, and for Hurwitt is “LM5”, the Little Man’s empty chair). The middle range seems to be more complicated for both of them.

 

Chris Jones, theatre critic for the Chicago Tribune
Photo by Chris Jones. 

For a show to appear in “Chris Jones Recommends,” on The Theatre Loop (the Tribune’s online theatre page) it needs 3 stars or above. So the threshold between 2.5 stars and 3 stars is a big division. A rating of 2.5 stars signifies that for Jones the show was, “almost there, there were significant things I didn’t like, but usually there was some really good stuff,” and a rating of 3 stars means that is overall good and that “people will enjoy it.”

This threshold seems similar in the Little Men between what Hurwitt calls “LM2” and “LM3” (“LM1” is the highest rating, where the little man is leaping out of his chair and applauding wildly and they each get slightly less excited until “LM5” as described above). He describes the two middle categories as both being broad, and LM2 as, “ranging from strongly recommended to well above average,” and LM3 as “for the full range of pretty good shows but ones I’d recommend only with reservations or only to people with specific interests; good plays that could be better done or excellent productions of not very interesting plays.”

None of these critics championed the review rating system. Marks doesn’t have to use it, and while Hurwitt doesn’t love it, he has found for himself an upside, “There are times when I feel having to justify a rating has sharpened my thoughts about a piece.” Jones likes that he can really push a play he truly loves, even if he thinks it might be challenging for some audience members, “when you can give 4 stars, you can make an impact.”

It’s true, the power of these ratings can make or break the sales of a show. Hurwitt explained, “One commercial producer, whose long-running show had an LM1 rating, told me years ago that during a two-week period when the rating had mistakenly slipped to an LM2, his box office dropped by almost 30 percent.”

And this is where producers seem to sit on the fence about how they feel about rating systems, at least the ones I spoke to, whose companies ranged from a small storefront to a major LORT theatre. The smaller companies in particular, who have little-to-no marketing budget feel the impact of the ratings. Peter Moore, artistic director of Steep Theatre in Chicago told me, “if we get a 4-star review [in the Tribune] the show has sold out sometimes in the matter of days, and gone beyond original run.” Nick Sandys, Artistic Director of Remy Bumppo Theatre in Chicago similarly told me that after the review came out for the four-star rated Our Class last season they had the biggest single-ticket sales day in their eighteen-year history. Would this have been possible without the graphic of the four stars?

a man looking at the camera
Nick Sandys of Remy Bumppo Theatre, Chicago. 
Photo by The Daily Herald. 

Hurwitt noted that, “although Bay Area theatre artists have complained—loudly and often—about the Chronicle’s rating system, many change their minds when they have a show running with a positive Little Man rating in the calendar.” I asked Carey Perloff, artistic director of ACT in San Francisco what she thought about this marketing push. “In the short-term it’s a good advertising campaign, but in the long run it won’t sustain the field. The work is worth encountering in all its complexity. Not just what is vetted and is a hit. What makes a relationship with theatre satisfying is a long-term belonging to a conversation, that’s what everybody wants.”

Which brings us back to the number one reason that both artists and critics dislike review ratings: they are reductive. They reduce the content of the critic’s opinion down to something a reader can glance at and know immediately if they want to read the review and/or see the play. Playwright Lauren Gunderson, (who also spoke very articulately about the artist/critic relationship at the American Theatre Critics Association conference last year) asserted that rating systems simplify the complexity that is the theatre-going experience, by reducing it to a good/bad paradigm, “…bad and good are meaningless, that’s not a way to talk about art.” Similarly, Marks told me that it conjures for him the image of someone “standing in front of a Van Gogh and thinking, ‘2.5 stars.’” Hurwitt wishes he didn’t have to rate plays with the Little Man, “I think any rating system is a disservice to the art form being reviewed, to the writer, and to the reader.  It encourages a simplistic approach to the work under consideration.”

a woman looking at the camera
Playwright Lauren Gunderson. Photo curtosey of Lauren
Gunderson. 

And while some papers have only come to use rating systems recently, and in some places, such as London, every single publication now uses a rating system, playwright Samuel D. Hunter told me that he feels review ratings are a relic of a time when theatre was mainstream entertainment. Understood this way, the rating of a play like the ones he writes on the same scale of the rating of a big, commercial musical just seems wrong, because they are two very different things. “It’s fine for someone looking for a great night on the town that picks the 5-star show,” he told me. Can one rate Jersey Boys and The Whale using the same rubric? Even if a critic likes a show for two very different reasons, doesn’t the stamp of a star rating then equate the two things?

Perloff said something similar about what gets rated in the Chronicle,

…it’s always been a real mystery and a great shame because the so-called ‘high arts’ have no rating system, as to say these are complex and serious art forms that require a complexity and ambiguity of approach whereas theatre and movies have a little man clapping. I have never understood why theatre is considered amenable to that reductive, while other art forms are considered more complex.

This made me consider something that had come up in many of my interviews: review ratings are effective when rating forms of entertainment, but not art. One can rate that on a scale how much they enjoyed their experience, but an art show is not about being “entertained,” so while the work can be reviewed, it cannot be simply rated. The experience of a piece of art is not necessarily about how much you enjoy it. Theatre exists somewhere in the middle of art and entertainment: some play-going experiences are entertainments that could be rated, but many are complex art works that the experience of cannot be reduced to a numerical value. Using a rating system generalizes the critic’s response instead of highlighting exactly what it is—an individual response to an individual artwork.

So given the issues that many have with review rating systems, why do they still exist in so many major publications? As Hurwitt mentioned, Perloff also spoke of the Bay Area theatre community asking the San Francisco Chronicle to stop using the Little Man for theatre, to which the paper argued, “it’s iconic, it’s what the readers look for, it’s what’s unique.” If the Little Man left the theatre section, would San Francisco residents mourn? Do they need that Little Man jumping up and down in order to know they want to see a show? Or, would they read the review? Chris Jones speculates that some people only read the extreme reviews, and when they see a 2.5-star review they don’t even read it, “I don’t have evidence, but I know there’s a danger.” Hurwitt is also concerned that even if they read the review, the rating “may discourage the reader from seeing something she or he might find more interesting than the critic did.” So the rating system simplifies not only the critical engagement with the piece of theatre, but the manner in which a reader engages with the review. So, if it comes down to serving the readers, is this what audience members are truly looking for?

I only spoke to artists, producers, and critics. I didn’t have any conversations with editors or theatre patrons. But this is the question that I’m left with, because everybody I spoke with found review rating systems problematic: do readers need them? If we took them out of our major publications for even, say, three months, as an experiment, what would happen? Would readers notice? Would they miss it? Would they adjust to reading reviews without them? Would they still come to the theatre?

I’d like to think that they would. That they would adjust to reading the content and deciding for themselves, without the guide of the rating, whether that show is something they’d want to see. That they would engage in the criticism, thinking about the critic as an individual and their experience of the production. And that maybe, this fuller engagement with theatergoers, might even encourage what Perloff called “belonging to a conversation.”

I want my productions to make deep connections with people on every level, whether they are viewing it or reading about it. I don’t want the response to my art to be a consumer guide, because audience members aren’t just consumers, they are a vital part of the conversation that is theatre. Review rating systems are dangerous because they not only cheapen the discourse on theatre between artist and critic, but they skew the perspective of the public, creating (or perpetuating) an expectation of the transaction between theatre-creator and audience, in which audiences sit back to be entertained, not sit forward and engage. Theatre-going is at best a complicated and beautiful experience that is created to be as individual as it is communal. The critic’s review is a vital voice that reports their individual experience to the public.

But the rating? It is a grade. And we all know from school that the feedback is far more vital than the grade itself, but the grade is what goes down on your permanent record. And the very fact that the least valuable piece of the review has the most value, needs to change.

Bookmark this page

Log in to add a bookmark

Interested in following this conversation in real time? Receive email alerting you to new threads and the continuation of current threads.

subscribe

Comments

14
Add Comment
Newest First

The ratings should not be taken seriously by artists. The reviews should not be taken seriously by the artists, and even awards should not be taken seriously by the artists (apart from any cash of course!). We all want the NY Times to 'notice' us. It's a human response - I must be important, I'm in the important paper. But the work can be quite brilliant and never noted in any public forum. We all know that is the truth. The critics are there to help sell seats, that is what they do. They are not the gods of Art Arbitration and never can be. So if some of them find a rating system helps them with their task of putting work in front of the public then don't go biting the hand that feeds you.Appreciate them for what they do do and find other ways of getting useful feedback on your creations.

Sure, yes, I realize I'm "not supposed to care" what critics think or how they rate my work. This did partially come from a personal place for me as an artist that I'm candid about in this article, as I'm tired of the constant pretending that reviews don't matter or get in our heads. It is, as you said, human. And, I don't think there's anything wrong with it, to be honest.

But again, I stand by the fact that the words of the review are one thing, the rating is another.

My point is that rating systems are reductive. Critics find them reductive for the most part, artists hate them, and the question that remains is: is it serving readers/audience? If the job is to help sell seats, is it the best way? Or is it oversimplifiying the critic's experience of the work to the point that it's not helping people find the theatrical performance that best matches their tastes? I suspect it isn't. That's all.

I'm certainly not "biting the hand that feeds". I am appreciative of critics and their words. I'm only questioning something that they are also questioning: the rating systems. And frankly, I think that sort of thinking is dangerously disempowering, to be quiet and appreciate whatever scraps of PR we can get without question.

Of course, they shouldn't be taken seriously by artists. But they are the message that gets sent out--the score--to anyone who searches for information about a play. And let us not be NYC-centric about this. A play in New York will be commented on by many voices (not as many in mainstream media as in the past, but...) Most plays outside of the major markets may have one or two voices commenting on them in the media. That star rating sticks. As an artist hoping and striving to expand the reach of my work, the star rating needs to be taken seriously not as guidance for the work but as a label that will likely be found well before a potential producer actually reads the play.

The website I write for, DC Metro Theater Arts (http://dcmetrotheaterarts.c..., uses a 5-star rating system on reviews, and like Robert Hurwitt, I've found that my appraisal of a show is often sharpened by having to select a star rating for it. But instead of reviewing a show, I will frequently contribute a column about it; and because columns have no rating, there's latitude to write what is in effect more of an appreciation and less an evaluation. I hate what I think of as the Consumer Reports function that gets attached to theater writing ("Is this show worth your time and money or not? Well, just check this handy rating and you need not bother to read more"). If I had my druthers, there would be no theater critics at all, only theater *appreciators*—people who can convey to audiences something of the *experience* they can expect once they attend a show, not its ranking on a not-very-informative scale.

Yes, and I like the Carey Perloff quote in your piece: “What makes a relationship with theatre satisfying is a long-term belonging to a conversation...” Another DCMTA writer, Michael Poandl, and I saw a play in DC the other night and afterward were pitched into an intense conversation that we decided to capture for others to overhear ("Post-Play Palaver: A Passionate Response to Forum Theatre’s ‘Passion Play’" [tinyurl.com/q4stocr]). Theater appreciation doesn't have to be only solo opining. Engagement with a theater experience can be expressed in actual conversations too!

***1/2 for a smart, wide-ranging intro to the topic. Some random thoughts from a critic/playwright/audience member--I had the luxury of creating a new arts section for my paper (www.ibj.com/arts) and, without pressure from my superiors, I was empowered to decide whether or not to use any kind of rating system. I happily chose not to because I believe they distract from the discussion, they create false equivalencies (that ***1/2 community theater production of oft-produced material is "better" than the **1/2 professional world premiere), and they dictate how the rest of the piece is read. --Star ratings can, as you said, have an immediate impact on butts-in-seats. Such ratings are also more likely to be social media pushed. My reviews are less likely, I believe, to be copied, pasted, and pushed than a quickie review with **** slapped across the top.--As a useful tool for playwrights, they are a double-edged sort with both edges blunted. Does a theater in Cleveland really believe that a production of a play truly deserved the **** it was given by a pub in another city? On the other hand, whatever is said in a review, no matter how thoughtful, gets wet-blanketed if there are only **1/2 stars on top.--As a member of ATCA's new play committee, I wish I didn't see the star rating at the top of a script nominated for the Steinberg or Osborn awards. But it's hard to unsee them if I read a review.--I do not see my job as a critic as being Consumer Reports for the arts. --When reading out of town reviews, without the benefit of knowing a critic's esthetic, such rating systems present an unfair shorthand.--Ultimately, they feel dishonest.Let the conversation continue....Lou Harry www.louharry.com

Not much of my own to offer, but your post is to some degree in conversation with something from the TCG Circle last week, so I wanted to introduce these posts to each other. Rating Systems, please meet Glorious Debate. http://www.tcgcircle.org/20... Both articles and Richard's comment below turn heavily on the distinction between critique and reveiws, a distinction that I as primarily an audience member rarely make, by the way, so it was interesting to think about.

Nice article! The perennial debate about the role of reviewers is a tough one. There was a really interesting discussion with playwright David Lindsay-Abaire at the Comparative Drama Conference, where most of the way through it, he said something like, "I love talking about my work here...mostly people say 'I liked it because of X' or 'I didn't like it because of Y', but here, mostly, people are saying 'This is what I think it means' or 'this is how it works'.... I wish more critics wrote like that..." For me, at least, I was thinking, "um...critics all write like that. You are thinking about reviewers." And that is the point. Criticism advances knowledge, while reviews are a service to the readers. Though, that does bring up the one complaint I have about your article: As you point out, you didn't talk to any actual readers of the reviews, and since they are the audience for the review, that is probably the most important constituency. If there is an answer to your headline-question, the answer would have to be 'readers who plan to make decisions about what to do with their time and money based on the review'. If they are being served, then that is all the _reviewers_ really care about.

I agree, that is a weakness with the article. I thought of this as a starting point, and think the next step would be to really figure out from publishers and their readerships if the rating systems are necessary. As a theater director myself, I talked to the people I had more access to and approached it as an artist.

As to then your answer to the headline question, I still don't buy that as an answer. Are they truly serving those people? Or are they getting in the way of them actually reading a review and seeing things based on what sounds interesting to them? Just because they can quickly glance at something and decide to see it or not, doesn't mean they are getting served. The reductiveness of ratings also hurts readers/possible theater goers of all kinds. I know plenty of theater artists that have even told me they were thinking of seeing something but it only got 2-stars so they didn't go. Did they read the review? Did they really know if it was something they wanted to see based on someone else's experience? No, they just looked at stars. And how can you not, when they're right there?

Fair points. Though I was not proposing that readers were being _well_-served... but they are the ones who the reviews are written for. I would be interested about what non-theater people think of the utility though. It is fine for us to wonder _whether_ the ratings serve them, but if we are going to make a claim that they _aren't_ being served by them (or aren't being well-served by them), at that point we would have to back it up not with what we think they want, but with some research into what they actually want. If the issue is "this is what they want, but it is bad for them" then we are getting into the pretty familiar but quagmire-like territory of the responsibilities of a corporation. Cigarette makers, fast food, etc.

True. I've been thinking much about this (clearly) and my proposal is that papers that use it try losing it for 3 months, and see what readers think. Do they complain? Do they read it more? Can we poll them after? And does theater attendance go down or up or stay the same?