Count What Matters Most
This week HowlRound will feature essays and interviews from the newly published anthology Counting New Beans: Intrinsic Impact and the Value of Art. This book includes the final report from research firm WolfBrown on their two-year study "Measuring the Intrinsic Impact of Live Theatre," as well as twenty-four interviews with artistic leaders and patrons, and essays by Diane Ragsdale, Arlene Goldbard, Clayton Lord and Rebecca Novick—plus a foreword by Ben Cameron of the Doris Duke Foundation. The following essay was adapted from “Sowing New Beans: The Making of Memory and the Measuring of Impact.”
When she was eleven months old, my daughter, Cici, learned how to ride a rocking horse. She was scared at first, but eventually got over that fear, and then rocked gently, then more forcefully, then like a little tiny madwoman racing away from a burning barn. Once she had learned it, she loved it. She did it often; she did it well.
About a month later, Cici received a large plastic dump truck for her birthday. We showed her how to put stuff in the back and flip it up, but when we left her alone she mostly ignored the truck, and when she did play with it, she would—instead of filling up the back—simply flip up the bright yellow bed, revealing the black undercarriage, straddle the middle of the truck and begin to bounce back and forth. Cici, it turns out, was interacting with the dump truck in the best way she knew how—by treating it as a rocking horse. When presented with a changed situation, she simply transferred the old idioms to the new obstacle.
That is, I think, where we are in terms of valuing and evaluating the impact of art. As an industry, the arts suffer from a value problem, because we keep trying to ride a rocking horse even when it has turned into something else.
All over the country, arts organizations attempt to justify their existence by talking about two things: anecdote and economics. That’s our rocking horse. We learned it thirty or forty years ago, after the great enlightenment of the Kennedy years gave way to the darkness after. And we’ve learned the lesson well. For nonprofits, whose true essence sits at zero dollars, artistry is now justified in terms of money in and money out. This is a problem, because the economics of what we do aren’t what matters, not really.
What we traffic in is memories.
We are memory makers, and it's important that we not forget that. Memory is what is held onto: people choose or don’t choose to repeat an activity based on the abstract feelings and impressions that are packaged together in a memory. We, as artists, make powerful, complete memories for people and, in so doing, we traffic in the making of meaning: the translation of the cacophony of life into understandable, memorable moments and the passing on of one person's transformative life experience to another person's narrative, forever.
So let’s talk about that.
And let’s talk about that in a way that makes sense to people seeking the “certainty” of numbers and graphs. Let’s turn an analytical eye on the previously “unmeasurable” parts of art and, well, measure them. Because measuring economic indicators as stand-ins for value forces arts organizations to shift away from their core missions. We need to start valuing ourselves in terms of how we impact people—but we need to do so using standard language and measurements, just like we would to count economic impact. And we can—it’s called “intrinsic impact.”
Research into intrinsic impact emerged, in a way, out of a single admonition from Ed Pauly, the evaluation director at the Wallace Foundation, to Alan Brown, the lead researcher for “Measuring the Intrinsic Impact of Live Theatre,” the major research report released last week along with 24 interviews from artistic leaders and patrons, and essays by the likes of Diane Ragsdale, Arlene Goldbard, Ben Cameron and others in the anthology Counting New Beans: Intrinsic Impact and the Value of Art.
Brown tells the story of sitting in a presentation by Kevin McCarthy, the lead investigator in the Gifts of the Muse study, at the Wallace Foundation offices in New York. Gifts of the Muse was the first articulate argument for shifting the conversation away from the “extrinsic” impact of art and toward the “intrinsic” impact. This was prior to the public release, and McCarthy gave a full presentation of the findings and then opened up the floor for questions.
Brown raised his hand and said, “After ten years of heavy emphasis on measurable outcomes, isn’t it funny that you’re telling us that the real benefits of the arts are intrinsic and can’t be measured?”
And there was more silence in the room. And then Pauly turned his head around and said to Brown, “Alan, if you can describe something you can measure it.”
This caught Brown off-guard, and it took him two years of close reading of the report to tease out what might be a manageable way to move forward with Pauly’s admonition. Many iterations of that effort later, we have come to this work, the most comprehensive analysis of the actual true intrinsic impact of live theater ever conducted.
The intangible aspect of the art, while never knowable in a complete way, is more knowable than we have assumed, and by learning to measure and talk about the intellectual, emotional, social and empathetic impact of art on an individual using standard metrics and a common vocabulary, we can move the conversation forward in a dynamic and new way.
Envision the artistic process as a beautiful, elusive wild animal, walking down a riverbed, stalking through the land so quickly that it’s impossible to catch a glimpse of the real thing. You can never see the animal itself, it’s gone. But the footprint is there, to measure and examine, and you can work from the footprint back to something of the animal itself.
We, as artists, make powerful, complete memories for people and, in so doing, we traffic in the making of meaning: the translation of the cacophony of life into understandable, memorable moments and the passing on of one person's transformative life experience to another person's narrative, forever.
The measurement of things-beyond-economics is like taking a step into Plato’s cave, where all we’re viewing is shadows and light, all we’re measuring is afterimages of something so unknowable and individual that we can never really get close enough to wrap a measuring tape around its form and write it down. We don’t have the animal. We have the footprint. So. Is measuring the footprint enough? What is the meaning made? What is the role of that memory in a life?
Over ten months from December 2010 to September 2011, the researchers worked with eighteen theaters in six cities across the country, looking at a total of fifty-eight productions. They distributed over sixty thousand surveys and received nearly nineteen thousand surveys back, an amazing number of responses. In the final research report, WolfBrown notes that this high response rate is indicative of a hunger on the part of audiences to engage with the work they’re seeing in this way. In the other direction, the authors of the study also point out that encouraging this kind of “audience feedback loop,” where staff is checking actual audience impact against the assumptions they made about what the work would do, also enriches the audience’s experience in a variety of ways.
Put bluntly, asking these questions educates the audience about how to engage with the art. Providing these questions, directing the conversation, allows a deeper engagement that may make the experience more lasting and the memory more powerful. The act of providing feedback is in itself a form of aesthetic development, allowing a more direct interaction with the piece over time. It also allows for an important milestone in the customer relationship.
In the course of developing the research methodology that formed the backbone of this study, WolfBrown has identified five constructs of intrinsic impact that interact to create the particular impression, memory or meaning of an artistic event: captivation, aesthetic growth, intellectual stimulation, social bridging/bonding and emotional resonance.
The particular mix of impacts among these constructs for a particular piece of art can (and should) vary. A musical should not have the same impact as a tragedy, one hopes. But with such a large data set, it is possible to see some common, fascinating trends in how live theater affects patrons. For the full set of results, take a look at the report, but here are a few:
- On average, single-ticket buyers report significantly higher impacts than subscribers. That is, our most frequent, loyal attendees are being impacted less by our work than people we’ve never had a relationship with before.
- Younger attendees are more socially motivated to attend theater and seem to have, generally speaking, a personal connection to the art form. Among the least frequent attendees, thirty-five percent came “because someone else invited me,” illustrating the power of social context to drive attendance among infrequent attendees.
- Women reported higher impacts than men across all fifty-eight productions, in particular feeling “emotionally charged” after a performance and “reflecting on one’s opinions.” Some of this difference may be explained by the fact that women were more likely than men to be sole decision makers (see next point).
- Decision makers reported higher levels of context and familiarity, and are more likely to prepare. All of this ties into generally higher levels of anticipation and impacts among decision-makers (presumably ticket buyers) compared to those who attend with them.
- Different types of productions generated different impacts. Plays were more intellectually stimulating while musicals were more captivating and emotionally charged. Comedies generally created more social bonding, while shows with challenging material made people flex more intellectually.
- Overall, thirty-five percent of respondents left the performance with unanswered questions. Respondents who reported having questions tended to have higher levels of familiarity with the playwright/composer or with the cast, but lower levels of familiarity with the story of the play. In other words, unfamiliar work generates more questions, which stands to reason.
- As familiarity with the work rises (people say they’ve read about it, they know the cast, playwright, etc), so do anticipation levels.
We need to start valuing ourselves in terms of how we impact people—but we need to do so using standard language and measurements, just like we would to count economic impact. And we can—it’s called “intrinsic impact.”
In her application to participate in this work, one artistic director wrote movingly about how, as a very small organization, she often was unable to secure reviewers to attend her work, which left her without the generally-requested “support materials” to demonstrate her organization’s value and impact on the community to grantmakers and other funders. Lacking those clippings, she instead often substituted a heartfelt personal artistic statement interspersed with quotes from patrons and artists about how the company had affected them, with mixed success.
This artistic director wrote about the hope she had that this work might revolutionize her ability to demonstrate her organization’s value specifically and realistically. While continuing to carry the testimonials into her writing, she envisioned a day where she could also pull a graph, for example for the question “How much were your eyes opened to an idea or point of view you hadn’t considered before?” and show that X percent of her audience had their worldview significantly expanded by seeing this show. That’s a new conversation, for her, and it does a variety of things.
It takes what was previously an esoteric, even touchy-feeling concept—empathy—and specifies it down to a particular question, a particular scale, a particular result. It turns the conversation from one about how many people were served into a conversation about how deeply they were affected. It makes a specific statement about her company, her production, her audience, her impact.
We’ve been using a one-size-fits-all frame in a conversation that really deserves to be custom made, and our hope is that working from a place of assessing the impact of the artistic product itself, we can get away from the generalist mentality. Earlier research indicates that increasing intrinsic impact can demonstrably increase the likelihood of audience members returning to a company, a form or a piece of work. In addition, a deeply impactful experience is also one that will remain in the memory more vividly and for longer. And since memory is the core of future decision-making, it is by increasing these impacts, and therefore increasing the resonance of the memories of that experience, that we can increase frequency of attendance, perceived value of the particular experience and of the arts in general, and most grandly, affect change in society in terms of the more extrinsic benefits that we all spend so much time trying to discuss now anyway.
All of this serves to change the conversation about the value of the arts. Five words, that’s the focus of our effort here. We want to make measuring intrinsic impact:
- Accessible. Intrinsic impact assessment used to be hard to do unless you managed to get into one of the studies being conducted by WolfBrown and its partner organizations. Theatre Bay Area and WolfBrown want impact assessment to be accessible by everyone, quickly and easily.
- Affordable. This research study had an overall price tag too high for any individual arts organization to really take it on. As part of this work, therefore, we have developed an online interface, available for all genres,, that will allow anyone with an internet connection, patron emails and as little as $1,500 to get easy, instant impact assessment feedback from their audiences.
- Standardized. We have to stop talking past each other, both inside organizations and when talking as a community outward. Part of this dissemination effort is to insert a set of important new words into the lexicon, and to give artists, arts administrators, stakeholders and funders a good understanding of what those terms might mean.
- Understood. People like graphs and numbers for a reason. Where there may be fewer nuances, there is more easy comprehension. We hope to link what might seem esoteric with concrete actionable items within an organization.
- Routine. Everyone knows how to talk about profit and loss. Everyone knows to count ticket stubs and to occasionally survey basic demographics. We want impact assessment to enter the lexicon of “the things you just always do” to keep tabs on your organization’s health.
Intrinsic impact is mission fulfillment, and mission fulfillment is increasingly core to the assessment of success. Mission fulfillment is about changing lives, creating memories, transforming people. It’s about providing the stepping stones through an artistic life, the touch points that bring us back to the world when we get caught up in things that don’t really matter.
We can know more about the consequences of what we do, the impact of what we do, the changes we make in the fabric of lives.
We can measure transformation. We can.
For the sake of the field, and the betterment of the people we serve, we need to get started counting some new beans, and we need to teach the people who control our funding how to understand the worth of those new beans. The drums are beating; our time may be short. But art is not optional. Art is life. It changes our rhythms, connects us to our humanity, teaches us to live and love. It makes us smarter, stronger, more coherent. It makes us care, and think, and innovate. It can, indeed, transform the way we think and act and live our lives.
So let’s put that center stage, and see where it gets us.
To order the entire anthology, Counting New Beans: Intrinsic Impact and the Value of Art, including the full-color final report from WolfBrown as well as new essays from Diane Ragsdale, Arlene Goldbard, Clayton Lord and Rebecca Novick, forewords by Ben Cameron and Brad Erickson, and interviews with twenty-four artistic directors and patrons including Diane Paulus, Todd Haimes, Oskar Eustis, Bill Rauch, Martha Lavey and more, or to view summary results of the research, visit theatrebayarea.org/intrinsicimpact. This research was generously funded by the Doris Duke Charitable Foundation, the Andrew W. Mellon Foundation, the National Endowment for the Arts, the Pew Center for Arts & Heritage, Theatre Development Fund, Arts Midwest, theatreWashington, LA Stage Alliance, the San Francisco Arts Commission and the City of San Jose Office of Cultural Affairs, and was commissioned by Theatre Bay Area.
The article is just the start of the conversation—we want to know what you think about this subject, too! HowlRound is a space for knowledge-sharing, and we welcome spirited, thoughtful, and on-topic dialogue. Find our full comments policy here
Clay- this is such important work you guys are doing. Thank you for your passion, tenacity, and clarity around it. One small comment here, then I look forward to engaging in the larger conversation around all the various platforms you have engaged for advancing these ideas. You write here and elsewhere that "mission fulfillment is increasingly core to any assessment of success." I think, in fact, it is more about RETURNING to the truth of this than it is about discovering it. The people who blazed the trails we're walking on now, in particular with respect to institutions and ideas about the "why" that Diane Paulus is talking about over on the blog here, have always known mission fulfillment was the only real measure and guide. We've wandered away from this core purpose- toward these other "beans" as you say, as the field has corporatized and our artistic inquiry commodified, but mission was both the basis and the point at the outset- and the path to fulfillment could only be discovered in the transformational interactions within a community of artists and audiences.