Reverse-Engineering Zoom with Isadora

Jared Mezzocchi has been systematically exploring what Zoom and the editing program Isadora can offer theatre practitioners during the pandemic. He has been working in the field for over a decade and has explored it from multiple points of view—designer, director, playwright, artistic director. As a theatre educator, he launched an MFA program at University of Maryland in projection design, and he recently designed and co-directed Russian Troll Farm, the highly acclaimed Zoom theatre production. We Zoomed in November to talk about the production, how to reverse-engineer Zoom, Isadora as the way forward, and more.

Barbara Fuchs: Tell me a little about the background you brought to this digital moment, to when COVID-19 shut theatres down.

Jared Mezzocchi: My ongoing mission has been: how can we organically bring digital technology into an analog space, if, first and foremost, the priority is analog storytelling. With the right sensitivity, I think one can use technology to enhance the analog space. When COVID hit, I immediately realized, “Ah, the tables have turned. Now, we are a digital space.” And, just as importantly, we have to find the analog in it. It’s a different stage, but the storyteller remains the same. And we have to figure out ways to keep that intact.

We called Russian Troll Farm a site-specific performance for the internet. We asked the same questions we would have if we had walked into a freezing empty warehouse with one outlet and no seating. We could say, “This is an impossible place to create performance” or we could say, “Okay, there’s one outlet. How much power does that pull? How many people does this space fit?”

On Zoom, we have to ask those same questions: What are the ins and outs of it? So I spent the entire summer reverse-engineering Zoom. “What does each button in the preferences mean?” Not what it does, but what it signifies.

Barbara: You’re creating a sign system for yourself, so that when a need comes up in production, you’ll know what the platform can do?

Jared: That’s right. Take hiding non-video participants. That’s not just a switch. It’s a potential tactic that we could use in storytelling to turn things on and off, and to be able to see that people are hiding in the space. That’s interesting, dramaturgically, even if I don’t have the script for it yet. I would much rather understand that than use Zoom to perfect one show. If I tell the actors to turn their camera on or off without giving a reason, they can’t be malleable in the space.

I’m very proud of Russian Troll Farm in the sense that we spent the first couple of weeks in calisthenics of Zoom know-how. It’s about technique, and we’re in such survival mode right now that no one’s thinking of technique because we’re thinking this is a band-aid until we get back into the theatre space. And I call foul on that. This is a really great opportunity to hone our craft, to tell stories no matter what condition, as site-specific performance.

Barbara: As theatre has moved on to Zoom, what have been the barriers for people to achieve this integrated mode you’re describing?

Jared: I’ll be the first to say that working on Zoom is exhausting and really hard. We’ve got to completely reinvent the whole process. Part of that is introducing the audience to a set of expectations and conventions—unlike theatre, unlike film, even unlike what they know of Zoom—so they can settle into a different, uncomfortable space. Right now, people are coming to the table thinking, “I know Zoom, I expect a very particular product.” To have that in your back pocket while creating is important. Not just to defy it, but to say, “Let’s start there. Let’s really work there.” And then lead the audience away from that.

We asked the same questions we would have if we had walked into a freezing empty warehouse with one outlet and no seating.

Barbara: Could you give me some more examples of the reverse-engineering of Zoom, of trying to identify the dramatic potential in the different functions?

Jared: It’s a little different with every update of the platform. To realize that I, the co-director and designer, can be in gallery and you, the performer, can be in speaker mode is also very helpful, so that we can set up our collaborative space for our specific needs. For Russian Troll Farm, I spotlit myself, so that all of the actors would see my spotlit screen, which showed them the final designed output of the show that audiences would be seeing. This was also helpful for all of the creative team, so that we could be responsive to a final product. Then I would remain in gallery mode so I could see everyone like a television studio. I could pull from everybody’s feeds and put it into my software to compose the designed video in real time. So the actors were seeing a monitor of the final product while I was seeing everybody all at once, in real-time.

In every scene, everybody’s cameras were always on and they were in gallery mode. On my keyboard, I had programmed 1 through 5 on my number pad to cut to each of them. The QWERTY row was two shots, and the ASDFG row was triple shots or creative shots for that scene. We never sequenced that—I just memorized it. Every night when we performed it, if the actor playing Steve introduced a guffaw for his character in a new moment, then I would cut to him, and he would see that. So that listen and response was there. Maybe I would cut back and forth a few times so he would explore and heighten his performance. It was basically me as director in the moment saying, “Keep scratching at that. Keep going. Keep going.”

Barbara: So the actors could also see when you were cutting to them?

Jared: Yes. Basically, let me turn this on…

[Note: Jared proceeds to demonstrate the editing software Isadora in our own Zoom meeting.]

Barbara: So you use Isadora as an editing capacity to enhance Zoom.

Jared: Exactly. Zoom basically sees Isadora as an additional camera. Right now, I’m actually in Isadora, and I have a live camera on, which means I can now do whatever I need to design out the space. Isadora can capture the Zoom feed.

[Note: My image appears in Jared’s Zoom box.]

So, now, I can have you here, and we can basically shape this out. This is the first cue, and I could create a second cue, a duplicate, in which you are full screen. So now we’ve got cue one, and then we can crossfade to cue two. Inside this, I can create a mixer board that can cut between each of these, instead of it being a sequence of cues. Inside one cue, I could have hotkeys that cut between two distinct visual looks that I am generating from our zoom frames. I can be a TV mixer and can play it differently each night. It’s like a keyboard.

[Note: Jared captures a smaller version of my Zoom window within his and plays with it: sizing, color, lighting, multiplication of the image—the possibilities seems endless.]

Barbara: What is the plan for getting the word out about this way of working? This is your own artistic practice, of course, but you also seem very committed to the development of the form.

Jared: I started a company in September called Virtual Design Collective, ViDCo, which is our tongue-in-cheek way of turning COVID around. I needed helpers, but I needed to train them in it. We’re creating a website and tons of documents on technical vocabulary. It’s like, “Hey, we learned that you can use Isadora as a TV mixer. And here’s the plugin for that.”

We sent it to Isadora so they could help us outsource it to anybody who needs it. To me, it’s about helping our industry survive right now to tell stories.

We’re already ten shows in, and we have a huge website update coming in the next month. We had received the same email from like seventy people on one Zoom issue. That’s why we’re making these PDFs, because now I can say, “You have a sound issue. People can actually talk over each other on Zoom, but you have to click on these five boxes.” I wanted to make space for this because no one realizes everyone’s going through the same struggle.

Most software for theatre is so proprietary and consumer-exclusive that it either costs an arm and a leg, which no one has, or requires ten computers, which no one can fit into their personal office space. Isadora is a simple software is $550 for an in-perpetuity license that can run on PC and Mac. It’s revolutionary.

Most software for theatre is so proprietary and consumer-exclusive that it either costs an arm and a leg, which no one has, or requires ten computers, which no one can fit into their personal office space.

Barbara: Accessible, relatively cheap, no additional hardware required, yet still enables editing on Zoom?

Jared: It’s all the editing function. Inside here [continues demonstrating], I can de-saturate your image. I can make it blue. All of this one could do in Final Cut Pro or Adobe Premiere, but you’d have to render it in fifteen-minute chunks. This is live. So, just like we’re working here now, we would have actors, my co-director, and the playwright all on Zoom watching me move our video feeds into different compositions, and we would all be responding in real time, asking, “Well, what about this?” We were all finger-painting together.

Barbara: You seem pretty certain that Isadora is the way forward.

Jared: For me, yes. The QLab program ran all of our sound. OBS is a great software used to stream the content generated live in Isadora—I had it running in the background. But Isadora, as a general engine, is very powerful, potent, and flexible. The problem with OBS is you can’t feed back into the system without a significant delay. Whereas with Isadora the entire creative team and ensemble are able to see exactly where you are in real-time.

I was able to visually place one actor in front of another actor, when they are in two different Zoom feeds and across the country. I’d build this and say, “This is how you are placed together into one visual frame for the scene,” and they were able to see my visual composition of them so they could collaboratively strategize how they could move around and relate to each other as if in the same room, while in their separate living spaces.

Barbara: Would you say this restores physical cues for actors, even though they aren’t in the same physical space?

Jared: Right. And they always had the cue there. Our actor Egor is always going to sit slightly differently, and Steve’s always going to move his chair slightly differently, no matter what the spike marks are. So to offer that feedback to them so they can say, “Oh, I’m a little off and I can adjust.” That’s liveness. That’s very exciting to me.

That’s where Isadora pushing back on Zoom, and letting the actors see their story be created and consumed in real-time, is very important, as opposed to the actors performing in real time and then us editing in real time, but not being able to feed the edited product back to the actor. That feels like a conveyor belt that actually destroys the liveness because the actor is never able to see the thing. The feedback loop is really important.

Barbara: And every performance is different.

Jared: With Russian Troll Farm, from start to finish I was sitting there with the actors, and every night was a different cut. A lot of people ask, “What’s the point of that? Just make a film.” But there’s something in the dust of that space, in the liveness, that I can’t stress enough. It’s this synergy. We are theatre artists—liveness is our expertise. Why turn that off to go into a different field and be filmmakers if the thing we have that no one else has is liveness? Let’s put it to the test and figure it out.