A Virtual Reality Climate Musical with Synchronized Swimmers

Tjaša Ferme: Hey theatre, science, and innovation fans, this is Tjaša Ferme, your podcast host for Theatre Tech Talks: AI, Science, and Biomedia in Theatre, a podcast produced by HowlRound Theatre Commons, a free and open platform for theatremakers worldwide. Tune in.

Mary John Frank is a New York-based director and choreographer. MJ's film and XR works have been shown at Lincoln Center, the Hammer Museum, Pioneer Works, film festivals, and more. She has been selected to competitive tech residencies, grants, and awards from Google Daydream, IBM, Panavision, and New Inc. And she also held production leadership roles at film studios like Warner Bros. and Paramount. Wow. Today we met to speak about her VR/360 project, From Sea to Rising Sea. 

Mary John, MJ, let's talk about your trajectory, how did you get from Warner Bros. and Paramount to dancing and now leading a dance tech collective, and everything in between?

Mary John Frank: Well, thank you for the question, Tjaša. It's so great to be here, and gosh, so I think these things kind of happened simultaneously. Dancing was always something I did growing up, it was just what I did in my room when I was by myself. I went to college for dance. And when I was in college, I actually struggled with an eating disorder. And so I put my dancing on pause and started focusing on internships and other things that I was interested in, in media.

So my first job out of college was at a film company, and I was an assistant. And then that led me to another company in Los Angeles, where I was an assistant at Lionsgate. I got invited to go to Paramount with my boss, who was co-president of Paramount Vantage, which was their independent film arm. It's no longer a part of Paramount, but it was really exciting at the time.

We distributed films like No Country for Old Men, There Will Be Blood. And when I was working there, I got promoted. I worked in acquisitions, so I was going to film festivals and watching films for various territories for Paramount. And then from there went to Warner Bros. and was working in more script development. And so that was sort of my day job all through my twenties.

And on the weekends, I slowly started dancing again. I was introduced, actually at Paramount, to a woman I was working with, her name is Elizabeth Tramontozzi, she was starting a dance company for people who had danced in college and pursued other careers. And so I just started very slowly. And I would just go to take a class with this company, and then I would teach a class, and then I started choreographing, and eventually the choreography gigs started coming in. I would choreograph a music video for something or a scene in someone's film, and it started taking up more and more time.

So during the day, I was an executive at Warner Bros. for a producer at Warner Bros. Her name was Polly Cohen, Polly Johnsen. And at night and on the weekends, I was choreographing, and I had to make a decision at a certain point of which path I wanted to take.

Tjaša: This is so much fun. And how many times success kind of comes from a back pocket. Do you know what I mean, the around ways? Thank you for sharing this by the way, that you were struggling in your own ways in the dance world, and then you started working for the film production company, and then this led you back to your true passion. I mean, that's amazing, but also I feel like so many people who are into film would love to have that day job. So sometimes when you're just so headstrong and so absolutely focused on the thing that you want, it's elusive. And then when you're more flexible and you just know, I love the arts or I love culture, I just want to have a job in this sector, then it's so much easier to attract opportunities.

Mary John: Sure. And I think I was thinking a lot about as I've gotten older, where can I be most useful, where can I be of service? I think at the time in my late twenties, I was thinking, you know what? If I want to be a choreographer, this is the time! I don't want to wait till I'm forty. This is the time to go for it. And so because I had had a stable job for many years, I think I felt comfortable making that leap. And once I made that leap, there were challenges too because I was going up for jobs against people who had been choreographing straight out of the gate from college. So I had a lot to learn about what it meant to be an independent artist, what it meant to start over. Grateful I took the leap, but it was definitely scary at moments.

Tjaša: Okay. So cool. Paramount, and then you started choreographing scenes in movies, and then more. And then how did you get into the tech aspects, or how did you get into VR and XR?

Mary John: So specifically VR and XR: I'd created an immersive dance performance that was in Brooklyn. And I was thinking about, this was like 2015, 2016, I was thinking about interesting ways to film an immersive experience, because flatscreen didn't really seem like the right medium. And so I learned about 360 filmmaking. And I just organically started going to this NYU VR tech meetup, and it was at ITP and it was open to the public, and I just was really interested in it. And I started talking to people about it.

And a friend of mine who was a producer, her name is Courtney Andrialis, and we'd gone to NYU together. She sent me a grant one day, and it said something like “grant and residency unicorn.” And it was a fully funded VR project with Google Daydream when they were coming out with their VR headsets.

Tjaša: Wow.

Mary John: And they chose three filmmakers to come in and make a project that was specifically about bullying. And it was around the time of Trump's first presidency.

Tjaša: So you got the unicorn, the VR unicorn, which was fully funded.

Mary John: Project grant, and created that 360 film project. And then from there, I mean, you know, once you've made something in one medium, you know how to do it. And so while I was working on that project, I started thinking, what would I really want to see in 360 and VR? And I was thinking a lot about... I was reading a book at the time by Jeff Goodell called The Water Will Come. It was about climate change. And I was just thinking, okay, well, VR is a great educational tool, it can immerse you in environments. What's an environment I would want to be immersed in? And underwater came up first for me.

And so from there, I created From Sea to Rising Sea, which—I say “created” From Sea to Rising Sea. It was a huge team effort. It started in 2017 when I wrote the script. I actually reached out to Jeff Goodell, and he read the script and gave notes, which was incredible. And then from there, the project was born.

Tjaša: Amazing. Yeah, I love this project so much. So we went to a Musical Theatre Factory’s event where you were showing the film, and I remember that the event had started announcements or whatnot, and I was wearing the headset, I just started the journey, and there was just a moment where a swimmer swam by singing to me, “call your local governor” or somebody like that. And I just bursted out laughing. I found that so hilarious. And the whole show stopped, and everybody looked at me. So it was dramatic to see it.

But I love this project, and it's also really interesting. So I've been thinking a lot about theatre and VR and the connection. And so one distinction I figured, okay, so this is not VR in theatre. This is theatre in VR. So yeah, From Sea to Rising Sea is basically a musical, which is also a VR 360 musical, which is also a PSA.

So again, tell me more. I've also been to your website, and I'm going to recommend everybody to check it out, fromseatorisingsea.com, because you also have amazing references and calls to action. People can, even if they're into sustainability, they can look for jobs. There's some book and podcast recommendations. It's really kind of like 360. I think that when starting to work on VR projects, you start thinking about...

Mary John: Definitely.

Tjaša: ...concepts in 360, which is amazing.

Mary John: The project was actually completed in 2021. However, we've continued to exhibit it because it's in a VR headset. So we can. It's not a live show.

So when we do present it, I try to update it as we go so that things are most relevant. But there's so many wonderful podcasts and book recommendations I would highly recommend currently, What If We Get It Right? by Dr. Ayana Elizabeth Johnson. And she was a huge inspiration for this piece, From Sea to Rising Sea.

And to your point about the action items, I think that's one of the reasons I wanted to make this was to create a climate film that was uplifting. The synchronized swimmers in it are sort of climate cheerleaders. “Let's not give up, y'all. Let's go.” I really wanted it to energize people. And I loved your response actually at the Musical Theatre Factory event because we heard you laugh, and you were laughing. No one else knew what you were laughing at, but I knew exactly what... I think. I knew it was probably the synchronized swimmers during their song, which it was.

Tjaša: Definitely, oh, yeah, definitely.

Mary John: And I love watching people look down at the synchronized swimming formations when we're doing the drone shot and look all around. It's really fun to watch people watching it, and I typically can tell where they are in the film based on the direction they're facing.

But to your point also about theatre in a headset, I love that you noticed that because the way that I learned to create 360 in VR was when I was doing that Google Daydream project, the three filmmakers that made projects with Daydream, we were given two weeks at a residency in Nantucket with Screenwriters Almanack, and they had different VR artists coming through and talking to us about how they work in the space. And one of them, Graham Sack, who's a director, he had really broke down how… He had a beautiful deck, and he was talking about just principles of theatre and principles of film and principles of choreography and how to direct the viewer's eye and what to do if you have a rebellious viewer who does not want to look at the main action? What easter egg can you plant for them so that wherever the viewer is looking, there's kind of something for them? So I had really great teachers and really learned to think about making this 360 film as if I was making a live show.

And the last thing I'll say about that, is one of the greatest differences, which I think really overlaps well with dance and choreography, one of the greatest differences in doing something in VR versus a flat screen project is there's less cutting. You don't want the viewer to get dizzy, so your scenes are longer. So you're designing these longer scenes, which when you're choreographing a scene on a flat screen, I often try to avoid cutting a lot. I love a one take. So I got to play with that in this 360 medium, which is great.

Tjaša: I love it. I mean, first of all, where is that deck where he was talking about the fortes or principles in theatre, principles in filming, principles of choreography? I want to learn about that. That's great. Who does that? And then second, I think this is so fun and so juicy. We've been talking about it in a little bit of a fragmented way, so I would love for you to just tell us the story.

Climate conversations like the one we're having today are, believe it or not, part of the solution.

Mary John: So From Sea to Rising Sea is a 360 musical experience about ocean-based solutions to the climate crisis. And in the project, we find a young woman who is totally bummed out about climate change. She's reading the news, she's getting updates on her phone, and she is like, "What can I do to help?" And she has an encounter with a marine biologist, and the marine biologist is like, "Do not lose hope. There are things that we can do, and here are some of those things."

And she's not putting pressure on just an individual, but she's talking about collective things. And so that was the part you laughed at when you watched it, “call your congressperson, march and demonstrations too.”

So the marine biologist and the concerned citizen go on this underwater journey and experience this encounter with synchronized swimmers who offer up a lot of these various solutions.

And then at the end of the film, which people... I don't want to give too much away, but because it's a PSA, I think it's okay to say that at the end you see our concerned citizen feeling a little more hopeful and taking a little action. She's texting with her mom, and you see her text an article to her mom, and “let's talk about this at dinner” and climate conversations like the one we're having today are, believe it or not, part of the solution.

Tjaša: Something I'm realizing that I really loved about this project was that we were in the natural environment. And oftentimes when you work with tech, everything is based in tech worlds and about tech, and it was so relaxing and liberating. Basically, the movie starts kind of on a dock, and then you go on a ship. So before you actually even get into the underworld or the undersea where the synchronized swimmers are swimming and there's some coral reefs and some fish swimming around, how beautiful. You're actually on this boat just cruising, and there's the wind blowing, and there's this beautiful musical theatre protagonist. So there's so many things to enjoy, and I think that sometimes we forget about this, and it's probably a forte of something that you can do in this medium, in a filmed medium, still using elements of theatre, so much so that this is primarily a musical, it's a VR musical. Amazing.

Mary John: Thank you. We really wanted to show off the beauty of the natural world. And so the location scouting took quite a bit of time to find these perfect locations. And by perfect, I just mean locations where you could literally do a 360 and there isn't going to be something behind you that isn't beautiful. So we wanted to show up the ocean’s various islands, and it was such a joy to film that way. But also, I was using my skill set. I'm slowly learning Unity and Unreal, but I knew how to film things with a camera and how to choreograph, and those were the two skills I was blending together for this. So really, again, it was wanting to show up the beauty of nature and just what are the skills I have that I can bring to this medium?

Tjaša: I want to, I guess, know the timeline of realizing this is a PSA. Is it like I want to do something that's going to inspire people to climate action, and I'm going to use musical theatre as a honey trap? Or was it like, I want to do this project, and then all of a sudden it was like, actually this was also a PSA, or we could make the medium of musical theatre, and the lyric is funnier if it's just such a strong appeal to action, which then you could call a PSA?

Mary John: I'm so glad you asked that question because I haven't thought about this in a long time, but this was the third of a series called “Musicals with a Message,” so they're not all in VR, but the first one was about our smartphones. The second one was about the Global Gag Rule, which was just reinstated a few days ago, which is really tragic. And the third one was about climate change.

So I had been working for a while on various musical calls to action prompted by the thing I did with Google, which was a 360 musical about bullying and cyberbullying specifically. And again, I think it goes back to what skills I have that I thought I could bring to the medium. And I think it's easier to digest challenging information when it's given to you as a song or as a dance. I think it's easier to talk about hard things sometimes, and I think that probably goes back to my childhood and how I learned to soothe myself as a kid was singing and dancing, and my sister and I would put on plays, and I think that's what it… And actually, yeah, music is just what came out of me when I was trying to understand, yeah, when I'm trying to understand the world, I think.

Tjaša: Wow, okay, this is how you perceive and think about the world. This is how you render. Bad joke, why am I trying to describe…

Mary John: I like your tech joke.

Tjaša: ...human processes, a technical parallel.

Mary John: How do you render? I know you're asking me questions, but I'm curious, how do you render, how do you process?

Tjaša: I feel like I run all the time. I'm incredibly kinetic. I bike, I listen to podcasts, and then, yeah, I think that by writing and talking, so it's like in conjunction with my body is the releasing organ feeling and releasing, and then my brain and my mouth come into massage these things so that they transmute and maybe transcend from just being felt notions, and they become tools or ways to think and act within the world.

Mary John: I think writing is such a great tool. I also start my days with writing. So I'm not starting my day jumping out of bed singing. I’m doing Artist Way, three pages. And you're right. I mean, I think it is things spilling out on a page and then perhaps it gets refined.

Tjaša: I was in this artist residency, and I remember how it was almost like somebody wanted to convince me that editing is writing, and I needed no convincing at all, because oftentimes my way to get into my writing is that I'm just... it feels like I'm doing miniature corrections, miniature, miniature, miniature, but from this miniature, miniature, miniature, I get deeper and deeper and deeper and deeper and more and more and more and more and more. So I never doubted that editing is writing. It's just a tool to engage with your material. And if you're really engaged, it sooner or later, it pulls you deeper.

The time we're living through now is remarkable. And I bet if we look back at our notes in three years from now, we'll be like “Whoa! We were living through some stuff.”

Mary John: Oh, I love that description. I've just revisited Joan Didion's Slouching Towards Bethlehem, and there's something about the way that she writes that is so, of course, sharp, but the details. She makes lists, lists of her packing. That's the one that stands out to me. And at first you think, oh, this is just a packing list. But you learn so much about her based on what she's taking, based on what she's not taking. And so even if my writing becomes a bullet pointed list, I'm like, maybe there is value in going back and seeing what I wrote down on that day about that thing. The time we're living through now is remarkable. And I bet if we look back at our notes in three years from now, we'll be like “Whoa! We were living through some stuff.”

Tjaša: Oh, yeah. This is great. You are revealing yourself so much through your work. I also know you personally, and I know that you're politically very engaged, so I guess there's no surprise that you're basically doing politically or sustainably with some kind of an action and impact in the real world in mind art pieces. I'm curious how this translates into your Dance Tech Collective, which is a new company that you've started.

Mary John: Thank you. Well, it's very connected in that when I was working in these spaces, whether it was film, VR, AR, different types of tech and the arts, I was often the only dancer in the room. And I would, after the project was over, offer workshops where I was teaching all the things I learned to other dancers, because I felt a little bit alone in it. And I think a lot of dancers I've met and female identifying creators have said to me, I'm not techy enough to try that thing, or I'm not technical. And to me, we're all techy enough. We are using Google Maps, we're all using AI, whether we know it or not. And I just want people who are interested in working with technology to have access to it and to be able to have conversations like we're having today and critical dialogue about the technologies, not just here it is, we're feeding it to you, you must use it… or do this upgrade.

So for me, while it seems quite niche, a dance company that's making work in digital and virtual spaces, holding workshops to talk about technology, blending spaces where technicians and dancers can work together. It sounds quite niche, but I think most dancers have to engage with technology at a certain point. Even this morning I was at Gibney Dance studio moving, and there was a dancer there who... her space anyway, it got messed up, and she ended up coming to the space with me, and I ended up filming her dance, because she needed someone to film her. And there were some technical glitches, the music wasn't working on her computer, so we put it on mine. And anyway, it was just every dancer is engaging with technology at some point, and I just want to make it a little bit easier for people.

Tjaša: I love it. So would you say that the mission is to build community, educate dancers, how to use the tools of technology and effectively live in this world, inhabiting it all? Is the point in making new productions using technology? What would you say that's the main goal of creating this company is?

Mary John: Right now there's four that I can easily speak to. I'm sure there's more, but the first bullet point that comes up for me, I don't love that phrase, bullet points, but first I guess pillars, definitely creating an original work, and that's why I've been doing fundraising and various things, is so that I can pay dancers and we can work together to create something with motion capture.

I also want to commission another choreographer to also make a new work with the technology they're interested in exploring, holding these workshops. And I would love for the workshops to be subsidized. So that's another part of the fundraising.

This is a little bit more commercial, but it's kind of the work I also do, which is work for hire. So if there's a group of us who've come together, if there's a music video or a commercial or something that needs dancers, what I'm typically doing is I'll get hired to come in and choreograph or direct. And before I get to the set, I'm working with dancers in a studio, working through some stuff.

So I would love to sort of take the budget that's coming to me for those jobs and bring it into the dance tech collective as opposed to pulling it out of my pocket and figuring that out.

Tech playground and making that more official. So app creators or friends of mine that work at tech companies, they'll be like, "Oh, it'd be so cool to do something with movement." And so I end up testing their equipment, and I just see that as another potential revenue stream for movers and dancers.

Oh, and lastly, this is another thing I'm just now thinking of, career development for dancers too. So one pathway for me in terms of income has been filming and editing, and while I had wonderful teachers, there's a lot that was self-taught, and I think it would be great to also offer some career development too.

Tjaša: I'm trying to think about VR and audiences. What do you think in terms of what's coming up, what's the future, what's compelling to you, how you'd like to use it, and what do you think the easter egg, throwing that back at you, the easter egg of audiences using VR is?

Mary John: Great questions. I don't know what the future is. What I feel is personally, Apple Vision Pro is really interesting to me. I've not experimented with it. That is something that I really want to look at, not only my own work in, but other people's as well. I loved hearing Jon Chu talking about editing the film Wicked. He used his Vision Pro to watch it and to give edits. And so I'm really curious about the capabilities of that headset.

My instinct says that moving away from headsets for immersive content is what will continue to be appealing to audiences. Not everyone has a headset, not everyone can afford a headset. We made a 360 VR version, sorry, 360 phone version of From Sea to Rising Sea for that reason. I think AR through our phones, I think glasses, that's what people I'm speaking to tend to be more attracted to.

Tjaša: Yeah. So it's like actually audiences want less tech. Audiences don't necessarily want to wear headsets, which is this is exactly how I'm feeling when I go into a theatre or in an experience, I want to feel connected, and not wearing an extra lens or not wearing a headset is what makes that happen. Actually, for the most part, VR headsets make me feel really insulated. But in your case, your VR project really worked, like I said, because it was a musical within a VR world. It wasn't also looking for a connection outside of that. It's not like there's five friends who are watching it at the same time and holding hands and banging their headsets at each other, right?

Mary John: I mean, I think one of the first questions to ask if you're interested in playing with VR is why does this need to be in VR? Because there is some technology that isn't totally streamlined yet that you're going to be working with. It isn't convenient, the stitching is challenging. I mean, it is not a convenient medium. I was really drawn to it for various reasons that were very specific to my trajectory and the causes I'm interested in. So it worked for me, but every project I do is not VR. In fact, the next one I'm working on is flat screen, and I'm sure I will work on other VR projects, but it needs to align, the subject, the context, it all needs to align for me to ask someone to put on a headset.

Tjaša:. And so I guess narratives that were about important issues in the world in which you had to put the audience member into experiencing something almost from a different person's point of view is useful.

Mary John: Oh, my goodness, yes. I mean, there are some beautiful 360 films like Notes on Blindness. I think films that take you to another location, maybe a world or whether it's imaginary or real that you've never been to before, that's a great use of VR. I think VR is obviously also a great, we mentioned this earlier, teaching tool for hands-on... for doctors, learning how to do certain procedures.

Tjaša: I haven't seen any of that, but that's probably because it's for doctors, not for me per se. Very cool. Okay.

Mary John: I think Cortney Harding at Friends With Holograms has done a lot of that kind of work.

Tjaša: Okay. Wow.

Mary John: Creative work too, but also very practical applications of VR.

Tjaša: Maybe if we can get a few links to these people and their works that you mentioned.

Mary John: Sure.

Tjaša: I would love to just kind of go...

Mary John: Yes, my pleasure.

Tjaša: ...take an hour of my life and be like, let me click and let me see what these people are up to. That sounds very interesting.

Thank you so much for your time and for this amazing work, and I love talking about exciting projects like that and figuring the veils and the paths of somebody's inner mind and how their life brought them to where they are at this moment. So thanks for sharing yourself.

Mary John: Thank you, Tjaša. This was a pleasure.

Tjaša: This podcast is produced as a contribution to HowlRound Theatre Commons. You can find more episodes of the show and other HowlRound shows wherever you find podcasts.

If you love this podcast, I sure hope you did, post a rating and write a review on those platforms. This helps other people find us. If you're looking for more progressive and disruptive content, visit howlround.com. Thanks for listening, and have an amazing day.