Back to stories

Orpheus VR project: Pushing the limits of opera

Vancouver, British ColumbiaVancouver (British Columbia)

Story Seeker: Anju Singh
Person interviewed: Debi Wong, Artistic Director
Interview date: August 3, 2021

Debi Wong, Founder and Artistic Director of Vancouver-based re:Naissance Opera, fell into the opera world through her vocal studies at the University of British Columbia. Her interest in opera was piqued by the discipline’s collaborative and multidisciplinary aspects, which incorporate a variety of experiences, backgrounds, and levels of expertise to execute successful and innovative productions. Debi founded re:Naissance Opera in 2017 to “produce modern-day operatic performances” that bring diverse people and communities together. It’s no wonder, then, that Debi’s company dove headfirst into an innovative tech collaboration to produce Orpheus VR, a virtual reality opera.

Orpheus VR “stemmed out of my own love for video games,” said Debi. “I grew up with every single game console you can think of, starting with the Super Nintendo. The classic Super Nintendo was my first one, and up to now we play [PlayStation 4] and my Nintendo Switch.” As a more traditional Western European art form, some of the approaches to producing opera are rooted in methods developed more than four centuries ago. Debi wanted to find new ways to engage audiences and encourage participation, play, and collaboration in opera making, thereby creating an interactive or “gamefied” experience for audiences. A conversation with a friend encouraged Debi to reach out to a virtual reality artist. After their initial discussions, she decided to “do it in VR and make it an immersive experience”, she said.

The Innovation: Gamifying opera

Debi’s passion for new ideas and finding ways to engage audiences fuelled her drive to pursue the creation of a VR opera. This innovation involved many layers for the company. As Debi noted, “If I were going to create a new piece of opera for a proscenium stage using the tried and true methods that have been used for a very long time, I would have a composer, a librettist, eventually I would bring in designers, conductors, stage directors.” Creating a VR opera, on the other hand, required a completely different process and approach:

Because we’re creating an interactive 360° experience, we have to really be in tune with what we think [audiences] are going to do, or how we think they’re going to engage with the piece, so that they don’t get lost in an endless VR world, or they don’t get confused and bored and just quit, or they can still understand the story. Because they have agency in the story.

Photo by Brian Topp. Photo of 3D artist Conrad Sly sitting on a stage in the centre of the photo wearing a zebra striped shirt and a virtual reality headset facing Debi Wong in the bottom left with a view of only the back of her head as she faces Conrad. The room is dark with a blue light and there is a computer monitor with the virtual reality imagery on the monitor in the background.

The creation of a VR opera requires a clear understanding of, and focus on, the intended audience. “The audience is a very crucial part — they are co-creators in a way; — that’s the experience we want for them, said Debi. “So we are very much thinking about the audience all the time, and testing with audiences all the time.”

This audience focus had an impact on the musical composition process, which differed from Debi’s experience with traditional opera development. The sound composition, created by Brian Topp, was entirely electronic and modular — designed in short sections, like in a video game — to allow for different audience reactions, such as the varied time spans that audiences would decide to spend in a scene (from 30 seconds to 5 minutes). It was also necessary for script writers to work in a way that predicted audience behaviour, finding creative ways to help audiences through the experience and to pique their interest throughout the piece.

Costume design was another area where the process took a detour from the more traditional opera-making approach. With avatars, the costume designer was actually a 3D artist, Conrad Sly, who would mold the characters using digital tools and brushes. For Debi, it was exciting to witness the artist wearing a VR headset to “create the world around him, painting the ground below and sky above”. Alongside Conrad, Orpheus VR’s creative team included a programmer/developer, Youhan Guan, with experience building video games, as well as an animator/motion capture specialist, Neel Nair, who played a role similar to a stage director and choreographer (i.e. instructing the performers on what to do).

While the ideation phase for Orpheus VR began prior to the pandemic, the interest in a virtual reality opera picked up significantly when in-person productions were essentially shut down. This turned out to be an opportune time for re:Naissance Opera to take on the project: it could focus on the VR project and make the production happen. The company completed a prototype of Orpheus VR that is currently being shown and tested alongside a companion program, “Live from the Underworld”, which features livestream performances using mythological avatars.

Debi noted that Orpheus VR has reached a different demographic of opera audiences. While in-person opera audiences tend to be over 55 years of age, Orpheus VR reached a primary audience between 25 and 44. re:Naissance Opera was pleased with this shift.

The company embraced the technology sector as a source of potential collaboration and new opera audiences. Debi shared her feeling that:

The tech sector is so creative, and the ways they go about problem-solving, and creating, and innovating, the processes for that are just things we don’t learn about in the classical music world. Just simple things like ‘design thinking’. I had never heard those two words together before I started working with people from the tech sector. It’s just interesting to find new ways to think about your own creativity from that kind of collaboration.

The choice of VR technology for this production was driven by its capability to deliver immersive audience experiences, with voice, environment, and movement all included in the expression of the piece. The company engaged a motion capture studio to fully grasp the vocal quality, voice, and body movement nuances and personalizations of the singers. In that way, these elements could be infused into the singers’ VR characters.

As motion capture technology quickly developed, re:Naissance Opera secured funding to buy a motion capture suit linked to an iPhone that would capture the performers’ voices, facial expressions, and movements, and then broadcast their avatars. Currently, re:Naissance Opera is using a video game engine called Unreal to create livestreams alongside the Orpheus VR project. This in-house technical production capacity provides re:Naissance Opera with substantial independence and flexibility to experiment and innovate without taking on considerable cost.

The Challenges: Experimenting with an emerging technology

One of the project’s main challenges is that VR is an emerging technology. There have been many steep learning curves where there was no precedent to rely on. The re:Naissance Opera team viewed learning as part of the excitement of the project, but this also added unanticipated time and costs to the project.

Choosing which headset to build for was a daunting task. After extensive research, the team settled on the Oculus headset due to its mobile feature, which eliminates the need for it to be plugged into a computer during use. This headset later became one of the more popular consumer VR headsets, which makes Orpheus VR accessible to a larger potential audience.

Photo of Eurydice Live From The Underworld. Photo by Brian Topp, Edited by Rebecca Smit. image of an artist wearing a black motion capture suit with a red stripe holding their hand out in front of themselves in front of a background of a pink, blue, and yellow abstract animated scene with a character/avatar in the forefront that looks alien with green pants that is holding its hand out the same way that the artist is.

Accessibility is an ongoing challenge, particularly the affordability of headsets. The Oculus costs between $300 and $500. To address accessibility concerns, re:Naissance Opera makes headsets available at their events for audience members who need them to experience the opera. While the headsets are a technical element of the piece, Debi felt it was important to maintain the spirit of experimentation in the creation of Orpheus VR. As such, she was not interested in being tied to one particular technology. Rather, she used the chosen technology to express her vision. For instance, “Live from the Underworld” — the livestreamed avatar opera — doesn’t require headsets to experience. Debi is exploring a move to multi-platform access points, including building the Orpheus VR world online, in a social media setting, and in other exciting ways that have yet to be dreamed up.

Another challenge has come from conventional views of the nature of opera. re:Naissance has sometimes come across criticism like “that’s not opera” or “that’s not classical music”. However, Debi believes that innovation and experimentation with the art form allow new ideas to flourish and provide new ways to grow opera, rather than taking away from or tearing down what has come before.

The Financials: Getting funders and in-kind support on board

re:Naissance Opera credits the Canada Council for the Arts as the first funding body to take a chance on the virtual reality opera project through a small research and development grant. As is often the case, it became easier to secure more funding for the project once the initial funder was on board. Creative BC’s Interactive Fund quickly followed with significant funding.

the Sawmill, a Vancouver-based motion capture studio, provided partial in-kind support in the form of expertise, consultation, and studio time. This support gave the project a level of technical skill and capacity that would have added significant cost.

Finally, the BC Arts Council provided a grant for the motion capture suit and other equipment needed to create content. The Support for Workers in Live Arts and Music Sectors Fund from Canadian Heritage allowed the company to include more artists in its productions.

In terms of earned revenue, the company offered a tiered ticketing system, allowing attendees to choose to pay more or less for tickets. re:Naissance always includes a pay-as-you-go option as well.

While the project received significant funding and support, Debi reflected that the team could have used much more, given the need for studio time, technical equipment, and a large team. The company was able to create the piece thanks in part to many in-kind hours from the creative team.

The Takeaway: Collaborating to build bridges and bind communities together

Debi is interested in using the collaborative nature of opera-making to find new ways to create, bring new perspectives and ideas into the discipline, include artists from different disciplines, and engage new audiences. She views these innovations as an opportunity to diversify the artform’s offerings. She also feels that this way of working “builds bridges between other industries, in the spirit of opera as a collaborative tool to bind new communities together and build new bridges between communities of artists and people.”

re:Naissance Opera’s innovative, boundary-pushing approach to opera makes it an exciting company to follow.

Photo by Brian Topp. Photo of Debi Wong and Mireille Asselin embracing one another and smiling facing the camera while wearing motion capture suits in a studio with computer equipment and artists in the background. There are boxes marked out in green tape on the floor.

More Stories

Orchestre symphonique de Montréal

The Orchestre symphonique de Montréal: It takes a village… to be resilient

The OSM team set up continuous programming, which was primarily online, knowing that the concerts…

Read the story

Alianait Festival

Alianait Arts Festival: Sounds of a virtual North

Adapting through pandemic restrictions, Alianait decided to take the festival fully online for th…

Read the story

Fredericton Playhouse

InterMISSION Residency Program: Respond first, finetune later

In June 2020, with touring at a standstill and the venue sitting dark, the organization had an op…

Read the story
Learn more about the project
Residents at Tiffany Village Retirement Residence in St. John’s, Newfoundland enjoy a digital performance of Handel’s Messiah. Photo credit: Tiffany Village Retirement Residence.Yellow Objects. Photograph by Derek Chan.