Narrative VR filmmaking

[Jimmy]

I've spent the last few months now really getting involved in VR... Learning everything I can and testing all the top (and low end) gear. My end goal is in the app side of things... But I wanted to make a post about 360 video and where I see VR filmmaking going.

VR is still very much in it's infancy and the tech is nowhere close to perfect, pixels are easy to make out and the heavy headset, tethered to a large computer*, is not the most comfortable way to be entertained (*Vive and Oculus)... But the magic of VR is clear the minute you put a headset on. It is a stunning experience and really does go some way to tricking your brain into feeling like you are in another world... Even senses like movement and touch can be tricked.

The first experience I had was in Oculus, with a game of sorts called "Life of Lon".... After floating around in the sky for a while, the spaceship I was in suddenly nose dived towards the ocean... I literally clung onto the desk in front of me (in the real world) as the sense of speed and dropping from the sky was incredibly real. This was when it struck me that VR as a narrative means will be huge. That sense of immersion is so well suited to certain genres.

My first experience of narrative VR was a trailer for Insidious 3... It is creepy as hell and the PERFECT use of the format... The feeling of something behind you or in your peripheral vision is palpable. They are also clever in their visual and audio cues to guide you to look in the right places... Essential for a 360 experience.

This brings me onto 360 video as a narrative tool.... For almost everything other than the first person view done so well with insidious... It is just a massive gimmick. It is nauseating, unnatural and, worst of all, boring. Most of the films on the Oculus video app are misusing the tech badly, making you swing wildly around, looking for the action. Narrative needs direction, 360 video takes that out of the equation, unless in very clever hands. The other downside is that production suffers, you can no long have lights, boom mics, dollies, cranes etc.

My view for the future of this tech is a 270 degree, ultra wide format that allows you to sit in a normal chair... look all the way left, right, up and down... This would still provide complete immersion, but allow for the so called "4th wall" to remain in place, so lighting, directing, sound etc become a little less stressful, allowing for far better production values.

Anyway, just some thoughts on what I think to be an amazing tech... I urge all of you who have yet to experience it to try and find a way. It is only going to get better and better.

 

[enny]

Unless porn industry takes and runs with it will be gone like 3d film or connect . not interested in VR meh not really. I been listening to cg garage podcast some really good interview on vr. huge population has problems using it but to trow in a joke when it comes out i will be playing Euro Truck simulator  many  hours i shall cruised through the French countryside delivering my load

[Jimmy]

Try it... then you'll understand this is far, far different to 3D films 

[Lintelfilm]

Thanks jimmy, very interesting points you make. I agree about the 270-degree fov. If VR tries to do away with a century of cinematic language it won't have a place in live action narrative. 

We have an IMAX nearby and I love seeing 3D epics there. Prometheus and Star Wars 7 were mind blowing. But on a smaller screen, even a cinema screen, 3D doesn't work imo. It needs that full fov element. So I agree with you , for me that's what VR offers narrative film - immersion. And only tasteful, appropriate use of viewer directed attention when the story calls for it. 

There is no doubt in my mind VR is here to stay. For games clearly, but also for certain factual stuff like fly on the wall docs there is huge potential. As the tech and language evolves and becomes widespread it will bleed into narrative fiction. I think the big unknown is wether people will actually want to wear headsets to watch TV drama etc. If not it may just become a parallel medium mainly used for games and big sci fi films, but personally I think there's huge potential for it to enhance and build upon existing cinematic language rather than turn it upside down.

New tech is always met with suspicion and naysayers, but filmmakers of any sort would be foolish to remain in denial. VR is huge and will take a long time to grow in narrative cinema but only because it opens up such a huge new palette of possibilities. For the immediate future I think it's main benefit for films will be to make 3D a worthwhile experience in the home, but over time viewer interactivity will undoubtedly bleed into cinema and that can only happen successfully through trial and error and a kind of rebellious approach to, but respect for, the long established language of film.

Right I'm off to buy a cardboard for my iPhone!

[Jimmy]

7 hours ago, Lintelfilm said:

I think the big unknown is wether people will actually want to wear headsets to watch TV drama etc. 

Yeah... it certainly isn't gonna replace a cinema experience of something like a romantic comedy.... it will be genre specific, for sure. Sci-fi , action and horror seem the perfect genres... as well as the really good imax "experience" films. I think Pixar are already on the case too.... though a world full of kids in vr headsets is a worrying thought!

BTW... Did you see the news today that imax and Google have paired up to create what they call the first cinematic 3d 360 camera.... I can only imagine the price of that!

Now go load up insidious on your cardboard.... headphones on... time to get scared!

[Daniel Nuss]

I'm excited as well by the potential vr has. I've been playing around just with google cardboard. I was watching some simple video clips of my son taken on my iphone. When played in the vr player with cardboard, it felt like i was there reliving the memory.

What i am most excited about 360 VR for family videos is perspective! Normally my videos are of my son or wife for instance. I am not in the video since im filming. With the ability to capture in 360, i can follow my son like normal, but if my son or wife or myself wants to watch it, lets say a few years from now. I/she/he can decide to watch watch what they're doing or watch the other people who were there.

It's funny looking at old photographs of myself when i was a kid. What the photograph shows might be me in front of a monument, but what i was actually seeing while the shutter sprung was the person with the camera, but i don't have that memory anymore. Only the photo, but, if you shot in 360. You could have both! 

I'm excited that by christmas there will be a couple affordable 360 cameras that will be out. The nikon action camera and the vuze camera are the two I'm most looking forward to. 

For narrative, I agree with you that for 360 fov is too much. It only works if you have a swivel chair and there is usually too much to focus on, so it's difficult as a filmmaker to have you audience look where you want them to, so they don't miss it. Also transitions will be a mess.  I think 180 FOV is where filmmakers should begin with for the reasons you mentioned. You still feel present with the VR experience, but you don't feel like you are missing something that could be behind you. And you won't look like a total idiot if you are using vr on a plane or in public. 

[fuzzynormal]

It'll be adopted as a viable form of entertainment or art, of that I have no doubt. 

I'm not particularly interested, but there are plenty of creatives out there with the skill to make it viable. 

As all nascent tech, when it's in the hands of the engineers to create content, or, worse yet, when non-creative "creatives" are making things, it's gonna kinda suck. Early motion pictures were kinda crap for more than a few decades, for example.

Heck, most commercial entertainment in motion pictures still is kinda lame, we just have a vocabulary for the craft now.

Ideas as presented in this thread will help define the potential for VR. 

[Jimmy]

I'm really hoping that now imax are stepping in the ring.... the idea of imersive narrative (180 to 270 deg) will squash the 360 gimmick.

Leave 360 for the sports stuff... weddings... family videos etc (note that I am not knocking those creative forms...I have preordered a gear 360 for family vids)

[KrisAK]

I think the concepts of "VR" and "narrative" are fundamentally at odds with one another. Watching a narrative...2D or 3D...is a passive activity; asking people to do more than turn their heads a bit is about as far as I think you're likely to go, in terms of what people find desirable.

But the IMAX deal makes perfect sense: a 20-minute VR "Experience" of flying in space, climbing Everest, diving the Mariana trench, etc., is a perfect application. As are VR promos for conventional narratives, as with your Insidious 3 example. (I keep meaning to buy one of those cardboard things.) I'm not sure there's anything new here, though, except another step towards the "feelies" of Brave New World.

I'm also guessing that after 20 minutes, or so, some sort of VR fatigue sets in. Last night I watched the Shanghai Tower "on the roofs" video, and was simultaneously fascinated and disturbed at just how visceral an experience it was. And that was on a measly 24" screen, in a well-lit room:

   

[Jimmy]

I've done an hour on Vive without any nausea or discomfort.... the latency and comfort is great ... though Gear, and especially cardboard, are uncomfortable after a while.

The tech, latency and headset comfort will get better and better though... 

[silvertonesx24]

likely, a completely new form of art will emerge for the technology. Something that combines the gaming and film narrative experience. Imagine Myst/Riven as a VR experience...

[Jimmy]

I actually fear a little for horror VR.... I was watching reactions of people using the paranormal activity game on Vive and some were breaking down in panic.

Super real immersion combined with horror could be a recipe for disaster

[IronFilm]

On 5/19/2016 at 3:07 AM, Jimmy said:

This brings me onto 360 video as a narrative tool.... For almost everything other than the first person view done so well with insidious... It is just a massive gimmick. It is nauseating, unnatural and, worst of all, boring. Most of the films on the Oculus video app are misusing the tech badly, making you swing wildly around, looking for the action. Narrative needs direction, 360 video takes that out of the equation, unless in very clever hands. The other downside is that production suffers, you can no long have lights, boom mics, dollies, cranes etc.

 

There are many VR approaches, each approach having their own sets of pros/cons. 

So 360 has their place as a tool in your toolbox, for when budgets are lower, deadlines are tight, or you need to capture a live event (such as a concert, or sporting events), or when you want to reach the maximum number of people possible (most VR flavors are not that the accessible to the average joe, not so for 360 video! Thanks to YouTube and Facebook, and smartphones)

And you can move the camera with 360 (I'll be doing this later in the year, maybe even next month), and you can control lights via practicals (or lights out of shot around a corner), but yes no boom mics so need to rely on lavs.

8 hours ago, Jimmy said:

I actually fear a little for horror VR.... I was watching reactions of people using the paranormal activity game on Vive and some were breaking down in panic.

Super real immersion combined with horror could be a recipe for disaster

Bet people said the same about movies in general over a century ago....

On 5/19/2016 at 4:18 AM, enny said:

Unless porn industry takes and runs with it

They are. 

On 5/20/2016 at 3:58 AM, Jimmy said:

BTW... Did you see the news today that imax and Google have paired up to create what they call the first cinematic 3d 360 camera.... I can only imagine the price of that!

 

The much much ***MUCH*** bigger news from that same day which is relevant for us EOSHD readers is that Yi & Google are partnering up to produce Jump! Even though we know nothing about specs or price yet, I'd say the odds are still relatively high that I'll buy one! :D I already have a Yi 360 degree rig. 

On 5/21/2016 at 9:46 AM, fuzzynormal said:

Early motion pictures were kinda crap for more than a few decades, for example.

Exactly, don't judge the potential of VR based on the few average examples you see now. 

[IronFilm]

13 hours ago, KrisAK said:

I'm also guessing that after 20 minutes, or so, some sort of VR fatigue sets in.

Same could also be said about the early days of movie making. Many many of them were very short! 

Why? Number of reasons. Average audiences probably lacked the mental stamina to take in 2hrs of a movie, as it was so new and overwhelming. And because it was so new... you didn't need an 1hr+ long movie to wow audiences! A few minutes could do it. Additionally the crew back then lacked the technical skills to make long movies efficiently, and they lacked the creative skills for looong form storytelling. 

All these points will find analogues in our situation now of very early days of VR filmmaking.

 

[Jimmy]

@IronFilm

Thanks for the input... what are you thinking will be a fairly future proof resolution? (Once stitched).... I'm thinking high quality 8k x 4x as bare minumum, but that bring a lot of pitfalls and headaches. The Facebook rig has been the best looking footage I've found, so far... 

If GH5 is 6k ... they could make a great rig.

[Chris Bernard]

It's early days but I do think narrative VR will emerge as a new genre that someday may even become the dominant one. We're still in the early stages. Sort of like when radio shows were first adapted for television and you just had folks reading in front of microphones. The design language that needs to be developed is how do you direct attention in the narrative and how do you help people have the experience when they are in an environment where they perhaps can't move around that much. I think there are ways around some of the production limitations, at least for mid-size and big productions with virtual sets, 3D environments or even the stitching together of environments that are shot conventionally, based shooting and relighting an entire room while only capturing 120 degrees at a time for example. I think the next big breakthroughs in this space will be VR narratives that supplement traditional storytelling. Imagine a bonus section of House of Cards where you could explore the White House for example or imagine a move like Inside Out where you could sort of go on the journey yourself.

If you really want to bend your head imagine what it would be like to attempt to tell narrative stories with augmented reality. Or leveraging the potential social aspects of this technology to create shared narrative experiences. Think a virtual version of Sleep No More, an interactive theatre experience. http://www.sleepnomore.com/

I'll be paying a lot of attention to both of these realms, I think they will be huge and I highly recommend that if you ever get a chance to try these experiences that you should. Even Google Cardboard is fantastically cool.

[Jimmy]

Yea, totally agree.... It's almost better to look at it as a new medium rather than VR filmmaking or 360 filmmaking.

CG and green screening will play a big part in these early days and I think breakthroughs in 360 binaural sound will play a big part in directing the viewer. As much as I enjoyed the Insidious trailer, the visual cues were obvious...

I just watched the Conjuring 2 VR trailer and their audio cues and slight camera movements (CGI) were a more natural way to direct the eye. Once again though, it showed that horror will be a serious, serious player in VR.... It is almost unbearable. It took all my nerve to not tear off the headset... I'm not sure I could cope once we are getting towards real life visual resolution.... and i'm a big horror fan.

I've been thinking alot about the sort of multiple narrative idea too, where your choices could change the experience. As VR is quite a single person experience, for now, something as looking in a certain direction at a certain point in the film could fragment it off... completely reworking the idea of linear, directed film making.

[Jimmy]

For those with a VR device:

 

[silvertonesx24]

On 5/22/2016 at 3:18 PM, Jimmy said:

I actually fear a little for horror VR.... I was watching reactions of people using the paranormal activity game on Vive and some were breaking down in panic.

Super real immersion combined with horror could be a recipe for disaster

Maybe, but consider this from The Great Train Robbery, 1903

"The final shot of a gun being fired toward the camera had a profound effect on audiences. As cinema was in its infancy, many people who saw the film thought that they were actually about to be shot."

[Jimmy]

There may have been a few heart attacks with that too!

Though my fear is more the psychological impact.... there is something intensly creepy about the two horror trailers I have mentioned... realism and technique will only multiply this.

That's without thinking about people who will inevitably add drugs to the mix