360° video and Virtual Reality

[HelsinkiZim]

Maybe I am a bit dim, but can someone explain the difference between 360° video and VR... or are they the same thing?

Is it like how corporate video production is content creation, or commercials are branded storytelling - essentially millennial terminology that is 'sexing-up' producing a shoot with a 360° camera?

edit: or is VR the technology that allows you to pan and zoom into a 360° video by moving your head (like you are a human mouse)?

I am honestly a bit confused, I am looking into it now but any thoughts from someone who knows would be appreciated.

[HelsinkiZim]

... also, does anyone see how a narrative would work in VR unless the audio changed with your perspective and proximity to what you are looking at? Is this something you can do... if you could, then it would be amazing to have mutiple things happening at once and you can walk around a scene and check out different storylines and view points.

[bunk]

1 hour ago, HelsinkiZim said:

I am honestly a bit confused, I am looking into it now but any thoughts from someone who knows would be appreciated.

A 360 camera allows you to play google maps in your own place. Google maps uses static footage (photos), the 360 camera records moving pictures. If you use four cameras yourself and stitch the footage together you get the same effect. Problem is you need something to project it on. Either it will be projected on a 3D cylinder in your computer and you can rotate a 3D camera within the cylinder ( the google maps way) or you need a special build theater to project the footage. some years back Peter Pan was performed in London on stage and the complete theater was enclosed by a 360 screen on which animated environments -rendered in a 3D program- were projected.

VR is basicly the same as a modern video game. The environment is created in a 3D program and imported into an engine that lets you navigate through that environment. The environment is rendered on the fly. And that is the reason you can remove 'reality' from virtual as it looks very artificial. Current computers and graphic cards are not fast enough to render reality in real time. But at least it's kind of "3D" ( you're still fooled with projected textures).

Oh and there is no such thing as a 3D movie or a 3D tv. That is stereoscopic we talk about and it has nothing to do with 3D. There are 3D movies but only because they were made with 3D programs, think of 'ToyStory' or 'Horton hears a who'. Also most fx shot are done in 3D programs nowadays. The footage from 3D programs is as flat as the footage coming from your camera and nothing in the world can change that ...at least not in the years to come.

[Cinegain]

You don't have to consume 360 video with VR goggles neccessarily.

However, if you want to immerse yourself into a world where you are the center of the 360 video by wearing a pair... that would be virtual reality.

Did you actually watch that Mini VR video I mentioned in the other topics? I think it's a good example of what touches on the possibilities of such a thing. Remember, this is a venue that is still pretty new and people are figuring out. Most videos just take you on a documentary type journey where you can follow along and look around, but that's just about it. It's more interesting when reality meets a first person video game concept. I can imagine some kind of Cluedo set-up. Where you perhaps hide some hints about who killed someone and if you were looking closely you might've spotted those clues, if not, you end up with different suspicions and a different experience. You could re-live it afterwards and change it up, suddenly have a whole other experience. You make your own stories. But you're limited to tech, especially the camera's location, you can't have your viewer just freely roam around. They control what they're looking at, but not where they are (because you can place the camera somewhere, but not everywhere). That would require an actual virtual reality, not a man made one. But yeah, there's still some unexplored territory here.

[bunk]

50 minutes ago, HelsinkiZim said:

... also, does anyone see how a narrative would work in VR unless the audio changed with your perspective and proximity to what you are looking at? Is this something you can do... if you could, then it would be amazing to have mutiple things happening at once and you can walk around a scene and check out different storylines and view points.

VR doesnt excist outside the computer. You can however render sound in 3D. Might be possible in VR as well ( on the fly)

[HelsinkiZim]

1 minute ago, bunk said:

VR doesnt excist outside the computer. You can however render sound in 3D. Might be possible in VR as well ( on the fly)

I think I wasn't clear, I was referring to sound in the headset - in the VR world, so if you turn your head and walk to another part of the room the sound of, say, a running faucet in the kitchen gets louder the closer you get to it. Or a conversation in a corner gets clearer when you move towards the chatters. I think for narrative human based VR, sound management would be key for an immersive experience... probably more so than anything else.

13 minutes ago, Cinegain said:

 

Did you actually watch that Mini VR video I mentioned in the other topics? I

I'll check it out now. The gamer benefits are pretty clear as VR is a natural progression from what have been developing in that arena for a long time. As to narrative work, as you mentioned, a 'choose you own adventure' style movie or short (with real humans interacting) would be a genuine breakthrough. Some people tried it with dvds and first person POV (I know porn tried it a loooong while back with My Plaything DVD series and it sounds like they are going to be the first to commercialise VR, so we should see what happens).

[Cinegain]

Speaking of which...

 

[IronFilm]

"Virtual Reality" is a very new term and can be a very broad term. So some people could arguably classify 360 degree video content under "VR", but other people who are VR snobs wouldn't regard it as "proper VR".

For "proper VR" they'll say you need to not just have 360 degree vision around yourself, but also have complete freedom to move through the space you're in as you wish (just like you can do in real life). 

So how can you create that freedom of movement?  Well you'll need to create all the items in that space (& the space itself) painstakingly from scratch, just like they do with video games. This can be very difficult and expensive (this is why video games can cost hundreds of millions of dollars to make, just like big blockbuster movies do).

And the downside is it still won't truly look life like!! Instead it will look like you're in a video game. 

360 degree video has the downside of you lose the freedom of movement, you are instead constrained to take the same path of movement through the environment as the filmmaking took during capture. It is like you're on an amusement park ride, once you're strapped in then you're going to go through it the same way the rides are going to go, following the tracks. 

But the upside is 360 degree video has the benefit of looking waaaaay more realistic, as you're filming real life itself! Also 360 degree video has the benefit that it is relatively "easy"-ish to create content. Well...  still quite tough! Just check out my other thread where I'm discussing hardware requirements:

Then you have software needs as well! Which are themselves quickly a very complex issue, and can be very expensive to purchase. 

A slight tweak to this 2nd approach is to do 3D 360 videos, this gives you stereoscopic vision in your VR headset, rather than seeing just a "flat" looking video. A potential big improvement for sure! However.... this means double as many cameras, which means more the double the hardware complexity, and more the double the difficulty in post production.

But all of this is a walk in the park compared to trying to do "proper VR" in a truly photorealistic manner. For that you need a capture rig kinda like the bullet time rig from "The Matrix", but even more complex and comprehensive. Then a vastly more complex software step to turn each capture into a moving 3D model with the filmed texture overlayed on top. 

Another big downside of this approach is you can't do large/complex scenes at once. For instance if you wanted a VR experience of a party via this method, you couldn't just a capture a crowd of people drinking, dancing, and having fun. Nope, instead you'd need to do each person one by one, then insert them one by one into the scene you're creating in the computer. On the upside, it does make it heaps easier to edit the scene in post if you wish to move or even remove elements from the scene! Want to move that idiot with the jerky  spazzy dancing to the back of the scene instead of at the front? Done!  

Maybe the promise of "The Edit Button" will come true one day... 

 
The other downside to this approach of using videogrammetry is that currently the quality you get is usually absolute rubbish, here you can see one I took of myself last month (it is a moving 3D model, which you can fly around it as you wish, or place it anywhere within a 3D space you create in Unity):

As you can tell, the quality looks like absolute trash compared to what we're used to from our BMPCC / URSA / Sony FS7 / Samsung NX1 / etc cameras!! However, that was just a very rough and quick test capture I did (for instance I didn't bother with lighting it at all), and I could significantly improve this. But it still is at best rather bad, unless you wish to pour millions into this! And even the companies which have tens of millions of dollars of funding behind them still are not hugely better than this, and have a looooong way to go. But I expect "soon" this approach will be mainstream (it is an open question as to when "soon" is, maybe just a couple of years away? Or maybe a decade away.... ).

So each of these three approaches have their own serious pros/cons list to consider, and I wouldn't consider any one "better" VR than the other. They're just different tools to use, depending on the task at hand to complete. 

 

Looking into the future, I think that although the 3rd approach is clearly the most undeveloped of the three, as tech improves it will likely become the main choice for creating high quality VR content. But all three approaches will remain valid approaches to VR creation for years to come.