VR180 audio: stereo vs. binaural vs. ambisonic

[gt3rs]

I had my ah ah moment while trying Vision Pro and watching Adventure a couple of months ago.
I then rented a Canon dual fisheye to use on my R5c, borrowed a Quest 3 and started experiment with some sports that I cover and so far, people are impressed and surprised.
Will this time take off?  I think we are still too early, but the experience is getting better. I wish Vision Pro would be priced lower and be more open to load content on it, something that with Quest 3 is super easy to do.
 

I kind of grasp now the filming, editing and delivery other than sound. I need ambient sound of the sport event, no dialogs etc... I would like to create a bit of the feeling to be there BUT I don't want to spend a lot of time managing the audio. KISS principle as for now is just a side thing. 

I see two options, simply using a Stereo Mic or using a Zoom H3 VR and use the "straight out of the device" binaural (not sure how it can be binaural if the mics are so close together).
I'm not too interested in dealing with the various ambisonic formats, editing, metadata etc... seems a huge pain and time consuming.

Would people perceive a big difference between a stereo mic and the H3 binaural? 
 

Any other ideas/opinions?

[BTM_Pix]

I did a first impressions thing here when I got my H3-VR.

It has examples of the auto down mixed binaural output for general city ambience and you can download the original ambisonic files to see the extent of the steering that you can do with them in post if you every wanted to.

I think it obviously punches above its weight and even in binaural mode it does add more pickup width outside of the visible frame as well as a certain degree of rear and height cues so for 180 video I think it might do the trick for what you are after without having to do anything in post.

The step up would be the Rode Soundfield mic paired with something like the Zoom F6 or even the new H6 Essential but, obviously, you will then need a more elaborate rig versus the diminutive H3VR and, of course, with no down mix you'd have to do that in post.

 

[kye]

One question that stood out to me immediately was how much you want the audio to move around when the person in the VR turns their head.

For example, if you record stereo and the VR person turns their head the visuals will all move but the audio won't change at all - I would imagine that to be unnerving and potentially ruin the immersion wouldn't it?

Does VR have a standard where you can record to that and then the headset will decode it to match where the viewer looks?

[sanveer]

Apparently this is very popular among people who are into paranormal activities and looking to recording on-field audio 

[ac6000cw]

17 hours ago, gt3rs said:

I see two options, simply using a Stereo Mic or using a Zoom H3 VR and use the "straight out of the device" binaural (not sure how it can be binaural if the mics are so close together).

It's possible to synthesize all sorts of microphone types/polar patterns, including things like perfectly co-incident stereo pair mics (which are impossible to physically build), from a B-format Ambisonics stream/recording -  https://en.wikipedia.org/wiki/Ambisonics#Virtual_microphones - and also produce a binaural stream - https://en.wikipedia.org/wiki/Ambisonics#Decoding

I think that post-processing flexibility is the real strength of using a Soundfield microphone for ambient sound recording.

[gt3rs]

On 4/27/2024 at 3:46 AM, kye said:

One question that stood out to me immediately was how much you want the audio to move around when the person in the VR turns their head.

For example, if you record stereo and the VR person turns their head the visuals will all move but the audio won't change at all - I would imagine that to be unnerving and potentially ruin the immersion wouldn't it?

Does VR have a standard where you can record to that and then the headset will decode it to match where the viewer looks?

I'm really a nob in this. But for VR 360 you are right, you need ambisonics, as your head may turn 180° the audio would come from the opposite side confusing you a lot, many VR players support some sort of ambisonics audio so they can move the audio following your head. 

From my limited research and understanding there are multiple standards so you may end up doing different edit+metadata for Youtube VR than for local players Meta Quest TV , Skybox etc.... Apple Vision Pro?
Also I think Resolve 19 now support ambisonics but not sure if you can do all in there, for sure not metadata injection.

Now for VR180° your head my turn max 45° as more than that half of your video will be black so it seems that you can get away with binaural or even stereo. 

I would prefer a stereo mic like the Sennheiser 440 or Rode Stereo Mic as they are not as tall as the H3 so I can use it on a gimbal.

[ac6000cw]

19 minutes ago, gt3rs said:

I would prefer a stereo mic like the Sennheiser 440 or Rode Stereo Mic as they are not as tall as the H3 so I can use it on a gimbal

Never done any VR stuff, but a few thoughts/ideas:

You could try a using stereo mic with a 120 degree (instead of the more usual 90 degree) angle between the capsules. That will give you a wider, more diffuse soundfield. Some stereo mics that use mid plus side capsules internally have a switchable 90/120 option (and/or an option to output the raw mid and side signals instead of L and R, enabling you to choose the virtual capsule angle in post using a mid-side to stereo conversion plugin).

Stereo field manipulation/enhancement plug-ins might also be useful to widen the soundfield and create a more spacious effect e.g. I've used  iZotope Ozone Imager (free), Nugen Stereoizer (in full and cut-down 'elements' versions), various Melda Production plugins to do this and/or create pseudo-stereo from mono sources.

[kye]

2 hours ago, gt3rs said:

I'm really a nob in this. But for VR 360 you are right, you need ambisonics, as your head may turn 180° the audio would come from the opposite side confusing you a lot, many VR players support some sort of ambisonics audio so they can move the audio following your head. 

From my limited research and understanding there are multiple standards so you may end up doing different edit+metadata for Youtube VR than for local players Meta Quest TV , Skybox etc.... Apple Vision Pro?
Also I think Resolve 19 now support ambisonics but not sure if you can do all in there, for sure not metadata injection.

Now for VR180° your head my turn max 45° as more than that half of your video will be black so it seems that you can get away with binaural or even stereo. 

I would prefer a stereo mic like the Sennheiser 440 or Rode Stereo Mic as they are not as tall as the H3 so I can use it on a gimbal.

That all makes sense, and like most things, it might be a while before all the tech is handled correctly so it just works and you don't need to troubleshoot the process at each stage.

I think that 180 VR video is likely to be a winning format.  In 360 video people don't really know where to look, and some of the experiments I've seen with it were really hit and miss if there was a narrative arc that you were meant to be experiencing.

Also, all these discussions were had when surround sound first came out and people didn't know what to do with it.  People were genuinely talking about mixing live concerts in such a way that put the listener in the middle of the band on the stage, or floating above the band looking down on them like you were in a box seat, only it was a live outdoor concert and there was no seating, etc.  
Eventually people settled down and realised that for the most part people don't really want those things, but they do want a "if I had been there" sort of experience.  So surround audio is mixed like you've got a great seat in the concert, and if there's picture too then the audio is mostly oriented around the point-of-view of that.

So, if it's a choice between supplying 180 video and a stereo mix, or 360 video and full ambisonic mix, just to be able to look behind you at nothing of any significance whatsoever, I think many will opt for the first one!

1 hour ago, ac6000cw said:

Never done any VR stuff, but a few thoughts/ideas:

You could try a using stereo mic with a 120 degree (instead of the more usual 90 degree) angle between the capsules. That will give you a wider, more diffuse soundfield. Some stereo mics that use mid plus side capsules internally have a switchable 90/120 option (and/or an option to output the raw mid and side signals instead of L and R, enabling you to choose the virtual capsule angle in post using a mid-side to stereo conversion plugin).

Stereo field manipulation/enhancement plug-ins might also be useful to widen the soundfield and create a more spacious effect e.g. I've used  iZotope Ozone Imager (free), Nugen Stereoizer (in full and cut-down 'elements' versions), various Melda Production plugins to do this and/or create pseudo-stereo from mono sources.

Having a wider spread seems sensible - if someone else hasn't done a bunch of tests then it would be relatively easy to do once you have the equipment.