What about software stabilization?

[Jay60p]

There is a lot of discussion about the “wobbliness” with the new IBIS in the X-T4.

What about software stabilization?
(No doubt this has been discussed before, I just can't seem to find it)

It seems Premiere, After FX, Resolve, FCP X all use the same basic functions
such as translation, rotation, scaling, cropping etc. and have the advantage
of selecting kinds of & amounts of stabilization after the fact.

Walking shots with a 10mm wide angle on the X-T3 are much improved with
much less “wobbliness” distortion than I see in the IBIS youtube videos, with
an acceptable amount of cropping.

“Acceptable” is subjective and I’m sure a lot of people avoid post production stabilization
completely. But the more I play with it the more I like it for certain kinds of shots.

Example: a zoom shot, on a locked tripod with a manual lens.  Trying to do an extremely
slow zoom by turning the ring by hand is difficult to do at constant speed throughout.
Turn on stabilization and an unsteady manual zoom runs as smooth
as a camcorder’s motorized zoom, with ease in and out added at the start and end.
And in my testing the cropping is almost nothing.

Any other suggestions, opinions?

[no_connection]

You need to correct the lens to something that can be stabilized before you run the stabilizer else you end up with "wobble" at the edges.
This may or may not be done automatically with the software assuming planar projection so you need to get your footage to that.unless it's smart enough and have a lens profile.

Also for IBIS you need a lens that have the correct projection as well, it's not as simple as "wide angle get wobble edges" cause that is just plain wrong. It's just that the lens had the wrong projection to work with focal plane shifting stabilization. Try this, take a picture, angle the camera slightly and take another. Now try to overlay and get them to match by just moving one of them. Spoiler but you probably figured out it won't match properly. I have not looked into exactly what projection is needed but I guess it's a bit fisheye with straight line being a bit bent as an object would have the same size in the center and edge allowing a simple movement to match both of them up correctly. You can always transform it back to planar afterwards.
Longer focal lengths notice this way less since even planar is pretty "flat" in the middle.

[androidlad]

https://***URL not allowed***/steadxp-first-look-review-a-promising-camera-stabilization-system/

[kye]

OIS and IBIS stabilise the image DURING the exposure, and software solutions stabilise the image AFTER the exposure.

If you shake a camera during the exposure (which is likely to be something like 1/50th of a second, which is quite a long time) then the camera shake will show itself in the motion blur.  Once it's in the motion blur it's baked into the footage.  There is nothing that can be done.

OIS and IBIS prevent the problem, EIS does its best to salvage what it can with what it gets.

[thebrothersthre3]

Yeah I don't see either being perfect ever. Unless hardware/software gets good enough to artificially erase motion blurring. 

[Jay60p]

Yes, you need to use higher shutter speeds. My walking shot I'm referring to was shot at 4k 60p at 1/125th sec,

so there is no overly blurred frames as I was taking steps.  Plus there is more for the stabilizer to work with

at 60fps compared to 24 or 30.

But even beyond that, the FCP X stabilizer must be doing more than what the merely mechanical IBIS

is doing when shifting axes. If I set it for "SmoothCam" motion, I do get some of the corner wobbles with

my 10mm wide angle, but setting it to "InertiaCam" motion reduces that wobble distortion. Then pulling the

amount down from the default "1" amount to say, "0.5" reduces the stabilization & the distortion & cropping

but still gives a major improvement over no stabilization, and at that point I am quite happy with it.

In fact I usually find the default "1" to be overkill.

 

 

[kye]

5 hours ago, thebrothersthre3 said:

Yeah I don't see either being perfect ever. Unless hardware/software gets good enough to artificially erase motion blurring. 

It's unlikely that software will do anything usable in that regard, at least for a long time.

The general approach would be to look to frames either side and kind of interpolate what a non-blurred frame would look like, but you'd need to track levels of blur across the whole clip (as it's not a "sharp" vs "blurred" situation) and if the frames either side are blurry then there's less information available to work with.

Plus, I don't see the market really demanding this.  More likely it's a case of just using it for cameras that expose with the shutter like action cameras and drones where EIS is commonly built-in already (eg, GoPro and Osmo Action).

4 hours ago, Jay60p said:

Yes, you need to use higher shutter speeds. My walking shot I'm referring to was shot at 4k 60p at 1/125th sec,

so there is no overly blurred frames as I was taking steps.  Plus there is more for the stabilizer to work with

at 60fps compared to 24 or 30.

But even beyond that, the FCP X stabilizer must be doing more than what the merely mechanical IBIS

is doing when shifting axes. If I set it for "SmoothCam" motion, I do get some of the corner wobbles with

my 10mm wide angle, but setting it to "InertiaCam" motion reduces that wobble distortion. Then pulling the

amount down from the default "1" amount to say, "0.5" reduces the stabilization & the distortion & cropping

but still gives a major improvement over no stabilization, and at that point I am quite happy with it.

In fact I usually find the default "1" to be overkill.

Yes, with short shutter speeds EIS starts to make sense.  I'm not sure why the various stabilisers out there can't correct the wobble-vision from IBIS and wide-angle lenses, the information is all there, it's just applying the algorithms to correct warp, which themselves already exist.

One thing that caught my eye recently was the idea of using a short SS, stabilising in post, then applying a fake motion blur to emulate a 180-shutter after the stabilisation has been done.  I think for my work, which isn't for commercial purposes, that might simplify things quite a bit and might be good enough from a quality perspective.

Of course, just holding the camera steady remains the best approach, if at all possible.

[gt3rs]

5 hours ago, kye said:

One thing that caught my eye recently was the idea of using a short SS, stabilising in post, then applying a fake motion blur to emulate a 180-shutter after the stabilisation has been done.  I think for my work, which isn't for commercial purposes, that might simplify things quite a bit and might be good enough from a quality perspective.

Of course, just holding the camera steady remains the best approach, if at all possible.

I'm experimenting a bit like this using high SS and adding motion blur back as I have sometime the requirement to have photo and video at the same time. I also use remotes for photo but still would be nice to combine both so kind of compromise SS 1/250 or so... The motion estimation of Resolve is not too bad, is not like the real thing but it can fool non critical eyes. I also add motion blur for time shifts or with 360 camera where there are no NDs....
I did also experiment a bit using 5.5k 60 fps and slowing down in Resolve like it was done 120fps and not always but many times the result is much better than 1080p 120fps of the 1Dx
The EIS in the 1Dx III with wide-angle lens and higher SS works quite well especially if you are not running around or so. Slowly  moving works quite well but definitely not like on a Gimbal. 

[Jay60p]

I just tried some more walking and running at 10mm with the 10-24mm zoom, with OIS on to help.

In FCP X you can see the difference between what's happening with SmoothCam vs. InertiaCam after analysis

by reducing the scale of the frame (I did 50%) and then playing the clip in slow motion (holding down K + L keys).

In SmoothCam the edges of the frame move up, down, sidways and rotate slightly as it compensates for the

walking motion in the shot. But the frame remains a rectangle, does not distort "out of square".

But in InertiaCam the frame does move out of square with edges stretching in or outwards as the clip plays.

This may be why a 10mm extreme wide angle will stabilize better with less warp in software than in the X-T4 IBIS.

Anyway, my running test failed miserably, that's just too much violent movement for the Software to fix!

Long ago I briefly considered a gimbal but realized the way I work, I'd rarely actually go to the time and trouble of

setting up & balancing the thing. Besides I don't do much running, I more of a biker.

13 hours ago, kye said:

One thing that caught my eye recently was the idea of using a short SS, stabilising in post, then applying a fake motion blur to emulate a 180-shutter after the stabilisation has been done. 

8 hours ago, gt3rs said:

I'm experimenting a bit like this using high SS and adding motion blur back as I have sometime the requirement to have photo and video at the same time.

Are you using 24 or 30fps and what motion blur solution works for you?

[gt3rs]

I normally use 24 fps timeline (some time if is for tv 25 fps I'm a PAL country). I use the following settings: Motion Estimation always Better, Motion Ra... depends how big are movements so either small, medium or large and then I play with the Motion Blur slider until I find the spot that looks good. Again it will never be the same as real as some motion are very complex. I should try to do a repeatable test with 180° and one with 1/250 and 1/500 + motion blur in post.

[Jay60p]

On 5/7/2020 at 3:24 PM, Jay60p said:

 

In FCP X you can see the difference between what's happening with SmoothCam vs. InertiaCam after analysis..

 

I just noticed you have to have FCP X viewer in "better performance" mode, otherwise the edges are cropped in playback

and you don't see the pre-cropped movements of the frame edges.

And for what it's worth, I found the "neck strap" method of hand holding during a walk helps reduce shake.

That is holding the camera in front of you far enough for the strap to be tight around the back of your neck

and viewing the shot on the LCD as you walk.

 

[androidlad]

The best software stabilisation is achieved by using gyroscope data and calibrating/compensating for rolling shutter.

Steady XP does all of that and I think it's better than any EIS or IBIS, and it does not have that gimbal floaty look.

 

[gt3rs]

15 minutes ago, androidlad said:

The best software stabilisation is achieved by using gyroscope data and calibrating/compensating for rolling shutter.

Steady XP does all of that and I think it's better than any EIS or IBIS, and it does not have that gimbal floaty look.

 

Afaik EIS also using gyro, canon, gopro, insta etc....

[androidlad]

1 minute ago, gt3rs said:

Afaik EIS also using gyro, canon, gopro, insta etc....

Yeah but they cannot remove movement generated rolling shutter.

[gt3rs]

3 minutes ago, androidlad said:

Yeah but they cannot remove movement generated rolling shutter.

Why would not be possible as camera makers know exactly the RS plus lens data etc so they could. Anything done by external software would be possible internally assuming enough cpu power.

 

[Jay60p]

On 5/6/2020 at 5:33 PM, androidlad said:

https://***URL not allowed***/steadxp-first-look-review-a-promising-camera-stabilization-system/

This looks like a great possibility for a professional who is doing a lot of run & gun work,
and an alternative to gimbals.  It is about $300 US.

The main limitation seems to be recommendation against using mirrorless cameras that have IBIS…

“The reason is simple: the sensor is stabilized inside the camera, the position of the sensor is not precise every time, and it can screw the post-stabilization software. In fact, during the test session that we did, I wanted to shoot a couple of test footage with my GH5, and I was strongly advised not to do so.”

So the best cameras to use this would include GH5S, X-T3 but not GH5 or X-T4 or the Sonys.
And it’s programmed for prime lenses, not for zoom shots.

I’d love to see the results compared with FCP X InertiaCam. (Like InertiaCam it works best at
shutter speeds of 1/125 or 1/250, and will crop into into the frame.)

[gt3rs]

Canon C500 II, C300 III, 1Dx III, Gopro, Insta (done in post with the gyro data) etc they do what steadxp is doing in camera. Sony FX9 collect the gyro and lens data and you can do it in post with their software. Maybe stedxp is more advanced but not so sure. 

There is no magic is not gimbal like but it is ok. wide lens + good dof + high ss + good light make the result better. 360 camera as they have the full image they are just above anything with EIS but also there no magic sometime it wobbles and the more low light the worst it gets (also due to the SS going down).

[kye]

On 5/12/2020 at 3:23 AM, gt3rs said:

Why would not be possible as camera makers know exactly the RS plus lens data etc so they could. Anything done by external software would be possible internally assuming enough cpu power.

Not true - one thing that cannot be done in real-time is predicting the future.

External software can (and does) analyse the entire length of the clip and take that into account.  You probably get most of the same benefits in-camera with a 2-3 second buffer on the stabilisation mechanism, but it's not the same.

I also find that external software gives you the option to try different ways to stabilise and to optimise parameters, so that's something you get without it being baked-into the footage.

Of course, the footage with stabilisation baked-in makes better use of the codec and bitrate as it's not having to compress an image that's jumping around all over the place, so that's a win for in-camera processing.

I've also considered there to be three time-scales to stabilisation:

• movement that occurs during an exposure
  this is blur or RS that can only be prevented with OIS / IBIS or having a huge camera rig - EIS might be able to help a bit with RS but can't help with blur during normal shutter times (ie, 1/50 not 1/4000)
• movement that occurs due to hand shakiness etc
  this is what light camera rigs have and heavier ones do not have - EIS can help a little with this
• movement that occurs slowly 
  this is where the 'floaty' look can come from with gimbals and forms the aesthetic part of the equation

My experience is that stabilisation of the first two is desirable and management of the third should be done to taste.

Stabilisation is a topic that is almost always oversimplified - please resist the temptation.....

[Jay60p]

One weakness of the FCP X stabilizer is that since there is no gyroscopic information,
It only has the pixels in the image to go by. So it must distinguish between camera motion
and subject motion. With the camera locked down during a zoom shot InertiaCam has occasionally
tried to stabilize wind blowing bushes or a person passing close across the frame and that resulted In some really weird distortions.

Sometimes switching to SmoothCam fixes this, but it can be impossible to get a good EIS stabilization with too much subject motion.
That doesn’t happen to me often, but it shows one advantage of motion sensor driven OIS and IBIS over EIS.

I don’t have a camera with IBIS so I haven’t experimented with that. But I did try this with
the Fuji 18-55 kit lens:  I taped a big post card on the end of my lens support rails to
cover the camera’s view, and turned on OIS while hand holding.  The OIS lens shifting can be seen on the video of the postcard (the OIS generated the image shake), proving that the actual video image has no effect on OIS.

[gt3rs]

40 minutes ago, Jay60p said:

One weakness of the FCP X stabilizer is that since there is no gyroscopic information,
It only has the pixels in the image to go by. So it must distinguish between camera motion
and subject motion. With the camera locked down during a zoom shot InertiaCam has occasionally
tried to stabilize wind blowing bushes or a person passing close across the frame and that resulted In some really weird distortions.

Sometimes switching to SmoothCam fixes this, but it can be impossible to get a good EIS stabilization with too much subject motion.
That doesn’t happen to me often, but it shows one advantage of motion sensor driven OIS and IBIS over EIS.

I don’t have a camera with IBIS so I haven’t experimented with that. But I did try this with
the Fuji 18-55 kit lens:  I taped a big post card on the end of my lens support rails to
cover the camera’s view, and turned on OIS while hand holding.  The OIS lens shifting can be seen on the video of the postcard (the OIS generated the image shake), proving that the actual video image has no effect on OIS.

I'm probably bad, but I cant get the same stabilization that EIS is giving me in post with Davinci stabilaizer