NX1 rolling shutter in live view vs. video standby

[MountneerMan]

Why is it there significantly less rolling shutter in the live view when you are not in video standby mode or recording? Is it because the camera is doing line skipping in live view giving for faster sensor refresh but when it records it down samples the entire 6.5K image?

I noticed this a long time ago and always just figured my assumption above was correct but never confirmed it.

Here is a test if you do not understand what I am talking about. Turn off video standby and pan back and forth or just look around relatively fast. As you are panning start the standby mode and continue to pan you will see the rolling shutter (as well as video lag and noise performance) significantly increase.

The reason why I have been thinking about this some more lately is because with a possible NX1 hack around the corner I have been thinking about possibility of being able to dump the live view output to the card similar to how ML gets RAW video and thinking about what kind of image we would get.

 

P.S I apologies in advance as I am sure someone has already asked this question or the answer is already on the internet but I could not find it.

 

EDIT: so basically what I am asking is

1. Why is the rolling shutter worse in video standby then regular live view
2. Why is their more video lag in standby mode
3. Why is the noise performance worse in video standby mode

[tugela]

Probably because stills are read completely into the buffer before being processed, whereas in video the raw data stream is read directly into the processor and processed on the fly - hence the rolling shutter.

The sensor can be read fast enough to avoid rolling shutter (that is why we can take stills), but it is the processing that causes the lag. Remember that there is very little rolling shutter in FHD mode, where the processing requirements are much lower.

[gsenroc]

From what I saw in the external monitor, my NX1 was outputting 1080p 60Hz via HDMI in live view mode, then switch to 4k 25Hz in video standby mode. I guess that's the reason it has less rolling shutter in live view mode.

[Phil A]

43 minutes ago, tugela said:

Probably because stills are read completely into the buffer before being processed, whereas in video the raw data stream is read directly into the processor and processed on the fly - hence the rolling shutter.

There's no rolling shutter because stills use the mechanical focal plane shutter while video uses the electronic shutter. It's quite obvious on cameras which can shoot both mechanical and electronic shutter (A7s, X-T1, ...), one has clear rolling shutter and the other not.

[Andrew Reid]

I'm afraid all the replies so far aren't right.

The NX1 uses a power saving live view mode when not in video standby mode.

In this mode for framing up a shot, there's hardly any rolling shutter skew because the sensor is doing a very fast sweep, missing out a lot of pixels and lines. We call that binning.

In video mode it does a 6K full pixel readout for the great quality 4K recording it has. That takes time... approx 30ms for the rolling shutter to expose the entire sensor, which is what creates the skew in video mode.

Switch to 1080/120fps and it reverts to pixel binning and there's less skew.

[DPStewart]

[MountneerMan]

4 minutes ago, Andrew Reid said:

I'm afraid all the replies so far aren't right.

The NX1 uses a power saving live view mode when not in video standby mode.

In this mode for framing up a shot, there's hardly any rolling shutter skew because the sensor is doing a very fast sweep, missing out a lot of pixels and lines. We call that binning.

In video mode it does a 6K full pixel readout for the great quality 4K recording it has. That takes time... approx 30ms for the rolling shutter to expose the entire sensor, which is what creates the skew in video mode.

Switch to 1080/120fps and it reverts to pixel binning and there's less skew.

Yea thats sort of what I always thought.

So just to wrap things up

1 hour ago, MountneerMan said:

EDIT: so basically what I am asking is

1. Why is the rolling shutter worse in video standby then regular live view
2. Why is their more video lag in standby mode
3. Why is the noise performance worse in video standby mode

1. The rolling shutter is better in regular live view because the camera is pixel binning the entire sensor at a faster rate. When recording the camera is down sampling the entire 6.5k sensor so takes longer to read out each pixel.

2. Same reason as number one. More processing power is required to down sample vs pixel bin.

3. When the camera is pixel binning the noise performance is actually = (current ISO)/(6480x3645)/(1920x1080). for example if the camera is set to ISO 3200 that would be equal to ~ISO 280 in live view

 

Am I correct?

[Andrew Reid]

1. It takes 30ms to read out the entire 6K sensor. More pixels to read out.

2. What do you mean by video "lag"? Lower frame rate?

3. I don't really get this... ISO 3200 does not = 280 due to pixel binning

[MountneerMan]

by lag I mean the time between things actually happening and when it appear on the screen. If you look into the EVF of a sony a6000 you would see what I mean by lag lol.

as for the ISO thing I am not quite sure what I am thinking. I guess what I was thinking is if the camera is binning the sensor resolution(6480x3645) down to 1920x1080 than it is grouping 11 pixels into one and since ISO is related to the sensor area by having a larger area representing each pixel it would lower the effective ISO. But I am sure I am wrong now that I think about it.

Why is it that when you enter video standby the noise increases then?

[tugela]

7 hours ago, Phil A said:

There's no rolling shutter because stills use the mechanical focal plane shutter while video uses the electronic shutter. It's quite obvious on cameras which can shoot both mechanical and electronic shutter (A7s, X-T1, ...), one has clear rolling shutter and the other not.

The sensor still has to be read. If you are shooting stills at 1/30th or 1/60th sec, it will be no different from a video frame. You don't see any sign of rolling shutter effects at those frame rates, so the method being used to read the sensor has to be different. The mechanical shutter has nothing to do with it.

If you shoot in burst mode at 15 fps you get full resolution images without rolling shutter, so that sets the minimum read speed the camera is capable of in a worst case scenario.

[Andrew Reid]

6 hours ago, MountneerMan said:

by lag I mean the time between things actually happening and when it appear on the screen. If you look into the EVF of a sony a6000 you would see what I mean by lag lol.

as for the ISO thing I am not quite sure what I am thinking. I guess what I was thinking is if the camera is binning the sensor resolution(6480x3645) down to 1920x1080 than it is grouping 11 pixels into one

It doesn't group 11 into one though. It discards them. That's what binning means. They go in the trash.

Fastest way to do it.

NX1's live-view is a low power mode designed for fast refresh rates at 640x480 just for composition on the EVF and LCD.

Only on a few cameras in video mode there is summing and averaging of groups of pixels into one. Not sure which ones do it, usually the ones that do cleaner 1080p like the Nikon D750 and 5D Mark II. I also think it helped explain why the GH2 had less moire and more detail than a 5D Mark II at the time which line skipped and didn't sum or average pixels together in an advanced way like the GH2.

6 hours ago, MountneerMan said:

Why is it that when you enter video standby the noise increases then?

Because the noise isn't diluted by taking ALL pixels and doing a clever downscaling.

Noise is amplified because of the pixel binning. Whole pixels become a spec of noise, rather than an average of a group of pixels where the signal would counterbalance the noise.

[tugela]

6 hours ago, Andrew Reid said:

1. It takes 30ms to read out the entire 6K sensor. More pixels to read out.

2. What do you mean by video "lag"? Lower frame rate?

3. I don't really get this... ISO 3200 does not = 280 due to pixel binning

The whole sensor is being read all the time when the computer is on, it doesn't care if you are taking a picture at the time or looking through your viewfinder. What is happening for the viewfinder is that only part of that data set is being processed, and the rest discarded. The reduced processing time is what allows the display to work without rolling shutter, and it is the same reason why FHD has minimal rolling shutter.

Try doing something on the camera that requires additional processing, while looking through the viewfinder. You will see the image slow or even stop while the camera catches up. This is due to the processing overhead and that affects refreshes.

Try taking a long exposure photograph. At the end of the exposure, see how long it takes before you can do anything again. It will take almost as long as the exposure itself. And the reason for that is that all those reads are being stored in buffer and it takes time for the processor to collate them all to generate the final image.

 

[Andrew Reid]

25 minutes ago, tugela said:

The sensor still has to be read. If you are shooting stills at 1/30th or 1/60th sec, it will be no different from a video frame. You don't see any sign of rolling shutter effects at those frame rates, so the method being used to read the sensor has to be different. The mechanical shutter has nothing to do with it.

If you shoot in burst mode at 15 fps you get full resolution images without rolling shutter, so that sets the minimum read speed the camera is capable of in a worst case scenario.

Actually a mechanical shutter has everything to do with it.

When we say the sensor has a rolling shutter, don't confuse this with a rolling readout.

It is actually doing a rolling exposure.

And unless you have a global shutter sensor (the NX1 does not), then any electronic shutter is a rolling one. Period. Whenever you use that instead of the mechanical shutter you will have to expose the sensor row by row and it will take a length of time determined by the speed of the sensor, to do this.... In the NX1's case 30ms to read out 24MP in 16:9, a bit longer to read out the full 28MP in 3:2. The top of the sensor will be exposed 30ms earlier than the bottom and you will get the RS skew.

The mechanical shutter allows the entire sensor to be exposed at the same time, then the readout is done by passing each row through an A/D.

10 minutes ago, tugela said:

The whole sensor is being read all the time when the computer is on, it doesn't care if you are taking a picture at the time or looking through your viewfinder. What is happening for the viewfinder is that only part of that data set is being processed, and the rest discarded. The reduced processing time is what allows the display to work without rolling shutter, and it is the same reason why FHD has minimal rolling shutter.

No, the discarding of pixels is done on the sensor and it only sends a subset of them out to the image processor to display on the EVF and LCD.

The NX1 only switches to a full pixel readout when you hit movie standby or start recording 4K.

Then the image is scaled on the image processor (in a more advanced way probably using bilinear or bicubic filtering) from 6K to 4K and there's also a more heavily scaled version for the LCD.

Quote

Try doing something on the camera that requires additional processing, while looking through the viewfinder. You will see the image slow or even stop while the camera catches up. This is due to the processing overhead and that affects refreshes.

Processing overhead... not necessarily. There are many things which can cause a delay with an EVF, slow panel refresh rate, contrast detect AF changing the sensor mode briefly, interruptions to the image pipeline caused by design. It's too simplistic to say it is a 'processing overhead' and without diving into the specifics in the case of the NX1 you're just guessing really.

Quote

Try taking a long exposure photograph. At the end of the exposure, see how long it takes before you can do anything again. It will take almost as long as the exposure itself. And the reason for that is that all those reads are being stored in buffer and it takes time for the processor to collate them all to generate the final image.

What does that have to do with video or live-view? Nothing. It's an in-camera post processing aspect and off topic.