My wishlist video feature for 8K sensors: realtime crop-zooming

[tellure]

A lot of us don't need more resolution at this point.  The idea of 8K sensors seems appealing not for resolution but for other potential features, like even better low light performance, as Andrew recently wrote about.

But what if your 8K sensor could zoom (by cropping) up to 2x, without any loss of quality, while outputting a 4K signal?  Right now many of us love that we can punch into APS-C from full-frame for an extra 1.5x zoom while still getting native 4K.  An 8K sensor could allow us to get to 2x while still getting full 4K resolution at the long end of the zoom.

Your expensive 24mm 1.4 prime could become a 24-48mm 1.4 (a lens which doesn't exist in full frame).  But to make this a really killer feature the zoom needs to be realtime.  Right now we have to stop recording to switch between full-frame and APS-C.  What if you could zoom while recording?  What if you could zoom in small increments, and control that zoom using any re-mappable dial on your camera, or even use your manual focus ring if you were using AF-C?

But wait, why not just shoot everything at 8K and then crop in post?  Composition, of course.  Being able to make those framing decisions in realtime and see it on the screen would be great, just like having a native zoom lens allows for more composition choices on the fly.  And of course you wouldn't need to store those larger 8K files.

Taking the functionality even further, if 8K sensors could do this cropping in realtime then it could even allow for a power-zoom feature - e.g. the ability to set a zoom rate and then do a slow crawl-in or even a smash-zoom in/out.

Sony's Clear Image Zoom is the closest thing we have to this right now, but of course it's not cropping, it's doing interpolation to give us an up-res'ed image from the same sensor area.  It's also sorely lacking in usability (can't assign it to any wheels or dials, and it only zooms in stepped 0.1x increments).  Worse yet, all the focus functions are disabled when CIZ active, so you lose the ability to set a specific focus point, change the focus point, or use object tracking.  Which is a shame because the quality is actually pretty decent, as shown below.

 

I imagine the processing horsepower needed to do a realtime crop smoothly across an 8K sensor is probably pretty huge, and I don't expect this kind of feature in the first or maybe even the second generation of 8K sensors but it seems like something that could happen eventually. Given how much I use the 1.5x APS-C crop now I would use this feature a ton.

[Kisaha]

This is happening, since 2015. I would expect Sharp will have something similar in their upcoming 8K camera.

http://pro.jvc.com/pro/attributes/4k/clips/prime_zoom_demo.jsp

[BTM_Pix]

Most of Panasonic's 4K cameras even the consumer ones also have a pretty neat live crop mode to fake zoom and movement.

The downside is that they are done in only two preset time periods so extending that out to a joystick control for manual operation would be a great option or even just making the time period fully adjustable would probably make it far more usable.

 

[tellure]

Cool, thanks for the info!  That JVC feature is exactly what I'm after.  Now they just need to get it in an 8K sensor outputting to 4K, as opposed to a 4K sensor outputting to 1080p :).  Still really promising that it's been done.  Makes me hopeful it'll eventually make its way to the 8K cams.  And they call it "prime zoom" no less.  The appeal of zooming with a prime seems pretty strong. 

[Mokara]

If you zoom in to 4K through a crop, you will actually be getting less than 4K due to debeyering. 

Lots of cameras have digital zooms, and have for a decade, it always compromises the image however.

[BasiliskFilm]

your 24 f1.4 prime would become the equivalent of a 24 f1.4 to 48 f2.8 zoom. Aperture equivalence is proportional to frame size.

[tellure]

@Mokara I agree digital zoom always compromises image quality, but we're not talking about a digital zoom here, which takes a fixed sensor readout and up-res's the image via interpolation.  We're talking about cropping an 8K sensor at variable sizes until you get down to 4K at the maximum crop (2x zoom).  So the final readout at maximum zoom would still be native 4K.

It's possible that downsampling at all the varying sizes in between 8K and 4K could introduce artifacts though.. just like downsampling an image in photoshop by 46%, for example, is never as clean as 50% or any multiples of 2.

 

@BasiliskFilm I had that same thought and I'm happy to be proven wrong here since I sat out the aperture equivalence wars but I don't think it applies here.  As I understand it, aperture equivalence should be calculated when comparing sensors of different sizes with equivalent framing.  But here we're talking about one sensor size - 8K - just cropped down to varying sizes and downsampled to 4K for the final output.

Think of it this way - imagine you took a series of 8K photos of a fencepost at 50mm f/1.4 and stitched them together into a time-lapse video.  There's lots of creamy bokeh in the background since you're shooting at f/1.4.  Because the final output of your video is 4K you have lots of room to crop in post.  So you add some zooms in your thrilling fencepost video to get more detail by cropping the image.  In the final seconds of the video you end at a 2x zoom, and now the *framing* (and only the framing) of the fencepost photo is the same as it would have looked like with a 100mm lens.  All that creamy f/1.4 bokeh in the original shot is completely unchanged since the contents of the original 8K photo are unchanged, it's just been cropped.

Unlike regular aperture equivalence calculations where the image being compared is being projected on different sensor sizes, in this (wishlist) feature the image projection is always over the whole 8K sensor.  Internally the camera would be cropping and downsampling the image before encoding it into the 4K video stream.  Hope that makese sense, but again happy to be proven wrong, I'm not an expert on the hotly debated topic of equivalence.

[BasiliskFilm]

2 hours ago, tellure said:

 

@BasiliskFilm I had that same thought and I'm happy to be proven wrong here since I sat out the aperture equivalence wars but I don't think it applies here.  As I understand it, aperture equivalence should be calculated when comparing sensors of different sizes with equivalent framing.  But here we're talking about one sensor size - 8K - just cropped down to varying sizes and downsampled to 4K for the final output.

Think of it this way - imagine you took a series of 8K photos of a fencepost at 50mm f/1.4 and stitched them together into a time-lapse video.  There's lots of creamy bokeh in the background since you're shooting at f/1.4.  Because the final output of your video is 4K you have lots of room to crop in post.  So you add some zooms in your thrilling fencepost video to get more detail by cropping the image.  In the final seconds of the video you end at a 2x zoom, and now the *framing* (and only the framing) of the fencepost photo is the same as it would have looked like with a 100mm lens.  All that creamy f/1.4 bokeh in the original shot is completely unchanged since the contents of the original 8K photo are unchanged, it's just been cropped.

Unlike regular aperture equivalence calculations where the image being compared is being projected on different sensor sizes, in this (wishlist) feature the image projection is always over the whole 8K sensor.  Internally the camera would be cropping and downsampling the image before encoding it into the 4K video stream.  Hope that makese sense, but again happy to be proven wrong, I'm not an expert on the hotly debated topic of equivalence.

You are turning a full frame camera into a crop frame camera - the centre 50% of the sensor is about the same size as a micro 4/3 sensor. The standard way of working out full-frame-equivalence is to multiply the focal length and aperture by the crop factor, in this case 2x. The actual focal length and aperture remain the same, obviously.

[tellure]

21 hours ago, BasiliskFilm said:

You are turning a full frame camera into a crop frame camera - the centre 50% of the sensor is about the same size as a micro 4/3 sensor. The standard way of working out full-frame-equivalence is to multiply the focal length and aperture by the crop factor, in this case 2x. The actual focal length and aperture remain the same, obviously.

@BasiliskFilm That would be true if we were only using the center 50% of the sensor here.  But in this proposed feature we are still capturing an 8K image from the entire sensor, and cropping it during the encoding process.  So the image circle projected on the sensor and sensor size remain the same.  It's literally the same as taking a full-frame 8K photo and then cropping it in post, which I'm guessing you'd agree wouldn't require any multiplication of aperture because the depth of field is already baked into the original full-frame 8K photo.

[BasiliskFilm]

4 hours ago, tellure said:

@BasiliskFilm That would be true if we were only using the center 50% of the sensor here.  But in this proposed feature we are still capturing an 8K image from the entire sensor, and cropping it during the encoding process.  So the image circle projected on the sensor and sensor size remain the same.  It's literally the same as taking a full-frame 8K photo and then cropping it in post, which I'm guessing you'd agree wouldn't require any multiplication of aperture because the depth of field is already baked into the original full-frame 8K photo.

The image from a crop sensor camera using a 50mm f1.8 lens will be identical to an image shot with a full sensor camera with the same lens cropped to match in post.

If you keep all variables the same it doesn't matter if you crop before or after shooting. The image is the same.

I honestly don't mind if you disagree with me though.