Recently Magic Lantern had the breakthrough discovery of beautiful pin sharp 2K sensor feed in raw format on the 5D Mark II and 5D Mark III.
This completely turns our understanding of DSLR video image quality on its head. Why is the video quality so far from what the camera is really capable of?
Disclaimer: Some of the following represents my thoughts and understanding of how camera technology works and some of it may be speculative as I am not a Canon engineer!
As I understand it the sensor does one sampling run to provide a feed that is used for both live view, video and HDMI. Magic Lantern has gained access to that image and it is very nice, lovely 2K full frame 14bit raw straight from the sensor via the camera’s buffer memory. The image processor (DIGIC) then has to produce an image for the live view LCD (approx. 640×480 or 1MP) and a second output for compressed 1080p video from that one high quality feed.
Alex of Magic Lantern told me DNG was ported from the Canon CHDK project and the camera can successfully write DNG frames to the buffer at 24fps to 30fps in 2K mode, which we now have access to (up to 30 frames at a time) in silent stills burst mode which is now working with Magic Lantern. Steps and research are being taken to see what the feed can be used for aside from silent stills, 30fps burst mode and timelapse with no mechanical shutter movement…
What this shows is somewhat revelatory – that the sensor isn’t the problem when it comes to moire or the general poor soft image quality in video mode.
So what is going on?
UPDATE: Sam (you may know him from the Flaat picture profiles) thinks the sensor is still line-skipping to produce the 2K raw image. He has run tests on 22MP 5D Mark III raw stills in Photoshop using the most basic nearest-neighbour downsampling to mimic 3:1 line-skipping on the sensor the end result looks very similar to the 2K raw output in Magic Lantern. In the DSLR community we have traditionally tended to blame line-skipping for soft resolution and moire in video mode, but maybe that isn’t the whole story. Sam’s tests seem to show that image quality doesn’t suffer too much with it.
The 14bit raw is turned into compressed 8bit 4:2:0 but rather than use all the native resolution of the 2K raw frames, the image processor is playing all kinds of tricks, at one point even downscaling to 1904 and then back up to 1920 again in the case of the 5D Mark III. Why does it do that?
Canon’s Chuck Westfall and Tim Smith say line skipping on the sensor was necessary to get down from the native sensor resolution of 21MP+ to 2MP, implying this results in moire and aliasing. However the 2K output of the sensor is much cleaner without the same moire issues as the final 1080p video, and resolution is much closer to 1080 lines than the mushy 5D Mark III 1080p is.
The softer image with moire and aliasing appears to be introduced by the image processor further (and crudely) resizing the 2K raw image, which seems unnecessary. All the camera should need to do is crop to 16:9 and compress the data, not reduce the resolution dramatically further like it is doing. It could be that in order to provide a simultaneous VGA live view feed and 1080p to the SD card, the 2K data has to be downrezed to a middle ground between 640×480 and 1920×1080, then upscaled back to 1080p but I’m purely speculating.
I asked Alex of Magic Lantern today how feasible it would be to get continuous raw video recording from the 5D Mark III and our conversation went like this:
“I think 1920×720 is doable on the 1000x card.
“In 720p, the raw data is 1920×670 or something around this.
“I also believe the video and live-view image is coming from that [2K feed], since they are perfectly synchronized.
“[Canon’s] image pipeline is probably optimized for fast math, not for high quality.
“They don’t do highlight recovery, they probably denoise too much… who knows?”
Disclaimer: I think it is important to keep expectations in check, nothing is confirmed with regards raw video and don’t forget to donate to Magic Lantern to keep the progress coming.
I will be trying the firmware hack later today.
UPDATE: I now have the burst mode silent pics feature running and sure enough I’m getting up to 30fps 2K raw from my 5D Mark III until the buffer fills up after only about a second. If the Magic Lantern developers can find a way to write this raw output at 24fps to the card despite the large data rates, even at reduced resolutions, we could have continuously recording raw video.
