Low light test of 5D Mark III raw vs H.264

[jgharding]

Well I have. These are my problems:

 

Cinestyle raises the blacks from 0 to 16. This is to match Cineon film scanners. It decreases dynamic range, mostly from the midtones as the highlights and shadows are compressed. Good highlight rolloff is gone. Cinestyle clips them ugly. You have to create them back with a gamma curve. It increases the noise in the blacks and when you pull those blacks down in post, the noise is there. And it doesn't really help in grading at all plus way more difficult to assess exposure. If you want milky blacks, just raise them afterwards.

 

For example the Flaat10p profile is lots better and also Prolost Neutral.

 

You personally don't see banding but that's because you use Neatvideo and it creates very good 16-bit files out of 8-bits. Neatvideo actually fills in the gaps. It's actually quite remarkable how much banding it can take away. But it will do that to any profile, not just cinestyle.

 

Those intermediate ISOs are cleaner in h.264 mode because they are digitally pulled down. This pull down only decreases DR in RAW files but because h.264 loses so many stops anyway, it doesn't really matter there. So basically, when shooting stills, try to go with the original ISOs and when doing h.264, drop down once to get the shadow noise gone and more for the highlights.

 

Do you use the LUT or are you yanking curves around yourself? It's supposed to be used with LUT. Prolost Nuetral was a joke of Stu's, it's just the settings people were already using before Cinestyle existed.

 

Almost every other flat profile is a poor thing made by playing with the packaged editor. some of them are horribly inverted and have no LUT to correct to a good starting point for post.

 

Neat Video solves the problem yes, which is why we use it. Any picture profile is 8-bit and will have banding, so I also solve it in post with Neat Video, it's smart to use the best tool for a job!

 

The only exceptions I've found are VisionColor, they're nice and have LUT built into FilmConvert. Technicolor spent about a year or so working with Canon to make Cinestyle according the PR, I'm not willing to laugh that off for these oddly named experiments.

 

I am willing to shrug it off for Magic Lantern Raw though. ;)

[ScreensPro]

I agree, ideally any tests should use the very best settings for the h264

[Tomas]

Could you inform us about the lens you´ve used?

[Tomas]

I made a first light grade of the image. The dng is simply just like a raw-photo (like the blackmagic). Absolute fantastic!

 

http://imageshack.us/f/69/iso800grade.jpg/

[Per Lichtman]

@EOSHD Andrew I'm about to read the rest of the article but the line " In its factory guise whilst not quite as clean as the Canon C300 or Sony FS100, it is the best DSLR for low light shooting (though the Nikon D5200 puts up a good fight)." needs to be updated, since it is factually inaccurate if not qualified.

 

The 1DX, 1DC , D4 and D3s all offer superior high ISO performance and the D4 and D3s stretch their max ISOs further than the 1DX in video mode. Now the 5DMkIII may be the "best DSLR for low-light shooting under x number of dollars" or the best combination of certain features in combination with lowlight shooting, but by every metric (including the posted comparisons of video clips, the scores for the sensors on DXOMark and the RAW stills posted on DPReview for the stills side) the sensors used in the cameras I mentioned outperform the 5DMkIII for lowlight/high ISO, both in stills and video in their compressed modes.

 

Looking forward to reading clarified version of your original remark on your blog and thanks for the early RAW footage and testing. :)

[peederj]

Well I have. These are my problems:

 

Cinestyle raises the blacks from 0 to 16. This is to match Cineon film scanners. It decreases dynamic range, mostly from the midtones as the highlights and shadows are compressed. Good highlight rolloff is gone. Cinestyle clips them ugly. You have to create them back with a gamma curve. It increases the noise in the blacks and when you pull those blacks down in post, the noise is there. And it doesn't really help in grading at all plus way more difficult to assess exposure. If you want milky blacks, just raise them afterwards.

 

For example the Flaat10p profile is lots better and also Prolost Neutral.

 

You personally don't see banding but that's because you use Neatvideo and it creates very good 16-bit files out of 8-bits. Neatvideo actually fills in the gaps. It's actually quite remarkable how much banding it can take away. But it will do that to any profile, not just cinestyle.

 

Those intermediate ISOs are cleaner in h.264 mode because they are digitally pulled down. This pull down only decreases DR in RAW files but because h.264 loses so many stops anyway, it doesn't really matter there. So basically, when shooting stills, try to go with the original ISOs and when doing h.264, drop down once to get the shadow noise gone and more for the highlights.

 

I completely agree that Neat Video is doing a lot more to restore the image and remove compression artifacts/banding than just NR. Anytime I'm doing a severe grade it's got to be on there...it's basically the best $99 you can spend in post. And far less hassle than RAW.

 

However I think it's laughable that you suggest ProLost Neutral is better than Cinestyle. And that Cinestyle has poorer highlight handling! Lol!

 

I will try Canon's new "X" style (released the other day but lost in the ML fuss) to see if that's like a Cinestyle that doesn't raise the blacks so much, saving a bit (update...it isn't but it's a nice look anyway). But for the moment I haven't found anything overall better. RAW will of course be the best we can do, but at what cost? Also note, "you can change the WB in RAW without losing anything"...no, your noise floor is going to shift with a WB shift, not much but some, so there will still be some loss...but the loss won't be any more than the loss would be if you set it properly in-camera. One thing the 5D3 lacks that the C100 has is the ABB (Automatic black balance) function that neutralizes the noise floor for a given ISO and WB setting...RAW will cover that too but again at enormous data and workflow cost.

 

Just give us a great internal codec, whatever you can, ML!

[userage]

Looking to buy a new camera, is there a chance this will come to the lower Canons? 550d/60d/7d ? Or shall I just go for a GH2/G6

[blackrat]

I completely agree that Neat Video is doing a lot more to restore the image and remove compression artifacts/banding than just NR. Anytime I'm doing a severe grade it's got to be on there...it's basically the best $99 you can spend in post. And far less hassle than RAW.

 

However I think it's laughable that you suggest ProLost Neutral is better than Cinestyle. And that Cinestyle has poorer highlight handling! Lol!

 

I will try Canon's new "X" style (released the other day but lost in the ML fuss) to see if that's like a Cinestyle that doesn't raise the blacks so much, saving a bit. But for the moment I haven't found anything overall better. RAW will of course be the best we can do, but at what cost? Also note, "you can change the WB in RAW without losing anything"...no, your noise floor is going to shift with a WB shift, not much but some, so there will still be some loss...but the loss won't be any more than the loss would be if you set it properly in-camera. One thing the 5D3 lacks that the C100 has is the ABB (Automatic black balance) function that neutralizes the noise floor for a given ISO and WB setting...RAW will cover that too but again at enormous data and workflow cost.

 

Just give us a great internal codec, whatever you can, ML!

There is no such thing as setting the WB in camera for RAW.

The camera records what it records and that is it.

 

NeatVideo won't bring back all the detail that got smeared away.

 

This was not carried out ideally, but look at this comparison and the detail at least is much better:

http://farm8.staticflickr.com/7286/8742046520_18fc4c218e_o.jpg

 

Granted a fair test does need the best profiles to be used in all cases and the best PP done, etc. but for detail at the least even a quick hack of a test makes the difference clear and it's clear that at the very least the RAW video files are much easier to deal with when messing around with tone response/color/WB/etc.

[Leica50mm]

Really want to see some anamorphic footage.

[peederj]

There is no such thing as setting the WB in camera for RAW.

The camera records what it records and that is it.

 

[i didn't suggest otherwise...]

 

NeatVideo won't bring back all the detail that got smeared away.

 

But what did the smearing? Everyone agrees that on the 5D3 the internal video recording sucks (regardless of picture style used) and 22MP RAW stills are the best you can do. However we can't get 24 22MP RAW stills per second, at least not unless there is some massive development on the CF card front (some have suggested an SSD adapter of some form). So there is detail that's being smeared away for the ML hack just as there is for the uncompressed HDMI out, which is a fairer thing to compare it with.

 

Is the ML RAW hack in its full sensor mode using the same 3x3 pixel binning downsampler that the uncompressed HDMI out is? Because if it is, that's the same detail damage. The difference in luma detail should then be minimal, unless Canon is damaging the HDMI image somehow. ML hasn't figured out the HDMI driver yet but they do suggest that some image processing is being done there (it does appear a bit brighter of an image I think). But is Canon intentionally crippling that image as they clearly did for the internal recording (40Mbps H.264 should look a LOT better than what they are giving us...)? I don't know but a fair, scientific, detailed comparison between the two would tell us. We know that chroma detail will be cut in half (422 vs. 444) and we know we will only be mapping the DR into 8 bits rather than 14, but that will only result in gradient accuracy being lost (i.e. banding) rather than detail and acutance/sharpness being lost.

 

So I want to know how bad the uncompressed HDMI actually is vs. the ML RAW...I predict it won't be noticeably worse when Cinestyle + LUT is used, and with Neat Video applied properly even a strong grade should hold up well. If not, if Canon really is damaging the supposedly uncompressed HDMI out beyond its 1080i60/8 bit 422 that we all know already, then we have a new thing to yell at them for and praise ML for saving us from. Otherwise we have a choice between buying a Ninja 2 and a couple cheap laptop drives and getting a usable field monitor in the bargain, or investing in a stack of the fastest CF cards available and trying to figure out a practical RAW workflow that's worth the hassle. Again I would have already answered this for you if I had the fast CF cards and a reliable ML build...asking if there's anyone out there who is trustworthy enough to do so for us now.

[xxbluejay21]

Exciting stuff! Will this ever be possible for the lower end Rebel cameras? Or is the processor too slow? Maybe soon we can have 24fps off the full 21mp sensor so we can have 6K+ footage!!! And maybe they can rewrite the code to reduce rolling shutter! That would be awesome! 

[eco_bach]

Thanks for sharing but have to agree 100% with Per Lichtman. "... the best DSLR for low light shooting" Huh? Not by a long shot!

[mike_tee_vee]

How would this compare to the output of the FS700?  According to ProVideoCoalition, the FS700 has 13 to 14 stops of dynamic range, similar low light to the FS100, and 800-1000 lines of resolution.

[blackrat]

But what did the smearing? Everyone agrees that on the 5D3 the internal video recording sucks (regardless of picture style used) and 22MP RAW stills are the best you can do. However we can't get 24 22MP RAW stills per second, at least not unless there is some massive development on the CF card front (some have suggested an SSD adapter of some form). So there is detail that's being smeared away for the ML hack just as there is for the uncompressed HDMI out, which is a fairer thing to compare it with.

 

Is the ML RAW hack in its full sensor mode using the same 3x3 pixel binning downsampler that the uncompressed HDMI out is? Because if it is, that's the same detail damage. The difference in luma detail should then be minimal, unless Canon is damaging the HDMI image somehow. ML hasn't figured out the HDMI driver yet but they do suggest that some image processing is being done there (it does appear a bit brighter of an image I think). But is Canon intentionally crippling that image as they clearly did for the internal recording (40Mbps H.264 should look a LOT better than what they are giving us...)? I don't know but a fair, scientific, detailed comparison between the two would tell us. We know that chroma detail will be cut in half (422 vs. 444) and we know we will only be mapping the DR into 8 bits rather than 14, but that will only result in gradient accuracy being lost (i.e. banding) rather than detail and acutance/sharpness being lost.

 

So I want to know how bad the uncompressed HDMI actually is vs. the ML RAW...I predict it won't be noticeably worse when Cinestyle + LUT is used, and with Neat Video applied properly even a strong grade should hold up well. If not, if Canon really is damaging the supposedly uncompressed HDMI out beyond its 1080i60/8 bit 422 that we all know already, then we have a new thing to yell at them for and praise ML for saving us from. Otherwise we have a choice between buying a Ninja 2 and a couple cheap laptop drives and getting a usable field monitor in the bargain, or investing in a stack of the fastest CF cards available and trying to figure out a practical RAW workflow that's worth the hassle. Again I would have already answered this for you if I had the fast CF cards and a reliable ML build...asking if there's anyone out there who is trustworthy enough to do so for us now.

As far as detail goes the ML RAW has MUCH more than using HDMI to Ninja 2. Absolutely no question at all. How much the final difference in DR will be I don't know. The RAW, at the very least, grades a lot more easily and with the extra detail it has richer color input too, you can easily pull sliders all over the place, raise exposure like crazy, pull shadows, change WB radically and it all holds together. I mean look at what RAW 22mP stills processing is like vs. manipulating an in cam JPG (and the in cam jpg generally seem hold together better than the video).

 

I have ML pre-alpha with RAW. I have 1000x cards. I have Ninja 2. Boulders that are waxy with internal or ninja 2 (and for fine static detail there is barely a difference at all between those two) suddenly have texture and subtle roughness, different color shadings using RAW Video for ML.

 

Just grab one Lexar 1000x 32GB card and the ML and try for yourself quickly if you don't believe any of us.

 

Sure the Ninja 2 collected footage is easier to handle. It goes straight to a nice larger HD and record clips longer than 50 seconds with ease and you don't need to pre-process it or anything. (OTOH ACR is so powerful that you can often skip having to use slow grading tools in premiere inwhich case you gain some back in saved render time there and it's not quite as bad a comparison overall). And the Ninja 2 takes way less space. But the detail just ain't there compared to the RAW ML Video.

[blackrat]

Thanks for sharing but have to agree 100% with Per Lichtman. "... the best DSLR for low light shooting" Huh? Not by a long shot!

Don't forget that the D4 video is pretty soft and weird while the 5D3 RAW footage is a million times more detailed. Maybe if you did a ton of NR on it and ended up with the same detail the D4 video shows it would be even or better.

[Per Lichtman]

Don't forget that the D4 video is pretty soft and weird while the 5D3 RAW footage is a million times more detailed. Maybe if you did a ton of NR on it and ended up with the same detail the D4 video shows it would be even or better.

The D4 video is generally considered soft when using the full sensor, but that criticism is not frequently leveled when the footage is shot at a crop instead.

And we aren't talking about overall quality - if the original statement was "combination of lowlight performance and detail" then it might be a bit less controversial as a statement. But the D4 is currently the only commercially available camera that shoots video at ISO 204,800 and that means that whether it does so well or not... it literally has no competition at the highest ISO (unlike stills, where the 1D X comes into play).

The next stop down we have ISO 102,400 where it's between the D4, D3s and 1DMkIV: the D3s shoots 720P instead of 1080P and the 1DMkIV has noticeably worse performance at the ISO than either the D3s or D4 - so the D4 is the winner for 1080P footage at that ISO again.

The next stop below that, we start to get some competition from outside the DSLR range coming into play, but the current Canon range (the 5DMkIII included) tops out in video mode at 25,600 (which one stop further down). So once again we are looking at the D4, D3s and 1DMkIV as the competitors for 51,200.

So if the 5DMkIII has 3 stops of ISO range where it is not even competing and some of those competing products have been demonstrated to match or exceed the ISO performance of the 5DMkIII at 25,600 how can the 5DMkIII be the "best"?

[Per Lichtman]

And just to be clear, I was very surprised that Canon did not include the six-digit ISO video support on the 1D X/1D C, etc. that they did in the 1DMk IV and had to get it confirmed by a Canon representative last summer.
http://www.eoshd.com/comments/topic/943-canon-1dx-v-canon-5d3/#entry14136

[peederj]

It would be easier for me to just take your word for it if you were really precise in making and backing up your statements. Shots of boulders? =) 

 

Also I am discounting your reports as I have confirmed quite a dramatic increase in quality using Ninja 2 with 5D3 HDMI out vs. the same settings internal. Look at the noise floor for instance. Look at color edges. There is no comparison.

 

I'm procrastinating getting a card seeing as the Komputerbay is so much cheaper (perhaps the savings is on poor QA?) and we're only now getting feedback on whether it works or not. The 128GB and 64GB cards are completely different implementations...and what's more, the production runs of these cards can be done in different factories with different components without notice. Furthermore I would rather just get a dependable bin of the ML that has more or less settled than a nightly I have to compile. My primary is the C100 + Ninja 2 and the 5D3 is my B cam and stills so I'm not in a mad rush. I'm also procrastinating getting a second Ninja as they just came out with the Samurai Blade with the far better screen and it shouldn't be long before they have an HDMI version rather than having to buy one of their HDMI converters (which are very nice looking btw, but $300 more, and only mount to their hardware).

 

As far as detail goes the ML RAW has MUCH more than using HDMI to Ninja 2. Absolutely no question at all. How much the final difference in DR will be I don't know. The RAW, at the very least, grades a lot more easily and with the extra detail it has richer color input too, you can easily pull sliders all over the place, raise exposure like crazy, pull shadows, change WB radically and it all holds together. I mean look at what RAW 22mP stills processing is like vs. manipulating an in cam JPG (and the in cam jpg generally seem hold together better than the video).

 

I have ML pre-alpha with RAW. I have 1000x cards. I have Ninja 2. Boulders that are waxy with internal or ninja 2 (and for fine static detail there is barely a difference at all between those two) suddenly have texture and subtle roughness, different color shadings using RAW Video for ML.

 

Just grab one Lexar 1000x 32GB card and the ML and try for yourself quickly if you don't believe any of us.

 

Sure the Ninja 2 collected footage is easier to handle. It goes straight to a nice larger HD and record clips longer than 50 seconds with ease and you don't need to pre-process it or anything. (OTOH ACR is so powerful that you can often skip having to use slow grading tools in premiere inwhich case you gain some back in saved render time there and it's not quite as bad a comparison overall). And the Ninja 2 takes way less space. But the detail just ain't there compared to the RAW ML Video.

[Husah]

There is nothing wrong with H.264. It's 8-bit that limits the DR. XAVC is also H.264

 based codec but uses 10-bit 

 

[Husah]

How would this compare to the output of the FS700?  According to ProVideoCoalition, the FS700 has 13 to 14 stops of dynamic range, similar low light to the FS100, and 800-1000 lines of resolution.

 

I seriously doubt that 8-bit FS700 has 13 to 14 DR. It's not mathematically possible. However, once FS700 is updated to RAW in July, the DR would be close to 14 stops -- and I am sure it would be higher than 5D III which (at best) has 11.7 stops according to dxomark