8-bit REC709 is more flexible in post than you think

[kye]

8-bit rec709 profiles are much more flexible in post than you might think.  I'm not saying they're as good as an Alexa or whatever, but they're a million miles better than people give them credit for.

Here's a latitude and WB torture test of the GX85, in the standard profile (8-bit rec709) customised to have reduced contrast but normal saturation.  
For each of the below, the left image is the properly exposed reference image, the middle one is the graded image, and the one on the right is the image SOOC.

Exposure latitude test - +3 stops:

Exposure latitude test - +2 stops:

Exposure latitude test - +1 stops:

Exposure latitude test - -1 stops:

Exposure latitude test - -2 stops:

Exposure latitude test - -3 stops:

Exposure latitude test - -4 stops:

Exposure latitude test - warm:

Exposure latitude test - cool:

Exposure latitude test - magenta:

Exposure latitude test - green (as far as it would go!):

Notes on the testing method:

• GX85 shot in manual mode on cloudy day
• Exposure varied by changing lens aperture
• WB varied by changing colour temp and tint
• Tools used in Resolve were mostly Lift/Gamma/Gain, saturation, and some had a bit of Shadows/Mids/Highlights

Notes on the results:

• If it's clipped then it's clipped, there's no getting around that
• If you've shot a whole sequence in the wrong WB then shoot a test chart replicating the error, spend some time on the correction, then apply to all the shots... a bit of work but worth it to rescue a days shooting
• If you've shot on auto-exposure / auto-WB and want to correct the small errors then this is easily possible - it won't have gotten it nearly as wrong as what I have shown above
• Most cameras shooting in rec709 will do funky stuff to colours depending on their luma value as part of the look of the profile, so when you over/under expose and then pull things back the hues and saturation levels will have shifted around in odd ways compared to a normal exposure, but if it's a real shoot then you most likely won't have many dominant hues in the image and you only have to correct the ones that are in the frame and distracting, so the above is far more work than normal shooting would be 

The days of needing RAW or even LOG to change exposure or WB in post are gone, and although the rec709 profiles often have lower DR than log profiles, they're much better than you think.

If anyone wants me to post full-sized versions with titles so you can flick back and forth then just let me know.

Happy shooting!

[FHDcrew]

Love this!  I am coming to the same conclusion as I shoot in 8-bit rec709 more and more.  And I find that having a color managed workflow helps a lot; the image is quite flexible when conformed to Davinci Wide Gamut; responds a lot like LOG does.  And if my exposure and white balance aren't too off, the grading experience is great.

[John Matthews]

Great work, @kye! I'd be interested at looking at the full resolution images if possible. After doing the WB tests, do you think there's more latitude in slightly blue, magenta, or green version. I remember that Gerald Undone did a test some time ago and found more latitude in a warmer WB than a bluer WB.

[kye]

12 minutes ago, FHDcrew said:

Love this!  I am coming to the same conclusion as I shoot in 8-bit rec709 more and more.  And I find that having a color managed workflow helps a lot; the image is quite flexible when conformed to Davinci Wide Gamut; responds a lot like LOG does.  And if my exposure and white balance aren't too off, the grading experience is great.

Actually, for these I didn't use Colour Management (I know this will shock people!) and it was a bit of a surprise to me too, but the LLG controls just worked better.  The fact they're designed for rec709 might have something to do with that, perhaps, maybe... ?

😆😆😆

I normally make my adjustments in real projects using Colour Management, and they're fine because they're not a torture test like this with huge adjustments, but maybe I'll keep this in mind and if I get difficult shots then maybe I'll change to rec709 for one node and do the adjustments in that.

8 minutes ago, John Matthews said:

Great work, @kye! I'd be interested at looking at the full resolution images if possible. After doing the WB tests, do you think there's more latitude in slightly blue, magenta, or green version. I remember that Gerald Undone did a test some time ago and found more latitude in a warmer WB than a bluer WB.

Cool, I'll work out the best way to post them.

TBH I don't know what "more latitude" really means.  For example it might be the amount you can shift something before clipping occurs, or before some level of visible undesirable aesthetic effect occurs, but this would be situation dependent and also user dependent.

If you can think of a way to test this then I'd be happy to have a look.  From memory, I think I set the blue image to a colour temp in the 6000s and the warm one to 3200, and neutral was in the middle somewhere.  The magenta and green images were setting the in-camera adjustment to the maximum it would go in those directions.  So, from that, these are very large WB errors to recover from, and (hopefully!!) much more than you'd ever encounter in real life.

[kye]

Also, if anyone can think of any other tests I should do then I'm all ears.  Perhaps the one that might be useful is including skintones, so maybe I'll have to make a cameo in the test images.  

Realistically, the ability to recover skintones is probably the most important thing, and perhaps the most fragile.

[FHDcrew]

Would love to see skin tones!  I need to learn how to work with them given 8-bit limitations. That indeed always tends to be the trickiest for me to work on. 

[kye]

15 minutes ago, FHDcrew said:

Would love to see skin tones!  I need to learn how to work with them given 8-bit limitations. That indeed always tends to be the trickiest for me to work on. 

Skin-tones are always the trickiest because we are much more sensitive to this area of the colour cube than any other.  If I make a cameo then you'll get to enjoy the full range of tones - from pasty to gaunt and all the way to magenta and red.  Lucky you!

In terms of working with skin-tones "given 8-bit limitations" I don't think there are limitations, at least if we're working with 8-bit rec709 images.  In broad terms, the danger of "breaking" the image comes from areas where the bits are too far apart and the associated artefacts become visible.  The bits are broadly as far apart in 8-bit 709 as they are in 10-bit LOG as they are in 12-bit RAW.  The difference that most people experience between those is likely to be the compression, considering that 8-bit 709 will mostly be 10-100Mbps, 10-bit log will mostly be 50-400Mbps, and 12-bit RAW will be uncompressed.

Perhaps the worst combination is 8-bit log, which TBH, is so bad it should be against the Geneva convention on human rights.  This is the worst combination because the bits in the final image are spread apart the most.

This is why I shoot rec709 with normal amounts of saturation - it gives more information for colours than if you reduce the saturation in-camera and then boost it in post.  It's also why the iPhone HDR implementation is so good - it is 10-bit but the levels of saturation in the files is close to 709 levels, so you have a really large amount of colour fidelity still retained in the files.

[kye]

Link to files.  The reference image is repeated so you can just go back and forth to compare.

Enjoy!

https://www.dropbox.com/scl/fo/vj7wthlzjc9ocqtphvclt/h?rlkey=ivcvcj3hmdrow1q0zsof0wc1p&dl=0

[John Matthews]

36 minutes ago, kye said:

Link to files.  The reference image is repeated so you can just go back and forth to compare.

Enjoy!

https://www.dropbox.com/scl/fo/vj7wthlzjc9ocqtphvclt/h?rlkey=ivcvcj3hmdrow1q0zsof0wc1p&dl=0

I downloaded all of them. My conclusion (as I thought): stay away from overexposure. Underexposure is scary, but noise can be dealt with. Overexposure results in lost colors and a strange halo softening that you don't have with underexposure.

[philipd]

It is going to be subject dependant to what can be recovered or not and how that recovery looks and test charts are always going to be easy to bring back from the brink, as you only have 24 values of colour represented.  As you stretch and squash things out to try and recover information, then there are gaps or data being lost, but you will not see this loss with a limited colour palate on a test chart. For example there are only 4 shades of grey, so you aren't going to notice that you've lost 100+ shades of grey due to having to grade it so heavily or due to under or over exposure.  Having a grey scale and a gradient of colours on a test chart will show up this loss.

[John Matthews]

4 hours ago, philipd said:

It is going to be subject dependant to what can be recovered or not and how that recovery looks and test charts are always going to be easy to bring back from the brink, as you only have 24 values of colour represented.  As you stretch and squash things out to try and recover information, then there are gaps or data being lost, but you will not see this loss with a limited colour palate on a test chart. For example there are only 4 shades of grey, so you aren't going to notice that you've lost 100+ shades of grey due to having to grade it so heavily or due to under or over exposure.  Having a grey scale and a gradient of colours on a test chart will show up this loss.

Those are good points. I would argue 10bit to 8bit comparisons almost never look like the 10bit file is 4x better. When shot properly, the 8bit file often indistinguishable to the 10bit file. The real question is: “Exactly how much can you screw 8bit up and still fix it?”

[Walter H]

This is interesting, @kye, thanks for putting 8-bit back out there. Nice to be reminded that there is more latitude than I've perhaps remembered.

I've shot log almost exclusively for the past five years or so except for under-lit situations. I found 10bit vlog so easy to work with that I came not to question the extra bit of workflow vs the headache of getting anything in 8bit wrong. 

[zlfan]

10 bit vs 8 bit. to make 10 bit really better than 8 bit, the camera has to be very quiet. when the codec is thick and the camera is at high iso, noise cancels the limited advantage of 10 bit over 8 bit. and most modern cameras shoot log at native iso of 800 to 2000, at which starting to show noise, even though the latest cams have good low light capabilities. 

for log shooting, in order to protect high light, it is typically to expose at about 60%-70% ire, and this is typically darker or underexposed, or ettl on the histogram. for most digital sensors, ettr when shooting and then push half a stop to one stop down in post to reduce noise makes the image looking darn good. shooting log ettl then pull up half a stop or a stop, you will see heavy noise in the shade frequently. 

personally, i think 12 -16 bit raw shooting is the best if wanting to do post cg. 10 bit log does not have much more benefit over 8 bit rec 709 if you shoot 8 bit rec 709 sooc. the reason why 10 bit log is useful is to match the color of different cameras in post. nowadays, 10 bit log is used as a quick leeway to 12 bit raw, which is not the right approach in most of the cases, imho. 

[kye]

6 hours ago, Walter H said:

This is interesting, @kye, thanks for putting 8-bit back out there. Nice to be reminded that there is more latitude than I've perhaps remembered.

I've shot log almost exclusively for the past five years or so except for under-lit situations. I found 10bit vlog so easy to work with that I came not to question the extra bit of workflow vs the headache of getting anything in 8bit wrong. 

Absolutely, if you can shoot 10-bit Vlog and are happy to grade it in post then there's no reason not to.

The main motivation for me is really in dealing with the older or smaller cameras that don't have a log profile built in, and being able to realise as much of their potential as possible.  The GX85 is my main camera now, simply because of size and form-factor, but as we have been reminded by @John Matthews in the "World's smallest DSLM that shoots 4k?" thread, there is still a lot of love for these small cameras.

[kye]

There is one point I forgot to mention in the first post, which is the WHY of this whole situation.

The point of looking at exposure and WB latitude isn't to correct these things in post.  It is to understand how the camera reacts to colour grading.

When we colour grade, we are taking the image and pushing things around.  In a literal sense, we are just raising or lowering the value of the R/G/B channels in pixels based on a variety of factors, but it's just that - raising or lowering the value of each channel.
So, if a camera falls apart when I try to make a tiny adjustment then I'll know that I can't colour grade it much.

Assuming you get the exposure and WB relatively close in-camera then the changes should be relatively minor, but some changes are larger or smaller than others.  As some examples, here are the operations that I would do across most shots in a grade of mine:

• Adjusting WB - this is raising or lowering the values of the RGB channels against each other
• Adjusting exposure - raising/lowering RGB channels together
• Adjusting contrast - raising/lowering RGB channels together but doing the shadows and highlights in opposite directions
• Adjusting saturation - raising/lowering RGB channels in a way that expands the differences between channel values
• Applying a subtractive saturation effect - lowering the RGB channels together based on the differences between channel values
• Brightening faces - raising the RGB channels in an area on and around the selected faces
• Massaging skintones - raising/lowering the RGB channels to subtly change the hue and saturation
• Cool shadows - lowering the RG channels in the shadows

etc etc..  as you can see, all these things are just subtle changes to the RGB channels, which relies on the latitude of the files.  Most of these changes will be adjusting values by 1 stop or less, and sometimes a LOT less (like skintones), but some might also be more (cooler shadows).  

I did an experiment on bit-depth and wrote a plugin that lowered bit-depth, and discovered that taking a final image down to 6-bits wasn't visible (and some were fine with 5-bits!) but if you tried to colour grade those 5-bit images then you'd discover pretty quickly that even very minor adjustments would annihilate the image entirely.

The other thing that occurs to me is that there are things you can do to help the image keep from falling apart too, like adding noise etc.  I should explore that in future tests.

[IronFilm]

And this is why ProRes 10bit 422 is plenty for even quite "high budget" productions to shoot in, and they don't "need" raw at all. 

[zlfan]

"

We did the color correction for "Key & Peele" on a Davinci Resolve.

Of note is the fact that our recording medium for the season (after a horrific trial with one of the external 10-bit 4:2:2 recorders) was 50 Mbs XDCAM disc. We never lost a frame and we have hard masters of all of our footage, but I didn't learn until we were in post that that is an 8 bit standard. However we were able to do a tremendous amount of push/pull and the 8 bit never seemed to create problems. There is, of course, a notable difference between 35 and 50 Mbs.

The shots we did in non s-log (60fps material) were far more problematic.

__________________
Charles Papert"

 

 

i think he did the whole season 1 on key and peele with f3 s log to 8 bit nano flash 50 mbits mpeg2 long gop. cg is really good. something to consider though, f3 has a nice sensor and good dsp; he had good budget on lighting, maybe he used one stop lower than the native iso to reduce noise; most of the scenes were staged so he could control dr. but the final results are amazing for 8 bit 50 mbits mpeg2 long gop. 

 

https://vimeo.com/channels/keypeele

 

 

 

[zlfan]

2 hours ago, kye said:

There is one point I forgot to mention in the first post, which is the WHY of this whole situation.

The point of looking at exposure and WB latitude isn't to correct these things in post.  It is to understand how the camera reacts to colour grading.

When we colour grade, we are taking the image and pushing things around.  In a literal sense, we are just raising or lowering the value of the R/G/B channels in pixels based on a variety of factors, but it's just that - raising or lowering the value of each channel.
So, if a camera falls apart when I try to make a tiny adjustment then I'll know that I can't colour grade it much.

Assuming you get the exposure and WB relatively close in-camera then the changes should be relatively minor, but some changes are larger or smaller than others.  As some examples, here are the operations that I would do across most shots in a grade of mine:

• Adjusting WB - this is raising or lowering the values of the RGB channels against each other
• Adjusting exposure - raising/lowering RGB channels together
• Adjusting contrast - raising/lowering RGB channels together but doing the shadows and highlights in opposite directions
• Adjusting saturation - raising/lowering RGB channels in a way that expands the differences between channel values
• Applying a subtractive saturation effect - lowering the RGB channels together based on the differences between channel values
• Brightening faces - raising the RGB channels in an area on and around the selected faces
• Massaging skintones - raising/lowering the RGB channels to subtly change the hue and saturation
• Cool shadows - lowering the RG channels in the shadows

etc etc..  as you can see, all these things are just subtle changes to the RGB channels, which relies on the latitude of the files.  Most of these changes will be adjusting values by 1 stop or less, and sometimes a LOT less (like skintones), but some might also be more (cooler shadows).  

I did an experiment on bit-depth and wrote a plugin that lowered bit-depth, and discovered that taking a final image down to 6-bits wasn't visible (and some were fine with 5-bits!) but if you tried to colour grade those 5-bit images then you'd discover pretty quickly that even very minor adjustments would annihilate the image entirely.

The other thing that occurs to me is that there are things you can do to help the image keep from falling apart too, like adding noise etc.  I should explore that in future tests.

help the image keep from falling apart too, like adding noise

true.

 

[zlfan]

5 minutes ago, zlfan said:

help the image keep from falling apart too, like adding noise

true.

 

because noise can overcast banding. banding is worse than noise. 

[John Matthews]

When weighing 10bit versus 8bit, the debate was particularly pronounced in the era of 1080p. While 1080p 10bit, depending on the camera, has issues like moiré and aliasing, these concerns are often gone with 4k 8bit. The redeeming quality of 10bit is in H.265, where file sizes remain more manageable, only marginally larger than 4k 8bit H.264.

IMO:

4k 8bit 4:2:0 > 1080p 10bit 4:2:2 due to:

• Absence of banding
• Diminished moiré
• Reduced aliasing
• Enhanced detail and cropping capabilities

While 4k 10bit holds an objective and mathematical superiority over 4k 8bit, its perceptible advantages are often overlooked by many.