ETTR: Noise, Dynamic Range and Skin Tones

[jonpais]

I just did a very quick test with the Lumix G85 in natural light which leaves little doubt that there is greater dynamic range and less noise in the shadows when exposing to the right. Colors are a little trickier, first of all, because I'm not a professional colorist, so I couldn't exactly match the ETTR clip to the one that was underexposed by 1/3 of a stop. Also, detail that is plainly visible in the shadows in the original ETTR clip is obscured somewhat by YouTube compression. But the most startling revelation is that noise in the shadows, which manifests itself as ugly macroblocking due to YT compression, is somewhat reduced by ETTR. Look at the back of the flapjack LED to the left in the ETTR clips, in which macroblocking is virtually absent, and compare to the clip that is underexposed by 1/3 of a stop. Many YT and Vimeo videos suffer from this unsightly macroblocking as a result of underexposure. Applying the Leeming LUT to the footage would have restored the proper rec709 color space. My takeaway is that, while correct exposure is critical, proper white balance in camera and in post is paramount for good skin tones. But the fact of the matter is, the G85 is a great camera for the money, content is king, and I wouldn't get my panties all twisted up in a bunch over exposure. 

 

[jase]

Thanks jon. I think I will try to do some tests on my own to see whether I can see the difference in my work. 

On a sidenode: god, whenever I watch your videos I feel the urge to buy the Summilux 12mm at an instant. As soon as we get the successor of the G85 that will enable the Teleconverter option also in 4k, this will be it for me.

[Fredrik Lyhne]

@jonpais Are you sure this is ETTR? Looks more like +/-0. Can you post a frame of the ETTR soon so I can see the histogram i Lightroom? 

[jonpais]

49 minutes ago, Fredrik Lyhne said:

@jonpais Are you sure this is ETTR? Looks more like +/-0. Can you post a frame of the ETTR soon so I can see the histogram i Lightroom? 

It is really very slight, as I say in the video, I actually opened up the aperture just until there were zebras (@100%) on my t-shirt and used that as my ETTR shot. I am working on another, far more detailed video, which I hope to post very soon. 

[jonpais]

@Fredrik Lyhne I uploaded to YT directly from within Final Cut. I will be uploading the master file to YT tomorrow to see whether the macroblocking is still a problem. 

[Axel]

19 hours ago, jonpais said:

But the most startling revelation is that noise in the shadows, which manifests itself as ugly macroblocking due to YT compression, is somewhat reduced by ETTR.

 

12 hours ago, jonpais said:

@Fredrik Lyhne I uploaded to YT directly from within Final Cut. I will be uploading the master file to YT tomorrow to see whether the macroblocking is still a problem. 

Macroblocking in the shadows is the encoder deciding that noise is no valid signal and overly simplifying the area. In case of a H.264 master, that distinction is in the upload file already and only gets worse when the original is reduced again by the YT processing. There are three strategies against that: 

1. Get rid of noise using Neat Video. The downside to this is that, no matter how well you know the expert mode of Neat, you will always also sacrifice texture detail. Which to protect by any means is the idea of 4k.

2. Dithering shadows by either keeping noise or applying grain. The first approach is unreliable and can't be fully controlled. The grain structure has been proven fail-save. The amount of the grain needs thorough testing, but you can do it before the upload. Philipp Bloom does this with FilmConvert. He also sometimes uploaded ProResLT, but this didn't make a huge difference - if at all - in my own tests on Vimeo. The grain (one can use the built-in grain effect of FCP) also reliably helps with fade-ins and fade-outs that can cause temporal banding.

3. ETTR. With less noise, there are less encoding errors.

[jonpais]

14 minutes ago, Axel said:

 

Macroblocking in the shadows is the encoder deciding that noise is no valid signal and overly simplifying the area. In case of a H.264 master, that distinction is in the upload file already and only gets worse when the original is reduced again by the YT processing. There are three strategies against that: 

1. Get rid of noise using Neat Video. The downside to this is that, no matter how well you know the expert mode of Neat, you will always also sacrifice texture detail. Which to protect by any means is the idea of 4k.

2. Dithering shadows by either keeping noise or applying grain. The first approach is unreliable and can't be fully controlled. The grain structure has been proven fail-save. The amount of the grain needs thorough testing, but you can do it before the upload. Philipp Bloom does this with FilmConvert. He also sometimes uploaded ProResLT, but this didn't make a huge difference in my own tests on Vimeo. The grain (one can use the built-in grain effect of FCP) also reliably helps with fade-ins and fade-outs that can cause temporal banding.

3. ETTR. With less noise, there are less encoding errors.

I am in the process of applying Neat Video to the shot that's underexposed by 1/3 of a stop (the one with the worst artifacts) and will be uploading it along with the same clip with no noise reduction to see how it affects macroblocking. Should be done in a few minutes, unless YT isn't cooperative this afternoon.  

[jonpais]

@Orangenz @Fredrik Lyhne @Axel

 

[Axel]

Not gone, but noticeably reduced to the point where it's no longer irritating. Agreed?

This is pathologic pixelpeeping anyway, isn't it?

One should try Neat, grain and a ProResLT. Maybe it's just that there are too few values to represent the 50 shades of grey of he flapjack, and with this data rate macroblocking can't be stopped at all.

[Orangenz]

47 minutes ago, Axel said:

This is pathologic pixelpeeping anyway, isn't it?

Quoted for truth. And as a famous person said:

Quote

But the fact of the matter is, the G85 is a great camera for the money, content is king, and I wouldn’t get my panties all tied up in a bunch over exposure.

It was worth it in terms of pointing out the bleeding obvious to those who won't care anyway that underexposing degrades a digital signal unnecessarily. It's really very similar to the recording scale on any digital recorder. There's usually a triangle indicator about 18 dB down and this is the sweet spot for audio. If you look at an ETTR histogram it maps the same distribution - max dynamic range.

[jonpais]

@Axel @Orangenz @Fredrik Lyhne I did another more pathologically detailed test this afternoon, this time, along with glorious screen grabs of the waveform monitor, and a couple shots of the vector scope readings for both the ETTR clip and the grossly underexposed one to show how exposure affects skin tones. To my way of thinking, the vector scope is showing far greater color information in the ETTR shot, and it's practically sitting on the skin tone line, so I don't see why it would give someone any trouble grading. As I've said before, I think proper WB in camera is key to good skin tones.

 

[mercer]

I really don't want to beat a dead horse, but what is your on camera monitor calling your ETTR shot?

Because to me, that isn't ETTR and that looks like +/- 0.

At 90% zebras, what f-stop did they disappear at?

I don't think you can truly ETTR by dropping a full stop. Those zebras could have disappeared at any point during that stop drop. 

Plus, you ETTR your subject, so technically the zebras should have been pulled back from your face, not your shirt. 

Which is another problem with ETTR on these 8bit cameras, you end up with a lot more work in post fixing your exposure. Not a big deal for small videos, but another point any beginners reading should be aware of. 

[Arikhan]

@jonpais

Quote

I've said before, I think proper WB in camera is key to good skin tones.

Exactly THIS is a very critical point for many Pana consumer cams.. AWB can be (specially in warmish sunset scenarios) a kind of poison for skin tones. You have to properly set your WB manually and reset every time the lighting situation (color temperature) changes - even as little bit. You can't trust Pana AWB in some lighting situations at all, the skin tones go so bad, as you can throw away your footage. In my case (sunset scenario) it helped to substancially cool up color balance in "Standard" mode. For Cine D, Portrait, etc., consistency didn't work at all, WB remaining very critical (concerning skin tones), even after cooling up colors. 

You have just to give a proper WB a little bit of love. AWB is generally completely unreliable when filming people with Pana consumer cams. You mostly get a mixed bag...I can mostly trust Nikon AWB, but never Pana (when filming)...

[Fredrik Lyhne]

2 hours ago, mercer said:

I really don't want to beat a dead horse, but what is your on camera monitor calling your ETTR shot?

Because to me, that isn't ETTR and that looks like +/- 0.

At 90% zebras, what f-stop did they disappear at?

I don't think you can truly ETTR by dropping a full stop. Those zebras could have disappeared at any point during that stop drop. 

Plus, you ETTR your subject, so technically the zebras should have been pulled back from your face, not your shirt. 

Which is another problem with ETTR on these 8bit cameras, you end up with a lot more work in post fixing your exposure. Not a big deal for small videos, but another point any beginners reading should be aware of. 

Doesn't look like ETTR on the histogram either and it sure looks a lot different from Noam Kroll's ETTR.

https://www.provideocoalition.com/use-ettr-technique-maximize-dynamic-range/

I believe when Paul Leeming used the term ETTR to explain how to use his LUT he should have explained that this is really ETTR light version or something. I took the shot off Paul holding the color checker and checked the histogram in Lightroom and it's very different from the one in his picture. Looks like he just used a another histogram to make his point, and this can be a little confusing when you're trying to match Paul's histogram in a shot with skin tones cause you could very easily ending up overexposing and possibly ruining the shot. ETTR is for raw, log and maybe jpegs/8 bit video without skin tones. 

If you look at Paul's images and explanation I would agree that you should adjust the exposure until the zebras disappears from the face, but that is definitely not a good idea. 

http://www.leeminglutone.com/#howtoettr

But's kind of funny how we're arguing over this, but we're actually kinda agreeing  

Thanks for the tests @jonpais!

[Fredrik Lyhne]

I'm really trying but there is no way I can fix that ETTR shot, especially the shirt. Could be my grading skills though...

 

 

[jonpais]

First of all, thanks everyone for taking the time to give feedback! Very interesting to read your responses, as well as being able to see comparisons between my work and that of Paul Leeming and Noam Kroll.

@Arikhan I don't use AWB since it doesn't give consistent results, and because I'm usually shooting outdoors where the light is constantly changing, even during the course of a single shot, it would make CC troublesome, though some people seem to be having good luck with AWB.

@mercer I use 100% zebras, not 90%, so I couldn't say at which f-stop the zebras would have disappeared in that scenario. For the very first exposure, my polo is completely covered with zebras on camera right. If the shirt had occupied a smaller portion of the screen, or if I'd been wearing a black shirt, I could have raised exposure even more. Because my understanding of ETTR is that you want to protect the important highlights, and that would include my shirt in this case. I changed exposure in increments of 1/3 of a stop, not full stops.

@Fredrik Lyhne Gosh darn it, you're right! Paul's histogram is a little misleading, since it's obviously not representative of the light values in the image. 

While we're all focusing on the highlights, it's also important to remember that we're still exposing for the shadows as well, and if you look at the waveform monitor for what I consider to be a proper ETTR shot, the shadows are raised considerably above 0 IRE, whereas in the underexposed shots, the information is all bunched up in the lower portion of the scope. And whether my version is ETTR Lite  or not, I hope these videos stimulate others to do their own tests.

 

16 minutes ago, Fredrik Lyhne said:

I'm really trying but there is no way I can fix that ETTR shot, especially the shirt. Could be my grading skills though...

 

 

I also think it's too bright, which I think proves that it is definitely not a normal exposure, and in practice, I would also have chosen to expose a little bit darker. No one should blindly follow any one method, you've got to use good judgement as well.

[Fredrik Lyhne]

44 minutes ago, jonpais said:

First of all, thanks everyone for taking the time to give feedback! Very interesting to read your responses, as well as being able to see comparisons between my work and that of Paul Leeming and Noam Kroll.

@Arikhan I don't use AWB since it doesn't give consistent results, and because I'm usually shooting outdoors where the light is constantly changing, even during the course of a single shot, it would make CC troublesome, though some people seem to be having good luck with AWB.

@mercer I use 100% zebras, not 90%, so I couldn't say at which f-stop the zebras would have disappeared in that scenario. For the very first exposure, my polo is completely covered with zebras on camera right. If the shirt had occupied a smaller portion of the screen, or if I'd been wearing a black shirt, I could have raised exposure even more. Because my understanding of ETTR is that you want to protect the important highlights, and that would include my shirt in this case. I changed exposure in increments of 1/3 of a stop, not full stops.

@Fredrik Lyhne Gosh darn it, you're right! Paul's histogram is a little misleading, since it's obviously not representative of the light values in the image. 

While we're all focusing on the highlights, it's also important to remember that we're still exposing for the shadows as well, and if you look at the waveform monitor for what I consider to be a proper ETTR shot, the shadows are raised considerably above 0 IRE, whereas in the underexposed shots, the information is all bunched up in the lower portion of the scope. And whether my version is ETTR Lite  or not, I hope these videos stimulate others to do their own tests.

 

I also think it's too bright, which I think proves that it is definitely not a normal exposure, and in practice, I would also have chosen to expose a little bit darker. No one should blindly follow any one method, you've got to use good judgement as well.

Haha, glad we're finally on the same page! Your test's have been very helpful and this really shows that one should pay more attention to exposure. 

Paul's histogram is indeed very misleading and I think it's just a case of an expert trying to explain something to non-experts but forget that they are not experts! 

The shots I graded are ETTR and -2/3 but I didn't try -1/3, maybe that one will work too? I find it a bit strange that your shirt was overexposed beyond repair when you followed Paul's advice with 100% zebras. Did you try to grade the different shots to see which was easiest for you? 

But you are right about the noise Jon, it's less noise in the shadows when done right, but unlike raw theres only a very small window where color, noise and highlights are perfect.

[jonpais]

@Fredrik Lyhne That particular shirt is pretty bright. I didn't try grading the clip, but I can tell just by looking at it that that particular shirt is ridiculously ETTR resistant.  For the record, up until only very recently, I'd been underexposing all my shots without being aware of it, so I confess to being a little gung-ho about ETTR. If only the G85 had just a touch more dynamic range! I'd really like to do a comparison between the Lumix and my X-T2 someday.

[jonpais]

I went ahead and shot a few more clips wearing the same notorious polo shirt. If anyone wants to download the file (actually, a screen shot), along with a picture of the waveform monitor and have a go at correcting it, feel free!

[jagnje]

Wow, you guys are really taking this to the next level

I have since forever exposed my footage to the right, even now that I shoot with log profile of the fs7. The reasson, it's how the companys colorist wants it. 

That being said, it's just preference. The differences in this cheap 8 bit cameras are so minimal that it hardly makes a difference. These are toy cameras that can give great results in the right hands and in the right circumstances. Expose as you wish and film what you like.