Are S-LOGS More Destructive Than They're Worth?

[cantsin]

12 minutes ago, maxotics said:

@cantsin and @iaremrsir you keep trying to explain this to me as if I just stumbled upon video technology, as if I haven't written software that works with RAW data.  I can't force you to think through the ramifications of S-LOG. 

I feel you keep bringing up issues that obfuscate the core question.  What does compression have to do with the amount of noise in sensor data or 8-bit color spaces? Nothing!  

Sorry Max, but you simply don't get the issue. "Compression" can not just refer to data compression of a codec, but also to:

- aspect ratio/perspective compression through anamorphics;

- dynamic range compression through log profiles.

Anamorphic aspect ratio compression can kill your image if you have too little pixel resolution - for example, if you shoot anamorphic in SD resolution and desqueeze, you end up with pixel mush.

Dynamic range compression can kill your image if you have too little color depth - for example, if you shoot log in 8bit and desqueeze into RGB, you end up with color banding and maybe 6bit/7bit actual color depth in the final image.

[maxotics]

1 hour ago, cantsin said:

Sorry Max, but you simply don't get the issue. "Compression" can not just refer to data compression of a codec, but also to:

- aspect ratio/perspective compression through anamorphics;

- dynamic range compression through log profiles.

Anamorphic aspect ratio compression can kill your image if you have too little pixel resolution - for example, if you shoot anamorphic in SD resolution and desqueeze, you end up with pixel mush.

Dynamic range compression can kill your image if you have too little color depth - for example, if you shoot log in 8bit and desqueeze into RGB, you end up with color banding and maybe 6bit/7bit actual color depth in the final image.

Hi @cantsin, I still don't see what I don't get? I understand that compression has many different meanings (like in audio and anamorphics), but isn't the core issue HERE how the S-LOG setting in a camera affects image quality?  I'll focus on "dynamic range compression through log profiles".  

The way of phrasing that misleads many people.  Most people have the idea that the right compression can improve video because, well, in general, it can!  So they read "A LOG profile allows you to get more dynamic range through your camera than you could using the camera's default settings (standard profile)."  Many people believe S-LOG is some new technology invented by Panasonic and Sony to get more dynamic range.  The fact is, it's been around professional 8-bit video for a long time.  As I discuss in my video, Canon and Nikon don't include S-LOG because they still don't believe the trade-off is worth it for most people.  That trade-off, again, is you get that extra dynamic range (though compression if you want to call it that) by substituting high noise visual data for low noise data.  There is no free lunch here.  If you come from a photographic background, you try to avoid that trade-off unless you absolutely can't help it.

Because most video has a washed out look these days people don't notice this.  But it's there.  And one day, when tastes change, and people try to re-grade their S-LOG video to new, fully saturated, noise-free tastes, they'll see exactly what I'm talking about

I can't stress this enough.  The world around us in in 20 stops of dynamic range.  But we watch ALL video in 6-stops of dynamic range.  Even if you could capture the physical world in all its 20 stops of dynamic range, with NO noise, you still have to fit your data into 6 stops of DR.  So again, if the end result in 6 stops, why wouldn't you want those 6 stops to be as fully saturated and noise-free as possible?  When one separates DR into those two worlds, physical OR perceptual (viewing) then they can understand the limitations/trade-offs of how technology translates one into the other.

In these three screen captures you can see an illustration of the issues.  The first is S-LOG interpreted under a linear (or standard gamut) curve, so looks washed out.  The second is that S-LOG frame "graded" so that it matches out visual expectations of full-color saturation.  The last image is a standard profile screenshot where you can see the benefits of full-saturated colors, both in color depth, and less-noise sharpness (in fact, you can see the aliasing from the monitor pixels!).

If you want to capture the monitor in full fidelity, why wouldn't you shoot a standard profile?  Who would honestly say the graded S_LOG is superior?  if you want to get my office out of the darkness (DR), see some detail there, then yes, the S-LOG does that!  

I'll say again, why would any DP use S-LOG to get the monitor and my office in the DR he/she wants when, if they had lighting, they could bring up my office into  the DR of the monitor and get EVERYTHING noise-free and fully saturated?  And, if they did want a scrubby look for my office, noise is easy to add in post.

[Deadcode]

1 hour ago, maxotics said:

 

In these three screen captures you can see an illustration of the issues.  The first is S-LOG interpreted under a linear (or standard gamut) curve, so looks washed out.  The second is that S-LOG frame "graded" so that it matches out visual expectations of full-color saturation.  The last image is a standard profile screenshot where you can see the benefits of full-saturated colors, both in color depth, and less-noise sharpness (in fact, you can see the aliasing from the monitor pixels!).

 

If you want to capture the monitor in full fidelity, why wouldn't you shoot a standard profile?  Who would honestly say the graded S_LOG is superior?  if you want to get my office out of the darkness (DR), see some detail there, then yes, the S-LOG does that!  

I'll say again, why would any DP use S-LOG to get the monitor and my office in the DR he/she wants when, if they had lighting, they could bring up my office into  the DR of the monitor and get EVERYTHING noise-free and fully saturated?  And, if they did want a scrubby look for my office, noise is easy to add in post.

Could you please upload this image  somewhere? i wanna try the same thing like you did. I think your SLOG grade is off, im curious if i can match exactly to normal profile then can i see any difference? if your explanation is correct we must see huge difference in tonality if the slog is perfectly matched to standard. But i think there wont be any difference, we will see.

[EthanAlexander]

The misunderstanding is from a misalignment of ultimate goals: I think you're just going about this as "how do I get the most saturated, punchy look?" in which case you can't really beat shooting a standard profile. But to my eyes, the last one of your examples looks like shit, and my goal in filming is rarely to make people's eyes bleed with color like that.

I'm being hyperbolic here, but really it's a matter of goals, and that's why people are disagreeing with you, @maxotics. Feature films don't look like rec709. They just don't. A lot of this has to do with the fact that movies are NOT supposed to look like real life. There's an art to how the colors of the set, wardrobe, lighting, and color grade create feelings, and S Log and wider color gamuts allow for this creativity whereas shooting a standard profile will make it look like a live TV production.

[iaremrsir]

5 hours ago, maxotics said:

@cantsin and @iaremrsir you keep trying to explain this to me as if I just stumbled upon video technology, as if I haven't written software that works with RAW data...

Which is why you need to work harder in thinking about how the image data gets to your NLE and how what you tell the camera to do, how to capture the data, what I'm calling its default "standard profile" or S-LOG, has different effects on the image then what curve you choose in post.  If the camera manufacturers had called S-LOG what it really is, mixed ISO shooting there might not be so much confusion.

Hi, I'm Eddie. I designed the Bolex Log color specification and the image processing pipeline for Digital Bolex towards the end of its production run. Also wrote the plugin that lets people process CineForm Raw color in Davinci Resolve as if it were DNG. And I'm not saying this to get into a pissing contest. I'm saying this as someone who is on the manufacturer side of things and has to know the ins and outs of the product and how it's used with other tools.

5 hours ago, maxotics said:

I feel you keep bringing up issues that obfuscate the core question.  What does compression have to do with the amount of noise in sensor data or 8-bit color spaces? Nothing!  

Shoot 8-bit H.264 from a Canon C100 or any of their DSLRs for that matter, then compare it to ProRes recorded over HDMI to an external device. You'll often seen that the ProRes has more noise and fine details/texture. This is because H.264 smooths out the noise in camera and HDMI outputs uncompressed data. You're speaking as if the main reason detail is lost in 8-bit is that it's the log curve, when in reality, the main loss of detail has been heavy compression. I already agreed that using a log curve in 8-bit will redistribute code values so that more space is given to the mids and shadows, meaning less codes per stop when storing HDR data. We're not debating sensor data, otherwise we'd be talking analog stages, 16-bit, 14-bit, uncompressed. So, in this case, compression has everything to do with the image data.

Quote

You wrote, "You're applying a line of thought that suggests these log specifications are applied at the analog stage before digitization"  YES I AM!  Though not in the way you say because "log specifications" are not a thing you apply in one place or another.  ALL DATA must be married to a curve in the end.  Where you think that curve should be applied is a philosophical one.  You talk about it as if it is some tool.  What I AM SAYING is that S-LOG pulls noisy data from the sensor outside the bounds of what it is engineered to do best and what its physical limitations are.  It has nothing to do with whether it is before or after digitization.  In a sense, you keep arguing that S-LOG can fix noisy data.  I say it can't.

If you apply a logarithmic gain at the analog stage (ignoring the electron multiplying case), noise would be much higher than if you applied the log curve after digitization. It's not pretty. Definitely not philosophical in any measure. And these log curves can't pull data that isn't there. It's well known that using a log curve will boost the appearance of recorded noise in any bit depth, color space, etc. I'm not arguing that log fixes noisy data. I'm arguing that recording log allows you to record data in a way that keeps more detail across the range and expose in such a way that allows you to better minimize the appearance of noise in post (ETTR). Trust me, I let out a heavy sigh any time I try to see someone compensating for low light scene or poor scene lighting by recording log.

Quote

I never said Cinema/TV doesn't use S-LOG.  Where did I say that?

I didn't say you said they don't. But you said professionals are lighting within 6-stop ranges. They have millions to pour into set design, lighting, and wardrobe. I was just pointing out the fact, that in spite of them lighting like that, they still shoot log or raw (which is later interpreted as log in the grading tool). Also, they're gonna be shooting 10-bit, 12-bit, and 16-bit more often than not, so they aren't worried about losing code words per stop, which takes away the main argument of using a standard profile.

Quote

And I certainly NEVER, EVER said it has been used in RAW because that is flat-out non-nonsensical.

I didn't say you did. But it isn't nonsensical. Canon raw has C-Log applied to it before being sent over SDI to external recorders for being saved as rmf. ARRI, BMD, CineForm, all write their raw formats with log as part of the specification. While it's not the Cineon-type log they use, it's log nonetheless.

Quote

 I would also never say "graded in log" because that is the same as saying "graded in graded" because you cannot do much grading without changing a curve.

But you can grade in a log color space. Hence Davinci Resolve Color Managed timeline and ACEScc/ACEScct. Whenever you grade on top of log image data, your working/"timeline" color space is log.

Quote

Again, if you get the results you want using your workflows, great!  In the end, that is ALL the matters.  But that doesn't mean what you do will work for the next guy.  Arguing that the other guy doesn't need mid-tone values because S-LOG can do this or that is, to me, artistically arrogant and indicative of a half-baked understanding of photography.  I am NOT saying you're a bad filmmaker because you use LOG, far from it!  But I will call anyone, even Oliver, on an argument that one can make S-LOG do anything if they know what they're doing.  If that' is true, I want to know how!  But you have to tell me how you get around the physics of image capturing as it exists today.  I have only heard rationales for the trade-offs, not reasons why S-LOG is always preferable.

This is where we get out of the realm of objectivity. Because there are technical trade-offs for both sides, it's up to who's shooting as to which is preferable. There is no clearly defined technical mathematical winner in this case (which is one of the reasons I'm happy I don't have to deal with H.264 compressed, 8-bit recorded data anymore). And I'm not saying there is one that checks all the boxes in this case.

3 hours ago, maxotics said:

Hi @cantsin, I still don't see what I don't get? I understand that compression has many different meanings (like in audio and anamorphics), but isn't the core issue HERE how the S-LOG setting in a camera affects image quality?  I'll focus on "dynamic range compression through log profiles".  

The way of phrasing that misleads many people.  Most people have the idea that the right compression can improve video because, well, in general, it can!  So they read "A LOG profile allows you to get more dynamic range through your camera than you could using the camera's default settings (standard profile)."  Many people believe S-LOG is some new technology invented by Panasonic and Sony to get more dynamic range.  The fact is, it's been around professional 8-bit video for a long time.  As I discuss in my video, Canon and Nikon don't include S-LOG because they still don't believe the trade-off is worth it for most people.  That trade-off, again, is you get that extra dynamic range (though compression if you want to call it that) by substituting high noise visual data for low noise data.  There is no free lunch here.  If you come from a photographic background, you try to avoid that trade-off unless you absolutely can't help it.

When I say "retain detail through compression" I'm talking about codec compression. Like I stated earlier, a standard profile combined with high levels of compression will reduce texture in the low mids and lows. It reduces flexibility and the overall naturalness of the image.

Quote

Because most video has a washed out look these days people don't notice this.  But it's there.  And one day, when tastes change, and people try to re-grade their S-LOG video to new, fully saturated, noise-free tastes, they'll see exactly what I'm talking about

That is an issue of grading, not the source material. When I grade log material, I have no issue getting thick colors from it.

Quote

I can't stress this enough.  The world around us in in 20 stops of dynamic range.  But we watch ALL video in 6-stops of dynamic range.  Even if you could capture the physical world in all its 20 stops of dynamic range, with NO noise, you still have to fit your data into 6 stops of DR.  So again, if the end result in 6 stops, why wouldn't you want those 6 stops to be as fully saturated and noise-free as possible?  When one separates DR into those two worlds, physical OR perceptual (viewing) then they can understand the limitations/trade-offs of how technology translates one into the other.

Again, log doesn't add noise. It just increases the appearance. Once the image is graded, the appearance of the noise will look similar to that of the standard profile, but the texture of the shadows will look more natural, especially in motion.

Quote

In these three screen captures you can see an illustration of the issues.  The first is S-LOG interpreted under a linear (or standard gamut) curve, so looks washed out.  The second is that S-LOG frame "graded" so that it matches out visual expectations of full-color saturation.  The last image is a standard profile screenshot where you can see the benefits of full-saturated colors, both in color depth, and less-noise sharpness (in fact, you can see the aliasing from the monitor pixels!).

But you didn't grade to match which means the higher contrast and extra sharpness of the standard profile will have increased perceived detail of the aliasing and text. And in spite of that, we can see that the aliasing that you point out in the standard profile is present in the log image! When graded down to match, you'll get similar, if not exact sharpness from the log image. The difference between the two being flexibility. I took the log one you posted and graded it down more closely to what your standard profile looked like without the boost in saturation and you can see the apparent sharpness is similar and the noise isn't as much as you were making it out to be.

 

[EthanAlexander]

^That's the most informative post I've seen in a very long time, @iaremrsir

[srgkonev]

Hot topic! Guys, so what about non-log "flat" profiles like Cine-D and Flat Nikon profile? If its a linear color space, do we completle lose those information opposite to logarithmic color space?

[maxotics]

3 hours ago, iaremrsir said:

Hi, I'm Eddie. I designed the Bolex Log color specification and the image processing pipeline for Digital Bolex towards the end of its production run. Also wrote the plugin that lets people process CineForm Raw color in Davinci Resolve as if it were DNG. And I'm not saying this to get into a pissing contest. I'm saying this as someone who is on the manufacturer side of things and has to know the ins and outs of the product and how it's used with other tools.

Shoot 8-bit H.264 from a Canon C100 or any of their DSLRs for that matter, then compare it to ProRes recorded over HDMI to an external device. You'll often seen that the ProRes has more noise and fine details/texture. This is because H.264 smooths out the noise in camera and HDMI outputs uncompressed data. You're speaking as if the main reason detail is lost in 8-bit is that it's the log curve, when in reality, the main loss of detail has been heavy compression. I already agreed that using a log curve in 8-bit will redistribute code values so that more space is given to the mids and shadows, meaning less codes per stop when storing HDR data. We're not debating sensor data, otherwise we'd be talking analog stages, 16-bit, 14-bit, uncompressed. So, in this case, compression has everything to do with the image data.

If you apply a logarithmic gain at the analog stage (ignoring the electron multiplying case), noise would be much higher than if you applied the log curve after digitization. It's not pretty. Definitely not philosophical in any measure. And these log curves can't pull data that isn't there. It's well known that using a log curve will boost the appearance of recorded noise in any bit depth, color space, etc. I'm not arguing that log fixes noisy data. I'm arguing that recording log allows you to record data in a way that keeps more detail across the range and expose in such a way that allows you to better minimize the appearance of noise in post (ETTR). Trust me, I let out a heavy sigh any time I try to see someone compensating for low light scene or poor scene lighting by recording log.

I didn't say you said they don't. But you said professionals are lighting within 6-stop ranges. They have millions to pour into set design, lighting, and wardrobe. I was just pointing out the fact, that in spite of them lighting like that, they still shoot log or raw (which is later interpreted as log in the grading tool). Also, they're gonna be shooting 10-bit, 12-bit, and 16-bit more often than not, so they aren't worried about losing code words per stop, which takes away the main argument of using a standard profile.

I didn't say you did. But it isn't nonsensical. Canon raw has C-Log applied to it before being sent over SDI to external recorders for being saved as rmf. ARRI, BMD, CineForm, all write their raw formats with log as part of the specification. While it's not the Cineon-type log they use, it's log nonetheless.

But you can grade in a log color space. Hence Davinci Resolve Color Managed timeline and ACEScc/ACEScct. Whenever you grade on top of log image data, your working/"timeline" color space is log.

This is where we get out of the realm of objectivity. Because there are technical trade-offs for both sides, it's up to who's shooting as to which is preferable. There is no clearly defined technical mathematical winner in this case (which is one of the reasons I'm happy I don't have to deal with H.264 compressed, 8-bit recorded data anymore). And I'm not saying there is one that checks all the boxes in this case.

When I say "retain detail through compression" I'm talking about codec compression. Like I stated earlier, a standard profile combined with high levels of compression will reduce texture in the low mids and lows. It reduces flexibility and the overall naturalness of the image.

That is an issue of grading, not the source material. When I grade log material, I have no issue getting thick colors from it.

Again, log doesn't add noise. It just increases the appearance. Once the image is graded, the appearance of the noise will look similar to that of the standard profile, but the texture of the shadows will look more natural, especially in motion.

But you didn't grade to match which means the higher contrast and extra sharpness of the standard profile will have increased perceived detail of the aliasing and text. And in spite of that, we can see that the aliasing that you point out in the standard profile is present in the log image! When graded down to match, you'll get similar, if not exact sharpness from the log image. The difference between the two being flexibility. I took the log one you posted and graded it down more closely to what your standard profile looked like without the boost in saturation and you can see the apparent sharpness is similar and the noise isn't as much as you were making it out to be.

 

Hi Eddie, a simple question, when an EOSHD reader shoots an aggressive S-LOG on their A6300/A6500, say, are they getting more noise and less color than what they would get shooting a standard profile?   You keep bringing up situations that is not consumer S-LOG, like uncompressed HDMI or RAW.   You keep looking for holes in my arguments instead of asking if they are correct in real-world shooting for consumer S-LOG.  You say you have designed log profiles for the Bolex.  COOL!!!!  What was the difference in noise when you grade with LOG in 10 stop dynamic range vs 6 stop dynamic range.  Putting aside the fact that a Bolex is hardly consumer.

You wrote "You're speaking as if the main reason detail is lost in 8-bit is that it's the log curve, when in reality, the main loss of detail has been heavy compression."  Fine.  How much!  I calculated 30% based on the number of unique colors I could count in each.  YES, they're H.264 compressed, but since they're BOTH compressed shouldn't the effect of compression be equal for each?  What are your statistics of noise for your LOG and linear profiles?  

Nice job on the image, yes, better than I could get!

[EthanAlexander]

Can this get deleted please, mod? I decided against my own post

[maxotics]

9 minutes ago, EthanAlexander said:

We've been answering your own question but it seems in reality you only wanted people to agree with you.

You're clearly a very intelligent person, don't get me wrong, but it seems like @iaremrsir has done his best to address all your concerns and you aren't listening.

I think we all want to be agreed with   That said, why is it I who is not listening?  What am I wrong about?  He has said himself he doesn't dispute much of what I've said.  He says that compression has more to do with noise than shooting S-LOG, fine, but that's his opinion!  He hasn't shown me any of his tests.  Why should I take his word for it over my experience?  Anyway, I've had to challenge him (us) to get that articulated.  We still have a ways to go because so far there are no facts about that, that we can both agree on.  We're getting there.  A little piss never hurt nobody   If I have to play the bad guy in this thread, so be it   I'm sure he's happy to continue playing the good guy!

[EthanAlexander]

Yeah, I was trying to delete that before you saw it but I guess my first response got cached in your notifications. I didn't add anything to the discussion.

I will say though, since you just said "why take his word over your experience?" that it does seem like the only way forward is for you to experience these things we've been talking about. And I do still think it's a matter of end delivery and the goal of shooting: whether it's to have the most "realistic" picture or to give the most latitude for creative looks - and the extra latitude is something that iaremrsir has addressed head-on multiple times.

[maxotics]

1 hour ago, EthanAlexander said:

Yeah, I was trying to delete that before you saw it but I guess my first response got cached in your notifications. I didn't add anything to the discussion.

I will say though, since you just said "why take his word over your experience?" that it does seem like the only way forward is for you to experience these things we've been talking about. And I do still think it's a matter of end delivery and the goal of shooting: whether it's to have the most "realistic" picture or to give the most latitude for creative looks - and the extra latitude is something that iaremrsir has addressed head-on multiple times.

Hi Ethan, couldn't agree with you more!   As soon as I get a break from my "real" work I'm going to fire up the C100, A6300 and a Ninja Blade so I can analyze un-compressed HDMI data, as Eddie suggests.  We all want the same thing.  We all agree high dynamic range shooting in 8 bit has issues.  How much?  That's the question.  Again, for all I knew when I started this post, I'm completely off.  But I think the issue remains so I will continue work on it with everyone's help here.

[ReinisK]

Very interesting discussion, although I haven't read it all completely (will definetely do, when it's not 1:00 at night). I, too, have always had the same impression as maxotics (at least from the part I read).

That is - to get the best picture from an 8bit camera, the contrast (and saturation!) levels should be adjusted individually to each scene to waste less luma and saturation values. Therefore, the highest recorded (in camera) luma value should be 255(and only one such value) and the lowest should be 0.

But in real world you cannot do this because it takes too much time and color correction would be too hard (imagine trying to match shots each with a different contrast and saturation settings in camera).

So it is easier to just shoot in s-log, not thinking about each scene individually, and color-correct it afterwards.

With 8-bit S-log you lose the middle luma values (after grading), but with a contrasty profile you lose highlights (blown highlights) or shadows, if the DR of the scene is too high for that contrast setting.

 

[EthanAlexander]

18 hours ago, ReinisK said:

Very interesting discussion, although I haven't read it all completely (will definetely do, when it's not 1:00 at night). I, too, have always had the same impression as maxotics (at least from the part I read).

That is - to get the best picture from an 8bit camera, the contrast (and saturation!) levels should be adjusted individually to each scene to waste less luma and saturation values. Therefore, the highest recorded (in camera) luma value should be 255(and only one such value) and the lowest should be 0.

But in real world you cannot do this because it takes too much time and color correction would be too hard (imagine trying to match shots each with a different contrast and saturation settings in camera).

So it is easier to just shoot in s-log, not thinking about each scene individually, and color-correct it afterwards.

With 8-bit S-log you lose the middle luma values (after grading), but with a contrasty profile you lose highlights (blown highlights) or shadows, if the DR of the scene is too high for that contrast setting.

 

Yeah this probably the biggest takeaway from everything. When shooting 8 bit in a controlled environment I try to get as close to the final look I can to make the best use of the codec (color charts help a lot with this).

With 10 bit though, I haven't found a reason not to shoot S Log (3) with wide color gamuts. It's lovely to work with and I much prefer it to baking in the image in camera.

[iaremrsir]

On 7/13/2017 at 0:15 PM, maxotics said:

Hi Eddie, a simple question, when an EOSHD reader shoots an aggressive S-LOG on their A6300/A6500, say, are they getting more noise and less color than what they would get shooting a standard profile?   You keep bringing up situations that is not consumer S-LOG, like uncompressed HDMI or RAW.   You keep looking for holes in my arguments instead of asking if they are correct in real-world shooting for consumer S-LOG.

I directly addressed the 8-bit H.264 log encoded footage. I already said detail in the lower end is more likely to be destroyed by heavy compression. That includes noise. They are getting a wider range of color intensities, but less color saturation when shooting a wide gamut color space.

On 7/13/2017 at 0:15 PM, maxotics said:

You wrote "You're speaking as if the main reason detail is lost in 8-bit is that it's the log curve, when in reality, the main loss of detail has been heavy compression."  Fine.  How much!  I calculated 30% based on the number of unique colors I could count in each.  YES, they're H.264 compressed, but since they're BOTH compressed shouldn't the effect of compression be equal for each?

No, the effect of compression is not the same in both cases because that's not how compression works. If the image isn't identical, it isn't going to compress the same.

On 7/13/2017 at 0:34 PM, maxotics said:

He says that compression has more to do with noise than shooting S-LOG, fine, but that's his opinion!  He hasn't shown me any of his tests.  Why should I take his word for it over my experience?

I said that compression smooths noise and details in the shadows. A log color space image is less likely to have compression artifacts and the strange blockiness, and will look more natural and retain the structure of the noise. The noise before compression is going to be the same before compression, regardless of color space. Only after a destructive process, like compression, do things like noise change.

On 7/13/2017 at 0:34 PM, maxotics said:

  Anyway, I've had to challenge him (us) to get that articulated.  We still have a ways to go because so far there are no facts about that, that we can both agree on.  We're getting there.  A little piss never hurt nobody   If I have to play the bad guy in this thread, so be it   I'm sure he's happy to continue playing the good guy!

There's no good or bad guy in this. It's just debate. The nature of the beast so to speak. I enjoy the mental exercise.

Here are some color managed samples from Dave Newman of CineForm from a decade ago. These show the effects of compression for data in the shadows. And before you say standard profiles aren't linear, I know they aren't. But they are fairly similar once you get below 3 to 4 stops below middle gray showing single digits per stop and being below 16/255. This is an extreme example being the equivalent of 10-bit 4:2:2 50 Mbps H.264, except these are wavelet compressed and generally handle noise a bit better than DCT based codecs.

Uncompressed Linear:

Compressed Linear CF Low and Linear J2K:

Cineon CF Low and Cineon J2K:

 

And some reading on the benefits of log curves compared to power curves.

https://cineform.blogspot.com/2012/10/protune.html

https://cineform.blogspot.com/2007/09/10-bit-log-vs-12-bit-linear.html

[cantsin]

Btw., I would much prefer if this thread wouldn't use "S-Log" as a denominator of logarithmic color curves, but would generally refer to "log" instead. "S-Log" stands for Sony Log, as "C-Log" stands for Canon Log, "V-Log" stands for Panasonic Varicam Log and "C Log" for Arri's Cinema Log. Most log curves are manufacturer-specific because they're optimized for particular sensors and their dynamic range.

[maxotics]

Thanks @cantsin.  I wasn't clear on the names, at all!  @iaremrsir I have been building a test-bed for these issues this past week.  I am breaking up all 16 million colors into 32 screens worth of data (4 pixels per color) and will video them and compare each profiles color-count and histogram of what the camera should be able to duplicate from a monitor.  I will shoot both in-camera AVCHD and ProRes on a NinjaBlade with a C100, and will shoot the profiles on the A6300.  Hopefully I can determine the percentage of color lost in the 5 stops surrounding an exposure when shooting LOG and the effects of compression.  I'm needing to do a fair amount of ffmpeg, imagemagick and Python/PIL stuff to get things set up.  That suggests to me that few have really studied this issue outside the manufacturer's facilities.

The GoPro articles are very interesting, and I'll have to read deeper later, but the author's goal is how to maximize image fidelity in their CODEC, real world of RAW to compressed video.  I have no idea what compression, or where, is applied in the data stream for Sony, Canon or Panasonic. (I'm only looking into the internal-compression vs HDMI feed because you said it is important).  My question is a practical one, believe it or not!  If you're shooting a person under an umbrella on a beach, how many shades/color values to you loose when you shoot a LOG curve? I want some real numbers.  Without real numbers many people assume LOG is okay because there's so much written about how to shoot it and grade it.  It's in no one's financial interest to make LOG look bad   And I'm not trying to make it look bad either.  I just want to put it in its place

[AndrewM]

So tell me if this simplification is wrong...

You have a black box. You point it at stuff and turn it on and it records movies. 

Depending on how you twiddle the nobs, it is only sensitive to a certain range of inputs. Things outside that range get misrecorded (clipped etc).

Inside the box are more black boxes. These encode and compress and store in ways that have particular costs. They produce artifacts, reduce resolution (temporal, color space, dynamic range etc). You only have so much recording power, however you spread it out by twiddling the nobs.

You have an output format (rec709, UHD Ultra, human eye, etc.).

You want to "realize your vision" and deliver the best possible output given these constraints.

Option 1: Make sure you point your black box at a rec709 scene (say), twiddle your nobs so it captures only what you will deliver and to put the limited recording power you have where it hides its weaknesses and puts its strengths where it most matters to you. (Maybe you like shadows? Maybe you like colors? Maybe you like motion?)

How do you get a rec709 scene? By going into a studio and having lots of lights and having lots of people who know how to use those lights, and running every shot five times before you shoot so you know exactly what will happen. Oh, and don't have anything reflective or any shadows. And hope you don't make any mistakes because if your exposure is off, well... you are screwed. We call this "making a Hollywood movie."

Option 2: Just go into the world and point your box at things. Take the scene as it is, twiddle the nobs so that you can capture everything in the scene that matters to you, and to spread your recording power to make it the least-worst you can, given your priorities. It is going to be "worse" than option 1 because you are spreading your power more thinly over a wider range, and there will almost certainly be losses when you stuff that into rec709 (say). But they may not be visually obvious, and it is probably better than just clipping 75% of your scene. (Remember miniDV?)

Option 3: Go into the world and do what you can to control the light with what you have - your judgement, your choices, some lights, some screens, some reflectors. Do what you can to not stretch your recording power too thin, and allow yourself a safety margin for when you don't twiddle the nobs quite right. Control your environment when you can to not put your black box in a situation where it is guaranteed to produce bad results. Learn how to twiddle your nobs so that you can expand the range of situations you can work in and better deliver the aesthetics you want in your final product. Save up for stuff (lights, better cameras. external recorders) that lead to fewer compromises and better results.

 

I don't think I understand the OP. We are all in option 3, right? And Log is a tool, designed to make the best of certain situations. If we are in option 1, then no log. If the costs of log in a particular situation outweigh the benefits, then don't use it. If you are just using log as a kind of "bracketing" to avoid not having your shot, then that is fine (but you can learn to do better in some situations, and you have to recognize the costs). If you like the "log look" and don't grade, then... that is your preference.

[EthanAlexander]

"So designing a log curve that is optimized for the camera's sensor and its compression processing will generate superior shadow quality through compression processing than will linear compression, but without visible impact to the highlights." https://cineform.blogspot.com/2007/09/10-bit-log-vs-12-bit-linear.html @maxotics (as posted by @iaremrsir)

 

 

[cantsin]

3 hours ago, AndrewM said:

So tell me if this simplification is wrong...

No this is excellent and should explain the issue to everyone.