SIGMA FP with ProRes RAW and BRAW !

[Ryan Earl]

9 hours ago, Llaasseerr said:

Alternatively it could just be that I'm taking the bits buried right at the left side of the histogram, and expecting there to be enough detail when expanded out by +4 stops into the midrange.

That could be a sweet spot, exposing 3 or 4 stops under at ISO 100 depending on the scene.  I've been sticking to ISO 800 and will go down 2 stops before it gets ugly. 

I made a rough example.  I think generally in daylight +4 at 160 ISO can be okay.  By 6 stops the ugly noise is very visible at 5 stops and by 7 stops under pretty bad.  My lens has veiling flare wide open and pink then stopped down greenish. 

ISO 160 Base Exposure - False Color "Green" in Upper Left Gray Card - No Push

ISO 160 - 4 stops underexposed and brought back

ISO 800 - 2 stops underexposed and brought back

ISO 160 7 stops underexposed and brought back

 

[HockeyFan12]

I believe the digital definition of ISO (depending on whether you place middle gray at 12.5% or 18% saturation) implies 2.5 or 3 stops of highlight detail. 

It's "highlight boost" effects like Canon had that would underexpose and then push a stop to improve this.

But clearly most video modes (as sensors have gotten ISO invariant) take this even further. Why is 640 the base ISO for the S1 in V Log but not in other picture styles? Because it's really exposing at 100 ISO probably. (Underexposing at 80 ISO then pushing three stops for a total of six stops of highlight detail.) 

https://www.photonstophotos.net/Charts/PDR.htm

The fp looks like it has a great image. I own an S1 and from what I can tell the fp has a better image.

Is there a more elegant way to redistribute dynamic range than underexposing three stops, though?

[Ryan Earl]

2 hours ago, HockeyFan12 said:

Is there a more elegant way to redistribute dynamic range than underexposing three stops, though?

Just shoot ISO 800 instead of ISO 100.

My example of ISO 800 pushed +2 would be rating it at 3200 ISO but using the lower ISO range instead of switching higher.  So there will be at least a visual difference between ISO 800 +2 and ISO 3200.

[Ryan Earl]

3 hours ago, Ryan Earl said:

I made a rough example.  I think generally in daylight +4 at 160 ISO can be okay.  By 6 stops the ugly noise is very visible at 5 stops and by 7 stops under pretty bad.  My lens has veiling flare wide open and pink then stopped down greenish.

Just editing my own comment, I meant to say noise is tolerable to me at 5 stops, then getting ugly at 6 and really bad at 7.  But that is from ISO 160 because it happened to be where I started to be able to stop down by 7.

[Llaasseerr]

12 hours ago, TomTheDP said:

Shooting at 100 iso I have found there to be an incredible amount of detail in the shadows. I did find banding and posterization to show up in 10 bit and 8 bit recordings.

The shadow control in the RAW tab on davinci can bring up banding posterization sometimes so I found it's best to avoid that. I found using the exposure control in the raw tab to make major adjustments and then using curves to make minor adjustments works well.

In terms of resolution and crop modes I found full frame 4k is noticeably cleaner than all the other modes. 4k crop mode, HD full frame and HD cropped all had more noise. HD is very noticeably softer. It isn't unusable but it's definitely not the golden standard for HD.

Thanks for your real-world feedback.

Yeah, I think my math was not sounds because I know the sensors are capable of capturing significant HDR values depending on the sensitivity and whether using ND filters. If middle grey is pegged at a code value of 727/4095 in a normal exposure, then that only gives about 2 2/3 stops over and obviously it can do way more than that. So there's something I don't understand about the way sensor sensitivity works, and how it translates to linear code values in a 12-bit raw file.

I'm talking about uniform exposure pushes rather than just affecting the shadows, but effectively it's the same area being influenced (shadows) in this example.

You own an Alexa Classic, right? Do you have any opinion on how far you can push the ISO on the fp to get a noise floor level like the Alexa which is relatively high? To me, that's still acceptable in a good many situations. Although the Alexa noise also looks pretty organic.

[webrunner5]

Sure, there is noise, but the colors hold up well and I like the look of all of the shots. Seems like a pretty dope camera.

[Llaasseerr]

11 hours ago, OleB said:

Took some time today to create a few clips to test noise of the different ISO settings. Found me a clear light bulb...@Llaasseerr so you can have a look in regards to the highlight clipping. 

You can download the files here: https://www.dropbox.com/sh/cgm80q9w682oc19/AADWHieOS-Wka9JuD-GnXerca?dl=0 

All ProRes RAW. Starting with ISO 100 up to ISO 12800 (keep in mind the metadata will tell you it is ISO 800 upwards)

Have fun playing around 🙂 

Thank you, this is freaking awesome! Will take a look. 

Also, thanks for the note about the metadata. So for whatever reason, Atomos decided to rate it 3 stops over. Can I assume that the higher base ISO is kicking in at 3200 (25600 in Atomos metadata)?

Do you think that this can be converted to DNG equivalence just by subtracting 3 stops? I'm unsure, and also I don't know if one or the other clips the highlights at a different value either.

 

[Llaasseerr]

5 hours ago, Ryan Earl said:

That could be a sweet spot, exposing 3 or 4 stops under at ISO 100 depending on the scene.  I've been sticking to ISO 800 and will go down 2 stops before it gets ugly. 

I made a rough example.  I think generally in daylight +4 at 160 ISO can be okay.  By 6 stops the ugly noise is very visible at 5 stops and by 7 stops under pretty bad.  My lens has veiling flare wide open and pink then stopped down greenish. 

ISO 160 Base Exposure - False Color "Green" in Upper Left Gray Card - No Push

ISO 160 - 4 stops underexposed and brought back

ISO 800 - 2 stops underexposed and brought back

ISO 160 7 stops underexposed and brought back

 

Thanks, this is very helpful! I can't believe how well this camera holds up when pushed 4 stops in daylight. Even the 7 stop push is usable with Neat Video, assuming a film grain emulation would be added afterward.

What I have not been clear on is if there's any effective reason to shoot ISO 800 vs just pushing ISO 100 by +3 stops, assuming ISO invariance. Definitely keeping an open mind on that though. I've only been able to check this with ISO 100,125 and 160, but the highlights clip at the same place if you take into account the ISO variation. But I haven't had a chance to confirm the behaviour of 200, 400, 800. This is assuming DNG images, not PRR from the Ninja V that I will check out with the files @OleB uploaded.

 

[Llaasseerr]

2 hours ago, Ryan Earl said:

Just shoot ISO 800 instead of ISO 100.

My example of ISO 800 pushed +2 would be rating it at 3200 ISO but using the lower ISO range instead of switching higher.  So there will be at least a visual difference between ISO 800 +2 and ISO 3200.

As I mentioned, I'm definitely interested to understand the real world differences between 100 and 800 if I can push the 100 to 800 in post. So basically, assuming ISO invariance. Like, what - if any - are the differences between the noise and the highlight clipping. It sounds like in your experience that there is an advantage from shooting 800 and only pushing 2 stops though, which sounds positive.

800 is obviously a comfort zone, given the Alexa and how other cameras have begun to settle on that as their rated base ISO even if the sensor native ISO is different.

There is a nice indoor scene of a musical duo shot at 800 in that raw footage I linked to off YouTube, and it does look good. It does seem like the highlights are clipping slightly higher than if ISO 100 was pushed +3 stops, but it could also be because of some random behavior by the highlight recovery in Resolve. I would need to do more tests.

https://drive.google.com/drive/folders/1-z-R-8377SCVI6OxVVa5AybmcerEpQ9h

Also, here's the original footage I screengrabbed where I saw the green tint and posterization in the shadows of the hair when I pushed the exposure up:

https://drive.google.com/drive/folders/1PqyifY3J9HLKEUTUhIr75SJU3uBKo8cv

 

[TomTheDP]

Did some pretty heavy dynamic range tests today. Shooting into the sun. The Sigma FP handled it like a champ. I am honestly slightly surprised how close it is the the Alexa with a bit of work. The reality is you don't actually need the Alexa's dynamic range that often. I had a scene recently where I exposed for the subject and there was a bright window in a portion of the shot. The Alexa retained the info in the window but in post you have to choose between keeping that window detail or keeping the subject nicely exposed. At the end of the day you have to light your scene.

That said there is something great about ARRI footage and how it just looks nice as soon as you throw on the 709 lut. I can also underexpose or overexpose the ARRI to hell and back and it still is usable. I shot both cams at native ISO and F8 indoors during the day. The ARRI was very noisy and had a a green tint, the Sigma completely fell about ugliest looking image you could imagine with no hope of NR repair. The Alexa footage might have been able to be cleaned up with NR but at the worst it looks like it could be a stylistic choice, not the same with the FP. That is dealing in extremes though.

But one thing you can't ignore is that if you are in a dark situation you can just bump up to 3200 iso or beyond with the Sigma FP and get a cleaner image than you could off the Alexa.

Only thing missing on the FP is 4k 48p, but hard to complain for the price. Also I hope that Micro hdmi holds up, using the onboard LCD in sunlight is close to impossible.

[Llaasseerr]

13 hours ago, OleB said:

Took some time today to create a few clips to test noise of the different ISO settings. Found me a clear light bulb...@Llaasseerr so you can have a look in regards to the highlight clipping. 

You can download the files here: https://www.dropbox.com/sh/cgm80q9w682oc19/AADWHieOS-Wka9JuD-GnXerca?dl=0 

All ProRes RAW. Starting with ISO 100 up to ISO 12800 (keep in mind the metadata will tell you it is ISO 800 upwards)

Have fun playing around 🙂 

I've had a chance to do some tests, here's what I found (let me know if this lines up with your own findings). I'm assuming -3 stops to correlate the Ninja V recordings to internal ISO values.

I ended up converting from PRR to Prores 4444 through EditReady, since it gave me consistent exposure for all clips except for EI 3200 (internal ISO 400?) which was a little darker. For some reason I had an issue with Assimilate Play Pro (possibly I made a mistake). I exported as Alexa LogC/AWG and reimported to ACES.

The highlights all clipped at around the same linear floating point value, in the 10.5-11.5 range which is a bit lower than the theoretical max based on the DNG clipping point. So the major difference is the noise floor. The most low noise (cleanest) image was EI 800 (internal ISO 100?). It has about 2 stops less noise than EI 6400 (internal ISO 800?), so it can be pushed +2 stops and have about the same noise profile. Here's a screengrab showing that the noise level is about equal for +2 stops more highlight capture. This puts the fp about 1.3 to 1/2 stop below Alexa highlight clipping point.

For comparison, I have a DNG ISO 800 shot I found and the DR in the highlights is clipping at about 17-18. This extra range must be because of highlight recovery in Resolve. So I would need to do more tests to determine what the relationship is between the DNG raw and the ProRes raw recordings, and if internal is better for pure image data.

Here's the internal ISO 800 clip I'm referring to:

https://drive.google.com/drive/folders/1-z-R-8377SCVI6OxVVa5AybmcerEpQ9h

Also, here's the clip I was referring to earlier, when I called out the green patches and quantizing in the shadows.

https://drive.google.com/drive/folders/1PqyifY3J9HLKEUTUhIr75SJU3uBKo8cv

 

[Llaasseerr]

1 hour ago, TomTheDP said:

Did some pretty heavy dynamic range tests today. Shooting into the sun. The Sigma FP handled it like a champ. I am honestly slightly surprised how close it is the the Alexa with a bit of work. The reality is you don't actually need the Alexa's dynamic range that often. I had a scene recently where I exposed for the subject and there was a bright window in a portion of the shot. The Alexa retained the info in the window but in post you have to choose between keeping that window detail or keeping the subject nicely exposed. At the end of the day you have to light your scene.
 

I can assure you that all that extra range is appreciated and used if you're not just shooting to edit for Rec709 delivery. Definitely in post and vfx it makes a huge difference, but also just for highlight rendering vs clipping. Just doing an accurate film emulation will very aggressively roll off the highlights in a way that would break the image in most 12 stop cameras. I always notice how clippy bright scene elements look for shows shot on Venice on Netflix because it doesn't have that extra range.

[Llaasseerr]

12 minutes ago, Llaasseerr said:

This puts the fp about 1.3 to 1/2 stop below Alexa highlight clipping point.

 

Meant to say 1/3 to 1/2 stop below Alexa sensor clipping point, but could not edit the post. This is obviously a very good result, if the shadow noise/detail is acceptable! And it might be higher with internal DNGs with highlight recovery in Resolve.

[OleB]

6 hours ago, Llaasseerr said:

Also, thanks for the note about the metadata. So for whatever reason, Atomos decided to rate it 3 stops over. Can I assume that the higher base ISO is kicking in at 3200 (25600 in Atomos metadata)?

I do not think that Atomos decided to rate the camera differently. My assumption is that the gain levels are not correctly transferred. Because if they would, also the Rec709 screen should correlate with the false colors, no?

Bottom line for what I do, as said before measure the real sensor clipping via false colors in the Ninja V set to PQ for max DR it can show. For instance ISO 100 shot on ISO 100 gives you exactly the clipping point as when you switch to ISO 800. The false color screen looks identical and due to PQ mode you can still see a somewhat correct preview picture. If you in post turn the ISO 800 back to ISO 100 it will look exactly the same as the one directly captured in ISO 100. This is, because you are not changing the light falling on the sensor with my method.

Afterwards I have increased the ISO values in full stops, so ISO 1600, 3200 etc. Camera switches to second native ISO when you go to ISO 3200 that is correct. However the real value behind is ISO 640. Take my table and try for yourself 🙂 By the way I have turned down the exposure a little to keep more details in the lightbulb. The false colors and the sensor can go up to super white, clipping point is at about 105 IRE.

Interestingly it can be assumed that the fp shares the same sensor as also the Lumix B1SH there Panasonic states ISO 100 and 640 as native values...and Sigma does it too, but only for their photo mode in the manual some pages above.

[OleB]

6 hours ago, TomTheDP said:

Also I hope that Micro hdmi holds up, using the onboard LCD in sunlight is close to impossible.

SmallRig has a nice cage for the fp and additionally a part for the side with the HDMI connector which includes a lock. That gave me back some confidence. 

[Llaasseerr]

6 hours ago, Llaasseerr said:

The highlights all clipped at around the same linear floating point value, in the 10.5-11.5 range which is a bit lower than the theoretical max based on the DNG clipping point. So the major difference is the noise floor. The most low noise (cleanest) image was EI 800 (internal ISO 100?). It has about 2 stops less noise than EI 6400 (internal ISO 800?), so it can be pushed +2 stops and have about the same noise profile. Here's a screengrab showing that the noise level is about equal for +2 stops more highlight capture. This puts the fp about 1.3 to 1/2 stop below Alexa highlight clipping point.

 

I just wanted to update this since I was really tired when I posted it. I made the observation that the Ninja V ISO 800 recording (internal ISO 100) +2 stops has what looks to me as the same gain level as the Ninja V ISO 6400 recording (internal ISO 800).

Obviously it's not that useful to push +2 stops if the image then looks as noisy as +3 stops ISO (800 vs 6400). It suggest that the 6400/800 image has better noise performance than 800/100 if pushing 800 to 6400 +3 stops.

I've taken another look, and this time I lowered the exposure of the 6400 image by -2 stops and compared to a similar luminance level area of the 800 image. Definitely the noise is a lot cleaner!

Note also that if this was -3 stops to reflect the exposure rating difference between ISO 6400 and 800, then the noise would be even cleaner.

So it seems best to shoot at 6400 (Ninja V)/800 (Internal DNG) and push +2 stops if you want, as per @Ryan Earl.

I checked 12800/1600 -1 stop vs 6400/800, and they seem to have the same gain structure, so the sweet spot seems to lie with 6400/800.

@Ryan Earl do ISO 800 internal DNGs also have lower gain than ISO 100, but with the same highlight clipping point? That would align with the fact you like to shoot at 800 internally and push +2 stops. That seems very good.

The caveat with the PRR tests is that there's no highlight recovery as far as I know, so if this gain behavior hold true, that may give an edge to the internal DNGs.

 

 

[Llaasseerr]

4 hours ago, OleB said:

I do not think that Atomos decided to rate the camera differently. My assumption is that the gain levels are not correctly transferred. Because if they would, also the Rec709 screen should correlate with the false colors, no?

Bottom line for what I do, as said before measure the real sensor clipping via false colors in the Ninja V set to PQ for max DR it can show. For instance ISO 100 shot on ISO 100 gives you exactly the clipping point as when you switch to ISO 800. The false color screen looks identical and due to PQ mode you can still see a somewhat correct preview picture. If you in post turn the ISO 800 back to ISO 100 it will look exactly the same as the one directly captured in ISO 100. This is, because you are not changing the light falling on the sensor with my method.

Afterwards I have increased the ISO values in full stops, so ISO 1600, 3200 etc. Camera switches to second native ISO when you go to ISO 3200 that is correct. However the real value behind is ISO 640. Take my table and try for yourself 🙂 By the way I have turned down the exposure a little to keep more details in the lightbulb. The false colors and the sensor can go up to super white, clipping point is at about 105 IRE.

Interestingly it can be assumed that the fp shares the same sensor as also the Lumix B1SH there Panasonic states ISO 100 and 640 as native values...and Sigma does it too, but only for their photo mode in the manual some pages above.

Thanks, I revisited your ISO equivalence table to better understand how the values map from the Ninja to internal as per your tests. If I factor out that Atomos may have the gain levels set incorrectly and also that the false colour is not behaving predictably, what I see is the same exposure and highlight clipping point (except for 3200, see below) and the only difference is with noise level.

I realise that you are stopping down the lens when changing the ISO level to compensate (I can see the iris change), which is really helpful in this case, so that means given the highlights still clip in the same place, the noise level is the only variable. 

The ISO 3200 clip was the only one with inconsistent exposure, which seems to be due to the switch to the higher base ISO 640. To make it consistent with the ISO 1600 clip, I adjusted the exposure by 1.67 stops (based on underlying ISO 200 vs 640), then lowered by -1 stop to go from 3200 to 1600. 

When accounting for the exposure difference, the highlight clipping point is about the same as ISO 1600, but with slightly more gain - so there doesn't seem to be an advantage to using the higher base ISO, meaning there was not a sudden drop in image noise when switching.

As I mentioned in my post above, from my tests I think the sweet spot is ISO 6400 (internal 800) based on the reduced gain level of the noise. What do you think?

 

[OleB]

6 hours ago, Llaasseerr said:

I realise that you are stopping down the lens when changing the ISO level to compensate (I can see the iris change), which is really helpful in this case, so that means given the highlights still clip in the same place, the noise level is the only variable. 

Indeed, I have at first used a vario ND to keep exposure identical and as this wasn't sufficient anymore I stopped down.

From my testings I will use the ISO 800 settings (so real 100), ISO 3200 (640) if I need more light, but I also felt comfortable to go to 6400 or even 12800. Any higher the noise would need more treatment in post than I like to do.

Next round of test files I will upload will be about increasing ISO without adjusting exposure. So you can see when the camera will clip ultimately so nothing can be recovered.

I have found that if I expose with the false colors and set the highlights to be at orange and avoid red, there is about 1 stop which can be recovered.

[Ryan Earl]

19 hours ago, Llaasseerr said:

Also, here's the clip I was referring to earlier, when I called out the green patches and quantizing in the shadows.

https://drive.google.com/drive/folders/1PqyifY3J9HLKEUTUhIr75SJU3uBKo8cv

This clip is 10 Bit.  It's hard to speculate about what's happening.  It seems like I should be able to dig out a little more.  I haven't encountered that in 12 bit but haven't shot any 10 bit to know if that's more common.

[Llaasseerr]

4 hours ago, Ryan Earl said:

This clip is 10 Bit.  It's hard to speculate about what's happening.  It seems like I should be able to dig out a little more.  I haven't encountered that in 12 bit but haven't shot any 10 bit to know if that's more common.

Thanks for flagging that! It makes me more hopeful that this isn't a widespread issue. Plus if pushing by only +2 stops at ISO 800 would mean more bits able to be allocated at the low end.