Sigma Fp review and interview / Cinema DNG RAW

[MikhailA]

On 12/31/2019 at 12:59 PM, paulinventome said:

Me too, my left eye doesn't need glasses and it's close to -2 for it to work properly and i'm short sighted in my left but still at -2 so i really don't know what that diopter is correcting for...

Also yes, the first time you view a DNG file it takes time to preview it. It's like there's a very low res thumbnail then it gets debayered on demand.

cheers
Paul

 

Yes, but why does it take place at all? It is hard to believe that it is the problem for the processor inside Sigma fp to display 24 mp still on the fly) e.g.10 years old canon 5d mark2 do this instantaneously. Moreover, it happens even with low-quality jpeg stills not DNG. Could it be the firmware issue?

[Lars Steenhoff]

Some nice footage from 
PhotoLuxuria

[mechanicalEYE]

2 hours ago, Lars Steenhoff said:

Some nice footage from 
PhotoLuxuria

consistent jitter/shake made me dizzy. Subtle at times but tripod needed.

[CaptainHook]

On 12/31/2019 at 10:46 PM, paulinventome said:

The BM Film setting is specifically designed for BMD sensor response. I believe someone from BMD also confirmed this. The fp sensor is not the same and making assumptions about that response will lead to colour errors - albeit perhaps minor.

"BMD Film" (version 1 which will be applied to non-BMD DNGs) will just apply a log type gamma curve and not touch colour (gamut). So you will be getting Sigma fp sensor RGB space in terms of gamut and no colour 'errors' except that a sensor response isn't meaningful on a display. To transform into a common display space would not be straight forward though unless Sigma or someone provided the correct conversion, so unless you could get something you're happy with by manually correcting the colours it might be easier and more straight forward to decode into a known defined space like 709 or something and go from there.

[rawshooter]

Since I found the YouTube comparison between the Sigma fp and the Blackmagic Pocket Cinema Camera 4K so worthless, I did a comparison test of my own.

I wanted to compare the cameras in two respects: 

(1) dynamic range when shooting a high-contrast subject under optimal light/with optimal exposure + robustness in grading the resulting image;
(2) low light/extreme high ISO image.

My setup was as follows:

• Completely dark room;
• Both cameras with the same lens, Tokina 28-70mm/2.8 (Nikon mount adapted to L-mount and MFT respectively) at the same aperture setting (f4) on both cameras, but at 35mm focal length on the Pocket 4K and at 70mm on the Sigma fp to compensate for the different sensor sizes;
• Record settings: UHD 23.98p on both cameras, full frame CinemaDNG 12bit on the Sigma fp and BRAW Q0 on the Pocket 4K (= best quality codec settings on both cameras); 
• For the high-contrast test: daylight-temperature LED fresnel with maximum focus/spotlight on a plastic appliance reflecting some of the light to create high contrast; both cameras at minimum/native ISOs exposed ETTR/to the right. Both cameras set to 11.25 degrees shutter angle (in lieu of an ND filter, since motion rendering is irrelevant in this test).
  (Note: Lowest, respectively native, ISOs on the two cameras are ISO 100 on the Sigma fp and ISO 400 on the Pocket 4K. Never mind that nominal difference, both cameras have almost identical clipping behavior at these settings, i.e. ISO 100 on the Sigma fp behaves like ISO 400 on the Pocket 4K. On both cameras, the zebras turned out to be reliable indicators for sensor clipping: To go absolutely sure that I would optimally expose the sensor, I also shot the scene at larger shutter angles - 22.5, 45, 90 and 172.5 degrees -, with zebras popping up as early as at 22.5 on both cameras. When looking at the material in Resolve, this was indeed where clipping had occurred and waveforms remained clipped in the RGB parade even when lowering exposure in Resolve's Raw control tab.) 
• For the low-light test: same 'scene' as above, but with the LED fresnel turned off and only a practical light in the background turned on. Both cameras set to maximum ISO (256.000), at f4 and 172.5 degrees shutter angle.

Treatment in Resolve:

• Basic image adjustments only in the Raw tab, interpretation in P3 color space with Rec. 709 gamma: adjustment of white balance/tint and exposure to make the Sigma fp and Pocket 4K footage match. (The raw material of the Blackmagic Pocket 4K was much warmer than that of the Sigma fp with the same Kelvin settings...) No highlight recovery. (Wasn't necessary anyway since there was no clipping in the images.)
• No noise filtering or sharpening, although the Pocket 4K's BRAW codec already has some baked-in temporal noise filtering.
• For the extreme grade, only a solarization-like custom curve was applied that pushed the shadows to the maximum and was meant to provoke banding in the material by pushing contrasts:
  

I exported 16bit TIFF screengrabs which can be downloaded here (6 TIFF files in a zip archive, 133 MB).

Here's how the well-exposed high-contrast scene looks like (25% downscaled images):

Sigma fp

Pocket 4K

- Note that the difference in sharpness may be my user error, and is also influenced by the different depth-of-field between 35mm/f4 on MFT and 70mm/f4 on full frame. The manual focus aides on the Pocket 4K are much better with latest firmware, so nailing focus without an external monitor was easier.

1:1 crops of the above two images:

 
Sigma fp (left) - Pocket 4K (right).

 

Extreme grade, with the same curve (as posted above) applied to the two above images:

Sigma fp

Pocket 4K

1:1 crops of the above:

Sigma fp

Pocket 4K

 

Low light, with both cameras at maximum (256,000) ISO, 172.5 degrees shutter and f4:

Sigma fp

Pocket 4K

1:1 crops of the above:

Sigma fp

Pocket 4K

So, to summarize, I think it's fair to say that the full-frame 12bit CinemaDNG material of the Sigma fp simply shows the benefit of a larger sensor and its lower image noise (even at base ISO if you compare the full-size TIFFs). It's thus only logical that it holds up better in extreme grades and in low light.

My likely user error in nailing the focus of Sigma fp also shows the strengths of the Pocket 4K, namely better camera assist functions and overally a better user interface/more practical user experience for video shooting. - But it's also nice to see that the Sigma fp has some genuine advantages over the Pocket 4K, especially for my own type of videomaking which revolves around event videos (concerts at indie/DIY venues) shot in extreme low light conditions.

 

[Lars Steenhoff]

Thanks for testing this !

Noise looks much more under control on the sigma

would you mind sharing a raw file of both cameras?

Not sure yet what to conclude on dynamic range

[rawshooter]

6 hours ago, Lars Steenhoff said:

would you mind sharing a raw file of both cameras?

Not sure yet what to conclude on dynamic range

Sure, here are 4 single DNG frames (35 MB zip archive) shot with both cameras in the two different light setups. The two DNGs of the Sigma fp were taken from the CinemaDNG folder, the two DNGs of the Blackmagic RAW were shot with the still function of the camera directly after shooting BRAW, with the same camera settings. (The BRAW files are too big to share here.)

To better judge tonality and dynamic range, I applied another 'extreme grade' solarisation curve which, this time, doesn't clip parts of the image and also compresses the dynamic range a bit so that highlight and shadow rendition become better visible:

Sigma fp

Pocket 4K

Full-resolution, 16bit TIFFs can be downloaded here (48 MB zip archive).

 

Highlight rendition is pretty identical with both cameras since they were both exposed to the right. So any difference in dynamic range should be visible in the shadows. A good indicator are the color gradations and red stripe on the lamp in the foreground that (still) can be seen in the low-light, low-contrast image:

In the high contrast light, neither the Sigma fp nor the Pocket 4K render these details anymore, but turn the lamp into one dark monchrome blob (with more image noise in the case of the Pocket 4K). So I think it's fair to say that both cameras have about the same dynamic range at optimal ISO settings, only that the Sigma fp renders smoother color gradations because of its lower sensor noise, as visible in the extreme grade above.

 

[Lars Steenhoff]

I made an extreme grade in Lightroom to see what information is in the shadows.

There seems a green tint on the sigma and a purple on the black magic

Also the black magic seems to be exposed different in Lightroom, not sure why this is.

[rawshooter]

1 hour ago, Lars Steenhoff said:

Also the black magic seems to be exposed different in Lightroom, not sure why this is.

Sorry, I had included the wrong DNG in the archive. This has been fixed now; if you re-download it, you should obtain the frame with the correct exposure. 

(Has now also been fixed in the originally uploaded archive.)

[rawshooter]

Here's another dynamic range shootout between the Sigma fp and the Pocket 4K, this time with the LED fresnel at full power to create maximum contrast and a hard shadow, and an X-Rite Color Chart Pocket Video in the shadow to have a better indicator of shadow dynamic range. 

Cameras were set to 11.2 degrees shutter, ISO 100 (Sigma fp) respectively ISO 400 (Pocket 4K). With this harsh light, I needed to set the Pocket 4K to f5.6 for ETTR exposure (i.e. exposure just below the clipping point) while the Sigma fp clipped at f5.6 and needed to be stopped down to f8 for correct ETTR exposure. (This is an indicator for camera's native ISO being 800 in log. In other words, the Sigma fp is about one stop more light sensitive at ISO 100 than the Pocket 4K at ISO 400.)

This resulted in the following pictures, with DCI-P3 color space + Rec709 gamma interpretation in Resolve and Color Temp/Tint and exposure sliders adjusted to have the two camera images visually match:

Sigma fp

Pocket 4K

 

And extreme lift of the shadows with the following curve...

...reveals the following:

 

Sigma fp

Pocket 4K

1:1 crops of the upper left color patches:

 
Sigma fp

Pocket 4K

I'd say that on the Sigma fp, three of the dark grey color patches can still be differentiated whereas on the Pocket 4K, they are undifferentiated. Since I don't have a precise-enough meter to measure the difference of the reflected light, I'd say in a completely unscientific way that the fp has probably one stop more dynamic range in the shadows when the picture is exposed ETTR.

It also seems as if in the extreme shadows, the temporal noise filter of the Pocket 4K's BRAW really kicks in, since single pixels on the fp's CinemaDNG are clearly differentiated whereas on the Pocket 4K's image, they are blurred. (This is why in this scenario/grade, the Pocket 4K's image is [surprisingly] less noisy than the fp's.)  - EDIT: indeed, if you compare it to the DNG still from the Pocket 4K where BRAW's processing isn't applied:

 

- This time, I created a complete zip archive (91 MB) with one frame of the Sigma fp's CinemaDNG recording, the Pocket 4K's original BRAW file (consisting of just 3 frames) + a DNG shot with the Pocket 4K, plus the full-resolution UHD 16bit tiff of all above images: 

 

 

[eleison]

I wonder how close the low light abilities are to each other if the pocket 4k has a speed booster.  I think a lot of pocket 4k users are also using speedboosters and this would probably be a fairer comparison.  The field of view would also be closer.  Also, a pocket 4k w/ speedbooster is about the same price as the sigma fp.

[rawshooter]

15 minutes ago, eleison said:

I wonder how close the low light abilities are to each other if the pocket 4k has a speed booster.  I think a lot of pocket 4k users are also using speedboosters and this would probably be a fairer comparison.  The field of view would also be closer.  Also, a pocket 4k w/ speedbooster is about the same price as the sigma fp.

The Speed Booster doesn't change dynamic range, nor does it change sensor noise at base ISO. So the above results would be the same - I would have only had to stop down the speed-boosted lens on the Pocket from f5.6 to f8 to avoid clipping.

In extreme low light settings, there wouldn't be much difference either because then the Pocket operates at its second native ISO with higher ground noise. You wouldn't gain much from lowering the ISO from 256,000 to 128,000 in that case. (In fact, you can test that yourself in Resolve with the supplied raw files.)

[Lars Steenhoff]

How did you measure when the fp clipped?  I think you could have exposed a stop more without breaking the highlights.   Maybe not but just a feeling

[rawshooter]

5 minutes ago, Lars Steenhoff said:

How did you measure when the fp clipped?  I think you could have exposed a stop more without breaking the highlights.   Maybe not but just a feeling

I made a series of exposures, by gradually opening the shutter and thus exposing beyond the clipping point indicated by the zebras.

So far, the zebras both on the Pocket 4K and the Sigma fp have been 100% reliable indicators of clipping. (At least with white light where RGB clipping is uniform.) Whenever zebras became visible on the camera display, I ended up having clipped waveforms in Resolve.

(Of course, "highlight reconstruction" needs to be switched off in Resolve in order to see sensor clipping.)

 

[eleison]

38 minutes ago, rawshooter said:

The Speed Booster doesn't change dynamic range, nor does it change sensor noise at base ISO. So the above results would be the same - I would have only had to stop down the speed-boosted lens on the Pocket from f5.6 to f8 to avoid clipping.

In extreme low light settings, there wouldn't be much difference either because then the Pocket operates at its second native ISO with higher ground noise. You wouldn't gain much from lowering the ISO from 256,000 to 128,000 in that case. (In fact, you can test that yourself in Resolve with the supplied raw files.)

Technically true.  Sensor noise is sensor noise.  However, are you trying to tell me that the noise of an image w/ a speed booster which allows more light in, is going to be the same for a given field of view?  For instance, if a pocket 4k is using a 35mm lens w/o a speedbooster as compared with using a 70mm with a speed booster to approximately give the save field of view and given that they are exposed correctly under the same lighting conditions and DOF; the graininess/noise of the images will be the same?  I was under the impression, these two images would have different noise levels.  The speed booster would have less noise since there are more photons hitting the sensor.

[rawshooter]

12 minutes ago, eleison said:

Technically true.  Sensor noise is sensor noise.  However, are you trying to tell me that the noise of an image w/ a speed booster which allows more light in, is going to be the same for a given field of view?  For instance, if a pocket 4k is using a 35mm lens w/o a speedbooster as compared with using a 70mm with a speed booster to approximately give the save field of view and given that they are exposed correctly under the same lighting conditions and DOF; the graininess/noise of the images will be the same?  I was under the impression, these two images would have different noise levels.  The speed booster would have less noise since there are more photons hitting the sensor.

In the example I posted last, the sensor is already maximally exposed, and if more photons hit the sensor, you would have a clipped image.

In that scenario, with a Speed Booster all you would gain is an extended field of view by stopping down the lens from f5.6 to f8. (Which however would introduce diffraction and is therefore not necessarily desirable.)

Graininess/noise of the image doesn't change with the lens' focal length or aperture unless you underexpose/boost ISO. 

Btw., use of a Speedbooster doesn't require to adjust your focal length from 35mm to 70mm, but from 35mm to 49mm if you use a 0.71x Speedbooster or to 54mm with a 0.64x Speedbooster. 

[eleison]

21 minutes ago, rawshooter said:

In the example I posted last, the sensor is already maximally exposed, and if more photons hit the sensor, you would have a clipped image.

In that scenario, with a Speed Booster all you would gain is an extended field of view by stopping down the lens from f5.6 to f8. (Which however would introduce diffraction and is therefore not necessarily desirable.)

Graininess/noise of the image doesn't change with the lens' focal length or aperture unless you underexpose/boost ISO. 

Btw., use of a Speedbooster doesn't require to adjust your focal length from 35mm to 70mm, but from 35mm to 49mm if you use a 0.71x Speedbooster or to 54mm with a 0.64x Speedbooster. 

I guess what I'm trying to say is if a short film was shot with a pocket 4k not using a speedbooster, would it have more noise than if it was shot with a pocket 4k using a speed booster ASSUMING the exact same shots (scene, fov, dof, etc.. basically the same movie shot for shot)?  I was always under the impression that the speed booster would give the film less noise.

I think I know what you are trying to say when you say sensor noise is static.  How sensitive a sensor is to light is pretty much set in stone, but when more light shines on the sensor (via speed booster), I'm assuming less noise for that imagine.  Basically, would a 49mm glass with a speed booster have more light gather abilities than a 35mm lens assuming that they created the same image with the same dof, fov, and exposure?

[rawshooter]

14 minutes ago, eleison said:

I guess what I'm trying to say is if a short film was shot with a pocket 4k not using a speedbooster, would it have more noise than if it was shot with a pocket 4k using a speed booster ASSUMING the exact same shots (scene, fov, dof, etc.. basically the same movie shot for shot)?  I was always under the impression that the speed booster would give the film less noise.

No, not in situations where the camera is correctly exposed at native/base ISO. (And not at all in situations where you use NDs anyway to keep light from the sensor - with a Speed Booster, you actually need stronger NDs.)

The only point where a Speedbooster helps against noise is in low light situations where you need to increase ISO/gain . Since the Speedbooster makes your lens one stop faster, you may be able to keep your ISO lower and thus end up with less image noise. 

All a Speedbooster does is make your lens faster - i.e. turn an f2.8 lens into an f2.0 lens (or f1.8 if it's a 0.64x Speedbooster) - and wider, on a smaller sensor than it has been designed for. Nothing more, nothing less... 

[eleison]

1 minute ago, rawshooter said:

No, not in situations where the camera is correctly exposed at native/base ISO. (And not at all in situations where you use NDs anyway to keep light from the sensor - with a Speed Booster, you actually need stronger NDs.)

The only point where a Speedbooster helps against noise is if you needs to increase ISO/gain in low light. Since the Speedbooster makes your lens one stop faster, you may be able to keep your ISO lower and thus end up with less image noise. 

All a Speedbooster does is make your lens faster - i.e. turn a f2.8 lens into an f2.0 lens (or f1.8 if it's a 0.64x Speedbooster), nothing more, nothing less. 

Ok, I think I get it.  Basically, you would like to compare sensors only at native/base ISO making the aperture and shutter speed subservient to the native/base ISO.  I think a more educational test would be to see how the pocket 4k w/speedbooster compares with the FP - shot for shot, (same Fov, dof and exposure); I feel the pocket 4k wouldn't be so bad at low light since the speedbooster/lens combo will be one stop faster and you can keep the ISO lower and thus end up with less image noise.

[rawshooter]

1 hour ago, eleison said:

Ok, I think I get it.  Basically, you would like to compare sensors only at native/base ISO making the aperture and shutter speed subservient to the native/base ISO.  I think a more educational test would be to see how the pocket 4k w/speedbooster compares with the FP - shot for shot, (same Fov, dof and exposure); I feel the pocket 4k wouldn't be so bad at low light since the speedbooster/lens combo will be one stop faster and you can keep the ISO lower and thus end up with less image noise.

Okay, made another test with the setup you proposed:

• Pocket 4K with 0.64x Metabones Speed Booster & Tokina 28-70mm/2.8 at 50mm and f2.8 [=actually at 32mm + f1.8], with camera exposed at ISO 128,000
• versus the Sigma fp with the conventionally adapted, non-speedboosted Tokina 28-70mm/2.8 at 70mm and f2.8 and ISO 256,000

Images:

Pocket 4K BRAW (= with some in-camera noise reduction), ISO 128,000 + 0.64x Speed Booster

Pocket 4K DNG, ISO 128,000 + 0.64x Speed Booster

Sigma fp CinemaDNG, ISO 256,000

1:1 crops:

Pocket 4K BRAW  (= with some in-camera noise reduction), ISO 128,000 + 0.64x Speed Booster

Pocket 4K DNG, ISO 128,000 + 0.64x Speed Booster

Sigma CinemaDNG, ISO 256,000

 

The result is pretty much as to be expected, with the speedboosted Pocket 4K vs. the Sigma fp performing more like an APS-C camera vs. a full frame camera in low light with high ISO. As soon you expose on native ISO (without pushing gain), you however will back to MFT vs. full frame sensor performance in regards to noise levels.