Grading S-Log2/S-Gamut in Adobe

[dhessel]

I have been frustrated by the fact that there is not really any good way of working with s-log2/s-gamut with Adobe software. So far I have not been able to find any way of working with default PP7 other than look luts, like Impulz. The problem is that I don't always want to use a look lut and I have found that the luts from Impulz don't seem to work that well with footage from the A7s anyway. The colors always seem off no matter what I had tried and grading without luts was even more problematic. 

 

So I have worked out a way to convert s-log2/s-gamut to Rec709 without any look luts. So far the results have been good but I have not had any time to shoot with our A7s myself so I have been using stills generously uploaded by Roman Legion on vimeo. So here it goes, it is two effects that must be done in this order.

 

First I have attached a lut to correct the s-log2 gamma curve to rec709. This is based off a formula I found from sony for its s-log2 curve. Took some tweaking but mainly just had to deal with using video levels instead of full range. It is a 1D lut so it only affects luma. Apply this lut using a lumetri/apply color lut effect.

 

This is a matrix transformation from s-gamut to sRGB we will use it to fix the colors next.

 

S-Gamut to sRGB

1.87785101 -0.79411894 -0.08373153

-0.1768095 1.35097992 -0.17417008

-0.02620544 -0.14844233 1.17464781

 

Now apply a channel mixer effect and enter the follows values. The channel mixer effect appears to only allow integer values from -200 to 200 but it seems to still work fine. So you can enter them like this in the following format, what out for the const fields and leave them at 0. 

 

Red-Red Red-Green Red-Blue

Green-Red Green-Green Green-Blue

Blue-Red Blue-Green Blue-Blue

 

188 -79 -8

-18 135 -17

-3 -15 117

 

That is it, so far this looks really promising but would like to hear how it works out for the rest of you. I am adding some corrected still from Roman Legion as well. These images are ungraded I only converted to Rec709 and adjusted exposure as they were under exposed a little.

 

 

slog2-to-Rec709-dhessel.zip

[jcs]

The example images look pretty good!

 

One of the challenges with S-gamut is the non-linear response (color changes with exposure), right? S-Log3/S-Gamut3 fix this issue (doesn't change color with exposure).

 

Since you're using the Lumetri effect to load a LUT (which will be fast and hardware accelerated), why not combine the affine color transform with the 1-D LUT into a 3D-LUT? Perhaps possible with SpeedGrade or Resolve?

 

What's the advantage of using S-Gamut vs. Pro or Cinema color mode? (I'm currently using Pro color mode with S-Log2).

[dhessel]

I have considered combining them but haven't yet since I am new to luts and wanted to keep the gamma correction separate from the color at this stage. If this looks worthwhile then that would be the next logical step.

Not sure what benifit s-gamut may have but I do feel the other modes like pro and cinema are somewhat looks and would just likento have the option of using s-gamut. Sony put that combo on their high end cameras for a reason I am sure.

[sunyata]

i researched this too and found these 2 color transform matrices (first looks to be the same):

 

SGamut to sRGB:

 

   1.87785      -0.79412       -0.08373

  -0.17681       1.35098       -0.17417

  -0.02621      -0.14844        1.17465

 

also, this may be useful for someone that needs to go to other color spaces from a neutral XYZ:

 

SGamut to XYZ

 

  0.706493      0.128799         0.115157

  0.270984      0.786595        -0.0575787

 -0.009678      0.00459997     1.09399

 

it's a good starting point but the colors can still be too saturated, especially in the blues. 

 

jcs-

I don't think S-Log3 will make it into the A7s because of it's steeper log curve and wider highlights... for a prosumer camera with 8bit output, that could increase the complaining about artifacts, but you never know. 

 

dhessel-

not baking in your 1D gamma lut with your color transform into a single 3D lut is smart, the 1D lut is invertable, you can modify w/o changing color, & the 3x3 transforms (which jcs is calling affine) can be inverted and more precise for linear transforms than 3D LUTs, which are bound by their sample resolutions and not reversible. 

[dhessel]

Thank you for all of that information. I believe it was some of your posts on the foundry forums that were a starting point for all this. I understand that the s-gamut offers a wider range of colors but what is somewhat unclear is what benefit it would have for grading compared to using a different color mode. I have not worked much with files outside the usual srgb colorspace.

 

Thank you for confirming what I suspected about a 3D lut not being as accurate as using a matrix, I didn't want to make that claim until I looked into it more. I am not real familiar with the all the options for 3D luts but was pretty sure I had not heard of any that have more than 64 samples per channel. That would result in quite a bit of interpolating compared to a matrix which has no interpolation and is lossless. The 1D lut I generated has 1024 samples so it has plenty of data for an 8bit source. 

[jcs]

When performed on a GPU, LUT interpolation via a cubic spline (Catmull-Rom, etc.) in floating point will have more than sufficient accuracy vs. a direct lookup table with no interpolation. In other words, a 4-point LUT with spline interpolation can look the same a 256 entry direct LUT for 8-bit (simple gamma curve). Before GPUs, the direct lookup table on the CPU was the better choice for performance reasons. Current GPUs can perform tricubic interpolation on 3D LUTs extremely quickly, so the performance and ease of use (saving time) is worth the minor differences vs. a 1-D LUT + affine transformation. A 64 3D-LUT is 2^6^3 = 262K entries which is overkill for most uses, especially if only needing a gamma curve + affine transform. All that's needed is enough points to represent the spline control points for the gamma curve- perhaps an 8 3D-LUT (2^3^3 = 512 entries).

[sunyata]

hey jcs!

 

it should be noted first off that a 3D LUT is not really used for the same purpose as a 3x3 transform matrix. the 3D LUT can include nearly all your non-temporal top level color grades and so is often used as a 1 way baked in final color look or pre-vis generator.. this 3x3 matrix is a color space transform and should sum across a single row to 1 (rounded to a precision of around 4-6 decimal points), and be a near lossless transform, and invertable, appropriate for cases where you might need to go from one space to another w/ minimal information loss, so they're apples and oranges really. in the case of what we're trying to do here, it makes sense to start with the gamma.. isolate that issue, then a 3x3 color matrix transform to get into the proper working color space, whatever that may be for you, then you can start tweaking your final grade, but from a (hopefully correct) baseline. so it's really about workflow flexibility and abstraction.

 

if you go from SGamut to XYZ with the 3x3 matrix above, you can then apply any number of XYZ to sRGB, XYZ to Adobe Wide, XYZ to ACES.. whatever you need.

 

it might make sense to make a .spimtx file though?

[dhessel]

Not familiar with that file format, can it be used in after effects and resolve? Also I was originally going to go from s-gamut to xyz and to rgb from there but it appears that the channel mixer has a maximum value of 200 which made it problematic for most of the xyz to rgb matrix. Also I felt a direct convert would be simpler for most purposes.

[sunyata]

not sure, it's an imageworks format that has a 3x3 and an offset..  but that's going towards mixing in all color decisions.

 

it looks like this internally (values are fpo):

 

0.880608          0.155864         0.017461313    0.0

0.033194133    0.96680668     0.0                    0.0 

0.013515          0.070699         0.933638605    0.0

[dhessel]

Ok I believe I have only seen that with nuke. But wanted to check.

[jcs]

Hey sunyata- a 3x3 matrix only has rows and columns which sum to 1.0 when it's a pure rotation matrix (no scale or shear) AKA orthoganol (orthonormal). A 3D LUT can perform precisely the same function as any 3x3 matrix. Visually, the 3D cube lattice points are each transformed by the matrix. When points (colors) are looked up in the transformed cube, the results are the same as the direct matrix computation (within float round off error). Now, if we transform the cube by a gamma function, we've got both steps baked into one transform, which can then be easily applied using the Lumetri effect in PPro. It's also very fast- GPU accelerated.

[sunyata]

jcs-

 

the main point is abstraction, but here we're summing left to right, not also top to bottom. if you apply this transform to a black and white gradient, or a plot scanline, you won't see color offsets (added or subtracted per channel). it's your basic color space transform. it can also be inverted. 

 

SGamut to sRGB

   1.87785      -0.79412       -0.08373    = 1

  -0.17681       1.35098       -0.17417    = 1

  -0.02621      -0.14844        1.17465    = 1

 

sRGB to SGamut

   0.566069      0.342758     0.0911729  = 1

   0.0769662    0.799066     0.123967    = 1

   0.0223549    0.108626     0.869019    = 1

 

since there seems to be a lot of variation with SGamut, camera to camera, ev to ev, handling gamma and color space separately lets you mix and match custom gamma profiles / color spaces / offsets in post. you could of course choose to wrap everything into a single baked 3D LUT...  whatever you prefer.

[jcs]

We can visualize what the SGamut to sRGB matrix is doing by drawing an untransformed basis (unit matrix) and also drawing the SGamut matrix (shown in brighter colors- a quick example in 3DSMax):

 

 

A pure red input pixel is the 3D vector {1,0,0}, and we if we perform the dot product for each row we see the new coordinates (color): 1.87785,  -0.17681, -0.02621, which matches the 3D drawing. The 3 points (darker colors) which represent the unit matrix (no transformation) also represent the edges of the color cube for a 3D LUT (which contain more points making up a full 3D cubic lattice). If we transform the points which make up a 3D LUT using the matrix, the resulting transformed cube will now exactly replicate (within float precision) the 3x3 matrix. Similar examples:

 

 

By increasing the density of the 3D cubic lattice, we can also accurately perform a gamma curve at the same time as the matrix transform. The accuracy can be high when the 3D-LUT uses tricubic interpolation. Premiere Pro doesn't appear to have a GPU accelerated method for matrix operations (Channel Mixer), so a 3D LUT which combines the matrix with the the gamma curve will not only be much faster (GPU accelerated), but also easier to use (just one effect). It will also be plenty accurate (you shouldn't be able to see a difference, even with pixel peeping). Putting the final result in a .cube file will allow the transformation to be used in any modern NLE/Color-correction tool. dhessel's LUT is already a .cube file (representing a 1-D transform):

LUT_1D_SIZE 1024
-0.00901794 -0.00901794 -0.00901794
-0.0088501 -0.0088501 -0.0088501
-0.00868225 -0.00868225 -0.00868225
-0.0085144 -0.0085144 -0.0085144
...

[sunyata]

i like your 3D jax illustration! i would repeat my previous comments though about the benefit of keeping gamma adjustments separate from your color space transforms. 

 

we use 1D LUTs for gamma because they can be inverted, while 3d LUTs can't. color space transforms are typically implemented (and documented) as a 3x3 matrix, they can be converted one direction, back, or to another space when it's necessary.

[mojo43]

I have been following this topic and I was wondering if somebody could help me out. I am editing mainly with Edius (which doesn't allow LUTs yet) and I would like to somehow convert sgamut to YUV colour space (which is what Edius works in).

 

Is there a way to determine the values needed? Or can someone explain to me how I can figure this out. All of this conversion process is a little over my head...

[sunyata]

might have to render an intermediate file with something like ffmpeg, which supposedly supports using LUTs, although i haven't tried it.. and it's freeeeee!

 

ffmpeg -r 24 -y -i my_sgamut_video.mov -vf lut3d="/path/to/my_downloaded_lut.3dl" -c:v prores_ks -profile:v 3 -pix_fmt yuv420p /path/to/render/drive/i_hope_it_works.mov

[jgharding]

It's pretty high tech in here! :0

 

I found these official Sony LUTs for more simle conversions, if they're any use:

 

http://community.sony.com/t5/F5-F55/NEW-3D-LUT-s-for-F55-and-F65/m-p/135617

[jcs]

Hey JG- as noted in the comments on the Sony forum, the F5, F55, and F65 are all different (as is the A7S). It would be helpful to have a LUT specifically for the A7S. I've had the best luck using PP5 or PP6 (stock) or Slog2 (PP7) with Cinema or Pro color modes (including Kholi's tweaks). Seeing proper handling of A7S Slog2 with SGamut with a mathematically correct conversion would be interesting.

[sunyata]

i tested these 3 gamma profiles and also 3 custom color matrix transforms (no channel offsets) on different A7s S-Log2 / SGamut ungraded footage.. one preset won't work obviously for all the different ev, lighting and wb settings. the first gamma profile is traced from the F5/F55 forum's cube file, the baked in color grade was removed. nuke is exporting 1D LUTs with 3 columns so i'm working* on a group node with an ocio wrapper to generate a correct 1D file.

 

 

* slowly

[jcs]

Hey sunyata- good point regarding needing different presets for different conditions. For controlled lighting / studio shots, setting manual ISO and WB is much better for post work. For fast turnaround and changing lighting-condition shoots I use auto ISO and AWB (especially when outside, even more so during sunset after golden hour). Since SGamut(2) changes color behavior with exposure (perhaps even non-linearly- can't be fixed with a linear transform?), I would expect (Slog+SGamut)2 to be best used only for fixed lighting conditions. It would appear Sony didn't provide (Slog+SGamut)3 for the A7S for business reasons: reserved for the pro cameras. (Slog+SGamut)3 doesn't (non-linearly?) change color behavior with exposure- a much better solution and far less work in post, especially for skintones, where accurate color is most important.

 

For final color grading of our first narrative, a Sci-Fi short called Delta, I calibrated 3 different displays with an X-Rite i1Display Pro (working on both Win7 and OSX). Two of the displays were Dells (2405 and 3007) with CFL backlighting and a Samsung D8000 HDTV with white LED backlighting. Even after repeated calibration, the displays weren't exactly the same, however color behavior was fairly close. I also checked color on uncalibrated MacBooks Pros, iPhones, and iPads. This is something I did with videogame development- testing on a wide variety of hardware to see how the product performs. Each display is different, and many times one of the displays would show something the others did not, requiring a grading adjustment. Most end-user displays aren't calibrated: this kind of testing is helpful in making sure the final product looks good in most conditions.

 

In my experience with recent Sony cameras (FS700 and A7S), Sony's sensor+software are 'unstable' around skintones. The color swings around magenta and green, making accurate and good looking skintones much trickier than with Panasonic or Canon. Even when skintones are 'correct' using the vectorscope, they don't always look correct with the naked eye. In other words, good looking skintones for Sony cameras have a small sweet spot, like the peak of a pointed parabola, with magenta on one side and green on the other. Panasonic goes green / orange with a more rounded parabola (more good skintone range), and Canon goes slightly-green / orange with the flattest parabola- the best skintone range with the least amount of post work.

 

I shot the first scenes of Delta with the A7S using stock PP6 (Cine 2 and Cinema Color Mode) outdoors with rapidly varying lighting conditions using auto ISO and AWB. In post I found the best skintones, especially for scenes of an alien planet, had a magenta bias:

 

The magenta bias helped set the mood for an alien world.

 

For the first Earth scene, I used PP7 with Slog2 and Pro Color Mode (essentially Kholi's settings with minor tweaks). The main shot used for this prototype poster is from a 1080p frame grab from the film:

 

The bottom images are GH4 with Natural profile from green screen studio shots. The wall image was shot on the A7S as a background plate during sunset on location.

 

Interestingly, scene to scene color variance was never mentioned during early screenings. While the current version matches more closely, it's not perfect and in the end the goal is for the color to affect emotion and scene energy more than perfect color matching. Note the character on the lower right has a magenta bias. When correcting for the white clothing, the skin ends up too green. The time-consuming fix is masking and correcting separately (I'll fix the colors for the final poster).

 

After working on this production with the A7S and GH4, I really appreciate how Canon handles skintones- the 5D3 requires far less work in post under a wide-variety of lighting conditions. For Sci-Fi, wild+interesting colors work well, however I'd prefer to have accurate+natural color straight from the camera and then make the colors into "Sci-Fi" colors in post. I much prefer the look of the 5D3 (including native H.264, RAW is amazing) vs. the C100/C300, and the 1DC doesn't provide decent price-performance, so we would not switch to the current Canon's C line. If in the unlikely event Canon produced a 5D4 with price+performance close to the A7S/GH4, we'd surely add them to our toolbox. The new Sony FS7 with 10-bit 422 XAVC and internal 4K is looking like a fine upgrade for our FS700+GH4 (the new PXW-X70 looks nice, though Sony's color science is still quite challenging).