Help grading HLG (GH5)

[Ki Rin]

Thanks for the replies guys. Based on some of the suggestions above, and reading the manual, I think I've come up with a workflow that works pretty well for me.

For anyone who is interested, the key steps are:
1) In project settings, use Davinci Color Managed. Set input, timeline, output to rec 709. Set "Timeline to Output Tone Mapping" to Luminance Mapping.
2) In media page, select all the GH5 clips, right click and set "input color space" to "rec 2020 Gamma 2.4".
3) Leave the other camera clips (in my case sony cine2 files) to the default project settings (rec 709).
4) Edit and grade as normal.

This way I get a decent starting point for the GH5 HLG clips, and I can actually see them properly while editing. And doing it this way instead of messing with luts, it seems like I don't lose any data, so I am able to recover highlights etc.

From my initial tests, this seems like it will work for my needs, and is the most simple and painless process I could find.

Of course, its still a bit of a pain to match the 2 cameras, but there's no way around that with the equipment I have, and on the bright side, its a good chance to practice my color correcting skills 


Just to re-iterate, I'm not delivering HDR. The only reason I am shooting in HLG is for the added dynamic range. I just didn't know how to get the footage to not look horrible since it is basically log footage, but there didn't seem to be a lut or color space specific to the GH5 HLG.
I'm still not sure if what I'm doing is "correct", but it seems to work for me, so I'm moving forward until I learn a better way.

[kye]

4 minutes ago, Ki Rin said:

Just to re-iterate, I'm not delivering HDR. The only reason I am shooting in HLG is for the added dynamic range. I just didn't know how to get the footage to not look horrible since it is basically log footage, but there didn't seem to be a lut or color space specific to the GH5 HLG.
I'm still not sure if what I'm doing is "correct", but it seems to work for me, so I'm moving forward until I learn a better way.

I shoot HDR for the DR too and tried to find a conversion for it, but couldn't find one either.

I did an experiment where I shot two clips, one with a SS that gave a normal exposure and one with a SS twice as long, so the second clip would be brighter by one stop.  I then tried to convert from <insert gamma profile here> to Linear, brightened the clip by a stop, then converted back from Linear to <insert gamma profile here>.  In theory if I picked the right profile then that transformation should make the first clip identical to the second one, but I tried all the profiles I could find and nothing matched.  I figured that if neither rec2020 or rec2100 matched the gamma of HLG then likely that the colour space wouldn't be an exact match either.

So no, it's not 'correct', but with grading if it looks good then it is good, so if it's working for you then go for it  

[Ki Rin]

1 hour ago, kye said:

I shoot HDR for the DR too and tried to find a conversion for it, but couldn't find one either.

I did an experiment where I shot two clips, one with a SS that gave a normal exposure and one with a SS twice as long, so the second clip would be brighter by one stop.  I then tried to convert from <insert gamma profile here> to Linear, brightened the clip by a stop, then converted back from Linear to <insert gamma profile here>.  In theory if I picked the right profile then that transformation should make the first clip identical to the second one, but I tried all the profiles I could find and nothing matched.  I figured that if neither rec2020 or rec2100 matched the gamma of HLG then likely that the colour space wouldn't be an exact match either.

So no, it's not 'correct', but with grading if it looks good then it is good, so if it's working for you then go for it  

If you get a chance, try the workflow I outlined in the previous post. It's actually really easy, and so far seems to be working well. 

As for shooting, I have found that it's really important to have zebras turned on, and also glance at the histogram from time to time to get the exposure right. You can't really trust what you see on the monitor in HLG. 
I basically made sure I was ETTR just under the point where zebras would show up. Doing that makes the clips "look" over-exposed, but all that data is there and when you set the color space, shifts back to a normal exposure. 

[stephen]

@Ki Rin
It is in principle correct. Davinci Resolve timeline by default is REC709 color space and gamma. Same for Adobe Premiere.
When you drop a clip Davinci assumes it is in REC709 color space and gamma. So without some form of transformation colors are not correct. You have to tell Davinci in what color space and gamma was your video clip shot in order to get the correct colors and gamma and use max quality of your source material.
That's the principle, well explained in Russian video.

As said one way is to use LUTs for transform, the other way is to user Color Space Transform or ACEs. Goal of all 3 methods is to have a good starting point with correct interpretation of colors.  For me Color Space Transform gives me best results and is the easiest. Makes matching BMPCC 4K clips shot in BROW and Sony A7 III clips shot in HLG relatively easy. That's my experience.  

 

4 hours ago, kye said:

 I figured that if neither rec2020 or rec2100 matched the gamma of HLG then likely that the colour space wouldn't be an exact match either.

@kye 
Don't agree with your logic. Same as saying: Don't like apples so figured that won't like oranges as well    They are different.
Second, let's be clear and precise. REC2020 and REC2010 are Color Spaces values in Color Space Transform effect and in general. REC2010 HLG and REC2010 ST2084 are gamma values. Now if you transform from REC2020 which is much wider color space to REC709 which is much narrower one (than REC2020) then go back it's no surprise result will be different (and worst). You basically destroyed quality of your video source. HLG -> REC709 -> HLG = REC709 colors. There is no way to get back colors which are absent in REC709. Same is true for Gamma. HLG gamma has 12 Stops of dynamic range ( given your camera sensor has this range). REC 709 6 stops, some sources claim up to 7 Stops. Now once you converted your video to REC709 gamma with 7 Stops there is no way to get back to 12 Stops. Dynamic range has been already destroyed. True at the end your video on the timeline in only REC709 color space and gamma, but one way you start from much wider color space and gamma and the other way you start with same limited REC709. 

In my understanding that's exactly what color space transform workflow tries to avoid.    

Here is a video from Juan Melara which goes on the opposite direction. By using Color Space Transform to wider color space and wider gamma he claims is able to get more dynamic range from BM Ursa Mini cinema DNG footage:

 

Am no expert in color grading/correction but Juan is. 

[kye]

3 hours ago, stephen said:

Don't agree with your logic. Same as saying: Don't like apples so figured that won't like oranges as well    They are different.
Second, let's be clear and precise. REC2020 and REC2010 are Color Spaces values in Color Space Transform effect and in general. REC2010 HLG and REC2010 ST2084 are gamma values. Now if you transform from REC2020 which is much wider color space to REC709 which is much narrower one (than REC2020) then go back it's no surprise result will be different (and worst). You basically destroyed quality of your video source. HLG -> REC709 -> HLG = REC709 colors. There is no way to get back colors which are absent in REC709. Same is true for Gamma. HLG gamma has 12 Stops of dynamic range ( given your camera sensor has this range). REC 709 6 stops, some sources claim up to 7 Stops. Now once you converted your video to REC709 gamma with 7 Stops there is no way to get back to 12 Stops. Dynamic range has been already destroyed. True at the end your video on the timeline in only REC709 color space and gamma, but one way you start from much wider color space and gamma and the other way you start with same limited REC709. 

In my understanding that's exactly what color space transform workflow tries to avoid.    

My logic of HLG -> rec709 -> HLG = HLG will work if you use the CST, which is what I was doing.  You're right that the HLG has colours or gammas outside the range of rec709, but Resolve retains those out of range values non-destructively - that's why Juan uses CSTs instead of LUTs.  

In terms of the logic about gamma vs colour space, I was interested in finding a technically correct way of converting as I am interested in doing some exotic things rather than just getting an image that looks good.  Those exotic things require an exact match and that's what I was looking for.  

One of the things that people don't realise is that with gamma curves, the LOG curves are all very similar to each other, with the main difference in DR.  If you haven't done it, try taking a shot that's been shot in LOG (of any variety) and make a bunch of copies of it on the timeline and grade each of them like this:

1. Curves adjustment where you adjust the highest point and lowest point for setting white and black points in final image
2. CST from <gamma> to rec 709
3. Add saturation

Make each clip have a different <log gamma> in the CST and adjust all of them to have the same white and black points.  Then go through them and see how each of them looks.  You may be surprised at how similar they are.  LOG curves are based on a mathematical function called "log" and while each manufacturer does push and pull their own versions for various reasons, they're very similar.  It's the same thing with RAW - all RAW is Linear and so you can use just about any Linear space on any RAW image.  They won't match exactly but the differences will only be very slight.

If you're just interested in getting something 'close enough' then Rec2020 or Rec2020 / Rec2100 HLG is fine, and is what was recommended to me over at LGG by the pros: https://www.liftgammagain.com/forum/index.php?threads/grading-gh5-hlg-for-rec709-output.11858/

It's great you're watching Juans videos, I've watched each of them dozens of times.  One thing worth noting is that there are two schools of thought about grading, one is Juans approach which is very technical and the other is completely non-technical and just involves taking any input and using the normal grading controls to make it look good without converting or anything.  I see advantages in both approaches, and it just depends on what you're trying to achieve.

[Ki Rin]

I read the section on color management in the resolve manual last night, and it seemed to be that CSTs are the preferred method over luts for this kind of thing. The reasoning being that CSTs are non-destructive so you can still get any data thats out of bounds (provided it wasn't clipped in the original footage). 

At least that was my understanding.

Color is incredibly complcated, and I can't say I understand it well. 
I'll see how I go with the approach I outlined above, and report back. 

 

[Mark Romero 2]

6 hours ago, kye said:

I see advantages in both approaches, and it just depends on what you're trying to achieve.

The one thing though is that in his Rebuilding the Linny Lut video, at the end he explains why he DOESN'T use the traditional Resolve curves tools (like hue vs hue, or hue vs luminosity). Unfortunately I can't find that video anymore (I think it was on Vimeo... looks like he took it down).

Anyway, he used one of those "LUT Stress Test" images from True Color to show how using curves to recreate the Linny Lut caused all sorts of artifacts.

Wish that video was still posted somewhere...

[kye]

2 hours ago, Mark Romero 2 said:

The one thing though is that in his Rebuilding the Linny Lut video, at the end he explains why he DOESN'T use the traditional Resolve curves tools (like hue vs hue, or hue vs luminosity). Unfortunately I can't find that video anymore (I think it was on Vimeo... looks like he took it down).

Anyway, he used one of those "LUT Stress Test" images from True Color to show how using curves to recreate the Linny Lut caused all sorts of artifacts.

Wish that video was still posted somewhere...

The video was on YT, but appears to have gone now.  I remember it well.

What I was talking about was that you can grade using technical transforms, or just with the tools.  Juan is obviously a fan of the technical conversions, but there are other more old-school pro colourists who have straight-out told me they just grade LOG footage by using contrast/pivot/saturation and then the LGG wheels.  What Juan was talking about in the Linny rebuild video was about being as accurate as possible in emulating something, but the guys who grade freehand are just looking for a nice image using the motto "if it looks good then it is good".  Colour grading can be as simple or complex as you like, and I've heard that most footage shot well just needs a WB, contrast adjustment, and primaries, and then everything (including skin-tones) will just drop right in place.

[Mark Romero 2]

37 minutes ago, kye said:

Colour grading can be as simple or complex as you like, and I've heard that most footage shot well just needs a WB, contrast adjustment, and primaries, and then everything (including skin-tones) will just drop right in place.

No final LOG to REC 709 CST node at the end???

I generally try to keep it simple and most LOG grades only have two nodes. First node is for LOG wheels and contrast/pivot/saturation, and the second node is usually CST for LOG to REC 709 (or I use the VLOG to 709 LUTS from panasonic). 

So the other colorists on the liftgammagain.com site are saying no need to do a conversion from LOG to 709 at the end?

[kye]

45 minutes ago, Mark Romero 2 said:

No final LOG to REC 709 CST node at the end???

I generally try to keep it simple and most LOG grades only have two nodes. First node is for LOG wheels and contrast/pivot/saturation, and the second node is usually CST for LOG to REC 709 (or I use the VLOG to 709 LUTS from panasonic). 

So the other colorists on the liftgammagain.com site are saying no need to do a conversion from LOG to 709 at the end?

I think so.

This post (https://www.liftgammagain.com/forum/index.php?threads/lab-processes-before-the-digital-intermediate.12923/#post-129058) from Marc Wielage describes a workflow that doesn't seem to include a conversion:

Quote

What I can tell you in a general way (for digital cameras) is I use the manufacturer's own color science to drop their material into basically a "Log Film" mode (kind of like RedLogFilm or Alexa Log-C), then I use Offset (essentially printer lights) to balance out the image, followed by a curve to tweak the gamma range, and an additional node to basically neutralize the material and get it into what I'd call a "Pseudo-Rec 709" range. I think the Editable Splines option in Custom Curves is useful for creating a smooth S-curve that's tweakable. From there on, I have three nodes for gain adjustment, an additional one for OFX plug-ins, several preset nodes for windows, a node near the end for clips, and a final node for a trim pass at the end of the session. Different sessions require different setups.

Assuming I am interpreting correctly, Marc may well use a conversion to get the files into a log format and then goes manual from there.  HLG is a log format but has a pretty extreme curve so that might not be the best curve to start with.

Having said that though, here's some tests.

GH5 HLG image with no processing:

GH5 HLG image with CST (rec2020/rec2100 HLG to rec709):

GH5 HLG image with contrast, saturation, and a slight hue rotation of all colours to get skin tones in range:

Are they the same?  No.  Is the manual one nicer?  Not really  But you can't deny that it's pretty similar to the CST.  They're just different.

As a more general comment, this test took be about 10 minutes and that included finding some music to put on and finding the project with these test shots in them.  Before you make a statement or ask a question, ask yourself if it's about something you can actually test yourself, and if you can, try it out and see what actually happens.  Grading is so easy like this because you don't need to actually go shoot anything, you just fire up Resolve / FCPX / PP and click a few buttons and see what happens  

[kye]

The colour science that manufacturers have created isn't as magic as most people think it is.  Yes, it's very nice, but you don't have to spend that much time grading to learn how to make well shot footage look pretty good even if you're only using manual controls.  

Yes, you'd have a pretty hard time replicating the CS from Canon or Arri, but getting 80% of the way there can be done just by shooting well and doing a few basic adjustments.

[stephen]

15 hours ago, kye said:

My logic of HLG -> rec709 -> HLG = HLG will work if you use the CST, which is what I was doing.  You're right that the HLG has colours or gammas outside the range of rec709, but Resolve retains those out of range values non-destructively - that's why Juan uses CSTs instead of LUTs.

 

Hmmm... Am not convinced this is true. And am not sure what non-destructive means. Everything that watch and read from professional colorists including Juan Melara's video point in the opposite direction. (HLG -> rec709 -> HLG = rec709)

Let's check Melara's video. He concentrates mainly on dynamic range (Gamma). Whole point of the video can be summarized like this:
URSA Mini is capable of 15 stops of dynamic range. But it looks you don't get all those 15 stops because BMD gamma is able to hold only 12 stops effectively cutting some of the dynamic range of the camera.

Later he proposes two methods / solutions:

    1. Play with curves. (But you still play with 12 stops of dynamic range at input)
    2. Choose Gamma=Linear. It looks Linear container(value for Gamma) can hold more stops of dynamic range. And you get full range of 15 stops, which yields better result at the end. 

    At 1.25: "My theory is that BMD 4.6K film curve as a container is actually not able to hold the entire 15 stops of URSA Mini..."

About color space:

    At 2:20: "Normally I'd recommend to output to the largest color space available and you can use P3 60... To keep things simple I'll choose REC709..."  

In my understanding he says: Get the widest color space you can. But here for simplicity I choose REC709 (narrower) because my main point in this video is about Gamma (dynamic range). 

Now if we assume what you say is true, then BMD Gamma should hold all values of URSA Mini dynamic range (15 stops). Why then go to Linear ? It would have been sufficient to choose it in color space transform and it would reveal the whole 15 stops. But that's not what Melara is saying. He's saying exactly the opposite. He is saying that BMD Gamma as container (Gamma value) is limiting dynamic range of URSA Mini sensor. And REC709 gamma has even less stops of dynamic range than BMD gamma ! 

Now let's go to HLG color space and gamma. As you said there are several approaches / methods to color correct and grade. Let's compare the one that plays with curves, saturation etc. with Color Space Transform.

When you place a GH5 or Sony A7 III HLG or LOG clip on timeline, Davinci assumes by default that your clip is REC709 color space and gamma. If you don't tell Resolve the color space and gamma your clips were shot it doesn't know and assumes REC709 (same as is your timeline by default). But REC709 color space and gamma are much more limited than REC2020 color space and REC2010 Gamma that HLG clips are. By doing so you effectively destroy the quality of your video. No matter what you do later, curves saturation, LUTs, your starting point is much lower. Yes at the end it's always REC709 but everything so far points that you loose quality when correct conversion was not done because you don't use the full range of colors and dynamic range your camera is capable of.

That's why they say that color space transform do this transformation non destructively. At least that's my understanding.   But may be wrong and am curious to hear others opinion. 

If your scene has limited dynamic range you may not see a difference between the two methods. But in extreme scenes with wide dynamic range it does make a difference. Same for colors. Some  color spaces approximate nicely to REC709, others (Sony S.Gammut) don't.  There is no surprise people are complaining about banding, weird colors and so on. Most of those problems could be resolved with Color Space Transform even for 8bit codecs footage. 

Second problem with using curves, saturation etc. is that this method is not consistent from clip to clip, between different lighting, etc. And involves a lot more work to get good results. At least that's my experience. Have to tune white, black point for each clip individually, then saturation etc. Change one color, another one goes off. Can't apply all settings from one clip to all especially when shooting outdoor in available light and different lighting conditions. It's a lot of work. Just read how much work was put to create Leeming LUTs, how much shots had to analyzed, etc. 

With Color Space Transform it takes me 3 to 5 min to have a good starting point for all my clips on the time line. Apply CST on one clip, take a still grab then apply the still (and CST settings) to all clips. And almost all of them have white black points more or less correct, skin tones are OK etc. It's much faster method. It was a game changer for me  

With BMPCC 4K clips can get away with the first method and not spend tons of time because Davinci knows quite a lot about their own video clips, when you place them on the timeline. It's much better than GH5 or Sony HLG. It is for those 8bit compressed codecs where CST method shines the most. Again at least that's my experience. Color matching BMPCC 4K BRAW and Sony A7 III 8bit HLG is for me now easier than ever.   
 

[kye]

10 hours ago, stephen said:

Hmmm... Am not convinced this is true. And am not sure what non-destructive means. Everything that watch and read from professional colorists including Juan Melara's video point in the opposite direction. (HLG -> rec709 -> HLG = rec709)

You are misinterpreting Juans video and confusing yourself.

Resolve retains all values, even when they are pushed outside the legal ranges.

Please open Resolve and do the following:

• Pull in a test clip
• Node 1: Adjust curves so that you make the image very bright and really really clipped
• Node 2: Adjust curves so that you darken the clip so the clip comes back into range

You will see by doing that test that even through the first node pushed the clip very far above the white point (100% brightness) that the data was still there and re-appears when the second node pulls the levels back down into the legal range.

Juans video on the URSA Mini is talking about many advanced concepts and he uses his words somewhat loosely which can be misinterpreted if you're not 100% clear on the order of operations in Resolve and what each operation does.  He moves very quickly and if that video was made by any other YouTuber it would have been 40 minutes as they clumsily explained each button-press and what was going on, whereas Juan just assumes you've got the knowledge and can keep up.

If you are going to look at Juans videos then I'd suggest doing what I did, and starting with the most basic ones, study them like you're at college/university and like there will be a test on them.  Recreate each node tree he shows, and play with it.  Read about each colour space and gamma curve that he mentions.  Read the Resolve manual for the Colour page.  If you didn't study the highest maths in high-school then go and learn the maths behind Linear and Log, learn what cartesian and polar coordinates systems are and how they work (RGB is cartesian and HSL is polar).  Read about human vision and hearing and understand the relationship between the physics of light (Linear), the mathematics of Log (ln(2)) and the perceptual experience of light and sound (ln(10)) and how they relate to Dynamic Range.  

It took me months to be able to wrap my head around what he's talking about in that video and I have a computer science degree and spent a decade in audio where I was reading about everything in the signal path including psychoacoustics, which there are strong parallels between the signal path of light and the signal path of sound, even deep into our perceptions.