Interpretation issues of Fujifilm video files in Davinci Resolve, Premiere Pro and Final Cut Pro

[androidlad]

1 hour ago, Llaasseerr said:

I've done some HLG tests with the caveat that I seem to be seeing this "video levels" interpretation issue with full range h.265s now in Resolve 16 (see my post above). So right now, my test is inconclusive even after rolling back to Resolve 15 to do a sanity check. However as a workaround I'm using the "full to legal" lut for now.

What I did notice is that the metadata tags for the internally recorded HLG are:

Color primaries : BT.2020
Transfer characteristics : HLG
Matrix coefficients : BT.2020 non-constant

For the externally recorded ProRes it's all BT.709 for those tags which is clearly incorrect. We weren't paying much attention to the Atomos setup so maybe there was a "HLG" switch on it that tells it to embed the correct metadata. 

I had also used Editready to convert F-log clips, and it fixed the problem I'm reporting about the incorrect video levels interpretation of h.265. Something interesting though is that if I convert the HLG h.265 to ProRes using EditReady, it preserves the metadata tags. It means the converted HLG doesn't match the Atomos recorded HLG. However I feel the recorded HLG is interpreted incorrectly by Resolve because of its incorrect metadata. So the EditReady converted HLG is probably correct, and it matches the h.265 once I add the "full to legal" lut to the h.265.

In Resolve, data/video level is arbitrary really, it's easily selectable in "Levels" clip setting. There's no need to use a LUT for that.

Also it's not really a problem for external ProRes for HLG to contain all BT.709 tags. HLG/BT.2020 tags are required for instant HDR playback on end user devices, or for automatic HDR to SDR conversion. Doesn't matter for post-production. Alexa Log-C ProRes carry all BT.709 tags as well, ARRI has to put additional metadata field to say it's Log C.

[Llaasseerr]

Quote

I didn't test HLG but if your issue with externally recorded files is metadata only, ffmpeg can change tags without re-encoding using the bitstream filter. I can help you with that if needed.

Thanks, that's great to know. Right now I installed ffmpeg through homebrew and it didn't configure with --enable-libzimg.

I'd be surprised if Atomos did not have some way of handling HLG more elegantly than what we did where we ended up with Rec709 tags. But we were shooting with an older 7" Ninja Flame I think, not the latest Ninja V. It did support HLG though.

I think the important thing for me is to know what my baseline correct colour handling is in HLG. I'm assuming that if the full range issue on the internal files is sorted out then that is my baseline.

[Llaasseerr]

47 minutes ago, androidlad said:

In Resolve, data/video level is arbitrary really, it's easily selectable in "Levels" clip setting. There's no need to use a LUT for that.

Also it's not really a problem for external ProRes for HLG to contain all BT.709 tags. HLG/BT.2020 tags are required for instant HDR playback on end user devices, or for automatic HDR to SDR conversion. Doesn't matter for post-production. Alexa Log-C ProRes carry all BT.709 tags as well, ARRI has to put additional metadata field to say it's Log C.

I assume you mean Clip Attributes->Data Levels, which is what I'm using. The issue is that all of a sudden full levels on an internal clip is the same as video levels on an external clip (for F-log). This seems to be a bug introduced in Resolve 16 beta. So right now, I need to add the compensatory LUT as a workaround to force the internal clip to full range.

I do realise the LUT is not necessary if everything worked properly, and that it's a very simple math function that could be written as a DCTL, or done in Nuke with a Grade node.

As for the Atomos HLG issue,  yeah I originally thought the same as you're saying that it wouldn't matter if all the tags are Rec709 so I'm glad you said that. And you're right actually. I just took another look at it and the issue is that if I set both the internal and external clips to full range, then I need to add the extra LUT twice to get the internal clip to match the full range luma levels of the ProRes clip. Obviously that's broken. So I don't know which one is correct. Possibly neither of them. I think this would have been easy to sort out in Resolve 15 prior to upgrading to 16, but now I've downgraded I'm seeing the same weird behaviour in 15 as well.

I'll admit I come from film where I'm mainly dealing with EXRs and DPXs and I never have to think about these kinds of issues. And I have not needed to look closely at broadcast HDR yet, but I love the fact you're getting a great result with HLG as a substitute log curve for X-T3 capture. I've had some promising early results too, but I'm working out a few kinks.

 

[androidlad]

4 minutes ago, Llaasseerr said:

The issue is that all of a sudden full levels on an internal clip is the same as video levels on an external clip (for F-log). 

I just took another look at it and the issue is that if I set both the internal and external clips to full range, then I need to add the extra LUT twice to get it to match the full range luma levels of the ProRes clip.

These issues you are describing are hard to grasp without any visuals or actual samples.

[Llaasseerr]

It's exactly as I say it is. You could take the same clip and set data levels to video for one of them and full for the other and that is the exact visual result. However what I'm actually doing is setting both internal and external clips to full, but the internal clip has the appearance of the external clip if it was set to video. This is only since Resolve 16 beta.

Edit. That's for F-log. For HLG, I would need to add the compensatory LUT twice, not just once, to get the internal clip to match the external.

[Attila Bakos]

1 hour ago, androidlad said:

Also it's not really a problem for external ProRes for HLG to contain all BT.709 tags. HLG/BT.2020 tags are required for instant HDR playback on end user devices, or for automatic HDR to SDR conversion. Doesn't matter for post-production. Alexa Log-C ProRes carry all BT.709 tags as well, ARRI has to put additional metadata field to say it's Log C.

There is a matrix in the ITU-R BT.2020 specification that's different from BT.709 and BT.601. If that doesn't matter, then this whole topic is meaningless

[androidlad]

3 minutes ago, Attila Bakos said:

There is a matrix in the ITU-R BT.2020 specification that's different from BT.709 and BT.601. If that doesn't matter, then this whole topic is meaningless

Yes, doesn't matter for post-production. We use HLG as an alternative log profile.

BT.2020 matrix coefficients tag is required for HDR playback, or on-the-fly conversion for SDR playback.

This is why in Scratch, X-T3 HLG files are default blown-out (on an SDR display) because Scratch respects the tags. If we change the decoding parameter to BT.709 SDR, the images return to the flat look as seen on camera.

Premiere ignores the tags and decodes everything in BT.709, so HLG files have that flat look ready for grading.

[Attila Bakos]

20 minutes ago, androidlad said:

Yes, doesn't matter for post-production. We use HLG as an alternative log profile.

BT.2020 matrix coefficients tag is required for HDR playback, or on-the-fly conversion for SDR playback.

This is why in Scratch, X-T3 HLG files are default blown-out (on an SDR display) because Scratch respects the tags. If we change the decoding parameter to BT.709 SDR, the images return to the flat look as seen on camera.

Premiere ignores the tags and decodes everything in BT.709, so HLG files have that flat look ready for grading.

Respecting or disrespecting the matrix coefficients tag won't result in blown-out footage. I believe you're talking about transfer characteristics, which is basicly the linearization function.
Disrespecting the matrix coefficients tag will result in different colors, as you have seen in my video.

I can show you what I mean. This is from a HLG video using the BT.709 matrix, extracted with FFMPEG. (Resolve has the same colors, I checked it.)

 

This is the same frame using the BT.2020 matrix, this is actually how FFMPEG extracts the frame without any extra options, since it respects the tags by default. But I also used zscale to do the scaling manually and the results are identical to what FFMPEG does by default:

 

Open the files in different tabs and switch between them. You might say that this doesn't matter to you, but the difference is there.

[Attila Bakos]

To me this seems to be the exact same issue as the one described in the video, but with different matrices. There are 3 matrices to go from YUV (YCbCr to be technically correct) to RGB. For SD usually the BT.601 matrix is used, for HD the BT.709 matrix, and there's the BT.2020 matrix which you can see in HLG for example. That doesn't mean that you can't encode an UHD resolution log format with the BT.601 matrix, Fuji does just that. And as androidlad mentioned, you can have Alexa log encoded with the BT.709 matrix. To have the right colors in RGB, you have to use the correct matrix. Then you can delog your footage and convert color space etc. But all this comes after the YUV->RGB conversion.

[Llaasseerr]

6 hours ago, Attila Bakos said:

To me this seems to be the exact same issue as the one described in the video, but with different matrices. There are 3 matrices to go from YUV (YCbCr to be technically correct) to RGB. For SD usually the BT.601 matrix is used, for HD the BT.709 matrix, and there's the BT.2020 matrix which you can see in HLG for example. That doesn't mean that you can't encode an UHD resolution log format with the BT.601 matrix, Fuji does just that. And as androidlad mentioned, you can have Alexa log encoded with the BT.709 matrix. To have the right colors in RGB, you have to use the correct matrix. Then you can delog your footage and convert color space etc. But all this comes after the YUV->RGB conversion.

I hope you're on to something, which is why I thought of posting in this topic. But I'm not sure where this additional transform is happening under the hood. I haven't updated my Mojave install since updating (then downgrading) Resolve, so I don't see how any OS -level libraries could do this. I've escalated this to BMD support as well, so I'll see what they say. 

I did install OpenImageIO via Homebrew, which installed ffmpeg, not sure if that could have changed something that Resolve also looks at.

I should describe my end usage scenario too. I'm planning for a SDR finish and I'm wanting to use HLG as a substitute log curve instead of F-log since it seems a better choice right now until maybe Fuji create an F-log2. I'm transforming it to a more familiar Arri LogC/AWG workflow once it's in Resolve. Thanks to @androidlad  for his work with HLG that brought it to my attention!

So basically if the HLG is living as a very flat log image on a 500 nit SDR display the same way a typical log captured image appears, then that is totally appropriate. I'm actually hoping that in a working install of Resolve 15, that this is all still behaving correctly.

Also can anyone recommend a 3x3 matrix DCTL or OFX plugin so I can directly apply a matrix in Resolve like I can in Nuke?? I used to have one made by Paul Dore (baldavenger) but I kind of lost track of where that is on his GitHub.

 

[Llaasseerr]

Re-reading what you're saying though, does this possibly account for the main issue I'm seeing with the video/full data levels not behaving predictably? That's the thing that's biting me the most, more so than colour shifts due to an incorrect matrix although that's important.

[Llaasseerr]

I posted this topic on the BMD forum too:

https://forum.blackmagicdesign.com/viewtopic.php?f=32&t=89799&p=503331#p503331

[Attila Bakos]

I understand your issue but I honestly don't know what's causing it, OSX is unknown territory for me. 

What kind of matrix do you need?

[androidlad]

Mystery Box offers LUTs that correct BT.2020 matrix decoded as BT.709

https://www.mysterybox.us/store/hdr-application-luts-v-11

[Attila Bakos]

Sometimes I feel stupid that I give away stuff like this for free.. 

[Llaasseerr]

20 hours ago, Attila Bakos said:

I understand your issue but I honestly don't know what's causing it, OSX is unknown territory for me. 

What kind of matrix do you need?

I also use Linux for work but I haven't gotten Resolve up and running on it yet. I was just thinking it would be handy to just work out the matrix whenever one is needed and then be able to apply it losslessly and mathematically with a Matrix node in Resolve. It's mind boggling that one is not included. I've implemented a static matrix transform as a dctl (Arri film matrix, just for fun) and that was easy, so in theory I could quickly do that for 601 to 709.

However I think I'm going to try and focus on HLG now, so I can probably use the CST to solve any colour space issues. I'm still talking to BMD about the data levels issue.

[Attila Bakos]

Just keep in mind that you can't fix a bad yuv to rgb conversion with cst. So if you know the math, create a dctl that undoes the 709 matrix and applies the 601 matrix. You can also create one for hlg that undoes the 709 matrix and applies the 2020 matrix. I think I'll create a package with all the possible fixes for this issue, in lut and dctl format, and I might ask a few dollars for it. 

[Llaasseerr]

On 4/27/2019 at 10:00 PM, Attila Bakos said:

Just keep in mind that you can't fix a bad yuv to rgb conversion with cst. So if you know the math, create a dctl that undoes the 709 matrix and applies the 601 matrix. You can also create one for hlg that undoes the 709 matrix and applies the 2020 matrix. I think I'll create a package with all the possible fixes for this issue, in lut and dctl format, and I might ask a few dollars for it. 

Yes. That was my understanding. For sure it would be good to get a readily available fix out there for those that don't want to delve into the maths.

[Attila Bakos]

I removed the free fix, and created a package with more options, including DCTLs: http://colorizer.net/index.php?op=technical

On the bottom of the page you'll still find an option to fix these issues for free using FFMPEG.
I can include more stuff in the package if any of you have suggestions.

[thephoenix]

17 hours ago, Attila Bakos said:

I removed the free fix, and created a package with more options, including DCTLs: http://colorizer.net/index.php?op=technical

On the bottom of the page you'll still find an option to fix these issues for free using FFMPEG.
I can include more stuff in the package if any of you have suggestions.

how does it work with resolve color management ?