Exclusive: here is the first app to resample Panasonic GH4 4K 8bit to 10bit 4:4:4

[Andrew Reid]

A few days ago Thomas Worth of Rarevision (5DToRGB and RAWMagic) asked me for some original files from the GH4 so he could develop an experimental app he had in mind. The result is an embryonic new command-line app for the Mac, which takes 8bit 4:2:0 4K files from the GH4 and resample them 2K 10bit 4:4:4.

Read the full article here

[dsavage]

Would this process also be possible with Cineform Studio Pro?

[Sean Cunningham]

I wonder how hard would it be to, instead of summing, do a poisson disc sample to arrive at the 2K image?  

[coyut]

I'd love to see a win version ;)

[MaxAperture Films]

Great job Thomas @ Rarevision... looking forward to seeing the results of 10 bit conversion.  In theory the files should be able to be pushed further in color correction, now we'll find out if that is actually true.  Any plans for PC supported script?

[Dekam]

Is this behaviorally equivalent to converting the 4K footage to 10-bit, then bilinear downsampling to 2K 10-bit?  Or has some additional special sauce been introduced?

[jcs]

I don't want to rain on the parade here (I championed post-sharpening for the 5D3 when folks didn't believe it would work), however the math doesn't actually predict much of an improvement for this process. First, we're not going to get a 10-bit 444 file. We'll get (at best) an 8.67-bit 444 file. Regardless of credentials and title and who said what, it's not possible to get 10-bit 444 from 420 8-bit (unless perhaps the 420 is taken internally).

 

Here again is what I wrote in the other thread (I'm a filmmaker, image-processing app developer, and game developer: I'm not a mathematician either :) ):

 

 

10 bits is 2 more bits vs 8, 2^2= 4, so adding 4 8-bit values together gives us a 10-bit result. If we add up all the values, then divide by 4, thus averaging the result, we'll still see the extra information in the fraction. So if we average the values together in floating-point, we've achieved the 'effect'. This is effectively what an NLE will do when rescaling, so we don't need any special extra processing for an NLE that works in 32-bit float.

 

420 AVCHD (H.264) is YUV. If we scale 4K YUV 420 to 2K 444 YUV, only Y is full resolution, and only Y will get the benefit of the 4-value summing and additional fractional bit depth. Luminance is more important than chrominance, so that's not so bad. So best case, we have 10-bits of information in Y and 8-bits for U and V. This is (10+8+8)/3 = 8.67 bits per pixel. If the NLE does the scaling after converting YUV to RGB, it's still 8.67-bits of information per pixel at best (no new information is added during the transformation).

 

This is why we won't see a significant improvement in color depth. Here's a challenge- create an example that shows the '10-bit' effect is significant (I agree it's there, but at 8.67 actual bits, it will be hard to see in most cases).

[Andrew Reid]

I'm no mathematician either JCS but my eyes tell me the GH4's 4K grades for 2K like a 10bit codec. It looks and feels completely different to the 8bit stuff I'm used to. Hopefully Thomas can put this to rest and explain the maths behind it in a way we can all understand?

[sunyata]

I don't want to rain on the parade here (I championed post-sharpening for the 5D3 when folks didn't believe it would work), however the math doesn't actually predict much of an improvement for this process. First, we're not going to get a 10-bit 444 file. We'll get (at best) an 8.67-bit 444 file. Regardless of credentials and title and who said what, it's not possible to get 10-bit 444 from 420 8-bit (unless perhaps the 420 is taken internally).

 

Here again is what I wrote in the other thread (I'm a filmmaker, image-processing app developer, and game developer: I'm not a mathematician either :) ):

 

I feel I should also add something of a warning for people that might think you can get great 10bit from 4k 8bit.. there are areas of quantization in your source 4k (large blocks of pixels that are forced to the same color values) that can be much larger than the pixel matrix that you are subsampling from, no matter what the algo is that you use, the blocks remain visually. in effect, you are smoothing just the contrast border areas of those quantized blocks... and later if you push the gamma in color correction, they will re-appear. 

 

This is not the equivalent of re-sampling the light at 2k, 10bit 4:4:4 native, but I commend the effort and am sure it will be useful.

[Andrew Reid]

Just tried it and works to output the DPX, the files look superb. I opened frames in Premiere, which does not treat it like a clip, but the frames grade superbly and look amazing.

 

However Resolve 10.1 gives me flickering playback with some frames shifted, some half black, to a random degree. Anyone else had success with Resolve yet?

[Andrew Reid]

"Luminance is more important than chrominance, so that's not so bad. So best case, we have 10-bits of information in Y and 8-bits for U and V"

 

I think Thomas did indeed mention 10bit luma in his original post.

 

The luma data alone is enough for a nice true 10bit black and white image. U and V contain the colour information. So luma (i.e. the bit important for smooth gradation and high dynamic range) goes 10bit and colour remains at 8bit. Am I correct here or talking out of my arse? :)

[johnnymossville]

I think another option would be to overlay a layer of 16 bit color, maybe a dithered 16 bit vignette created in photoshop, to the 4k file to increase it color depth and cover those areas of the image that have one one 8 bit color. Render that out in 444. Then down sample/size to 2k. Actually add color to the 8bit 4k file manually. If that makes sense.

[Michael Ma]

Need A windows method.  More importantly, need a GH4.   ;)

[rygenova]

Maybe I'm completely wrong, but the math makes sense. Let's look at a really simple example, say converting a 2x2 pixel video to 1 pixel video with 2-bit luma channel.  4K is roughly 4 times the resolution of 1080p, so in this example it is the same down-coversion.  For 2-bit luma we can have a value of 0, 1, 2, or 3.  Let's say we take our 2x2 video and the pixels end up with luma values of 2, 2, 2, and 1.  Now, let's average these value for our 1 pixel, down-converted video and we end up with a luma value of 1.75, a value that can't be represented by our 2-bit luma values.  For this averaged, down-coverted pixel we can have values of 0, 0.25, 0.5, 0.75, 1, 1.25, etc.  In fact we can have 16 different values of luma now, a 4-bit value.  By averaging the luma values, we've definitely gone from 2-bit (4 possible values for the luma) to 4-bit (16 values for luma).  Reduction from 4K to 1080p does increase the number of possible luma values by a factor of 4, hence 8-bit to 10-bit.

 

Could be interesting that using a Shogun could yield 12-bit 1080p.

[sunyata]

that is correct but, in your example, you would never have a better dataset to sample from than that 2-bit data, so you would always represent  your higher bit data as a different representation of the original 2 bit data. so something that might have millions of shades of luma would always appear as 1.75 (in your example), you would never be able to see a clearer representation of the original light... even with more pixels.

 

if you think about a building way off in the distance with dimly lit windows and someone standing inside: even with infinite pixel density, we might never see that person w/o enough levels of luma... no matter how many times you sample that data, that person is not visible.

[jcs]

We are all in agreement that we'll get 10-bits of luma information and 8-bits of chroma. Now, think about photons hitting the sensor. If the photons hit the 4-pixel group evenly, or with minor variance, we won't get much out of our 10-bit final result. Interestingly, noise will help get more variance. Will the noise act like dither or will the low-pass filter effect just smooth the result? Intuitively, I don't think there will be enough variance for the 4-pixel subgroup to have much of a visible effect in helping to reduce banding or in increasing grading latitude vs. 10-bits from the sensor.

However, if you guys are having fun exploring through experimentation, carry on! :)

[theSUBVERSIVE]

I have a couple of practical questions:

 

- What will grade better? Andrew already said that the 4K file holds pretty good when grading, so is it better to convert it right away and then grade it? Or to grade in 4K, render and convert it to 2K using this app - supposing this app is not exclusive to the GH4?

- Will using the 10-bit 4:2:2 to convert it get what? Closer to 10-bit 4:4:4 or just something else? How much better will it hold for grading?

[see ya]

I mentioned this in another thread, perhaps someone could clarify whether the fact 8bit 4:2:0 doesn't use the full 8bit levels range unless its JFIF standard then should these calculations include that?

Also the GH4 keeps getting talked up regarding the conversion to 10bit but to clarify surely any 4k cam output could do the same in theory.

Not sure whether Thomas is using QT or FFmpeg for decompression but these GH4 movs do appear to be JFIF and flagged full range so QT appears to immediately rescale levels to 16-235 at decompression and clip any non flagged full range sources? Certainly looks that way from Driftwoods GH4 native source and buggers muddle of a full range encode on Vimeo.

[sunyata]

thesubversive -- so i think you're going to get better results grading after converting, because not only will you be in a higher depth, but those border contrast areas will be smoothed out a little, but if you want to resolve to 1080 and get dynamic range to work with, i think you should look at a native 10bit capture solution instead... which could be the GH4 with HDMI, or BMPCC etc.

[Thomas Worth]

I mentioned this in another thread, perhaps someone could clarify whether the fact 8bit 4:2:0 doesn't use the full 8bit levels range unless its JFIF standard then should these calculations include that?

Also the GH4 keeps getting talked up regarding the conversion to 10bit but to clarify surely any 4k cam output could do the same in theory.

Not sure whether Thomas is using QT or FFmpeg for decompression but these GH4 movs do appear to be JFIF and flagged full range so QT appears to immediately clip outside of 16-235 at decompression? Certainly looks that way from Driftwoods GH4 native source and buggers muddle of a encode on Vimeo.

 

The GH4 files are flagged Rec. 709, which implies broadcast range levels. I use the Rec. 709 matrix in gh444, which seems to be correct so this all looks like good intel. However, I've seen cameras that output full range even though they're flagged broadcast range. As you mentioned, QuickTime clips the data which is why 5DtoRGB is a good tool to recover the lost highlight detail.

 

FFmpeg's H.264 decoder doesn't care about levels/ranges. It just outputs whatever is in the file. It's up to the programmer to make sense of that depending on what type of file they're writing. In this case, it's uncompressed 10 bit RGB (full range) so I leave the levels alone. That said, nothing is getting clipped with DPX, obviously.