Optimising resolution & sharpness in post

[kye]

There is an optimum resolution and sharpness.  More is not always better.  
This is why movies aren't all shot with the highest MTF lenses currently available - DoPs choose the optimal lenses and apertures for the scene / project.

However, I shoot with cheap cameras (iPhone, GX85, GH5, etc) which are far too sharp, and look video-ish.  Luckily, we can reduce this in post.

This thread is me trying to work out:

• What the range of optimal resolution / sharpnesses are actually out there (from serious professionals, not moronic camera YouTubers or internet forum pedants)
• What might be a good point to aim for
• How I might treat iPhone / GX85 / other cheap shitty video-looking footage so it looks the least video it can be

These techniques will likely apply to all semi-decent consumer cameras, and should be able to be adjusted to taste.

I'm still at the beginning of this journey, and am still working out how to even tackle it, but I thought I'd start with some examples of what we're talking about.

Reference stills from the Atlas Lens Co demos from their official YT channel, shot on Komodo and uploaded 6 months ago:

(You have to click on these images to expand them, otherwise you're just looking at the forum compression...)

Reference stills from the Cooke SP3 demos from the official Cooke YT channel, uploaded 11 days ago:

I've deliberately chosen frames that have fine detail (especially fly-away hair lit with a significant contrast to what is behind it), in perfect focus, with zero motion blur.  I think this is the most revealing as it tends to be the thing that is right at the limits of the optical system.

So, what are we seeing here?

We're seeing things in focus, with reasonable fine detail.  It doesn't look SHARP, it doesn't look BLURRED, it doesn't look VINTAGE, it doesn't overly look MODERN (to me at least) and doesn't look UNNATURAL.  It looks nice, and it definitely looks high quality and makes me want to own the camera/lens combo (!) but it basically looks neutral.

But, that's not always the case.  This is also from the same Cooke SP3 promo video:

The fine detail is gone, despite there being lots of it in the scene.  Is this the lens?  Is this the post-pipeline?  We don't know, but it's a desirable enough image for Cooke (one of the premier cinema lens manufacturers in the world) to put it in their 2.5 minute demo reel on their official main page.  It also has a bit more feel than the previous images.

Contrast that with these SOOC shots from my iPhone 12 Mini:

I mean....  seriously!  

(If you're not basically dry-wrenching then you haven't opened the image up to view it full-screen.. the compressed in-line images are very tastefully smoothed over by the compression)

More:

and my X3000 action camera also has this problem:

Those with long memories will recall I've been down this road before, but I feel like I have gained enough knowledge to be able to have a decent stab at it this time.  We'll see anyway.

Follow along if you're open to the idea that more isn't better...

[kye]

Going in a different direction slightly, let's eliminate some variables and look at some controlled tests.  Luckily the absolutely spectacular YT channel ShareGrid has hundreds of lens tests on their controlled setup, so we can compare..

https://www.youtube.com/@ShareGrid

Here's one of the highest quality lenses, the Zeiss Master Anamorphic:

it looks detailed and clean, even wide open.

Contrast that to the image of the Hawk V-Lite Anamorphics...  

No, this isn't motion blur - that's how the lens actually is.

You might be thinking that this is just a relic of history, but you'd be wrong.  These were used on:

• 13 Reasons Why (2017)
• Atomic Blonde (2017)
• John Wick (2014)

Serious films with serious budgets that could have chosen other lenses, but used these.  I think this shows we have some serious latitude in terms of resolution and sharpness.

[kye]

One thing I've been experimenting with is to simply lower the timeline resolution, and then upscale on export.

Not only does this smooth over the fine detail, but it means there's less pixels being processed in the NLE, so double-bonus!

Here's a comparison from that iPhone shot.

This provides a hard-limit on the amount of fine detail in the shot, and is a pretty extreme thing to do, especially considering that 960x540 is one quarter the resolution of 1080p, one sixteenth the resolution of 4K, and less than 2% of the resolution of 8K!

But it's not as awful as you might think, here's that shot from the Cooke SP3 promo video again:

There is a difference, but it's incredibly minor, and more likely to just be smoothing rather than removing detail.  

What happens if we add sharpening?

I can't tell any difference between that and the original, maybe you can, but it's not creatively relevant.

So, does the above image, which is on the promo from one of the premier lens companies in the world, have a resolution approaching SD (480p)?  Yes, yes it does.  Is this a bad thing?  No, it's a creative choice.

Would I go this far in my own projects?  I haven't answered this definitely yet, but I think it's probably a bit too far for my goals of creating a non-video look, but it's an interesting thing to observe.

There are lots of other ways to control resolution and sharpness, and I'll be covering those too, but this one has the distinct advantage of making my computer more powerful, so it's a fun one! 

[BTM_Pix]

1 hour ago, kye said:

I can't tell any difference between that and the original, maybe you can, but it's not creatively relevant.

Its there if you are really, really bored on a Sunday morning and really, really look for it.

At 300% (😂) here is the eye.

Here is it with the lower resolution one on another layer with blend set to difference.

It is definitely there but certainly close enough for jazz in terms of how keen you have to be find it.

However, it is far more pronounced when doing the same difference layer process with your iPhone footage.

This is at 80%

And at the same 300% as the Cooke sample.

Its difficult to know if there would have been a similar outcome if you had access to higher quality source material with the Cooke footage.

[kye]

5 hours ago, BTM_Pix said:

Its there if you are really, really bored on a Sunday morning and really, really look for it.

At 300% (😂) here is the eye.

I used the built-in soften/sharpen thingy in Resolve for that one, and I think this was 4 clicks of sharpening, and 3 and 5 were definitely not right!  There have been some feature requests to give more granular control on that one!!

5 hours ago, BTM_Pix said:

Its difficult to know if there would have been a similar outcome if you had access to higher quality source material with the Cooke footage.

This was the video with the shot in it...  35s mark:

I mean, maybe the discussion went like this...  "All I could get was an SD quality shot of this guy - can we get a better version?" ... "Nah, it's only for our demo reel, just upload it, image quality isn't that important".

....  but I don't think so!

[kye]

It's also worth pointing out that I've played the game of Difference blending mode before, and tried to replicate images using that mode alone, by matching the response across smaller and larger blur sizes.

Sadly, it didn't work.

[BTM_Pix]

27 minutes ago, kye said:

This was the video with the shot in it...  35s mark:

 

Don't know whether you would get more joy quality wise with the Vimeo version.

I think if you have a paid plan you can download it and might be higher resolution options.

 

28 minutes ago, kye said:

It's also worth pointing out that I've played the game of Difference blending mode before

It was just a quick lash up in Photoshop to see if there were differences that were there but imperceptible.

Have to say if they are imperceptible then its probably moot anyway.

Although the difference with the iPhone shots were far more perceptibly imperceptible 😉 

[kye]

Next strategy is to blur the image and put the original over the top with a small opacity.

The theory behind this is that if you blur the image slightly and put the original over the top then you're essentially "turning down" the differences, which will be the fine detail and sharpness of edges etc.  I've been reviewing a lot of material shot on 35mm film and I suspect that this is what film does - beyond a certain point the finer the detail the more it is attenuated.  
This is different to the shots above with lowered resolution - in those the detail is eliminated completely, but in this method the detail is attenuated but still present.  In theory we can have a larger radius blur using this method, because we're still adding in the original so the resolution isn't actually reduced at all but the sharpness definitely will be.

I've used the built-in Blur tool in Resolve (which has very very course adjustment) because it is faster than the Gaussian Blur OFX and the results are indistinguishable at the right settings.  I've lowered the opacity of the effect, so a 90% blur is 90% blurred image and 10% the original image.  The Blur tool is neutral at 0.5, blurs with higher values and sharpens at lower values.

Here are some sample shots.  These are between the perceptual sharpness of the original image and the 960x540 image above, which I think is too soft for my tastes and only relevant when going for a strong retro/film look.

Once again, you have to open the full res versions in a separate window in your browser to see any differences.

I'm curious to hear any impressions people have.  

I'm also aware that streaming compression will have an effect, but I'm also aware that perceptual resolution seems to be increased when viewing moving images rather than stills, as if the eye is interpolating between frames.  This is a whole other thing, but it's definitely an effect.  When I start narrowing in I'll start exporting sample videos, and add grading too - the pale green of iPhone footage isn't so useful.

[kye]

I've been looking at film, not as the ideal reference to replicate, but as a data-point to help understand what we are seeing.

What I have been noticing is that on film, finer detail is present but at a lower contrast ratio, whereas digital keeps the same level of contrast regardless of the size of the detail (up to the sensor resolution limits, and subject to the lens characteristics of course).

For example, here's an image from Game of Thrones, shot on Alexa/RED:

and here's a closeup:

Notice that the individual strands are basically as bright as the larger areas of his hair, right up to the resolution of the file.

Same with this closeup of an image I posted earlier:

There is a little bit of contrast loss in these examples, but lenses and compression are also in the mix too.

Contrast that with this shot from the original Blade Runner, which was obviously shot on film:

Here's a closeup from the above:

Note how the individual strands have far less contrast with the background than the more solid areas with many strands of hair.

Here is the MTF chart of Kodak Vision3 500T, which confirms these observations.

So, what is the actual resolution?

<snip> see below.....

Ironically, digital sharpening methods have a completely inverse response - they increase the contrast on fine detail rather than decreasing it.  

So, digital cameras take a readout from the sensor that has an essentially flat MTF curve, and then apply a transfer function that does the exact opposite of what film did.

Awesome..... screw you too!

Food for thought.

[kye]

Crap.

[kye]

Sorry - just realised I screwed up the math so cut it out of the above posts while the edit window was still open, then re-thought it again.  Here's the corrected version....

I'm confused about the math, depending on how you define "sine waves".  Here's what I think is the correct one:

At 10 cycles/mm, which is where there is no softening, on Super35 3-perf which is ~25mm x 14mm, that would be a horizontal resolution of 500 x 280.  The curves end at about 75 cycles/mm, which is ~3750 x 2400, where the contrast has dropped to between 40-15% of the contrast in the scene.

There is a chance they mean it in a different way, which doubles the resolutions to 1000 x 560 up to ~7,500 x 4200.  I think the above one is more likely to be correct though.

These curves are obviously dropping as the level of detail gets smaller, but if you were to extend them then they might have a 1% contrast response at over 600 cycles/mm, which would be approaching 30K resolution.  This is an incredible assumption and should be taken with an entire bag of salt that you buy at the hardware store to put in your pool, but still.....  this is why specifying the resolution of film in pixels is an inappropriate measure, it simply doesn't work like this.

All this is just the resolution of the negative.  I've heard knowledgeable people say that the resolution of film is a lot, but by the time that you print the negative, cut it up in the edit (in a purely analog pipeline), then duplicate that as a master, then take several prints off that to then duplicate to make the prints that go out to each megaplex, the resolution has been reduced to approximately 2K.  This is why DI systems were at 2K for so long during the days of film acquisition and distribution.  During that period people didn't suddenly start saying that movies were blurrier and asking for their money back - a 2K DI was sufficient.

[kye]

Just found this video outlining a bunch of new features in Baselight (the other worlds best colour grading platform) and it confirms exactly what my thoughts were about sharpening and blurring, starting at 31m:

He set up a frequency generator that starts at low frequencies on the left and goes higher towards the right, then uses the waveform monitor to show the amplitude, essentially showing a frequency response plot.

The plot looks like this before any transformations:

Adding sharpening to the image increases the amplitude of smaller frequencies:

and a simple Gaussian blur reduces them:

which is what we want to counteract the effects of in-camera sharpening, as well as replicate the decreasing response of film:

I'm HUGELY encouraged now that I've worked out that a blur is a reasonable approximation of a more analog look, and that it also reverses sharpening.

Another thing he demonstrates is that by increasing contrast on the image, you are amplifying the contrast of both the fine detail as well as the lower frequencies, so you're changing the texture of the image in ways that may not have occurred had the scene been lit with a greater contrast ratio.

At 46m he shows an example (from Mindhunter, which was graded heavily in Baselight, fun fact!) with this being how it was shot:

then you might add an effect in post like there was an additional light source:

which has the effect of amplifying the contrast on everything, including fine details (ie, what sharpening does), but then he uses a special tool they've developed that only adds contrast to the lower frequencies and doesn't amplify the finer details, and voila:

Now it looks natural, like that could have been how it was on set.

I suppose you might be able to build this operation using frequency separation, but in Baselight it's just a slider.  I guess that's one of the many reasons why instead of buying Resolve for $299 you'd buy Baselight, which is more of a "mortgage your house" sort of thing.

The rest of the talk is excellent and worth watching, and this one that is more recent has some new stuff that is also fascinating:

[Beritar]

On 9/17/2023 at 8:41 AM, kye said:

There is an optimum resolution and sharpness.  More is not always better.  
This is why movies aren't all shot with the highest MTF lenses currently available - DoPs choose the optimal lenses and apertures for the scene / project.

However, I shoot with cheap cameras (iPhone, GX85, GH5, etc) which are far too sharp, and look video-ish.  Luckily, we can reduce this in post.

This thread is me trying to work out:

• What the range of optimal resolution / sharpnesses are actually out there (from serious professionals, not moronic camera YouTubers or internet forum pedants)
• What might be a good point to aim for
• How I might treat iPhone / GX85 / other cheap shitty video-looking footage so it looks the least video it can be

These techniques will likely apply to all semi-decent consumer cameras, and should be able to be adjusted to taste.

I'm still at the beginning of this journey, and am still working out how to even tackle it, but I thought I'd start with some examples of what we're talking about.

Reference stills from the Atlas Lens Co demos from their official YT channel, shot on Komodo and uploaded 6 months ago:

(You have to click on these images to expand them, otherwise you're just looking at the forum compression...)

Reference stills from the Cooke SP3 demos from the official Cooke YT channel, uploaded 11 days ago:

I've deliberately chosen frames that have fine detail (especially fly-away hair lit with a significant contrast to what is behind it), in perfect focus, with zero motion blur.  I think this is the most revealing as it tends to be the thing that is right at the limits of the optical system.

So, what are we seeing here?

We're seeing things in focus, with reasonable fine detail.  It doesn't look SHARP, it doesn't look BLURRED, it doesn't look VINTAGE, it doesn't overly look MODERN (to me at least) and doesn't look UNNATURAL.  It looks nice, and it definitely looks high quality and makes me want to own the camera/lens combo (!) but it basically looks neutral.

But, that's not always the case.  This is also from the same Cooke SP3 promo video:

The fine detail is gone, despite there being lots of it in the scene.  Is this the lens?  Is this the post-pipeline?  We don't know, but it's a desirable enough image for Cooke (one of the premier cinema lens manufacturers in the world) to put it in their 2.5 minute demo reel on their official main page.  It also has a bit more feel than the previous images.

Contrast that with these SOOC shots from my iPhone 12 Mini:

I mean....  seriously!  

(If you're not basically dry-wrenching then you haven't opened the image up to view it full-screen.. the compressed in-line images are very tastefully smoothed over by the compression)

More:

and my X3000 action camera also has this problem:

Those with long memories will recall I've been down this road before, but I feel like I have gained enough knowledge to be able to have a decent stab at it this time.  We'll see anyway.

Follow along if you're open to the idea that more isn't better...

About the GH5 being too sharp, have you tried the Open Gate mode of the GH5 ? I don't speak about the 6K photo but the 6K anamorphic. I've found the amount of details very good without too much sharpening with this mode.

This is just my opinion but I don't like the internal details rendering of the new Panasonic cameras like the S5II, I find the details "rough", like if a mix of noise reduction and sharpening is used.

[kye]

4 hours ago, Beritar said:

About the GH5 being too sharp, have you tried the Open Gate mode of the GH5 ? I don't speak about the 6K photo but the 6K anamorphic. I've found the amount of details very good without too much sharpening with this mode.

This is just my opinion but I don't like the internal details rendering of the new Panasonic cameras like the S5II, I find the details "rough", like if a mix of noise reduction and sharpening is used.

I have, and you're right that it's less sharpened than the 16:9 modes.  

I'm still trying to work out what is what, but my current thinking is:

• Analog has a decreasing contrast on fine detail
• RAW unprocessed digital has approximately no (or very low) lowering of contrast on fine detail
• Processed digital has a rising contrast on fine detail

By this measure, even if there was no sharpening on it (there is, it's just less) then that would still look digital, it would just look less digital than the over sharpened stuff.

I've been paying attention to peoples hair while watching TV and movies over the last few weeks, especially where there is edge lighting, and most of the material shot with high end equipment has significantly less contrast on fine detail than the test videos published by the camera and lens manufacturers.  I suspect this is deliberate, so I suspect that even shooting digital requires some correction in order to not look digital.  The ARRI videos demonstrating their lens lineup look quite digital to me, and not in a good way - this might be why.

[kye]

I've developed a more sophisticated "false sharpness" powergrade, but it was super tricky to get it to be sensitive enough to tell the difference between soft and sharp lenses (when no sharpening has been applied).

Here are some test shots of a resolution test pattern through two lenses - the Master Anamorphics which are gloriously sharp, and the Hawk Vintage '74 lenses which are modern versions of a vintage anamorphic.

ARRI ungraded with the false colour power-grade:

Note that I've added a sine-wave along the very bottom that gets smaller towards the right, and acts as a scale to show what the false sharpness grade does.

Here's the Hawk:

and the Zeiss one with a couple of blur nodes to try and match the Hawk:

 

Here's the same three again but without the false sharpness powergrade.

Zeiss ungraded:

Hawk ungraded:

Zeiss graded to match the Hawk (I also added some lens distortion too):

Interestingly, I had to add two different sized blurs at different opacities - a single one was either wrong with the fine detail or wrong on the larger details.  The combination of two blurs was better, but still not great.  I was wondering if a single blur would replicate the right shape for how various optical systems attenuate detail, and it seems that it doesn't.  This is why I was sort of wanting a more sophisticated analysis tool, but I haven't found one yet, and TBH this is probably a whole world unto itself, and also, it's probably too detailed to matter if I'm just trying to cure the chronic digitalis of the iPhone and other digital cameras.

....and just for fun, here's the same iPhone shot from previously with the power-grade:

If I apply the same blurs that I used to match the Zeiss to the Hawk, I get these:

It's far too "dreamy" a look for my own work, but the Hawk lenses are pretty soft and diffused:

 

[foliovision]

Hi Kye,

Your work on analogising iPhone footage looks like just what the doctor ordered (I have a 14 Pro, which doesn't offer the Prores log option, which in turn suffers from crazy space requirements).

How did you do that powergrade? It looks great (albeit very strong as you point out – all the better for showing the effect as before/after and in combination with the Hawk V-Lite real world examples).

[kye]

On 9/30/2023 at 11:33 PM, foliovision said:

Hi Kye,

Your work on analogising iPhone footage looks like just what the doctor ordered (I have a 14 Pro, which doesn't offer the Prores log option, which in turn suffers from crazy space requirements).

How did you do that powergrade? It looks great (albeit very strong as you point out – all the better for showing the effect as before/after and in combination with the Hawk V-Lite real world examples).

The Hawk "emulation" was simply two blur operations, each at a partial opacity.  In Resolve:

• Node 1: Blur -> Blur tool at 0.53 with Key of 0.6
• Node 2: Blur -> Blur tool at 1.0 with Key of 0.35

The first one (0.53) is the small radius blur that knocks the sharpening off the edges, and if you were just using this one on its own you might even want to make it closer to 90% opacity.  The second one is a huge blur (1.0) that provides the huge halation over the whole image.

I use the Resolve Blur tool because it's slightly faster than the Gaussian Blur OFX plugin on my laptop, but the OFX plugin allows much finer adjustments so that might be easier to play with.  You can also adjust the size of the blur and the opacity in the same panel, so it might be easier to get the look you want using it.

What are you grading?  I'd be curious to see any examples if you're able to share 🙂