ProRes and H.265 Comparison

[KnightsFan]

I did a quick comparison between ProRes and H.265 encoding, and thought I'd share the results with everyone. I grabbed some of Blackmagic's 4.6k RAW samples and picked a 3 second clip. In Resolve, I applied a LUT and then exported as an uncompressed 1080p 10 bit RGB 444 file. This is my reference video. From this reference file, I encoded two clips. One clip is H.265, and the other is ProRes SQ, which I compared to each other and the reference video.

The reference video (which I did not upload) was 570 MB.

One of these files was created from the reference file using "ffmpeg -i Reference.mov -c:v libx265 -crf 20 -preset slower -pix_fmt yuv422p10le H265.mov". The file size is 7.81 MB (1.4% of the reference)

The other file was created using "ffmpeg -i Reference.mov -c:v prores_ks -profile:v 2 ProRes.mov". The file size was 44 MB (7.7% of the reference, or ~5x the size of H.265)

In order to keep the comparison blind, I then converted both the ProRes and H.265 files to uncompressed 10 bit 422. So you shouldn't be able to tell which is which from the metadata, file size, playback speed, etc. You can download these files (380 MB each) and do extreme color grades or whatever stress tests you wish, and compare the quality difference yourself.

https://drive.google.com/open?id=1Z5iuNkVUCM9BgygGkYRXizzr6XnSXDK7

https://drive.google.com/open?id=1JHkmDKZU4qdS7qO_C0NwYH2xkey0uz7C

 

I'd love to hear thoughts or if anyone would be interested in further tests. Keep in mind that there are different settings for H.265, so this test doesn't really have implications for camera quality. since we don't know what their encoder settings are. However, it could have implications for intermediate files or deliverables, especially to sites with file size limits.

[thebrothersthre3]

did you come to any conclusions yourself?

[OliKMIA]

So basically you did RAW > ProRes 444 > h265 / ProRes SQ > ProRes

This is 3 steps of conversions. Not sure it will change much but why didn't you convert the RAW directly to h265 and ProRes SQ without the 444 step?

Also, in terms of size, a 3 sec clip is too little to really asses the h265 size. The sample should be longer with a variety of motion. But thanks for the sample, it's interesting.

[KnightsFan]

@thebrothersthre3 I find them to be equal for all practical purposes, even with extreme grading (by extreme I mean waaay out of the realm of usefulness). There are differences, but neither seems more accurate. I haven't tried green screening, and that would be an interesting test, but I don't have a RAW camera so I'm limited in the scenarios I can test. And naturally it defeats the purpose of the test if I start with anything less than a RAW file.

Interestingly, despite having a hard time making any substantive distinction between the two, when comparing the PSNR (peak signal to noise ratio), ProRes is higher than H.265 at this compression level. H.265 needs to get up to around 50% of the bitrate before the average PSNR is the same--and even then, the PSNR on I frames is higher than ProRes, but the PSNR on P and B frames is lower. That's to be expected. I could also push the preset to be even slower, or use two pass encoding for better results on H.265 as well.

5 minutes ago, OliKMIA said:

So basically you did RAW > ProRes 444 > h265 / ProRes SQ > ProRes

This is 3 steps of conversions. Not sure it will change much but why didn't you convert the RAW directly to h265 and ProRes SQ without the 444 step?

Also, in terms of size, a 3 sec clip is too little to really asses the h265 size. The sample should be longer with a variety of motion. But thanks for the sample, it's interesting.

No, I did RAW -> Uncompressed RGB 444

And then Uncompressed -> H265 -> Uncompressed

And also Uncompressed -> ProRes -> Uncompressed

I did it this way so that I would not be limited by Resolve's H.265 encoder, and so that I could do PSNR tests from Uncompressed, without having to futz with debayering messing up the PSNR comparison. I did a short clip to keep the file size manageable at only 380 MB each. I originally did 4k, but I figured no one wanted to download that! If there is interest I am happy to do more extensive examples.

[Deadcode]

2 hours ago, KnightsFan said:

I did a quick comparison between ProRes and H.265 encoding, and thought I'd share the results with everyone. I grabbed some of Blackmagic's 4.6k RAW samples and picked a 3 second clip. In Resolve, I applied a LUT and then exported as an uncompressed 1080p 10 bit RGB 444 file. This is my reference video. From this reference file, I encoded two clips. One clip is H.265, and the other is ProRes SQ, which I compared to each other and the reference video.

The reference video (which I did not upload) was 570 MB.

One of these files was created from the reference file using "ffmpeg -i Reference.mov -c:v libx265 -crf 20 -preset slower -pix_fmt yuv422p10le H265.mov". The file size is 7.81 MB (1.4% of the reference)

The other file was created using "ffmpeg -i Reference.mov -c:v prores_ks -profile:v 2 ProRes.mov". The file size was 44 MB (7.7% of the reference, or ~5x the size of H.265)

In order to keep the comparison blind, I then converted both the ProRes and H.265 files to uncompressed 10 bit 422. So you shouldn't be able to tell which is which from the metadata, file size, playback speed, etc. You can download these files (380 MB each) and do extreme color grades or whatever stress tests you wish, and compare the quality difference yourself.

https://drive.google.com/open?id=1Z5iuNkVUCM9BgygGkYRXizzr6XnSXDK7

https://drive.google.com/open?id=1JHkmDKZU4qdS7qO_C0NwYH2xkey0uz7C

 

I'd love to hear thoughts or if anyone would be interested in further tests. Keep in mind that there are different settings for H.265, so this test doesn't really have implications for camera quality. since we don't know what their encoder settings are. However, it could have implications for intermediate files or deliverables, especially to sites with file size limits.

With ridiculous amount of crop there is a difference in texture detail. "B" is ProRes (left) and "A" is h265. (right)

Thanks for the test!

 

[Emanuel]

Exactly my findings : -)

[zerocool22]

Whenever prores is not available, DNxHD is quite good also. h264 for corporate/event stuff though

[KnightsFan]

Spoilers for anyone who wants to do a blind test!

 

 

It turns out that A is ProRes and B is H.265. As @Deadcode points out, B (H.265) has noticeably more detail. However, the ProRes file retains a tiny bit more of the chroma noise from extreme shadows the original video. As you can see in the 300% crop below, all that sparkly noise is simply gone H.265, and while the ProRes has some obvious blockiness, you can still faintly see the noise. This is of course a VERY extreme grade (see the curve from Resolve).

With extreme grades the other direction, the ProRes looks better up until the VERY extreme, when its blockiness shows. As I said before, encoders have many options so this doesn't mean much for cameras. However, if you are uploading to a site with file size limits, H.265 is a good option. Also, as @OliKMIA pointed out, this is not a comprehensive test: it's 3 seconds of HD, and only looks at one scenario. This shot is quite stable, so the interframe flavor of H.265 that I used has an easier time. Shakycam will reduce the quality of H.265.

Furthermore, the encode time on ProRes was significantly less: 11s vs 68s. I could have used a faster preset for H.265 which would speed up the encoding at the expense of quality (my guess is that cameras, tasked with real-time encoding, are not as good as the preset I used). On the other hand, I have a 2013 CPU. Perhaps newer CPUs with hardware encoding could narrow the gap? I'm not sure.

 

If you want to look at the original files (before I turned them into 380 MB monsters!) One final note: I accidentally encoded both with AAC audio, so ~6kb of the file size of each is an empty audio track.

ProRes: https://drive.google.com/open?id=102Ivc9Xa1Z7mPzCK8TgTqwh2GExMPOkZ

H.265: https://drive.google.com/open?id=1NxMofYvrHcP6DHx5VECRwMNw8qA4KXOD

[Deadcode]

1 hour ago, KnightsFan said:

It turns out that A is ProRes and B is H.265. As @Deadcode points out, B (H.265) has noticeably more detail. However, the ProRes file retains a tiny bit more of the chroma noise from extreme shadows the original video. As you can see in the 300% crop below, all that sparkly noise is simply gone H.265, and while the ProRes has some obvious blockiness, you can still faintly see the noise. This is of course a VERY extreme grade (see the curve from Resolve).

If you want to look at the original files (before I turned them into 380 MB monsters!) One final note: I accidentally encoded both with AAC audio, so ~6kb of the file size of each is an empty audio track.

ProRes: https://drive.google.com/open?id=102Ivc9Xa1Z7mPzCK8TgTqwh2GExMPOkZ

H.265: https://drive.google.com/open?id=1NxMofYvrHcP6DHx5VECRwMNw8qA4KXOD

These results are shocking for me ? h265 > ProRes ?! Insane. I think i will redo this test with my own RAW footage.

[KnightsFan]

40 minutes ago, Deadcode said:

These results are shocking for me ? h265 > ProRes ?! Insane. I think i will redo this test with my own RAW footage.

ProRes is built for editing speed, whereas H.265 was designed to maximize quality on low bitrate streaming, especially 4k and 8k. I knew that at some lower bitrate, H.265 would retain more data than ProRes, but I was surprised that it does so well with just 1/5 the data. Of course, it took 6x longer to encode!

 

I would love to see your results, if you publish. Keep in mind that not all encoders are equal, and H.265 has a LOT of options. ProRes is easier in that regard, as you just pick from Proxy, LT, Normal, HQ, and XQ with predictable size and quality.

 

Edit: the difference in detail is negligible between ProRes HQ (180 Mb/s) and the H.265 file. Also I didn't see any real difference between ProRes and DNxHD at equivalent bitrates.

[Deadcode]

39 minutes ago, KnightsFan said:

ProRes is built for editing speed, whereas H.265 was designed to maximize quality on low bitrate streaming, especially 4k and 8k. I knew that at some lower bitrate, H.265 would retain more data than ProRes, but I was surprised that it does so well with just 1/5 the data. Of course, it took 6x longer to encode!

 

I would love to see your results, if you publish. Keep in mind that not all encoders are equal, and H.265 has a LOT of options. ProRes is easier in that regard, as you just pick from Proxy, LT, Normal, HQ, and XQ with predictable size and quality.

 

Edit: the difference in detail is negligible between ProRes HQ (180 Mb/s) and the H.265 file. Also I didn't see any real difference between ProRes and DNxHD at equivalent bitrates.

Zoom in and hit play. There is the difference.

I created a sample myself from my old 5D2 RAW footage. Real codec killer due to the heavy moire and aliasing all over the image. If you zoom in and hit play you can see the ProRes holds up much better. But actually the h265 smoothens out some of the smearing and moire, so it's actually better looking ? h265 100Mbps vs ProRes 422HQ

[KnightsFan]

What encoders did you use? Your H.265 is 16 Mbps (Mine came out to 22), not 100. It's also 420 instead of 422, so that could be part of the chroma difference. But yes, with your settings the ProRes HQ certainly is much better!

[Deadcode]

19 minutes ago, KnightsFan said:

What encoders did you use? Your H.265 is 16 Mbps (Mine came out to 22), not 100. It's also 420 instead of 422, so that could be part of the chroma difference. But yes, with your settings the ProRes HQ certainly is much better!

Thanks for checking, this was Resolve's default export for h265. Now it's corrected to 100Mbps

[KnightsFan]

22 minutes ago, Deadcode said:

Thanks for checking, this was Resolve's default export for h265. Now it's corrected to 100Mbps

Cool, less difference now, but ProRes still looks like it retains more detail. I've never liked the results from Resolve's H.265 encoder, though, and I think ffmpeg's generally produces better results. But clearly H.265 struggles a lot more with this busy scene than with the one I used.

[KnightsFan]

You know, I just used Resolve's H265 encoder and I can't see much difference compared to ffmpeg on the car scene I've been using. But I am 100% sure I got some really bad results on a Resolve project I exported not too long ago. Maybe that car scene is just really easy for H265--though to be fair, I downloaded it for these tests before I had seen it. Someday I'll have to borrow an Ursa and shoot my own tests. I'm also interested in finding out whether coming from a 444 source improves a 422 file vs. coming from a 422 source.

[Mokara]

Why not just shoot with a H.265 capable camera such as an NX1 and then record the same scene with a Ninja recorder? You would be able to compare the two directly then without any intermediate steps.

[KnightsFan]

47 minutes ago, Mokara said:

Why not just shoot with a H.265 capable camera such as an NX1 and then record the same scene with a Ninja recorder? You would be able to compare the two directly then without any intermediate steps.

I wanted to test an H.265 encoder where I can adjust parameters and see how they effect size and quality. I didn't publish all my tests, but I've actually tried dozens of combinations of settings, both from ffmpeg and Resolve, and also tried different ProRes and DNxHD encoders. The NX1 specifically has a hardware encoder from 2014 designed to run in real time, whereas I was more interested in how good H.265 could look if you give it, say, 68 second to encode a 3 second clip. Hardware encoders are often not as high quality because they have a more limited number of algorithms available, and can't be updated as the specification is changed. Since 2014, we have had several updates to the HEVC standard. The test you propose only compares the NX1's encoder vs. the Ninja's, not the possibilities of H.265 as a distribution codec (which was my interest).

Furthermore, I do not own a Ninja. In fact, one of the main reasons I didn't invest in one is because, through my tests, I have not seen ProRes as a huge benefit.

[MeanRevert]

Hi again guys,

Thought I'd post here instead of the X-T3 thread ..

If I wanted to use ffmpeg to transcode H265 to ProRes, is the command line example given in the first post the correct one I should use?  I've never used ffmpeg before.

Thanks!

[KnightsFan]

@MeanRevert Yes, you can use the command line I gave in the original post to convert from H.265 to ProRes 422.

ffmpeg -i Reference.mov -c:v prores_ks -profile:v 2 ProRes.mov

"Reference.mov" is the original file, and "ProRes.mov" is the new file you will be creating. Change the 2 after profile:v to a 3 if you want ProRes 422 HQ (or 1 for LT, or 0 for Proxy).

If you are transcoding an entire project, you will want some way to batch the files. If you have an HEVC hardware decoder (many modern GPUs and CPUs have hardware decoders now), you can use cuvid to speed up the decoding process.

Keep in mind that ProRes 422 will probably lose some quality, though really unnoticeable in ordinary viewing. To me, 422 HQ was visually lossless compared to H.265, even under scrutiny.

[MeanRevert]

23 minutes ago, KnightsFan said:

@MeanRevert Yes, you can use the command line I gave in the original post to convert from H.265 to ProRes 422.

ffmpeg -i Reference.mov -c:v prores_ks -profile:v 2 ProRes.mov

"Reference.mov" is the original file, and "ProRes.mov" is the new file you will be creating. Change the 2 after profile:v to a 3 if you want ProRes 422 HQ (or 1 for LT, or 0 for Proxy).

If you are transcoding an entire project, you will want some way to batch the files. If you have an HEVC hardware decoder (many modern GPUs and CPUs have hardware decoders now), you can use cuvid to speed up the decoding process.

Keep in mind that ProRes 422 will probably lose some quality, though really unnoticeable in ordinary viewing. To me, 422 HQ was visually lossless compared to H.265, even under scrutiny.

Hi KnightsFan,

Appreciate the prompt response.  I was able to transcode one file but batching them would be preferable.

I found a FFMPEG GUI here:

https://sourceforge.net/projects/ffmpeg-batch/

When I load it up, there are some presets and one of them is for ProRes MKV with these command line parameters:

-c:v prores_ks -profile:v standard -vendor:v ap10 -pix_fmt yuv422p10le -c:a pcm_s16le -chunk_size 64K

 

Is MKV doable for ProRes editing and rendering?