Is anyone else interested in a screen-less (small) external recorder?

[androidlad]

2 hours ago, KnightsFan said:

GoPro makes a camera the records 4k60 for hours on end while enclosed into a waterproof case with no external heatsinks, at under half the weight of the Ninja V. I'm sure it's quite possible to build a screenless recorder smaller than the Ninja V.

😂

Does GoPro record ProRes HQ or RAW?

H.264 and HEVC codecs can be heavily hardware accelerated, using off the shelf chips (GoPro branded them GP1).

Chips to encode ProRes have to be built from scratch (essentially a CPU), hence the size and thermal requirement.

[Video Hummus]

On 9/5/2021 at 5:00 AM, TrueIndigo said:

and was surprised there weren't many products to choose from and somewhat mystified that the Ninja Star had not been updated.

If the product doesn't exist from any manufacturer there is probably some kind of liability or patent around such a concept. Otherwise there would be something from Atomos and 10 Chinese knock-off products for cheaper.

 

[Video Hummus]

7 hours ago, androidlad said:

Chips to encode ProRes have to be built from scratch (essentially a CPU), hence the size and thermal requirement.

P4K has this and it uses a FGPA. I think Apple has a ASIC ProRes chips but probably isn't sharing.

[KnightsFan]

9 hours ago, kye said:

No, I absolutely do not, under any circumstances, think this is even remotely true.  Once you see it, you can't unsee it.  I'd suggest if you're not seeing it then I wouldn't go looking...

I am the owner of a GH5, which does 200Mbps 1080p, which is one of the very few cameras that give a bitrate MORE than the ~176Mbps bitrate of Prores HQ, and yet, I am still interested in something that can record Prores externally that isn't a large screen.

So you're saying that if I transcode a ProRes file to H.264, you will 100% be able to tell which is which in a blind test?

[kye]

8 hours ago, KnightsFan said:

So you're saying that if I transcode a ProRes file to H.264, you will 100% be able to tell which is which in a blind test?

I might or might not be able to.  

The challenge with such a test would be that it's not an apples-to-apples comparison.  You'd never get something that was RAW->Prores->h264, so the Prores step may obscure the comparison.  If you were to encode the h264 using a computer I'd suggest that the h264 would likely be very high quality (although Resolves h264/5 is notably poor compared to ffmpeg for example).

However.....  none of this matters.  I care about the real world.

In the real world, I'd like to be able to choose my camera independently of the codec and processing pipeline.  There are cameras like the A7S2 which had a lot going for them but topped out at 100Mbps 4K, which is (quite frankly) pathetic.  Then there are cameras with internal Prores like the P4K that lack IBIS and are huge.

If I was taking the HDMI out then I would also bypass all the horrible sharpening and NR and awfulness that cameras typically apply to their h264 captures, including even the GH5 and it's high-bitrate ALL-I modes.

If there was a Atomos Ninja Star 2, which was relatively compact, I could add it to any camera that had a clean HDMI output and get the codec/bitrate I want, regardless of the paltry codecs they decided to cripple-hammer the camera with.

Unless you know of some way for me to automatically beam the output from my cameras sensor to your computer to be compressed in h264 in realtime and then written back to the SD card in the camera?  That would be pretty spectacular and I'd be keen to see the h264 files where we get to decide the entire imaging pipeline....

[KnightsFan]

1 hour ago, kye said:

In the real world, I'd like to be able to choose my camera independently of the codec and processing pipeline.

But what you're saying in this topic is that you'd like a 3rd party to make an HDMI recorder, putting you in their hands for recording format instead of the camera's. Unless you're envisioning the external recorder space getting large enough that there is legitimate competition with a variety of different options. Why not wish for camera companies to allow for more flexibility in choosing codec, and higher quality instead of adding a whole extra layer of compatibility, accessories, and specs.

Remember when Panasonic added 10 bit to the G9 years later via firmware? Remember when hackers added compressed raw recording to the 5D3? Or when Blackmagic made a pocket camera that shot ProRes and Raw at a fraction of the price of contemporary DSLR's, without fans and heatsinks? My sarcasm in this topic is because, in my opinion, wishing for small external recorders is a roundabout solution that would also encourage the implementation of bad internal video. I'd rather pay $100 for firmware that unlocks pro video formats, than $600 + batteries for a recorder even if it's the size of a battery grip.

2 hours ago, kye said:

Unless you know of some way for me to automatically beam the output from my cameras sensor to your computer to be compressed in h264 in realtime and then written back to the SD card in the camera?  That would be pretty spectacular and I'd be keen to see the h264 files where we get to decide the entire imaging pipeline....

This is the exact reason I hope for the Octopus cinema camera to turn into a real product, or why I'd like to see cameras running vanilla android or Linux OS's. Getting on my soap box here, but the migration away from standardized computers (both hardware and software) to dozens of different devices running proprietary firmwares is a harmful trend.

[kye]

34 minutes ago, KnightsFan said:

But what you're saying in this topic is that you'd like a 3rd party to make an HDMI recorder, putting you in their hands for recording format instead of the camera's. Unless you're envisioning the external recorder space getting large enough that there is legitimate competition with a variety of different options. Why not wish for camera companies to allow for more flexibility in choosing codec, and higher quality instead of adding a whole extra layer of compatibility, accessories, and specs.

Remember when Panasonic added 10 bit to the G9 years later via firmware? Remember when hackers added compressed raw recording to the 5D3? Or when Blackmagic made a pocket camera that shot ProRes and Raw at a fraction of the price of contemporary DSLR's, without fans and heatsinks? My sarcasm in this topic is because, in my opinion, wishing for small external recorders is a roundabout solution that would also encourage the implementation of bad internal video. I'd rather pay $100 for firmware that unlocks pro video formats, than $600 + batteries for a recorder even if it's the size of a battery grip.

This is the exact reason I hope for the Octopus cinema camera to turn into a real product, or why I'd like to see cameras running vanilla android or Linux OS's. Getting on my soap box here, but the migration away from standardized computers (both hardware and software) to dozens of different devices running proprietary firmwares is a harmful trend.

I agree with you - I'd love to see all the camera manufacturers implement better internal codecs.  and by better, I mean:

1. Scalable compressed internal RAW (best)
2. Prores internal
3. Scalable uncompressed internal RAW
4. Unscalable uncompressed internal RAW
5. High-bitrate low-processed high-quality h264/5
6. High-bitrate low-processed typical-quality h264/5
7. Whatever h264/5 implementation the marketing department thought was good

What I mean by "scalable" RAW is the ability to read the whole sensor, downscale in-camera, and write that in a RAW format to the card.  I believe the P6K can do this?  It's how you can get different resolution RAW files with the same crop factor.

Unfortunately, most cameras are "7" on the above scale, with only a few exceptions here and there, and those are often very compromised in other ways.  Effectively, an external Prores recorder upgrades all the "7" cameras into "2" cameras.  

Cameras are always a compromise, some have this function but not that function, others are the other way around, etc.  I'm looking to eliminate one of those variables.

[TomTheDP]

On 9/7/2021 at 12:05 AM, kye said:

I agree with you - I'd love to see all the camera manufacturers implement better internal codecs.  and by better, I mean:

1. Scalable compressed internal RAW (best)
2. Prores internal
3. Scalable uncompressed internal RAW
4. Unscalable uncompressed internal RAW
5. High-bitrate low-processed high-quality h264/5
6. High-bitrate low-processed typical-quality h264/5
7. Whatever h264/5 implementation the marketing department thought was good

What I mean by "scalable" RAW is the ability to read the whole sensor, downscale in-camera, and write that in a RAW format to the card.  I believe the P6K can do this?  It's how you can get different resolution RAW files with the same crop factor.

Unfortunately, most cameras are "7" on the above scale, with only a few exceptions here and there, and those are often very compromised in other ways.  Effectively, an external Prores recorder upgrades all the "7" cameras into "2" cameras.  

Cameras are always a compromise, some have this function but not that function, others are the other way around, etc.  I'm looking to eliminate one of those variables.

The Pocket 6k cannot downscale the RAW data to a lower resolution. The only way to do that with RAW usually is line skipping which the 5DMK3 does with magic lantern or the Z6 shooting prores RAW 4K. 

The URSA 12K does have RAW with the full sensor not line skipped at different resolutions, its using a non traditional method to do it though and has a very unique sensor. 

[kye]

1 hour ago, TomTheDP said:

The Pocket 6k cannot downscale the RAW data to a lower resolution. The only way to do that with RAW usually is line skipping which the 5DMK3 does with magic lantern or the Z6 shooting prores RAW 4K. 

The URSA 12K does have RAW with the full sensor not line skipped at different resolutions, its using a non traditional method to do it though and has a very unique sensor. 

I would encourage you to think about this as independent of the sensor.

Every camera does the following:

1. read RAW data off the sensor
2. de-bayer that data
3. rescale the image (optional)
4. process that data with colour profiles and other processing (eg, sharpening)
5. encode into a file format (compressed or not) and write that to the card

Any camera can be built to rescale the image and write it to the card using a low/zero compression file type.  

Most cameras simply take the sensor readout and pipe it to the card or to a hardware chip for resizing and encoding to h264, but rescaling the image and compressing/encoding can be independently controlled.

You have to also realise that cameras are already doing this internally, by taking the sensor data, rescaling it, adding an OSD overlay, and piping that to the LCD/viewfinder/HDMI/SDI port.  The data stream used for monitoring is often rescaled and would not be compressed.  Ironically, that's the stream I'd like to grab using a small Prores recorder.

[KnightsFan]

I think the question is whether its still called "raw" if it's been scaled, and as a separate question, to what degree scaling individual color arrays before debayering is better or worse.

[Video Hummus]

External recorders are like the donut spare tire of the camera world. Like @KnightsFan said, I would much rather pay several hundred dollars for some firmware upgrade for internal features than bolt on external products through flimsily breakable ports, etc…

Perhaps they could pay the patent royalty man and then pass the charge on to me plus some markup for profit to have internal RAW in some future mirrorless cinema camera. That would be great. Everybody makes money and I get what I want for a cost I could determine worth it or not for myself.

Back to the topic, most video is going to get compressed to some shitty h264 format because most video is going to be compressed or streamed in some way. Also, is some bride or corporate manager going to spot the difference between ProRes and a good internal H264 (which most 2 year old cameras have) if the material you give them is lit, shot, and composed well?

I just don’t think it’s worth the time to fret over and really the only benefit of products that shoot ProRes is the editing performance and availability of cheaper capture media. All of that will improve over time with internal H265 capture as CFExpress gets cheaper and new CPUs hardware accelerate. 

[kye]

23 hours ago, KnightsFan said:

I think the question is whether its still called "raw" if it's been scaled, and as a separate question, to what degree scaling individual color arrays before debayering is better or worse.

Maybe RAW is the wrong word, but with all the compressed-RAW variants out there, it's tough to come up with a phrase that means the right thing.

Perhaps the way to understand it is this.. there are two groups of codecs.

(all bitrates below are for DCI4K 30p)

The first group are unsharpened (for RAW or uncompressed formats) or perceptually unsharpened (Prores is sharpened to match the perceptual sharpness of RAW but not more, and requires this because the compression softens it a little bit).  The goal of these formats is to provide high-quality capture / DI formats that are perceptually as neutral as possible and where image quality was robust enough for further processing prior to distribution (eg, colour space transforms)

These have high bitrates:

• Uncompressed is 8492Mbps
• Blackmagic RAW 3:1 1088Mbps
• Apple ProRes 422 HQ 943Mbps

The second group are sharpened (which obviously emphasises edges) and are compressed with an algorithm that also emphasises edges (also perceptually sharpening the footage).  

These have much lower bitrates:

• GH5 max h264 bitrate 400Mbps
• Sony A7S3 max bitrate h264 300Mbps
• Sony A7S2 max bitrate h264 100Mbps

The goal of the h264/5/6 codec development was to provide a high-quality image at low bitrates for video distribution at the end of an imaging pipeline, where image artefacts would not be emphasised by further processing (eg, by adding large amounts of contrast via LOG to 709 conversions etc).  Manufacturers of consumer cameras choose h264/5 formats because they provide good-enough image quality when combined with in-camera colour profiles that are properly exposed, and provide file sizes that are manageable by non-professionals.  This is why the GH5, in combination with its 400Mbps modes, also include many 100Mbps 8-bit modes, whereas the lowest Prores flavour (not explicitly for proxy use) Prores LT, is 437Mbps.

Essentially I'm interested in the first group.

The first group is superior because the bitrates are significantly higher, but also because the sharpening is not ridiculously overblown.  Mathematically sharpening is the opposite of blurring, so you should be able to un-do sharpening with blurring, provided that the strength, radius, and distribution are matched.  However, the h264 footage is sharpened and then compressed, which means that the sharp edges create 'ripples' in the footage, for example the band of lighter coloured sky below is a compression artefact:

by the time you apply enough blur to the image to have that bright spot darkened by the blur of the nearby darker object, the image is blurry as all hell and not suitable for use.  Prores exhibits very little of these types of artefacts.

Interestingly, in my comparisons of codec quality, h264 is more mathematically accurate (using the mathematical model that approximates visual similarity) than Prores at a given bitrate, but I find that the perceptual difference is significant.

A parallel might be in cooking.  If I made a custard recipe perfectly, separated it into two halves, and in one I added 100g of cooked apples and in the other 25g of raw garlic, you could say that the one with 100g of apples is 4x less 'accurate' and therefore should taste 4x worse, however the fact that apples are a relatively benign flavour and garlic is much stronger flavour would mean the garlic one would likely be inedible.

This is the challenge with Prores vs h264 - the artefacts from Prores are relatively benign (and aesthetically are similar to film) but h264 the artefacts are very 'digital' and aesthetically are the opposite of filmic.  I suspect this is one reason that people making films (where image sharpness is kept to minimal levels) are interested in RAW/Prores and those making video (where sharpness = quality for most clients) are just fine with h264/5.

14 hours ago, Video Hummus said:

External recorders are like the donut spare tire of the camera world. Like @KnightsFan said, I would much rather pay several hundred dollars for some firmware upgrade for internal features than bolt on external products through flimsily breakable ports, etc…

Perhaps they could pay the patent royalty man and then pass the charge on to me plus some markup for profit to have internal RAW in some future mirrorless cinema camera. That would be great. Everybody makes money and I get what I want for a cost I could determine worth it or not for myself.

If we're going to play hypotheticals, then sure.  But why stop there?  Why not just wish for a camera that flies around automatically taking the footage you want, edits it in-camera, and just writes the final render to an SD-card when it flies back to its charger?

Back in the real world we have cameras with good features and poor codecs or good codecs and poor features, but external recorders is a way to get the best of both worlds.

14 hours ago, Video Hummus said:

Back to the topic, most video is going to get compressed to some shitty h264 format because most video is going to be compressed or streamed in some way.

You're confusing capture codecs with DI codecs with delivery codecs.

Your argument suggests that there's no point shooting with anything above 8Mbps because YT and streaming platforms distribute in that format.  

I'd suggest you watch some comparisons where people compare capture codecs and the differences are clearly visible, despite the low bitrate you're watching them in.  One of the reasons is that the capture codec gets heavily manipulated after it's captured and before distribution, and another is that a poor quality copy of a good source will always look better than a poor quality copy of a poor quality source.  That's why a photocopy of a photocopy of a photocopy of a document was almost unreadable.

14 hours ago, Video Hummus said:

Also, is some bride or corporate manager going to spot the difference between ProRes and a good internal H264 (which most 2 year old cameras have) if the material you give them is lit, shot, and composed well?

Why would I care if some mythical bride I don't know likes my home videos?  Unless you're talking about my wife?  If so, she can tell footage from Alexa/RED vs h264 cameras while watching Netflix far better than I can!

[Video Hummus]

14 hours ago, kye said:

You're confusing capture codecs with DI codecs with delivery codecs.

Your argument suggests that there's no point shooting with anything above 8Mbps because YT and streaming platforms distribute in that format.  

I'd suggest you watch some comparisons where people compare capture codecs and the differences are clearly visible, despite the low bitrate you're watching them in.  One of the reasons is that the capture codec gets heavily manipulated after it's captured and before distribution, and another is that a poor quality copy of a good source will always look better than a poor quality copy of a poor quality source.  That's why a photocopy of a photocopy of a photocopy of a document was almost unreadable.

No, that is not my argument. Of course sourcing video at 8Mbps is stupid and laughable today. I'm saying as you go up, the differences become less and less to the point of not making any meaningful IQ difference whatsoever. The lions share of benefits are on the backend.

I'm posing the question to you, if I capture the same scene in ProRes HQ and say the R5's 480Mbps HEVC, and then upload it to YouTube or Vimeo will you be able to tell which one is which at normal viewing distance on a normal laptop or monitor? Was it worth your effort and time (strictly comparing IQ; not editing performance) to have a $600 external recorder, with SSD, and HDMI cable plugged in to capture something that is 0.02% better than internal 480Mbps HEVC at the end of the pipeline? The answer for me is no, its not worth it at all for image quality.

14 hours ago, kye said:

Why would I care if some mythical bride I don't know likes my home videos?  Unless you're talking about my wife?  If so, she can tell footage from Alexa/RED vs h264 cameras while watching Netflix far better than I can!

My scenario was hypothetical for people that use their camera to make more than home videos....

[Video Hummus]

14 hours ago, kye said:

Back in the real world we have cameras with good features and poor codecs or good codecs and poor features, but external recorders is a way to get the best of both worlds.

In the real world, today, we have cameras with good features and good codecs that make external recorders less valuable if IQ is a big concern. Tomorrow, we will have better cameras with better codecs that make external recorders pretty bulky and not worth it.

|--8Mbps H264-------------------------------150Mbps h264----480Mbps HEVC--1000Mbps ProRes--|

[KnightsFan]

@kye I think you should be clearer when you are talking about properties of the codec, vs. properties of the camera. Not having a GH5, I'll take your word that the internal recording is over sharpened vs. the uncompressed output.

However, Z Cam (for example) provides sharpening options that are used with whichever codec you are on, all the way down to None which is genuinely blurry compared to any other camera I've used, including Blackmagic cDNG or ProRes. These are minor details in the firmware, from an implementation perspective, that you are suggesting we solve with huge amounts of extra hardware. I'm saying let's vocalize our desire for these options to be exposed as internal settings, rather than vocalize our desire for a separate company like Atomos to come up with their  interpretation of what good quality means.

16 hours ago, kye said:

If we're going to play hypotheticals, then sure.  But why stop there?  Why not just wish for a camera that flies around automatically taking the footage you want, edits it in-camera, and just writes the final render to an SD-card when it flies back to its charger?

Back in the real world we have cameras with good features and poor codecs or good codecs and poor features, but external recorders is a way to get the best of both worlds.

The difference is that the hypothetical we're talking about--good internal recording options--is entirely possible via simple firmware changes. The reason they aren't included is to sell us something else, whether it's a higher end Panasonic camera, or Red locking out compressed raw to sell you their brand.

[kye]

8 hours ago, Video Hummus said:

No, that is not my argument. Of course sourcing video at 8Mbps is stupid and laughable today. I'm saying as you go up, the differences become less and less to the point of not making any meaningful IQ difference whatsoever. The lions share of benefits are on the backend.

I'm posing the question to you, if I capture the same scene in ProRes HQ and say the R5's 480Mbps HEVC, and then upload it to YouTube or Vimeo will you be able to tell which one is which at normal viewing distance on a normal laptop or monitor? Was it worth your effort and time (strictly comparing IQ; not editing performance) to have a $600 external recorder, with SSD, and HDMI cable plugged in to capture something that is 0.02% better than internal 480Mbps HEVC at the end of the pipeline? The answer for me is no, its not worth it at all for image quality.

My scenario was hypothetical for people that use their camera to make more than home videos....

Meanwhile, back in the real world......    

1) I care about both IQ and editing performance

2) The Atomos Ninja Star cost roughly 10% the cost of an R5, and if you take into account the total cost of selling everything I own, buying an R5, re-buying all my lenses, batteries, accessories, well, your "but the R5 has a nice codec" argument is really "why bother with a recorder when you can pay 20x for a completely new camera setup.....  just to get a better codec"

8 hours ago, Video Hummus said:

In the real world, today, we have cameras with good features and good codecs that make external recorders less valuable if IQ is a big concern. Tomorrow, we will have better cameras with better codecs that make external recorders pretty bulky and not worth it.

|--8Mbps H264-------------------------------150Mbps h264----480Mbps HEVC--1000Mbps ProRes--|

Once again, you suggest that instead of buying an accessory that I re-buy my entire setup.

6 hours ago, KnightsFan said:

@kye I think you should be clearer when you are talking about properties of the codec, vs. properties of the camera. Not having a GH5, I'll take your word that the internal recording is over sharpened vs. the uncompressed output.

However, Z Cam (for example) provides sharpening options that are used with whichever codec you are on, all the way down to None which is genuinely blurry compared to any other camera I've used, including Blackmagic cDNG or ProRes. These are minor details in the firmware, from an implementation perspective, that you are suggesting we solve with huge amounts of extra hardware. I'm saying let's vocalize our desire for these options to be exposed as internal settings, rather than vocalize our desire for a separate company like Atomos to come up with their  interpretation of what good quality means.

The difference is that the hypothetical we're talking about--good internal recording options--is entirely possible via simple firmware changes. The reason they aren't included is to sell us something else, whether it's a higher end Panasonic camera, or Red locking out compressed raw to sell you their brand.

That is a very interesting set of features from ZCam, but it proves my point, which is that there are cameras that take image quality seriously but lack features, and cameras with great features that don't take image quality as seriously.  

The ZCam is in the former camp, considering they're modular, lack IBIS, and are generally designed for high-end use on official shoots.  Cameras that are designed to be small and convenient to use typically don't have the better codecs.

[kye]

Anyway, it seems pretty obvious that @KnightsFan and @Video Hummus don't shoot for an organic look, and so in answer to the original question of "is anyone interested in a small external recorder" basically could have answered "no".

I do find it odd that people often reply to questions by criticising the preferences of the OP, or invalidate the original scenario posed.  In this case I stated that I wanted Prores codecs in a small camera, and the responses were "you don't actually want Prores, you don't actually see what you see when you look at footage" and "you're wrong for wanting a small camera - buy a large camera without Prores instead".

Very odd.

[KnightsFan]

9 minutes ago, kye said:

Anyway, it seems pretty obvious that @KnightsFan and @Video Hummus don't shoot for an organic look, and so in answer to the original question of "is anyone interested in a small external recorder" basically could have answered "no".

Not sure how you came to that conclusion, afaik neither Video Hummus nor I mentioned anything about what look we preferred, except for the statement (not by me) that the last 0.02% of quality wasn't that important.

10 minutes ago, kye said:

do find it odd that people often reply to questions by criticising the preferences of the OP, or invalidate the original scenario posed.  In this case I stated that I wanted Prores codecs in a small camera, and the responses were "you don't actually want Prores, you don't actually see what you see when you look at footage" and "you're wrong for wanting a small camera - buy a large camera without Prores instead".

No, that's not what happened at all. You said you wanted ProRes in a small camera via a bolt on accessory, I said it makes more sense to want ProRes in that small camera via firmware.

[TomTheDP]

12 hours ago, KnightsFan said:

 No, that's not what happened at all. You said you wanted ProRes in a small camera via a bolt on accessory, I said it makes more sense to want ProRes in that small camera via firmware.

Maybe prores in camera but will settle for small external recorder if that won't happen. 

[Video Hummus]

On 9/10/2021 at 11:00 PM, kye said:

2) The Atomos Ninja Star cost roughly 10% the cost of an R5, and if you take into account the total cost of selling everything I own, buying an R5, re-buying all my lenses, batteries, accessories, well, your "but the R5 has a nice codec" argument is really "why bother with a recorder when you can pay 20x for a completely new camera setup.....  just to get a better codec"

Never did I say you needed to buy an R5 or that even R5 was superior in anyway. Not sure what you are reading.

On 9/10/2021 at 11:00 PM, kye said:

Once again, you suggest that instead of buying an accessory that I re-buy my entire setup.

Again, no I did not. I suggested today’s cameras and future ones will make external recorders less useful in the IQ department which is good because I don’t like the concept of external recorders at all and find latest internal codecs get me 95% or more there. I would welcome any smallish mirrorless camera with internal ProRes. 

On 9/10/2021 at 11:05 PM, kye said:

Anyway, it seems pretty obvious that @KnightsFan and @Video Hummus don't shoot for an organic look

Come on, like a codec is the only measure of the “organic look”. I think we both know it’s far more complicated (and subjective!) than that.