Shooting Internal RAW Video on nearly any camera by means of frame interpolation

[FHDcrew]

For a while I’ve been contemplating an experiment. Many cameras have burst modes where they write RAW stills to the card at a certain framerate. My Nikon Z6 for example can do 12fps at full resolution in 12 bit RAW. My thinking is as follows:

Theoretically we could shoot in the RAW burst mode, and then use state of the art motion interpolation software, such as RIFE or DAIN, to fill-in the missing frames to get us to a usubale video framerate. 
 

Believe it or not, it works. I was able to shoot at the 12fps burst and interpolate the remaining frames, and the video sequence looked somewhat convincing. I was actually very impressed.

 

Our problem here is buffer size. On many cameras, the high-speed modes only last a few seconds. This is the sad reality, as then the framerate slows down. A way around this is to shoot at a lower framerate to keep things continuous. My Z6 can do 5fps full-sensor RAW to an XQD card with no apparent limit. However, with tue current state of frame interpolation, 5 FPS just seems too low for anything other than simple slider/dolly moves. I filmed myself talking in the 5fps mode, and the result after frame interpolation was very unnatural movement. Where things are currently, 5fps is just too low. You really need at least 12fps to make things look decent. 
 

The other thing that could be tried, and I have tried this, is to lower the resolution/use apsc crop mode to basically shrink the file size down. I still couldn’t hit a sustained 12fps, but was able to get probably around 7fps continuous all-day. Sadly though, still not enough data to interpolate effectively. I’m curious if upgrading to a very fast CFAST 2.0 card would give me constant 12fps in this crop mode?

 

I can think of cameras that would work well with this technique. The 1DX Mark II comes to mind, with an unlimited 14fps buffer using electronic shutter. That should bode well with frame interpolation. 
 

I’m also keenly watching frame interpolation advancements. I hope that within the next year or so, new advancements in the FOSS interpolation research unfold, so as very low framerate content (like that unlimited 5fps mode) can be interpolated with very natural motion. 
 

Curious to hear your guys thoughts. Manufactures are lame and don’t like to give us an uncompressed dump of the RAW video all of these cameras are capable of. Magic lantern proved all of these cameras could be giving us internal RAW video without stupid patents and licenses and proprietary codecs and Jeremy Youngs and bulky external recorders. 
 

Any ideas on not-crazy-expensive cameras that have a decently high burst-rate, without a buffer limit?

[BTM_Pix]

The state of the art - which still doesn't actually match the requirement of full RAW shooting at 24fps - would be the Sony A1 which in APS-C mode can shoot at 20fps in compressed RAW with an unlimited buffer.

Not a small price but neither is the requirement.

There is also the matter of the memory card usage and battery life restricting the practical usefulness of it.

The €400 160gb CFExpress card will give you five minutes of that 20fps APS-C compressed RAW storage but the bigger issue being the battery on the A1 being rated at 430 shots which will give you about 21 seconds of footage before it gives up the ghost.

It may be that interpolation improves dramatically over the next few years but it will likely still take something fairly high end (i.e. a sports camera) to make it viable because even at a reduced frame rate it will still need a big buffer so until then the closest option is paying €7.5K for a camera that shoots just above the frame rate of a Super 8 cine camera and chews memory cards and power at an alarming rate.

There is, of course, a stills camera that exists that gives you a "burst rate" of up to 30fps RAW in full frame with very modest power requirements and far more affordable media where that €400 will buy you closer to five hours of footage rather than five minutes.

For about 20% of the price too.

To demonstrate the diminishing returns of stills RAW burst vs internal RAW on the same camera, I've just taken this comparison of an uncompressed RAW in stills mode on my Sigma Fp with the 12bit cDNG RAW frame in cine mode.

For me personally, such differences that are there are definitely insufficient for me to take the ball ache of trying to make the stills mode into a cine mode.

[webrunner5]

2 hours ago, BTM_Pix said:

The state of the art - which still doesn't actually match the requirement of full RAW shooting at 24fps - would be the Sony A1 which in APS-C mode can shoot at 20fps in compressed RAW with an unlimited buffer.

Not a small price but neither is the requirement.

There is also the matter of the memory card usage and battery life restricting the practical usefulness of it.

The €400 160gb CFExpress card will give you five minutes of that 20fps APS-C compressed RAW storage but the bigger issue being the battery on the A1 being rated at 430 shots which will give you about 21 seconds of footage before it gives up the ghost.

It may be that interpolation improves dramatically over the next few years but it will likely still take something fairly high end (i.e. a sports camera) to make it viable because even at a reduced frame rate it will still need a big buffer so until then the closest option is paying €7.5K for a camera that shoots just above the frame rate of a Super 8 cine camera and chews memory cards and power at an alarming rate.

There is, of course, a stills camera that exists that gives you a "burst rate" of up to 30fps RAW in full frame with very modest power requirements and far more affordable media where that €400 will buy you closer to five hours of footage rather than five minutes.

For about 20% of the price too.

To demonstrate the diminishing returns of stills RAW burst vs internal RAW on the same camera, I've just taken this comparison of an uncompressed RAW in stills mode on my Sigma Fp with the 12bit cDNG RAW frame in cine mode.

For me personally, such differences that are there are definitely insufficient for me to take the ball ache of trying to make the stills mode into a cine mode.

One click on Auto in Lightroom gives this result. Not that big of chore doing it in batch files. Sure, the video one is better but who is going to notice it not side by side.

 

[QuickHitRecord]

I've been thinking about this since my first test with RIFE. The problem is that none of the old CCD cameras I would want to do this with can capture more than about 5fps.

[Andrew Reid]

 

The E-M1 Mark II does 60fps RAW. You get very short clips that you slow down to about 2 sec. max.

Didn't really bother repeating the experiment as you're better off using Magic Lantern or a Sigma Fp.

It has no legs really.

There are so many better options available in 2022.

[webrunner5]

That is some pretty damn good looking footage though, Shame it wasn't taken further by Olympus, but I guess they had enough on their plate we didn't know about at the time. Yikes.

[FHDcrew]

The Sigma FP is a solid option. But we don’t get good AF or IBIS on the camera. I do really wish all of these large-sensor cameras with fast CFExpress throughout would allow us to access the raw feed of the sensor and just dump it into the memory card. Similar to what magic lantern did. But yes I suppose this whole burst-mode 2 video, isnt worth pursuing in light of the Sigma FP, let alone all the feature-packed 10 bit cameras we can choose from. 

14 hours ago, BTM_Pix said:

The state of the art - which still doesn't actually match the requirement of full RAW shooting at 24fps - would be the Sony A1 which in APS-C mode can shoot at 20fps in compressed RAW with an unlimited buffer.

Not a small price but neither is the requirement.

There is also the matter of the memory card usage and battery life restricting the practical usefulness of it.

The €400 160gb CFExpress card will give you five minutes of that 20fps APS-C compressed RAW storage but the bigger issue being the battery on the A1 being rated at 430 shots which will give you about 21 seconds of footage before it gives up the ghost.

It may be that interpolation improves dramatically over the next few years but it will likely still take something fairly high end (i.e. a sports camera) to make it viable because even at a reduced frame rate it will still need a big buffer so until then the closest option is paying €7.5K for a camera that shoots just above the frame rate of a Super 8 cine camera and chews memory cards and power at an alarming rate.

There is, of course, a stills camera that exists that gives you a "burst rate" of up to 30fps RAW in full frame with very modest power requirements and far more affordable media where that €400 will buy you closer to five hours of footage rather than five minutes.

For about 20% of the price too.

To demonstrate the diminishing returns of stills RAW burst vs internal RAW on the same camera, I've just taken this comparison of an uncompressed RAW in stills mode on my Sigma Fp with the 12bit cDNG RAW frame in cine mode.

For me personally, such differences that are there are definitely insufficient for me to take the ball ache of trying to make the stills mode into a cine mode.