Video Data Fundamentals

[maxotics]

A WORK IN PROGRESS.  Everyone, feel free to correct, add, subtract...

 

Storage, power and bandwidth constraints necessitate the need for video compression.  It's easier to understand the trade-offs, and issues, once you understand the ideal world.

 

In the ideal world, you would work with all the data recorded by the camera. 

 

• The total pixels in a frame of 1,920 pixels wide, and 1,080 pixels high is 2,073,600, or about 2 million pixels.
• In one second, we watch 30 of those frames, so that 2 million times 30, or roughly 60 million pixels per second.
• For a minute we’d need 60 million times 60 seconds, or 3,600,000,000 pixels per minute, or 3.6 billion.
• When you’re watching your HD-TV your eye is viewing 3.6 billion pixels every minute.
• What makes up a pixel? A color. Colors are often described in their red, green and blue components. That is, every color can be separated into a red, green and blue value, often abbreviated RGB.  
• Most cameras record each color as a brightness value from 0 to 16,383 (14 bits).
• You need three sets of numbers, red (0 to 16,383), green (0 to 16,383) and blue (0 to 16,383) to numerically describe ANY color that the camera has recorded.
• Some simple math tells us that we will get a range of values between zero and 4.3 trillion.  (16,383 times 16,383 times 16,383)
• To make matters REALLY confusing, cameras only shoot one color at each pixel location (red, green or blue (or yellow, magenta or cyan), in a "bayer" pattern.  So each pixel is only accurate about 25% of the color at that location.  It assumes that two pixels near it can give the correct color information to create a full color, through "de-bayering".  This trick of borrowing color information from nearby pixels is ALSO used in video compression in a completely different way. Too complicated to get into here.

 

We can only "see" about 12 million colors.  We don't need 4.3 trillion.  

That is, we don't need 14bit * 14bit * 14 bit, we need 8bit * 8bit * 8bit (which actually gives us about 16 million)

 

Therefore, for viewing purposes, we can throw out most of the recorded data

 

Let’s go back to the optimum image we’d like to see, 3.6 billion pixels per minute times 24bits (3 bytes). That would be 10.8 gigabytes per minute. As you know, you’re not streaming 10 gigabytes of video to your TV every minute. Video compression does a marvelous job of cutting that down to a manageable size

 

HD 720p @ H.264 high profile 2500 kbps (20 MB/minute)
HD 1080p @ H.264 high profile 5000 kbps (35 MB/minute)

 

If your compressed image "overexposed" your original data, you cannot get back the correctly exposed data from the compressed video.  You would want the original data.  

 

Put another way, in compressed video you are starting out with 24-bit pixels (8/8/8).  In the original data, you have 42bit pixels (14/14/14/).  Those 42bits aren't all equal (the sensors aren't as accurate at the extreme ends of their readings), but this should give you an idea of why RAW sensor data is the ideal.

 

REFERENCE

 

BAYER SENSORS

http://www.siliconimaging.com/RGB%20Bayer.htm

 

DEBAYERING

http://pixinsight.com/doc/tools/Debayer/Debayer.html

 

PATENT BELIEVED BEHIND CANON's CURRNET VIDEO FOCUS PIXEL TECHNOLOGY

http://www.google.com/patents/US20100165176?printsec=abstract#v=onepage&q&f=false

 

SIGMA/FOVEON NON-BAYER SENSORS (not currently used in video due to technical problems)

http://en.wikipedia.org/wiki/Foveon_X3_sensor

 

CAMERAS MUST DUMP IMAGE DATA IN REAL-TIME.  Or, not all SD cards created equally

http://en.wikipedia.org/wiki/Secure_Digital

 

VIDEO COMPRESSION

 

http://tech.yanatm.com/?p=485

 

Oh this stuff makes my head swim ;)

[fuzzynormal]

Oh this stuff makes my head swim

 

That's why I don't really care all that much.  I'm the type that would just rather use the dang things to make something interesting and call it good.

 

I mean, it's curious and cool to know the tech, but hardly a priority for making something artistic.

 

Good for you if you want to delve in though.  Lord knows I'm not inclined to be an engineer.

[maxotics]

That's why I don't really care all that much.  I'm the type that would just rather use the dang things to make something interesting and call it good.

 

I mean, it's curious and cool to know the tech, but hardly a priority for making something artistic.

 

Good for you if you want to delve in though.  Lord knows I'm not inclined to be an engineer.

 

fuzzynormal, think about what you're saying.  Why do you set the aperture to be 2.8 vs f8.  Why might you not use f22 if you want everything in focus?  Why would use use 25fps in Europe, but not the U.S.?  Why would you turn sharpness down in the camera?  Why wouldn't you expect to use your pancake lens with an adapter on a camera not designed for it?  Why might you use a Blackmagic camera for stuff you plan on showing at your local theater, but would use a GH3, say, for an on-line video series?  You think more like an engineer that you realize ;)

 

Andrew has argued this before.  You can't separate the technical from the artistic.  Yes, you DO NOT have to be a technical expert to create great art.  That's why movie-making is the most collaborate of efforts.  No one can know/do it all.  You have to have multiple experts.  However, if you are doing this yourself, YOU want to know as much as possible.  For the guy-and-a-dog filmmaker, this site is an oasis.

 

Most people here are not learning the tech to be "curious and cool".  They're learning it to be better artists.  

 

This is how I got here.  I've never liked skin tones in compressed video.   I come from a film background.  I tried all kinds of things, but nothing worked.  Then I read a blog post here by Andrew on the 50D and how it was shooting RAW video (which I had no idea about).  So I bought his 50D guide, a camera, and tried it.  I just followed Andrews step-by-step instructions.   The first clip changed my life.  And I've been here every since, learning and sharing my knowledge and clips with others.  

 

I admit that I get lost in the weeds in the technology, which becomes counter-productive artistically.  We all do.  I think that's why Andrew shot his latest video in the dark with a non-RAW camera using internal stabilization.   That's a real video, a real work of art, for a real client.  Andrew has amazing cameras.  He could have shot with RED.  But he shot with the Olympus because his technical knowledge told him what would be THE BEST EQUIPMENT TO REALIZE HIS ARTISTIC VISION.  

 

You can get into the technology and still use a super-8 film camera!  One does not preclude the other.

[fuzzynormal]

Andrew has argued this before.  You can't separate the technical from the artistic.

 

No, not in DIY film making, that's for sure.  But I've mired myself in the tech for decades.  I do enjoy it and don't find it daunting.  

 

It's just that, for me, digital imaging has finally gotten to the point where it's so technically democratized that I feel I absolutely have to focus on the more important aspects of the craft.  I've too often ignored the art to play with the newest buttons and switches.  I'm now feeling that such propensity is not going to serve me well in a DIY film making career moving forward.  

 

Those switches and buttons are not as expensive and exclusive as they used to be.  Anyone can afford them.  I just saw a 5DII for sale on craigslist for $350.  Put that camera and a cheap 50mm prime lens in the hands of a talented artistic kid and there's no limit to what she/he could do.  And the gear is so good, even if it's not "the best", it will support and even improve their creativity...as you allude to.

 

Sorry. I'm hijacking the thread.  I know gear talk is the prevue of this forum and sensor stuff is the main part of it.  Have at it and I'll shut up.  

[AaronChicago]

No doubt that these technical charts are essential to top quality performance. Many find it very interesting, and worth a "hobby." When I see this stuff though it's a BUZZKILL.

[maxotics]

No doubt that these technical charts are essential to top quality performance. Many find it very interesting, and worth a "hobby." When I see this stuff though it's a BUZZKILL.

 

Hi Aaron, it's not about "top quality performance."  It's about getting the quality YOU want for your artistic expression.  So again, Andrew shot his latest video with the Olympus (which he and others slam for its video quality) because it has superior in-camera stabilization.  If he really bought into what you're saying he would have shot with a RAW based camera.  If you look at the video, you will see that he needed to get close-up to his subject, and need to focus on composition than his camera settings.  EVERY shoot is a trade-off!  

 

What is a buzzkill is listening to people argue about seemingly arcane stuff.  I wouldn't argue with you there :)  Keep in mind that the manufacturers want to sell cameras.  They do not want you to know the truth about their weaknesses.  So everyone here tries to figure out what is what.