115 mbp/s??? CANON DSLR

[FilmMan]

For those interested:

1%, one of the Magic Lantern Programmers, has posted another video. Progress is being made on the bitrate investigation!!!

The video he posted is 159 MB and it is 11 seconds in length.

Check my calculations and correct me if I'm wrong.

159MB x 8 = 1272 mb (convert Megabyte to Megabit)

1272 divided by 11 seconds = [size=5][b]115.6 mbps[/b][/size]

[size=5][b]Here's the download to the video (note it is a test and they are still investigating)[/b][/size]
[size=5][b][u]Careful on clicking on the correct download[/u] or else you'll get spam.[/b][/size]
[i][size=5]Samples:
Here is a Gop4 video (~150MB)
[url="http://www.sendspace.com/file/fipvng"]http://www.sendspace.com/file/fipvng[/url][/size][/i]

[Andrew Reid]

Great work as always by the time.

Bare in mind that bitrate isn't the main problem with Canon DSLRs though. So it is unlikely to have any impact on image quality compared to the default 40Mbit or so.

When Magic Lantern comes to the 5D Mark III however, then things will get interesting...

:)

[FilmMan]

I was initially thinking the same as you as the bitrate may not matter. But this go around these guys are making visual progress! I downloaded the above footage and played with it. Surprisingly, the image looks to my old eyes to be pretty darn solid. On the ML site they had posted some images of a leaf at 2 different bitrates and there is a big difference in detail (for example, the lower bitrate leaf looks to be blurry in the middle whereas the higher bitrate has very noticeable fine details showing). It'll be interesting to see what they can pull off. Cheers.

[FilmMan]

I think there is an improvement after watching this video! It was just posted. This is getting really fun now.

roughly 116.89 mbp/s


As per 1% posted on the ML site:

[i]I have a similar for gop3, just have to upload it.
Here: [url="http://www.sendspace.com/file/mgj8ig"]http://www.sendspace.com/file/mgj8ig[/url][/i]

[matt2491]

Wow - there IS a noticeable difference!

Can't wait to see what ML brings to the 5D3.

[FilmMan]

Matt,
The programmer 1% posted some videos. Night time and raining - Lamp shining on a brick wall. 4 videos (gop1/3/6/12) I looked at 2 of the 4 videos (gop1 and gop3). The gop3 video setting looks to me like the famed gh2 for clarity! They are using this program for analysis purposes too - [url="http://www.jongbel.com/?page_id=573"]http://www.jongbel.com/?page_id=573[/url] This is starting to get exciting because 100 mbps to 140 mbps are being discussed. Seems to be stable? Hopefully the links work.

Gop1
[url="http://upload.g3gg0.de/pub_files/d4c92e3c7850a189c1c192a86ace2f77/Lamp_Gop_1.MOV"]http://upload.g3gg0.de/pub_files/d4c92e3c7850a189c1c192a86ace2f77/Lamp_Gop_1.MOV[/url]

Gop3
[url="http://upload.g3gg0.de/pub_files/8e06487dd60083d1e42bd73197ec441c/Lamp_Gop_3.MOV"]http://upload.g3gg0.de/pub_files/8e06487dd60083d1e42bd73197ec441c/Lamp_Gop_3.MOV[/url]

Gop6

[url="http://upload.g3gg0.de/pub_files/1507715cede6b8b58d476acb7c2bc3ec/Lamp_Gop_6.MOV"]http://upload.g3gg0.de/pub_files/1507715cede6b8b58d476acb7c2bc3ec/Lamp_Gop_6.MOV[/url]

Gop12
[url="http://upload.g3gg0.de/pub_files/6186c340ceffd9fd7b4b6b940c14a0f5/Lamp_Gop_12.MOV"]http://upload.g3gg0.de/pub_files/6186c340ceffd9fd7b4b6b940c14a0f5/Lamp_Gop_12.MOV[/url]

[jgharding]

After using CBR 2.8 in ML2.3 on 550D recently, it makes a huge difference to high ISO footage, so if we can get lower GOP lengths and higher bitrates still I'll be happy. The noise is much easier to separate form the image at these rates as it isn't quite so baked in, so you end up with even better low light capability. My CBR 2.8 shots at ISO3200 were running around 98mbps using ML2.3. It looks great when you noise reduce! There's no contest compared to standard bitrate, you keep a lot more detail as the noise isn't blended into your picture so much.

The main issue is still the sensor binning and scaling inside the camera, but these ML guys are amazing. I've been using this camera two years or so and every time I think of dumping it they make a breakthrough of some kind. I never went to GH2 in the end despite the bitrate, because of all the little niggles like the band at high ISO, some of the ligfht halos, the sensor crop, lack of cinema profiles (not ultra flat which can look awful, but just reasonably flat) and some stability issues some people were having with the hack, so I've just ended up sticking with 550D, but I have to say that because of ML I'm still happy! Those guys are great, I'm happy to donate!

We should definitely support our hackers, they give us so much.

[jgharding]

Also, the highlight roll-off is smoother than before with ML 2.3. It's subtle but noticeable when you're pushing it. I'm always preaching about ML2.3, it's absolutely essential. It turns the 550D into a monstrous tool, I'm always making 7D owners cry.

[FilmMan]

The Magic Lantern Team are impressive. The programmer 1% had some footage cranked to a whopping [size=5][b][u]200 mbp/s! [/u][/b][/size]
[size=5]There are definitely onto something with the bitrates. Whether they get it workable for the masses remains to be seen. I'm hoping. Cheers.[/size]

[jgharding]

Would you be able to post a link to the thread where these GOP settings and bitrates are being discussed on Magic Lantern please? I get a bit lost on their forum...

[FilmMan]

http://www.magiclantern.fm/forum/index.php?topic=1565.0

Things looked promising however, issues arose. They can pump the bitrate, but the quality they want to achieve isn't there. Alot of different ideas are being tossed around. Still hopeful.

[jgharding]

Awesome cheers! :)

[FilmMan]

Interesting...1% thoughts from the Magic Lantern Site...422 colour would be nice.

[i]Size of Simple 5x5 .422 is ~3mb. If we get it down to 1.0 mb or less we could record it fairly well. Maybe could be put through jpeg path but I don't know if any of them keep the 4:2:2. At 3mb would need like 480 megabits. My card does about 19MB write.

Even written as 1 frame of jpeg, could be made as image sequence in AE and then wav sound added. If there really is intermediate jpeg step already it would be even easier, just stop it from being passed to the H264E/Movie writer. I bet overhead goes down too.

I still think some buffer overruns are because compressor stop compressing. Buffer start filling from 1%-5% to suddenly 40, 60, 75, stop. Like new frames stop being written out. How can it do that at <90mbps when it records steadily at 120-130+ for minutes at a time with quality constant.[/i]

[FilmMan]

Interesting stuff.

The Magic Lantern guys seem to be understanding things more. So will tweaking a few items in the encoder bring about better things? I think we know the answer! Understanding how to change the setting is the difficult part. Hey, talk of 422 is still going on. Will changing a few parameters do the trick?

From developer 1%
[i]Also, quite hilarious:[/i]

[i][url="http://www.learn.usa.canon.com/resources/articles/2012/ipp_ipb_all_i_compare.shtml"]http://www.learn.usa.canon.com/resources/articles/2012/ipp_ipb_all_i_compare.shtml[/url][/i]

[i]So 600D is 5dMK2.5? I guess this means all I is the best for quality (horrible for file size), especially with a fixed qp slice. Also explains why buffer is written immediately, there is no additional pass or prediction being done.[/i]

[jgharding]

Awesome. I reckon intra-frame codecs are the best bet for film like quality, that or very short GOP.. Big files are fine, and buffer bypass is a benefit as far as I can see. Then the only cap is the speed of the card writer in the camera. Let's hope it's fast enough. There's still more life in the older EOS cameras yet ;)

4:2:2 would be so nice. The deeper they delve the more they unearth it seems... Great stuff!

[FilmMan]

For those interested. Here's the latest conversation:
[color=#ff0000]Leon[/color]

[color=#ff0000]@1% : I think the "5x5" part only applies to Hi-Res. If you select "Simple" it seems to completely ignore whether it is 5x5, 2x1, etc. I just had a wee play with the options, and an 18MP Hi-Res 5x5 comes out at 34MB, so it does seem to be uncompressed with 4:2:2 subsampling – I got something muddled up before and thought the full-res pics were much smaller.

Can burst not be done at higher quality and faster? With a UHS-1 Class 10 card, I only get 1.6 fps at only 1056x720 resolution. If the H.264 encoder is getting 1080p (or are they 972p at this point?) .422, can these not be written directly to the card? [b][i][u]What's limiting?[/u][/i][/b]

So if the .422 files aren't compressed, and they are the "input" to the H.264 encoder, what is this jpcore slice qp setting that controls "input" quality, and why does it not seem to exist for x264?[/color]

[color=#0000ff]1%[/color]
[indent][u][i][b][color=#0000ff]What's limiting?[/color][/b][/i][/u][/indent]
[color=#0000ff]The zooming.

I'm not sure if h.264 is fed from the LV buffer which is only 720x480? or from full size yuv buffer. You'd have ML graphics on the video, would you not? The simple pics come out worse quality than the encoded frames so something must be missing. If I'm not mistaken, the high-res ones clear the overlays too.

I think there are several steps to compression as there are multiple passes and mention of jpeg encoding being finished. So it could go YUV -> jpeg path -> h264E. You're not going to alter the quality of the raw YUV, just the compressor input.

The YUV has to be compressed with SOMETHING before we put it to the card. The simple frames are 3 MB. Card writes at ~20MB... raw would give you 6FPS only. The buffer is 1GB big and if not much is being used in ALL-I, the frames must be much smaller. I don't think the camera could take raw YUV, compress it with H264 and run several passes (for P frames) then write it to the card at over 30fps without some intermediate step.

x264 has slice QP parameter but it will not go below -10 on videos from the camera. But bit rate does rise if lower jpcore slice like 90/95 is used vs 100/110 and recording stops sooner filming the same thing. Q-scale -16 is only 112[/color]

[FilmMan]

And Leon's response:

[color=#ff0000]In burst mode, there isn't any zooming though, is there? It's only managing to write 2MB per second in silent frames to my camera when I use burst mode.

Seems so strange that it would waste processing encoding to jpeg and then decoding it to send it to the H264 encoder, unless it is somehow forced by the way their hardware is wired up and communicates. If the jpcore part is already a bit compressed, [i][u][b]maybe that can be written directly to the card bypassing x264. We only need a 7:1 compression overall (or 4:1 compared to 4:2:2) to get 1080p down to being 20MB/s.[/b][/u][/i][/color]

[kabuto1138]

This is really exciting. But they haven't written anything in a few days.

If they can pull this off, it will change the game.

[FilmMan]

[b]Some Positive news![/b]

Programmer 1% comments:


Found new thing about CBR mode. If you're using ALL-I CBR will not adjust qscale/slice quality. This means that next qscale value is calculated from P frames not I frames. Can anyone confirm? Set slice 0, reboot, turn on debugging and see if slice quality changes in all I. Even in gop2 qscale changes, gop 1 is a default of 120, I think thats -8.


*Edit:

More good news... slice QP can be adjusted by as much as you want. For now I set it to drop by 4 when buffer gets to warning level and no errors + no buffer overruns but quality drops down quickly so I need to find a good mathematical solution either by buffer or bitrate.

Something quick:

[url="https://bitbucket.org/OtherOnePercent/tragic-lantern/downloads/autoexec.bin.quickcbr"]https://bitbucket.org/OtherOnePercent/tragic-lantern/downloads/autoexec.bin.quickcbr[/url]

If buffer warning drop slice q by 2, if BR < 75mbps raise by 1. Its working at ISO5k in crop mode but its too dark right now for me to test + fine tune. Mainly doing this for ALL-I ... I'll try in longer gops next.

[FilmMan]

For those wanting to read about X264. Interesting read.

[url="http://mewiki.project357.com/wiki/X264_Settings"]http://mewiki.project357.com/wiki/X264_Settings[/url]