My crowdsourced rolling shutter sensor readout speed repository

[horshack]

FYI, I've started a GitHub project to create a repository for sensor readout speeds of all cameras, for both stills and video, using a standardized measurement method that involves a $17 USD Arduino board.

The project's homepage is:

https://github.com/horshack-dpreview/RollingShutter

The project has a link to the current database, a primer about rolling shutter artifacts, source code, and collection details for those who would like to contribute their camera's images to be measured and added to the database.

Direct link to current results:

https://horshack-dpreview.github.io/RollingShutter/

 

[KnightsFan]

Nice! I have my own database of rolling shutter values that I can get to you. The one column I would add to the table is the ratio of the rolling shutter to the frame rate. That value normalizes the skew per frame.

[horshack]

4 hours ago, KnightsFan said:

Nice! I have my own database of rolling shutter values that I can get to you. The one column I would add to the table is the ratio of the rolling shutter to the frame rate. That value normalizes the skew per frame.

Thanks! To keep the results in the table fully comparable I would like for all of the cameras to be measured using an identical methodology.

[KnightsFan]

2 hours ago, horshack said:

Thanks! To keep the results in the table fully comparable I would like for all of the cameras to be measured using an identical methodology.

In that case none of my values will go into your table. Seems like a waste, though--it's a raw speed so there's no subjectivity.

[horshack]

53 minutes ago, KnightsFan said:

In that case none of my values will go into your table. Seems like a waste, though--it's a raw speed so there's no subjectivity.

I agree, the sensor speed is what it is. However there is a moderate amount of variability in the readout speeds posted online, owing to differences in methodology. I'm looking to avoid that variability in this repository of measurements.

[Al Dolega]

The methodology from this thread:

https://www.dvxuser.com/forum/hdslrs/hdslr-general-other/307186-measuring-rolling-shutter-put-a-number-on-this-issue

seems to at least be consistent, if not totally scientific. Hopefully it helps a bit.

[kye]

+1 for having a consistent methodology.  

It might not be exactly what each person will get in their own setup, but it allows direct comparison between brands.

The parallel is DR, which has so many nuances in testing that you can't compare measurements that come from different sources, making the data almost completely useless unless it's part of a large database all from the same source and methodology.

[KnightsFan]

Here's my list

https://outerspaceoatmeal.com/tools/RollingShutterComparison.html

You can add or not , up to you. Most of my numbers come from DVXuser, CineD, and a couple from other primary sources where the test method has been shared. Global shutter cameras are self explanatory so I link to the product page.

10 minutes ago, kye said:

+1 for having a consistent methodology.

Yeah I mean it's ideal to always do it the same way, I'm just not sure many people will buy a specific arduino to fill out this table.

Difference with DR is that it's extremely subjective. Rolling shutter is not. People can measure it incorrectly-- which they can do whatever their intended test method is -- but they can't measure it correctly and then arrive at a different conclusion than someone else.

Edit: And to be clear, measuring signal to noise ratio is also objective.

[horshack]

8 hours ago, KnightsFan said:

Here's my list

https://outerspaceoatmeal.com/tools/RollingShutterComparison.html

You can add or not , up to you. Most of my numbers come from DVXuser, CineD, and a couple from other primary sources where the test method has been shared. Global shutter cameras are self explanatory so I link to the product page.

Yeah I mean it's ideal to always do it the same way, I'm just not sure many people will buy a specific arduino to fill out this table.

Difference with DR is that it's extremely subjective. Rolling shutter is not. People can measure it incorrectly-- which they can do whatever their intended test method is -- but they can't measure it correctly and then arrive at a different conclusion than someone else.

Edit: And to be clear, measuring signal to noise ratio is also objective.

Thanks. The results represented in that table is part of the reason I started my project - some of the measurements are off by significant amounts. For example, the A7 III 1080 is listed at 8.7ms - my measurement is 7.10ms - that's a 22% error in the 8.7ms measurement. There is simply too much slack in the varied methodologies being used to be considered objective measures.

[KnightsFan]

3 hours ago, horshack said:

Thanks. The results represented in that table is part of the reason I started my project - some of the measurements are off by significant amounts. For example, the A7 III 1080 is listed at 8.7ms - my measurement is 7.10ms - that's a 22% error in the 8.7ms measurement. There is simply too much slack in the varied methodologies being used to be considered objective measures.

Sounds like either you or the other person measured it wrong (or possibly both of you did).

[horshack]

2 hours ago, KnightsFan said:

Sounds like either you or the other person measured it wrong (or possibly both of you did).

Agreed. The difference is the methodology I'm using has been fully disclosed, along with all materials needed to generate it, the most important of which is the light source, the frequency of which has been independently verified with an oscilloscope. That means anyone can verify the math behind the measurement and reproduce it themselves using a $17 Arduino board.

Again, that was the impetus behind the project. To finally standardize a measurement around a transparent, opensource-veified methodology.

[KnightsFan]

1 hour ago, horshack said:

Agreed. The difference is the methodology I'm using has been fully disclosed, along with all materials needed to generate it, the most important of which is the light source, the frequency of which has been independently verified with an oscilloscope. That means anyone can verify the math behind the measurement and reproduce it themselves using a $17 Arduino board.

Again, that was the impetus behind the project. To finally standardize a measurement around a transparent, opensource-veified methodology.

I fully support your endeavor! I'm not negative on what you're doing. Ideally it is best to standardize, I just worry that the list won't grow very large, because of the purchase requirement. Unless you can get buy in from a big reviewer who gets their hands on a lot of models (or maybe you are a reviewer who gets your hands on lots of models personally)

[horshack]

3 hours ago, KnightsFan said:

I fully support your endeavor! I'm not negative on what you're doing. Ideally it is best to standardize, I just worry that the list won't grow very large, because of the purchase requirement. Unless you can get buy in from a big reviewer who gets their hands on a lot of models (or maybe you are a reviewer who gets your hands on lots of models personally)

Thanks, I appreciate that. I have a running thread on Fred Miranda soliciting submissions and so far the response has been good. To encourage participation I've been buying the Arduino boards on Amazon and having them shipped directly to members who have cameras I would like tested. Going forward my hope with the crowdsourcing is that the group interested in these kinds of measurements would be enthusiastic about having their own reliable method for measuring readout speeds whenever they need, so that the small investment in the Arduino board wouldn't be an impediment. Time will tell if that turns out to be true.