Petition for Samsung NX1 hack

[Marco Tecno]

I searched. Pro video quality was added with fw 1.2.

[kidzrevil]

10 hours ago, Pavel Mašek said:

5D has option for lower resolution "RAW" (it is not exactly the RAW becuase of resolution, but bit depth is same as original RAW). That is someting that NX1 does not have. Moreover - NX1 would need also electronic shutter to achieve "video RAW in photo mode". It would be complicated even for Samsung engineers...

Let's focus on bitrate first... (...crop mode, lower sharpness or noise reduction)

Agreed

18 hours ago, sandro said:

could be because the processor has been developed to downscale in hardware?

The processor is doing a full pixel readout and then downscaling it. The high rs is the end result of this

[Marco Tecno]

I just talked with a computer scientist. He states that reverse engineering would be possible, but very time consuming, especially not knowing exactly the hardware. But he thinks there must be some visible numerical parameters, since those are variables for the main software. Unfortunately he hasn't got time for delving into this and he's not a camera enthusiast, so no personal interest into this :-( 

[Syme]

I don't think I currently have the experience or time to successfully modify the NX1/NX500 firmware. I don't even have one of those cameras. However it would be a shame to let the interest in this die, so I guess I'll post some of my notes/thoughts about the firmware in hopes that it might help keep the ball rolling.

Note: if you aren't into technical minutiae, the only interesting part of this wall of text is the bulleted list at the end detailing what is and is not possible in my opinion.

Here are the files listed in the header of nx1.bin (version 1.4.0):

version.info:    offset=0x0          size=0x3F   (same as found in /etc/version.info)
linux image:    offset=0x0130       size=0x00624748
idk:                   offset=0x624878     size=0xD8E9
linux image:    offset=0x632161     size=0x3192E8
linux image:    offset=0x94B449     size=0x638518
idk:            offset=0xF83961     size=0x01FF10
idk:            offset=0xFA3871     size=0xB35140
rootfs:         offset=0x1AD89B1    size=0x117A89FF     (lzo compressed ext4 filesystem image)
opt:            offset=0x132813B0   size=0x58E91C   (lzo compressed ext4 filesystem image)
pcachelist:     offset=0x1380FCCC   size=0x7000 (PAGECACHELIST, preceded by a header, I think)
idk:            offset=0x13816CCC   size=0x35BCC44  (lzo compressed. header indicates swap image?)

Anyone with the skills to reverse engineer a camera could figure this out pretty easily, but it was fairly tedious so maybe this will save someone 20 minutes of poking around in a hex editor. If anyone knows what's up with the files I've labeled "idk," I would love to hear about it.

The checksum algorithm is fortunately unchanged from the NX300 as far as I can tell.

As documented at sites.google.com/site/nxcryptophotography/diy-firmware
"width=32 poly=0x04c11db7 init=0xffffffff refin=true refout=true xorout=0x00000000 check=0x340bc6d9 name="JAMCRC""
"jacksum -x -a crc:32,04c11db7,ffffffff,true,true,00000000 [file]"

The main camera app binary is (I'm pretty sure) located at /usr/apps/com.samsung.di-camera-app/bin/di-camera-app in the rootfs. It seems to access the hardware through a relatively high-level API with the /usr/lib/libmm* libraries. libmmf-camcorder.so is particularly interesting. The function I've focused my attention on is mmf_camcorder_set_attributes(), which comes from libffm-camcorder and is used repeatedly in di-camera-app. It conveniently (and strangely IMO) takes strings as identifiers for the attributes that are apparently being set (why not just an enum? I suppose I shouldn't look a gift horse in the mouth...). Some of those attributes include "target-time-limit," "audio-encoder-bitrate," and "video-encoder-bitrate." The guy who successfully removed the recording time limit on the NX300 did it by modifying the instructions that set the variable being passed along with "targe-time-limit." I found the control flow instructions he mentioned in that thread, so it should't be hard to get rid of the time limit on the NX1, provided the camera accepts the modified firmware. The NX500 is probably similar. The "video-encoder-bitrate" attribute also looks promising, though it would take some more advanced reverse engineering to figure out where the values are being set.

So from what I've seen and read, here is what I think is and is not possible to modify on the NX1 and NX500:

• Remove time limit: Highly likely. Seems to be the same as the NX300. Pretty easy too, if there aren't any new security measures in place.
• Increased bitrate: Possible. Needs some real reverse engineering to find where the rates are set for each resolution and quality.
• Noise reduction and sharpening: Possible. Haven't seen anything that looks like it's controlling these, but if setting the bitrate works, this should be possible too. FWIW I think that increasing the bitrate would help with the noise reduction issues. H.265 tends to smooth things out a lot to achieve low bitrates.
• Re-enable 2.5k on NX500: Plausible but difficult. It depends on whether they just removed it from di-camera-app, or if they removed it from the underlying libraries as well. Either way it would likely require actually adding control flow to the binary, which opens a whole new can of worms. Beyond my current ability, for sure.
• Focus peaking for NX500 4k: Maybe? I have no idea, really. There might be a good reason they didn't include it, there might not.
• 4k crop on NX1: Plausible but even more difficult. We know the hardware can do it, but it was probably never implemented on the NX1, even in pre-production.
• Gamma profile on NX500: Plausible. Similar to porting the NX500's 4k crop to the NX1, I think.
• 6k 24fps H.265: Highly unlikely. The H.265 encoder would have to support frame sizes larger than DCI 4k and be able to handle twice the pixel rate (clock speed) of 4k. Furthermore it would require implementing a brand-new video mode at a very low level. I can't say for sure it's categorically impossible, but don't get your hopes up.
• 10bit H.265: Nope. The H.265 standard does indeed allow for 10bit encoding, but I highly doubt Samsung would include the significantly larger (wider busses, more complex encoding) hardware necessary to do it. It would be a miracle if Samsung had really decided to go to all that effort and not use it.
• 6k or full sensor 4k at more than 30fps or 1080p at (significantly) more than 120fps: Impossible. The image sensor simply isn't fast enough. If you hope for this you will just be disappointed!
• RAW Video: Not really. It might be theoretically possible to dump the live-view feed as in Magic Lantern. Who knows how fast the SD card interface is, though. Certainly no more than 1080p. I can imagine tricking the GPU into packing 12bit 4k RAW into 1080p 444 HDMI like Apertus, but consider that a pipe dream. Don't get your hopes up.

Anyway, that was longer than I expected, but I enjoyed poking around in the firmware, so I don't regret it even if it comes to nothing. It's a shame Samsung appears to be dropping the NX line; they are cool little cameras.

p.s. If you know anything I've said is wrong, please correct me; I'm learning this as I go along.

[Pavel Mašek]

Thank you very much for you hard work.... I think this is definitely best summary related to possible hacking of NX1! 

Let's hope that someone else will be able to continue this way to higher bitrate in NX1. 

[Marco Tecno]

Very interesting. I still think 1080@240fps could be possible for the hw, due to what the engineers said during nx1 launch. But I understand the possible limitations. I agree with you on all the rest.

 

It'd be interesting to put the hands on the beta fw for nx500, with 2.5k resolution. And would be even more interesting to compare the fw of nx1 and nx500, to understand where the missing parts are.

[MountneerMan]

@Syme

Have you seen otto K's two posts on dpreview regarding his investigation into the NX500 software (linked below).

http://***URL removed***/forums/post/57133589

It seams to me that if the two of you shared notes we could have something.

 

EDIT: Fixed link

[sandro]

Higher bitrates will help in 120fps setting otherwise at 3200 ISO 4k the noise will be as bad since you get those horrible blue lines that have nothing to do with NR... it's the sensor.

[Mattias Burling]

Getting the 2.5K back in the NX500 would make my day.

[undecided]

15 hours ago, Syme said:

I don't think I currently have the experience or time to successfully modify the NX1/NX500 firmware. I don't even have one of those cameras. However it would be a shame to let the interest in this die, so I guess I'll post some of my notes/thoughts about the firmware in hopes that it might help keep the ball rolling.

Note: if you aren't into technical minutiae, the only interesting part of this wall of text is the bulleted list at the end detailing what is and is not possible in my opinion.

Here are the files listed in the header of nx1.bin (version 1.4.0):

version.info:    offset=0x0          size=0x3F   (same as found in /etc/version.info)
linux image:    offset=0x0130       size=0x00624748
idk:                   offset=0x624878     size=0xD8E9
linux image:    offset=0x632161     size=0x3192E8
linux image:    offset=0x94B449     size=0x638518
idk:            offset=0xF83961     size=0x01FF10
idk:            offset=0xFA3871     size=0xB35140
rootfs:         offset=0x1AD89B1    size=0x117A89FF     (lzo compressed ext4 filesystem image)
opt:            offset=0x132813B0   size=0x58E91C   (lzo compressed ext4 filesystem image)
pcachelist:     offset=0x1380FCCC   size=0x7000 (PAGECACHELIST, preceded by a header, I think)
idk:            offset=0x13816CCC   size=0x35BCC44  (lzo compressed. header indicates swap image?)

Anyone with the skills to reverse engineer a camera could figure this out pretty easily, but it was fairly tedious so maybe this will save someone 20 minutes of poking around in a hex editor. If anyone knows what's up with the files I've labeled "idk," I would love to hear about it.

The checksum algorithm is fortunately unchanged from the NX300 as far as I can tell.

As documented at sites.google.com/site/nxcryptophotography/diy-firmware
"width=32 poly=0x04c11db7 init=0xffffffff refin=true refout=true xorout=0x00000000 check=0x340bc6d9 name="JAMCRC""
"jacksum -x -a crc:32,04c11db7,ffffffff,true,true,00000000 [file]"

The main camera app binary is (I'm pretty sure) located at /usr/apps/com.samsung.di-camera-app/bin/di-camera-app in the rootfs. It seems to access the hardware through a relatively high-level API with the /usr/lib/libmm* libraries. libmmf-camcorder.so is particularly interesting. The function I've focused my attention on is mmf_camcorder_set_attributes(), which comes from libffm-camcorder and is used repeatedly in di-camera-app. It conveniently (and strangely IMO) takes strings as identifiers for the attributes that are apparently being set (why not just an enum? I suppose I shouldn't look a gift horse in the mouth...). Some of those attributes include "target-time-limit," "audio-encoder-bitrate," and "video-encoder-bitrate." The guy who successfully removed the recording time limit on the NX300 did it by modifying the instructions that set the variable being passed along with "targe-time-limit." I found the control flow instructions he mentioned in that thread, so it should't be hard to get rid of the time limit on the NX1, provided the camera accepts the modified firmware. The NX500 is probably similar. The "video-encoder-bitrate" attribute also looks promising, though it would take some more advanced reverse engineering to figure out where the values are being set.

So from what I've seen and read, here is what I think is and is not possible to modify on the NX1 and NX500:

• Remove time limit: Highly likely. Seems to be the same as the NX300. Pretty easy too, if there aren't any new security measures in place.
• Increased bitrate: Possible. Needs some real reverse engineering to find where the rates are set for each resolution and quality.
• Noise reduction and sharpening: Possible. Haven't seen anything that looks like it's controlling these, but if setting the bitrate works, this should be possible too. FWIW I think that increasing the bitrate would help with the noise reduction issues. H.265 tends to smooth things out a lot to achieve low bitrates.
• Re-enable 2.5k on NX500: Plausible but difficult. It depends on whether they just removed it from di-camera-app, or if they removed it from the underlying libraries as well. Either way it would likely require actually adding control flow to the binary, which opens a whole new can of worms. Beyond my current ability, for sure.
• Focus peaking for NX500 4k: Maybe? I have no idea, really. There might be a good reason they didn't include it, there might not.
• 4k crop on NX1: Plausible but even more difficult. We know the hardware can do it, but it was probably never implemented on the NX1, even in pre-production.
• Gamma profile on NX500: Plausible. Similar to porting the NX500's 4k crop to the NX1, I think.
• 6k 24fps H.265: Highly unlikely. The H.265 encoder would have to support frame sizes larger than DCI 4k and be able to handle twice the pixel rate (clock speed) of 4k. Furthermore it would require implementing a brand-new video mode at a very low level. I can't say for sure it's categorically impossible, but don't get your hopes up.
• 10bit H.265: Nope. The H.265 standard does indeed allow for 10bit encoding, but I highly doubt Samsung would include the significantly larger (wider busses, more complex encoding) hardware necessary to do it. It would be a miracle if Samsung had really decided to go to all that effort and not use it.
• 6k or full sensor 4k at more than 30fps or 1080p at (significantly) more than 120fps: Impossible. The image sensor simply isn't fast enough. If you hope for this you will just be disappointed!
• RAW Video: Not really. It might be theoretically possible to dump the live-view feed as in Magic Lantern. Who knows how fast the SD card interface is, though. Certainly no more than 1080p. I can imagine tricking the GPU into packing 12bit 4k RAW into 1080p 444 HDMI like Apertus, but consider that a pipe dream. Don't get your hopes up.

Anyway, that was longer than I expected, but I enjoyed poking around in the firmware, so I don't regret it even if it comes to nothing. It's a shame Samsung appears to be dropping the NX line; they are cool little cameras.

p.s. If you know anything I've said is wrong, please correct me; I'm learning this as I go along.

One way to see what is going on would be to be able to have a development kit. Like an emulator for the camera firmware to run on. Not sure if such a beast ever existed

[SMGJohn]

1 hour ago, undecided said:

One way to see what is going on would be to be able to have a development kit. Like an emulator for the camera firmware to run on. Not sure if such a beast ever existed

Basically what you want is the Samsung NX1 SDK, right? And that exists including a 91 page PDF document for it.

[Syme]

Has anyone taken a look at what exactly is in that SDK? I was under the impression it was just for developing remote apps.

I think the Nikonhacker folks made an emulator for some Nikon DSLRs to help understand the firmware. I'm not sure how much they have done with it.

For the NX1/500 getting root on the physical camera and running a debugger there would probably be easier and more fruitful. Nikon, Canon, Sony, Panasonic, and most other camera manufacturers use custom (or semi-custom) operating systems, which make standard debuggers incompatible. Since Samsung uses a Linux kernel it should be possible to run a standard debugger if you have root, and use that to see the instructions executing on the real hardware.

[MountneerMan]

 

13 hours ago, Syme said:

Has anyone taken a look at what exactly is in that SDK? I was under the impression it was just for developing remote apps.

I was under the same impression.

@SMGJohn et al, where can I download the SKD and 91 page PDF? I would be interested in giving it a look over. USB 3.0 has a transfer rate of 5Gb/sec witch is more than enough for a 4K RAW stream. I am imagining being able to plug in a laptop(with an SSD) via USB and using it like an external recorder except you would have pretty much complete control of the camera(except zoom and aperture depending on the lens) as well.

Does any one think this would even be possible? I am not a CS expect.

 

13 hours ago, Syme said:

For the NX1/500 getting root on the physical camera and running a debugger there would probably be easier and more fruitful. Nikon, Canon, Sony, Panasonic, and most other camera manufacturers use custom (or semi-custom) operating systems, which make standard debuggers incompatible. Since Samsung uses a Linux kernel it should be possible to run a standard debugger if you have root, and use that to see the instructions executing on the real hardware.

Did you read the dpreview posts I linked yesterday above? otto K said he thinks its possible to run scripts from the SD Card witch would allow for the use of debuggers.

Also because the NX1/NX500 are Tizen based it sounds like it could be quite easy(relatively speaking) to create an emulator https://developer.tizen.org/development/tools/common-tools/emulator . Again I am far from a CS expect and I am way out of my depth and really have no idea. Anyone who knows better please correct me.

[Marco Tecno]

Nobody compared fw 1.01 and fw 1.2 or higher? As said, 1.2 is where higher bitrate was introduced.

[Syme]

3 hours ago, MountneerMan said:

USB 3.0 has a transfer rate of 5Gb/sec witch is more than enough for a 4K RAW stream. I am imagining being able to plug in a laptop(with an SSD) via USB and using it like an external recorder except you would have pretty much complete control of the camera(except zoom and aperture depending on the lens) as well.

Does any one think this would even be possible? I am not a CS expect.

Did you read the dpreview posts I linked yesterday above? otto K said he thinks its possible to run scripts from the SD Card witch would allow for the use of debuggers.

Also because the NX1/NX500 are Tizen based it sounds like it could be quite easy(relatively speaking) to create an emulator https://developer.tizen.org/development/tools/common-tools/emulator . Again I am far from a CS expect and I am way out of my depth and really have no idea. Anyone who knows better please correct me.

I did read otto k's posts. Currently I'm going about things from another angle, but there is definitely some interesting stuff there. Running scripts on the card would be the first step on the way to getting root.

Alright, now time for some unfounded wild speculation =)

As far as USB 3.0 RAW goes, it really depends on a lot. Keep in mind I have no idea how to actually do any of this; I'm just considering the capability of the hardware. First of all USB 3.0 cannot achieve 5Gb/s in real life. The real maximum data transfer is somewhere around 3.6Gb/s after the overhead is removed. Even then most implementations of USB 3.0 are a lot slower. Point Gray, a machine vision camera company did some extensive testing in which they found the maximum transfer rate with various computers and USB 3.0 cards, so if you want to know more, google that. Fortunately with custom drivers the integrated USB 3.0 ports on modern Intel processors appear to be just barely fast enough for 12 bit 4k RAW at 24fps. However the implementation of USB 3.0 on the NX1 isn't guaranteed to be fast enough, since not all USB 3.0 devices are. Someone would have to test.

The other high bandwidth external interface on the NX1 is HDMI, which has a maximum pixel clock of 340Mhz. That would be enough for 6.5k 12 bit 24fps RAW video if it were possible to pack two 12 bit pixels into each RGB444 pixel like Apertus is trying on their cameras. Of course it is possible that the NX1 doens't support the full bandwidth of HDMI 1.4, in which case the maximum rate we know it can handle is 3.40Gb/s for DCI 4k 422. That is less than the theoretical bandwidth of USB 3.0, but it is completely stable and guaranteed, unlike USB, and it would be enough for 5k at least. It would require a custom FPGA-based recorder to capture it (or support from Convergent Design or Atomos), but that's a lot easier than making a fully custom camera.

There are some big fundamental issues hanging over either strategy for RAW video. The first is whether or not the general purpose processor can actually access the raw sensor readout at full resolution. That isn't necessary for the normal operation of the camera, so there is no guarantee it's possible. If the general purpose ARM cores in the DRIMEv5 can't even access the raw data coming off the sensor, neither strategy would work. The other big question I can think of is what layout the data would come in memory if the ARM cores can in fact access it. If it is packed tightly it might be possible to just pass that buffer to the GPU and tell it to treat it as 24bpp RGB, or just copy it out the USB port as fast as possible. The wimpy little ARM cores are probably fast enough for that. On the other hand it might be aligned to 16bit boundaries for fast memory access to each pixel, in which case it would have to be packed tightly before transmitting it. I'm not sure if the general-purpose cores can do unaligned memory accesses or bit-packing operations fast enough (but again I'm no expert). Magic Lantern found that the processor in a Canon camera isn't fast enough to convert 14 bit to 12 bit at just 720p24. Actually on second thought the GPU might require 24bpp RGB to be padded out to 32bits, in which case some messy unaligned copying would be unavoidable.

Back to more realistic considerations, that Tizen emulator might be good for testing what the menues do. It probably could't emulate the camera-specific low-level stuff, but that doesn't mean it's not useful.

1 hour ago, Marco Tecno said:

Nobody compared fw 1.01 and fw 1.2 or higher? As said, 1.2 is where higher bitrate was introduced.

I might get there eventually. Currently I'm working on unpacking the NX500 firmware to compare it to the NX1, but I'm pretty busy with the work I'm actually supposed to be doing, so I'm not going to be making any rapid progress.

[tugela]

On ‎2‎/‎17‎/‎2016 at 0:40 AM, Syme said:

 

• 6k or full sensor 4k at more than 30fps or 1080p at (significantly) more than 120fps: Impossible. The image sensor simply isn't fast enough. If you hope for this you will just be disappointed!
• RAW Video: Not really. It might be theoretically possible to dump the live-view feed as in Magic Lantern. Who knows how fast the SD card interface is, though. Certainly no more than 1080p. I can imagine tricking the GPU into packing 12bit 4k RAW into 1080p 444 HDMI like Apertus, but consider that a pipe dream. Don't get your hopes up..

The sensor can do full reads at 240 fps, so it isn't an issue with the sensor. The reason for higher fps not being enabled is almost certainly because the processor can't do it or becomes unreliable due to bandwidth issues. The sensor itself is capable of doing 6K and any fps up to 240.

What comes out of live view is not RAW video. It is uncompressed, but still processed. It will still be 8 bit 4:2:0.

[Phil A]

Well, he's capable to do that in a laboratory probably... with god knows what system connection and active cooling.

[Marco Tecno]

About 240fps. It's a four cores arm cpu with additional co-processors. If the sensor can handle it, most certainly can the cpu. I'm not sure for how long, but...at 240fps I would not want more than 10 seconds anyway. Actually 5 would already be more than enough for most purposes.

[undecided]

7 hours ago, MountneerMan said:

 

I was under the same impression.

@SMGJohn et al, where can I download the SKD and 91 page PDF? I would be interested in giving it a look over. USB 3.0 has a transfer rate of 5Gb/sec witch is more than enough for a 4K RAW stream. I am imagining being able to plug in a laptop(with an SSD) via USB and using it like an external recorder except you would have pretty much complete control of the camera(except zoom and aperture depending on the lens) as well.

Does any one think this would even be possible? I am not a CS expect.

 

Did you read the dpreview posts I linked yesterday above? otto K said he thinks its possible to run scripts from the SD Card witch would allow for the use of debuggers.

Also because the NX1/NX500 are Tizen based it sounds like it could be quite easy(relatively speaking) to create an emulator https://developer.tizen.org/development/tools/common-tools/emulator . Again I am far from a CS expect and I am way out of my depth and really have no idea. Anyone who knows better please correct me.

Very interesting, I wonder if the firmware can execute on this VM

[tugela]

34 minutes ago, Phil A said:

Well, he's capable to do that in a laboratory probably... with god knows what system connection and active cooling.

No, those modes were used for debugging.

32 minutes ago, Marco Tecno said:

About 240fps. It's a four cores arm cpu with additional co-processors. If the sensor can handle it, most certainly can the cpu. I'm not sure for how long, but...at 240fps I would not want more than 10 seconds anyway. Actually 5 would already be more than enough for most purposes.

240 fps would be 8X the data load that the current 4K format handles. HD can be done at 120 fps, but at reduced quality, so the processor is choking on the data somewhere between 60 and 120 fps. My guess is that the thermal envelope limits how much they can push the processor.

It probably could handle higher loads if it had external cooling however.

And if you stuck the sensor/processor in a pro camcorder (where the thermal envelope is greater) it would likely be one of the most capable cameras in that segment as well.