Advice for buying SSD?

[kye]

I'm contemplating buying and SSD for editing, and potentially in future for USB-C recording from a camera if I ever go that direction.

Any reason I shouldn't buy the Samsung T9 drive?  Pricing is not much more than the T7 and it's faster and more future-proof.

[IronFilm]

22 minutes ago, kye said:

Any reason I shouldn't buy the Samsung T9 drive?  Pricing is not much more than the T7 and it's faster and more future-proof.

I remember reading there was so sort of compatibility issues with BMD cameras and the newer Samsung drives?

But checking it out quickly shows this post from a couple of days ago, saying T9 is all good:

https://forum.blackmagicdesign.com/viewtopic.php?f=2&t=190756 

This post says T7 is bad:

https://forum.blackmagicdesign.com/viewtopic.php?f=2&t=165083

But now the SamsungT7 Shield Portable SSD 1TB / 2TB / 4TB is listed as supported:

https://www.blackmagicdesign.com/support/faq/59025 

So they found a way?? And updated the firmware. (as the 2nd forum post was a little while ago) 

Thus either would be fine??? If the price difference is only small, get the T9

Edit:

ahhhh... it was about the Shield vs non-Shield versions of the T7:

https://www.reddit.com/r/videography/comments/17kglkn/blackmagic_ursa_mini_pro_46k_g2_samsung_t9_ssd/ 

[ac6000cw]

No experience of recording to them from a camera directly, but I use a 500GB T7 (non-Shield) at home for video files, and we use several T7 Shield drives at work for daily off-site backups. Performance is good (and doesn't slow down noticeably when they being worked hard and get hotter) and so far they've been reliable, They also seem well built.

[Jedi Master]

9 hours ago, kye said:

Any reason I shouldn't buy the Samsung T9 drive?  Pricing is not much more than the T7 and it's faster and more future-proof.

I have quite a few of both, and the T9 is more robust than the T7. Whenever I copy files to an external SSD, I do a rigorous bit-by-bit compare of the copied files with the originals. On numerous occasions, I've found cases were there were bit errors in the copies, and even cases were entire files were not copied to the SSD. I have never had this issue with T9 drives, only T7 drives. I have seen this on several T7 drives, so it's not an isolated occurrence. It happens regardless of whether I format the drives as ExFAT or NTFS.

This is with Windows 10 and 11. I don't know if MacOS would have a similar issue. YMMV.

[kye]

1 hour ago, Jedi Master said:

I have quite a few of both, and the T9 is more robust than the T7. Whenever I copy files to an external SSD, I do a rigorous bit-by-bit compare of the copied files with the originals. On numerous occasions, I've found cases were there were bit errors in the copies, and even cases were entire files were not copied to the SSD. I have never had this issue with T9 drives, only T7 drives. I have seen this on several T7 drives, so it's not an isolated occurrence. It happens regardless of whether I format the drives as ExFAT or NTFS.

This is with Windows 10 and 11. I don't know if MacOS would have a similar issue. YMMV.

I am stunned that this would happen at all - perhaps the most fundamental principle of digital electronics is perfect replication of digital information.  To have bit errors in something as simple as a file copy over a USB cable is staggering.

Where the hell are the error detection mechanisms in these situations???  

Did the OS detect the errors, or was it only your verification that revealed them?

[Jedi Master]

1 hour ago, kye said:

Where the hell are the error detection mechanisms in these situations???  

Did the OS detect the errors, or was it only your verification that revealed them?

Only the compare revealed the copying errors. I too was shocked when I saw them and when I used Window's Event Viewer to look at the error logs, there was nothing there to indicate any device errors. The first time this happened, I wiped the T7 by reformatting it and tried the copying again, and got the same results, although the copy errors were in different files and the missing files were different. I tried this several more times with two other T7 drives and got similar results. The files being copied were about a TB of RAW files from my Canon still cameras along with associated .PSD files.

In all cases, the drives were connected to my PC using USB C to a native USB C port on the PC. I tried four different USB cables to rule out a bad cable and it didn't make a difference.

I tried the same tests with several T9 drives and never saw any binary copy errors or missing files. At this point, I have zero confidence in the T7.

[kye]

8 hours ago, Jedi Master said:

Only the compare revealed the copying errors. I too was shocked when I saw them and when I used Window's Event Viewer to look at the error logs, there was nothing there to indicate any device errors. The first time this happened, I wiped the T7 by reformatting it and tried the copying again, and got the same results, although the copy errors were in different files and the missing files were different. I tried this several more times with two other T7 drives and got similar results. The files being copied were about a TB of RAW files from my Canon still cameras along with associated .PSD files.

In all cases, the drives were connected to my PC using USB C to a native USB C port on the PC. I tried four different USB cables to rule out a bad cable and it didn't make a difference.

I tried the same tests with several T9 drives and never saw any binary copy errors or missing files. At this point, I have zero confidence in the T7.

That is bizarre.

I happened to see a menu item in Resolve yesterday and looked it up and it's a feature where you can set a source folder and one or more destination folders and Resolve will duplicate the contents, including creating a log file in each location containing some sort of checksum, so I imagine there must be some reason they decided to implement that verification step.

[kye]

Thanks all ...  I've ordered a T9 2Tb.

Rationale:

• Tried and tested brand and series
• I was concerned that the odd shape of the T9 would be a problem (the T5 and T7 can fit into a slot for camera mounts) but I figured that it's so common that people will make some kind of mount for it
• T9 wasn't much more expensive than older generations but will be more future-proofed

In case anyone is curious about the business case and stakeholder management aspects, I explained it to the wife using this argument:

1. I tried editing the honeymoon trip from the HDDs I have, but they can't keep up
2. I have an SSD, but it's not big enough to hold the whole trip
3. The SSD I have is still a great performer, even though I bought it many many years ago, so they last
4. Apple charges $600 per TB for their internal SSDs, external ones are a fraction of that and transfer from one computer to the next
5. I just got paid, so......

[Jedi Master]

17 hours ago, kye said:

That is bizarre.

I happened to see a menu item in Resolve yesterday and looked it up and it's a feature where you can set a source folder and one or more destination folders and Resolve will duplicate the contents, including creating a log file in each location containing some sort of checksum, so I imagine there must be some reason they decided to implement that verification step.

Yes, very bizarre. I've never seen anything like it in recent memory and am puzzled where the issue is happening--on the SSD itself, in the transfer process across a USB cable, on the motherboard in the USB hardware, transferring data from the USB controller into memory, a bug in the OS that's corrupting memory involved in the file transfer, or a bug in the program I use for the binary comparisons.

I did a test yesterday that adds to the confusion. I took my newest T7, reformatted it, and copied all of my digital photos (about 1TB worth) to it. I then ran a bit-by-bit binary comparison and saw several bit compare errors and several missing files in several directories. I ran the same compare two more times and saw the same kinds of errors, but they were in different files! Yes, files that didn't compare previously were now comparing correctly, and files that compared correctly previously were now not comparing correctly.

It's always good to compute checksums for important files to verify their integrity. They're not quite as good as bit-by-bit comparisons, because two totally different files have a tiny, but non-zero, chance of hashing to the same checksum. I think I'll try calculating MD5 hashes on all of my digital photo files from their usual location (a 10TB WD hard disk) and do the same for the copies on the T7 SSD and see if the MD5 hashes are always the same. I'll run this in a loop and log the results.

[kye]

6 hours ago, Jedi Master said:

Yes, very bizarre. I've never seen anything like it in recent memory and am puzzled where the issue is happening--on the SSD itself, in the transfer process across a USB cable, on the motherboard in the USB hardware, transferring data from the USB controller into memory, a bug in the OS that's corrupting memory involved in the file transfer, or a bug in the program I use for the binary comparisons.

I did a test yesterday that adds to the confusion. I took my newest T7, reformatted it, and copied all of my digital photos (about 1TB worth) to it. I then ran a bit-by-bit binary comparison and saw several bit compare errors and several missing files in several directories. I ran the same compare two more times and saw the same kinds of errors, but they were in different files! Yes, files that didn't compare previously were now comparing correctly, and files that compared correctly previously were now not comparing correctly.

It's always good to compute checksums for important files to verify their integrity. They're not quite as good as bit-by-bit comparisons, because two totally different files have a tiny, but non-zero, chance of hashing to the same checksum. I think I'll try calculating MD5 hashes on all of my digital photo files from their usual location (a 10TB WD hard disk) and do the same for the copies on the T7 SSD and see if the MD5 hashes are always the same. I'll run this in a loop and log the results.

I wouldn't pretend to know where such an error would be, but if it happened across multiple drives then it wouldn't be on the SSD unless there was an issue with the firmware / controller.  

Maybe try the same drive in a different computer?

My dad used to be in IT at a large educational institution and they regularly had issues when buying custom high-powered PCs (such as servers or classrooms for processor intensive applications).  The issues were always that some piece of hardware was incompatible with one or more other pieces of hardware.  At one point they bought a server and there was some problem, so he went online and apparently because it was a combination of the latest components it was common for people to have that problem.  Each of the manufacturers were blaming each other and claiming their product was fine.  After a couple of months of it not being solved they just told their supplier to take it back as a return, which they did because the college purchased a huge amount of equipment from them each year.  Dad explained to me that manufacturers frequently build components that follow a standard but will deviate from it in little ways that are advantageous for their product.  Most of the time this is fine, but occasionally there will be two or more products that took each took a liberty that clash, and so the combination of the two won't work.  Of course, then both manufacturers will claim the issue is with the other product because their product works with lots of other alternative products.

[Jedi Master]

On 1/6/2024 at 6:42 PM, kye said:

Maybe try the same drive in a different computer?

This has got me curious enough to dig deeper. I'll try with several other computers running various operating systems, different cables, etc. Hopefully I can identify the root cause of the miscompares.

[kye]

My Samsung T9 finally arrived***
(*** here in Western Australia (WA) people joke that the WA stands for "wait awhile" because we're 3500kms/2000mi+ from the main cities but when things are shipped from overseas like this SSD, this waiting applies to all of Australia....)

Plugging it straight in to a dedicated port on my MBP with their supplied cable, it gets about 850MB/s read and write.  The manufacturer claims 2000MB/s.  My internal SSD gets 1700MB/s write and 2300MB/s read, so the computer probably isn't the bottleneck.

Is there anything I should do to get better speeds?  Samsung says on their website that "the UASP mode must be enabled" - is that something I should look at?

Disk Utility says it's formatted with ExFAT, so that probably doesn't need reformatting.

I'm testing the Resolve Clone Tool to copy a 219Gb folder to it from my Media HDD, and I think the tool does a checksum of each, so this will verify if the copy was bit-perfect @Jedi Master.  It's got 30 mins to go so I'll post again once I've done some further testing.

 

 

[kye]

My 219Gb copy completed successfully, with Resolve saying the MD5 checksum validation was good.  It has a range of options for validation, with MD5 being the default:

I googled what an error looks like, and Resolve makes it pretty obvious that the job failed, so it seems like a good tool.

[kye]

I hit an error.  

Turns out Resolve has great logs.   Go to Help and select Create Diagnostics Log On Desktop and it will create a set of logs.

In one of the files, ResolveDebug.txt, the last line is:

Quote

Checksum verification failed, target can not be read: Clone Error Path = [/Volumes/T9/Footage/2018-11-10 India big trip/2018-11-10 India/._P1033157.MOV]

I tired it twice, it failed twice, and the same file is the one listed in the log as the one that gave the error.  Interestingly, that file doesn't exist, at least not starting with a "._" prefix.  There is a file there with that name without the prefix though, and it plays fine.

The source and destination folders are the same size and contain the same number of items, once you remove the log files it puts in there.

I tried syncing each of the subfolders individually, and the subfolder with the issue generates an error, and the other ones don't, so it seems to be consistent.  Odd.  Considering this is copying files from my master location to an SSD for faster editing, and that the SSD copy will be deleted when no longer needed, I'll just move on.  I have noticed that OSX occasionally hits an error in moving files because a file will be locked for some reason, and there are other strange little hiccups that happen.  I guess nothing is perfect.

[ac6000cw]

5 hours ago, kye said:

Plugging it straight in to a dedicated port on my MBP with their supplied cable, it gets about 850MB/s read and write.  The manufacturer claims 2000MB/s.  My internal SSD gets 1700MB/s write and 2300MB/s read, so the computer probably isn't the bottleneck.

Which version of the MBP have you got?

The internal SSD uses a PCIe interface (as is normal for modern internal SSD). The T9 supports 'USB 3.2 Gen 2x2' (20 Gbps) but your MBP might not support that on its expansion ports. The speed you're getting suggests it's running at 'USB 3.2 Gen 2x1' 10 Gbps connection speed.

[Jedi Master]

5 hours ago, kye said:

I tired it twice, it failed twice, and the same file is the one listed in the log as the one that gave the error.  Interestingly, that file doesn't exist, at least not starting with a "._" prefix.  There is a file there with that name without the prefix though, and it plays fine.

Interesting. I wasn't aware of Resolve's logs--I'll have to check it out. In my case, the errors I was seeing didn't involve Resolve because I was copying files using Windows Explorer, not Resolve. The two types of error I saw, file mis-compares, and missing files, did not show up in any Windows log (as viewed in Event Viewer).

Many Windows applications create temporary files with prefixes and/or suffixes added to the eventual permanent file name when doing certain operations on files, so perhaps Macs do something similar and that file you saw beginning with ._ was a temporary file that was later removed. 

[kye]

14 hours ago, ac6000cw said:

Which version of the MBP have you got?

The internal SSD uses a PCIe interface (as is normal for modern internal SSD). The T9 supports 'USB 3.2 Gen 2x2' (20 Gbps) but your MBP might not support that on its expansion ports. The speed you're getting suggests it's running at 'USB 3.2 Gen 2x1' 10 Gbps connection speed.

I have the MacBook Pro, 13-inch, 2020..  specs here.

The specs page quotes it as "USB 3.1 Gen 2 (up to 10Gb/s)" so it sounds like you're exactly on the money, and if my math is right then 10Gb/s is 1,250MB/s which seems to align with the 850MB/s I was seeing.

I'm not worried by this - I have an SSD that's faster than my computer and easily faster than I need for editing.  The files I edit are typically 200Mbps ALL-I or 100-150Mbps IPB codecs, so the speed issue is when there's a cut the computer has to read the file to find the previous keyframe, then decode the file from there to catch-up to the first frame in my timeline, which obviously you want to happen fast enough so that the timeline plays flawlessly.

9 hours ago, Jedi Master said:

Interesting. I wasn't aware of Resolve's logs--I'll have to check it out. In my case, the errors I was seeing didn't involve Resolve because I was copying files using Windows Explorer, not Resolve. The two types of error I saw, file mis-compares, and missing files, did not show up in any Windows log (as viewed in Event Viewer).

Many Windows applications create temporary files with prefixes and/or suffixes added to the eventual permanent file name when doing certain operations on files, so perhaps Macs do something similar and that file you saw beginning with ._ was a temporary file that was later removed. 

I copied about 800Gb across multiple projects without errors, then I hit my first error and kept hitting them after that.  I tried copying a few more projects and each failed for some reason.  Overnight I unplugged the drives and this morning I reconnected them and am now trying again, so we'll see if it was just a random thing or if it's repeatable.

TBH, the Clone Tool seems great when it works but it's a bit of a PITA when it doesn't.  Having to export the logs, decompress them, then open up a file is very cumbersome.  I also think it just hits an error and stops, whereas I'd rather it keep going and give an error log of all the files that failed and do so in a way that's easy to access.  If you want to try again there's no way to only validate without copying the files again, or to resume a job, etc.  

If it keeps failing I think I'll look up what unix commands I can use to compare the files, I'd imagine there's a unix utility that will compare two folders and give me a nice report listing all the failures.  

[IronFilm]

On 1/19/2024 at 6:12 PM, kye said:

My Samsung T9 finally arrived***
(*** here in Western Australia (WA) people joke that the WA stands for "wait awhile" because we're 3500kms/2000mi+ from the main cities but when things are shipped from overseas like this SSD, this waiting applies to all of Australia....)

hey there, Adelaide is "nearby" and is the largest city in the state ( and the fifth-most populous city in Australia). Merely a couple of thousand kilometers away (closer to Perth than we in Auckland are to Sydney!)

[kye]

So, I realised that it kept failing on these files beginning with "._" I realised that they were actually there, despite Finder not showing them to me when I put it into the mode for showing hidden files, so it was only the command line that revealed their existence.  Turns out they're some sort of Apple metadata file for storing info on lesser file systems.  This makes sense that they'd be present in older projects but not in new ones, as I must have moved to ExFat at some point and Apple stopped creating them.

After a bunch of problems trying to work with them, since some commands can see them and others can't, I found the dot_clean command, which merges all the metadata from these files back into the file it should be in and then removes them.  So I ran it over my whole footage archive.

I then successfully copied one project that had been failing before.  Unfortunately, the next project I tried copying failed on a real file, so not just those ones.

Further investigation revealed that my spinning disk was getting hot, really hot.  It's a "Seagate IronWolf 12TB 3.5" SATA Internal NAS Hard Drive HDD 7200RPM 256MB Cache" mounted vertically in one of those USB docks, so probably had enough passive cooling for anything except going flat out for multiple hours without a break.

I kept getting an error on a good file and couldn't figure it out, but eventually I worked out that the "Couldn't read the source file" was actually a very poor way of saying that the target drive was full.  FML!  Anyway, 2Tb of footage copied successfully!

[kye]

5 hours ago, IronFilm said:

hey there, Adelaide is "nearby" and is the largest city in the state ( and the fifth-most populous city in Australia). Merely a couple of thousand kilometers away (closer to Perth than we in Auckland are to Sydney!)

We might be closer to Adelaide than you are to Sydney, but have you been to Adelaide?  From any kind of perspective it's a non-place, so I'd rather be slightly further away from Sydney!