benjiro3000

It'll definitely do that if you keep your database on a network share with spinning disks.

Database and Nextcloud where on a 4TB NVME drive ... in Mysql with plenty of cache/memory assigned to it. Not my first rodeo, ...

Even if you ran a basic sqlite nexcloud, if properly optimized, you can deal with millions of files like its nothing. And that is the issue, the bugs and lacking optimization..

4650g + 64GB ram + Mysql and it was file locking on just a 21k 10GB folder constantly.

I have written apps (in Go) that do similar and process data 100 times faster then nextcloud. Hell, my scrapers are faster then nextcloud in a local netwerk, and that is dealing with external data, over the internet.

Its BADLY designed software that puts the blame on the consumer to get bigger and better hardware, for what is essentially, early 2000 functionality.

Any media you want to long term store, you want to rotate. Aka, move data from A, to B, to C, B to C to A, C to ... Any storage media not specific designed with redundancy in mind, will see bit rot over time, depending on humidity, heat, location (if you live 2k high vs sea level), etc.

I always moved my media around like that, never had a issue with data that goes back 20 years.

For me any media is no different then having a car parked out for 6 months. Sure, it will work, but your battery may (will) be death, you brakes may get locked up, etc... Or leaving a house or apartment unattended for 6 month.

SSD for cold storage is just as good. Been using some SSDs going back to when 256GB was expensive. But like with HDDs, Tapes, etc... rotate and rewrite.

One of the biggest mistakes i have seen in a government job, was making backups, never checking / rewritting those backups and the day they needed those backups. Well, the taps had damage (despite only being 2 years cold stored). So the loads of money they spend on a dedicated guy, who's job it was to do backups every day, the expensive tap machines, etc, was all for nothing. Simply they did not do checks/rewrites. And nobody was able to blame him, because he was following the exact instructions for making those backups. He even brought up the issue but ... "instructions are made by people more qualified then you". O, they tried to blame him but he stepped to the union and it was quickly resolved in a "nobody was to blame" (because can not blame the managers above him, now can we ;0 ). His instruction got changed to: Verify every tape at minimum X times per year, and rewrite on new taps, with backups for every tape (also increased budget to buy more tapes instead of recycling). And that was like i said, government ;)

Any storage media will fail, and its up to the end user to ensure you have regular rewrites, checks, recoverable parity of the files itself, ... Say this as somebody that lost months of programming work, when his primary HDD failed and multiple backups ended up useless ( in the old floppy days ). Multiple backups helps but even that can be fraud. I have seen written DVDs go bad all at the same time (one reason i never backup to DVD or Bluray)...

So far i found SSDs more reliable then most people realize. As long as i rewrite them, every so often.

The trick is to get the lightly used 12TB drives from the people who just upgraded to 20TB drives.

Good luck finding those in Europe. Second hand stuff is being sold here at almost new prices (or above new!). Been so many times that it was not worth buying second hand because the price difference was barely 5 a 10% vs New + 2 year warranty + 2week return.

Over the last 2 years, I've noticed that I spend WAY more time carefully cataloguing my collections of digital media (games, anime) than actually experiencing those media.

Collecting is the fun part. Its the keeping up to date, that is the mindboggling dull part.

If you collect good comic series, well, ... trying to keep track of new releases that you want to read is annoying. Same with web series or web novels. Note: Non-US (EU) comic series are much harder to keep track off, as most programs focus on US comics.

The price premium for high capacity HDDs is the reduced number of required disks. Hence, less power consumption and simpler hardware to host the HDDs (e.g., 4 slots instead of 6 or 8).

But also increased risk... Its way easier to secure 8* 5TB drives, then 2* 20TB drives.

A 2* 20TB means you need to go Raid 1, where as 8* 5TB drives means you can go 2 Parity and 6 data drives. Aka 30TB usable and 4 times shorter rebuild times. And this also scales to the same effect...

Combine that with something like Unraid, where you do not strip your data over a raid, your chance of a catastrophic loss of all data is WAY less with more smaller drives + parity (and the benefit of more independent filesystems, then one big bulk one. Speaking from experience).

For companies, sure, those guys just make massive arrays with multiple redundancies but for self stores More and smaller is better (in my opinion), what is in major conflict because ironically, the cheapest drives in Germany are in the 14 to 20TB range. And you pay price premiums on the 12 to ... 4TB drives.

Interesting how prices different per region. Here in Germany, its mostly 14~20GB drives that are the cheapest, and 12~1TB are not worth buying.

https://geizhals.de/?cat=hde7s&xf=3772_3.5

Ignore the first SAS one, that is one seller on Amazon.

And second hand HDDs are even worse priced on places like ebay. To the point there is little reason to even look at second hand drives. Same applies to a lot of PC hardware, that tends to be more expensive second hand on ebay then brand new from stores.