this post was submitted on 22 Nov 2023
1 points (66.7% liked)

Data Hoarder

170 readers
1 users here now

We are digital librarians. Among us are represented the various reasons to keep data -- legal requirements, competitive requirements, uncertainty of permanence of cloud services, distaste for transmitting your data externally (e.g. government or corporate espionage), cultural and familial archivists, internet collapse preppers, and people who do it themselves so they're sure it's done right. Everyone has their reasons for curating the data they have decided to keep (either forever or For A Damn Long Time (tm) ). Along the way we have sought out like-minded individuals to exchange strategies, war stories, and cautionary tales of failures.

founded 1 year ago
MODERATORS
 

Hey friends, looking for advice on how I can possibly get the best of both worlds out of this situation...

Some Background:

I've been running a Plex server for about 2 years now. 2018 Mac Mini, DAS: Pegasus R4 with 4x 12TB drives configured in RAID5. It's worked great, no complaints. I'm nearing the 36TB capacity of my array and have been researching the best course to upgrade for a couple months. Very interested in the idea of building a proper server and moving to Proxmox so I can start doing some other homelab stuff with the machine as well.

Yesterday, I got a new job which starts in February, so I have a little bit of time to figure things out.

For the new job, I will need to build a RAID with 70-100TB of usable space. I'll be offloading 1TB and 2TB NVMe's to 2 destinations. Destinations are the HDD RAID I'm building as well as to a 4TB NVMe external drive. Everything must be run through an xxhash64 checksum. One downside, is I'm required to use MacOs for the job. The offload has to be pretty fast, without breaking the bank.

The cheapest/easiest solution I've come up with is:

- Buy Mac Studio M1 Max ($1,579 refurbished from Apple)

- Buy OWC Thunderbay 8 enclosure ($899.99)

-Fill the Thunderbay with 20TB IronWolf Pro drives ($1800 for 6x 20TB)

Pros to this setup: Relatively cheap given the parameters. No building anything or relying on my (lack-of) skill in building a DAS/NAS. Good warranties on the computer and enclosure if something goes wrong and needs replaced fast.

Downsides to this setup: I really don't like the idea of using Softraid. I would much prefer a hardware raid. The Thunderbay is big and power hungry, and I'd prefer something rack mounted.

I considered the idea of building a NAS...But then to move files from the source drives to the NAS, would I just need a 10Gb switch to establish a local network on my work truck? (I've never done this, only used DAS for work) I also have no idea what kind of read/write speeds I could expect from a NAS spinning the IronWolf Pros.

What I'm looking for is advice on possibly building a DAS or NAS that I could use for the job and, when the job is over, reuse as much of it as possible in my new Plex server / Homelab build. Ideally, it would be compatible with both MacOS and Linux, have hardware RAID control, minimum 8 bays, be rack mounted and power efficient.

Based on the price of the above setup, I'll say my budget is $2,500 excluding the cost of HDDs.

Things I already own that may or may not be helpful: 2018 Mac Mini i7 w/ 16GB DDR4 RAM, Pegasus R4 with 4x 14TB IronWolf Pro Drives

Thanks for your time and I appreciate any feedback you can give!

you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 1 points 11 months ago

Other commenters have good advice re software RAID. I’ll just reiterate you should investigate that. TrueNAS is what I use, and it’s great.

I’ll add some thoughts about the application. Is this for professional work, as in, someone is going to be counting on you to forever more have the files you mention available? If so; you have basically just one copy (I think) of the info, that’s what gets backed up in the NAS/RAID. if people are needing this data to be safe and protected against all manner of things that happen to data, you’ll need to consider another backup option, offsite, like a cloud provider. Something like Backblaze, AWS, or Google or something. Another great feature of software RAID solutions like truenas or unsaid is that they make it easy to setup rsync or something similar to automate these cloud (or local) backups for you.

If this is just you trying to make your workflow more convenient, that’s a bit different, but still know you’re a little light on the backup side of things. No matter what RAID type you use, it’s not a backup.