I use rclone with encryption via cloud and also endpoints backup.
this post was submitted on 15 Nov 2023
0 points (50.0% liked)
Data Hoarder
170 readers
1 users here now
We are digital librarians. Among us are represented the various reasons to keep data -- legal requirements, competitive requirements, uncertainty of permanence of cloud services, distaste for transmitting your data externally (e.g. government or corporate espionage), cultural and familial archivists, internet collapse preppers, and people who do it themselves so they're sure it's done right. Everyone has their reasons for curating the data they have decided to keep (either forever or For A Damn Long Time (tm) ). Along the way we have sought out like-minded individuals to exchange strategies, war stories, and cautionary tales of failures.
founded 1 year ago
MODERATORS
The 2 in this rule isn’t clear: 2 different media?
Why is it important if it’s DVD & HDD or SSD & HDD?
For home use, it's just two different devices that can be the same type, like harddrives in two servers, but not redundant data storage in one device, like RAID or just having two copies of files on the same drive. For corporate, most will probably interpret it as two different media types, like harddrive and tape. You want them seperate to prevent accidental deletion of files, ransomware and such.
Diversity in Failure Modes: Different storage media have different failure modes and life expectancies. For example, a hard disk drive (HDD) might be susceptible to mechanical failure, while a solid-state drive (SSD) might have limitations in terms of write cycles. By diversifying the media types, you reduce the risk that a single failure mode (like a power surge, mechanical wear, or temperature sensitivity) could compromise all of your backups.
Reducing Common Points of Failure: If all copies of your data are stored on the same type of device, they may all be vulnerable to the same type of failure. For example, if you have all your backups on different HDDs from the same manufacturer and there's a manufacturing defect, all your backups could fail simultaneously.
Technology and Ageing: Different technologies age and become obsolete at different rates. By using multiple types of media, you're less likely to find yourself in a situation where all your backups are stored on outdated or unsupported technology.
Physical and Environmental Threats: Different types of media have varying levels of resilience to physical and environmental threats like fire, water damage, magnetic interference, etc. By diversifying, you increase the chances that at least one of your backup mediums will survive a catastrophic event.
Data Recovery Options: In the event of a failure, different types of media may offer different data recovery options. Some media might be easier or more cost-effective to recover data from than others.
CSVs ftw
Mandatory to follow. Just thinking about it makes me want to check everything again
I haven't set up a proper backup yet because I'm not sure how much storage I'd need. My NAS has ~34 TB of usable space and I don't want to spend too much money. The most critical data fits within 1-2 TB so that should be doable, and I really need to get around to it!
In my case most of the backups are done to local drive and to Acronis Cloud, some of these backups tasks are also going to external drive and FreeNAS share. Lastly, quarterly backups are going to Glacier Deep Archive in encrypted containers.
What do you use for encrypted containers? Veracrypt?
I have 3:2:1 for my crucial data ( pictures of family and travels and digital documents like tax returns). Basically one main copy another copy on an old nas with shucked drives not connected to the internet and one cloud copy. It is worth it because I would hate hate to lose that data.
I have 2:1 for my media. Just a local copy . If the apartment goes up in flames or a freak lightning burns it down I will have to re-download it again or I will live without it and ghats fine. For a long time the media had no backup but just raid and snapshots to protect against hard-drive failures and dumb user errors.
It's all about your means and risk appetite.
Yes, I follow the 3-2-1 backup rule
1st copy is my working data
2nd copy is to an external HDD
3rd copy is offsite to a cloud site
My backup copies are incremental, using a backup service (Arq)
I also have device backup, with my data auto-sync’d between devices (Mac and iPad)
You may laugh but I burned all my super important stuff(hi res scans of photos going back to wheN they got off boat and lot S of other pics and vid’s ) onto 50 Blu-ray disks. Since I have no friends I’d trust I found a piece of 6 inch pvc pipe glued cap on one end and threaded plug on other leak checked it put disks in it and buried in the yard a foot and half deep with p gravel around it. My main worry is Forrest fire I live in woods. Only thing in my backyard is me and the deer and a big ass raccoon. It doesn’t need updating. It’s been underground 5 yrs now I should dig it up and have a look I suppose. My daughter that lives a few hundred miles away knows where it is incase I get ate by raccoons. But not what’s inside of it. She has the originals. So not a good place for digital copies. There’s also a note in my will for someone to go dig it up. It’s not hard to find has a paver stone as x marks spot.
So it's 6 inch PVC... how long is it? We need to know it's capacity in TB.
Have you considered encrypting the data and storing it with a cloud provider?
No I don’t want to have a forever reoccurring bill. And who knows ten years from now the encryption method we use now can be cracked by some method we don’t know of now. Or the password is lost to time. And I dont trust the cloud much I use it for some stuff but not the Crown Jewels. My biggest fear is that I die they sell house new owner finds it that’s why I have it in my will to dig up before house is sold. But I’ll be dead and won’t care also. But they don’t need to see a video of my daughter when she’s two months old pooping on me.
you definitely sound like someone who lives in the woods
Optical disks buried in a back yard... do you check them for disk rot? If you aren't testing the restore, you don't have a backup!
I don't have too much critical data to backup. I have a low power thin client (lenovo m720q with a 2tb sata drive) I park at a friends house on their network. In exchange I let them borrow a bit of space on it too.
Kinda-sorta...? The most sensitive data was more like 5-3-1: 5 copies (in my twin laptops, in my HDD-based NAS, one in my mobile external HDD, and one 'offsite' with my family back in my hometown). Then for my work data it was 3-2-1: main laptop, my NAS backup, and offsite (updated every holiday). For entertainment which are mostly loot from the seven seas, 2.5-1 rule: I stash bulk of them in my NAS (which I mostly stream to my laptop) and copy what I need to share with folks to the mobile HDD) and the other copy is also offsite
For everything that is personal, such as photos, videos, file backups, etc, I follow 3-2-1.
I have a raid1 array with all my files, and any new files are pushed to aws S3 in a weekly basis. its weekly as i may have plenty of GB of photos and videos after a big trip and with my current internet speed 24 hrs may not be enough to upload everything :/
For cold storage stuff, aka everything until last year I also copy it at an old nas at my parents house that is only connected to power and network when i need to add the backups and 2-3 times per year otherwise.
It's also good to keep a raid aray protected with a surge protector and UPS.
3-2-1 is a suggested default backup strategy.
3 backup copies. 2 different types of media. 1 copy stored remotely.
I use a mix of backup strategies. 8-3-3 to 1-1-0, depending on what it is I backup. For example, I have two internal SSDs on my PC. Every boot a new updated versioned rsync snapshot of the primary SSD is automatically created on the secondary SSD. Only new and modified files are actually copied. Files present in the previous snapshot are simply hardlinked. So each snapshot looks like a full copy, but takes up very little storage and is very fast to make.
In addition I have two large DAS, a small NAS, a small cloud account and various external drives and devices, some stored with relatives. They are also used for backups.
4-3-1 for important data
2-2-0 for data that would be inconvenient to lose
"YOLO" for everything else
Could you explain the numbers?
I don't. My backups are encrypted on my cloud provider. My cloud provider takes care of the systems, snapshots and files so I highly doubt Google will lose my files. But besides that it's always a good idea if you backup locally to follow the 321.
yes, i am over 100TB and backup everything on two sets of external disk arrays. one is at my house only powered when i perform my monthly backup and the other is at the in-laws house. i swap the two sets every 3 months
here are the enclosures i use
https://www.amazon.com/gp/product/B07MD2LNYX. between all my backups i have 4x of these enclosures and 32x drives
backup 1
--> 8 bay USB disk enclosure #1: filled with various old disks i had that are between 4TB and 10TB each. the total USABLE space is 71TB
--> 8 bay USB disk enclosure #2: filled with various old disks i had that are between 4TB and 10TB each. the total USABLE space is 68TB
Backup 2
Exact duplicate of backup #1.
i have windows stable bit drive pool to pool all of the drives in each enclosure. i also use bitlocker to encrypt the disks when not in use.
I'm not even sure what my rule is.
I use iCloud, OneDrive, and Sync on all devices. Photos are in iCloud.
All cloud services are set to fully download to an M1 Mac Mini that functions as file server/Plex server.
That Mac Mini has a Sabrent 5 bay enclosure attached with an SSD and two hard drives in it.
Everything on the SSD gets copied to the first of the hard drives nightly, and then that hard drive is cloned to the second drive.
Used to have a fourth drive for Time Machine but it died, will replace it with a Black Friday deal, maybe.
The entire Mac Mini is backed up by Backblaze, even the Plex Media.
I have random old hard drives lying around with the most important stuff (photos, docs) back up, and I am in the middle of preparing M-Disc backups of this as well.
Future plans:
Still might get a NAS if I have money burning a hole in my pocket.
Going to get a third 8TB hard drive on Black Friday and do a monthly backup, stored at the in laws, rotating with one of the drives here. I used to do this with a safe deposit box but it cost too much and was too much of a pain in the ass to get to.
I don't follow it for everything, but I follow it for stuff I don't want to lose
At work, yes and hell yes.
At home, only irreplaceable data like photos and personal documents. The media is not as important because if I downloaded it once, I can download it again. I don't need the same RTO so I can reimage a box and bring down data in a catastrophe. I'm still rockin' an LTO4 tape drive at home.
photos and personal documents, some other stuff. is triple back up.
then other data.
has 1 copy some where due to size of data to re downloaded.
other is takes me 5 mins to re downloaded.
If you have reduntant storage spaces and places to store the 3-2-1 comes almost automatically. Once you start taking backups there is no need to not backup it to several places if the data is truly important.
- NAS with R1 of 2x SSDs for all my important data (NAS also does less important data on some R10 HDDs)
- Second NAS with similar layout as the first, back up daily.
- External SSD backup of NAS1 important data (about once a month and kept in my car)
- Offsite NAS to specifically back up important data once a week)
- Wife’s PC keeps a local copy of raid important data and stays sync’s daily.
Mine is more 4-2-2, but yes.
No. My media data is not important enough
Important photos are triple cloud backed up.
#*POOF*
OK who rubbed the lamp? Of course I follow the rule religiously!
It's something that everyone should follow.
It is also something that everyone can absolutely afford to follow, for at least some of their data.
Take this for example.
A 5 pack of 128GB USB drives is dirt cheap.
Encrypt them all, keep two plugged into a USB hub. One in a drawer, one you keep in your car (who cares if it dies) and store another in a safe deposit box/friend's/family member's house.
If your house burns down you get to keep that 128GB of data, if you want more, pay more, but this is available for under $45 so yes everyone should do it for at least some of their data. There's no excuse.
My music, photos, and documents are backed up remotely (Dropbox).
Everything else is just backed up to another machine.
For me it's cost. 80tb wouldn't be cheap enough for me.
I get down on my knees every month just to pray that I don't need to use my back ups. Then, when the inevitable happens, I get down on my knees and pray thanks that I have my back ups.
More religious than anything else in my life. I have had numerous events occur over the past 2 decades and can confirm that restoring is so much easier and better than installing from scratch. Also data( in my case the usual pictures/movies/documents/etc) are at least duplicated on other media/devices/etc.
3-2-1 is the minimum I follow for anything important.
1 copy is the working data, 1 copy is a full system image stored on a NAS with incremental backups done nightly with Veeam, and 1 copy is on Backblaze B2 with incremental backups done nightly with Restic,
I follow it for the most critical data, other data get just one copy (but those data is not important to me)
I follow it personally, backup locally to a WD drive, and cloud is idrive
I’m a photographer with almost 25TB of photographs.
Primary storage: diy truenas On-site backup: off the shelf branded nas Off-site backup: cloud storage.
Just a note: any automated backup you need to be 100% sure you have set it up to not sync deletions.
Not yet. My 2nd form of media will be Blu-ray 100GB Discs, and second location will probably be another house 30 minutes away. I DO have about 3-4 copies of my most important data.
Locally, I have RAID on my NAS, my sentimental stuff is mostly synced with other systems through seafile (similar to nextcloud), and is also backed up to backblaze.
For everything else, it's just RAID.
I do main data at home, external HD as backup offsite (I update maybe 1-2x a year otherwise it's turned off/unplugged), and any new files not on the backup are in cloud storage + local HD, separate from main data.
If either drive failed I'd just order a new one since the odds of both failing within a couple days would be low.
I have a 2-2-0 for now. The problem is with 100 TB of data, it's hard to find an offsite back up that is reasonable priced.
Everyone else seems to have parents or these things called "friends" that they can ask to hold onto. Wonder where I can find them.
There are excellent articles that go over all this. Do a something search.
Bottom line, yes, you should at least do 3-2-1 methodology. More than that is gravy.
view more: next ›