It's A Digital Disease!

20 readers
3 users here now

This is a sub that aims at bringing data hoarders together to share their passion with like minded people.

founded 2 years ago
MODERATORS
101
 
 
The original post: /r/datahoarder by /u/AxlJones on 2025-04-02 19:32:16.

Does anyone have nice tool to scrap everything off their youtube account? Favorite videos, Subscriptions, uploaded videos etc?

102
 
 
The original post: /r/datahoarder by /u/ArtLongjumping487 on 2025-04-02 16:02:40.

Is there anyone one here with a good lead

103
 
 
The original post: /r/datahoarder by /u/daxliniere on 2025-04-02 18:10:37.

Hey everyone,

I am trying to purchase a pair of Toshiba MG drives for a small NAS. I had an 'experience' with a seller on Amazon (Toshiba drive sealed in WD anti-static bag, recovery software R-Studio showed extensive usage, and S.M.A.R.T. data had been reset! 😯 (thankfully refunded)), so I'm now wary of third-party sellers on Amazon/eBay, even if they claim them to be new and with warranty.

I'm looking at sizes between 12 and 16Tb and having trouble finding reasonable prices. There seems to be a monopoly in the UK, as Scan seems to be one of the only known companies able to get stock of these sizes.

Can anyone recommend a good deal from a reliable outlet, please?

Thank you for reading this far. :)

104
 
 
The original post: /r/datahoarder by /u/SuperCiao on 2025-04-02 17:34:09.

https://preview.redd.it/qv6j6cp6jgse1.png?width=866&format=png&auto=webp&s=70fde72ef385583dfb49acc9ad6a2a7ba839372a

Xreveal PRO stuck at 99% but operation success. It's normal?

105
 
 
The original post: /r/datahoarder by /u/Dnasrphotography on 2025-04-02 17:06:56.

I'm photographer and currently have about 6.5TB of data, nothing crazy yet. I have a 4TB SSD which I edit off of that's formatted exFAT, a Seagate 8TB for archiving data that's exFAT (which is a backup of my SSD and everything) and another newer Seagate AFPS 8TB that right now is just a second backup of the SSD. I have the exFAT HDD and 4TB SSD backed up on Backblaze as well. I'm wondering if I should backup the first Seagate exFAT HDD to the Seagate AFPS HDD and have Best Buy (I have a plus membership so this would be free) offload the data and reformat the ExFAT Drive to AFPS and reload the data. I do have a Windows Laptop, but seldom use it and figured the SSD could be kept as exFAT for that reason. I wanted to get some suggestions, feedback. Wasn't sure if having both HDD drives as AFPS would make space on the drives more optimized and the read/writing speed a lot faster or worth the trade off in my situation.

106
 
 
The original post: /r/datahoarder by /u/kschaffner on 2025-04-02 15:47:00.
107
 
 
The original post: /r/datahoarder by /u/kcombinator on 2025-04-02 14:52:06.

I’m trying Archivebox, and it has a lot of nice ideas, and it is completely inadequate. I need something that can fetch in parallel. I have ~25k unique bookmarks dating back almost 15 years and just want to preserve what I still can. Does anyone have any recommendations?

108
 
 
The original post: /r/datahoarder by /u/TheFeshy on 2025-04-02 14:32:28.

I've seen the "Do I need ECC RAM" question come up from time to time, so I thought I'd share my experience with it.

The common wisdom is this: cosmic ray bit flips are rare. And the chances that they happen in a bit of memory you actually care about are rarer still. And from a data hoarder perspective, the chances that they occur in a bit of memory you're just about to write to disk are vanishingly small. So it's not really worth the jump in price to enterprise equipment, which is often the only way to get ECC RAM (Even when the RAM itself isn't much more expensive.)

Well, I've been data hoarding since the late 90's, and all but the last 5 on consumer-grade, non-ECC equipment. And I've finally gotten around to using a program that will go through my hoard, and compare it with existing Linux ISO torrent files, to see if I've got the same version. Then I can re-share stuff that's been sitting around for a decade or more. It's been a fun project.

This program allows you to identify less-than-perfect matches, in case you've got a torrent with many Linux ISOs and only one doesn't match, or there are some junk files you've lost track of, or whatever.

I was finding that, sometimes, I'd get a folder of Linux ISOs where they all match except one. And stranger still, I'd get some ISOs that were showing 99% match, but only had one file! So I started looking into this, and did a binary comparison of a freshly downloaded copy and my original. I found they didn't match by a single byte! But all these files were on ZFS initially, and now Ceph - both check for bitrot on every read, and both got regular scrubs to check as well. So how could I be seeing bitrot?

What I found is this (four random examples from my byte by byte comparisons.) See the pattern?

Offset    F1 F2
--------- -- --
5BE77DA0  29 69
1FF937DA0 A8 E8
234777DA0 24 64
29DE37DA0 0B 4B
2B7537DA0 3A 7A
2F88D7DA0 9F DF

If you do, consider your geek card renewed. The difference between the byte from the first copy and the byte from the second copy is always 0100 0000.

I notice another thing: All the files have write dates in 2011 or 2012.

That's when it hit me: I RMA'd a stick of ram about that time. Late 2012, according to my email records.

I had been doing a ZFS scrub, and found an error. Bitrot! I thought. ZFS worked! During the next scrub, it found two such errors, and I started to worry about my disks. Then it found more in a scrub later, and I got suspicious. So I ran memtest on the RAM for 12 hours, and it showed no errors. Just like when I tested it when it was new. Maybe it really is my disks then?

Then I did another zfs scrub, which found more errors, so out of paranoia I ran memtest for 48 hours. That was many loops through all its tests, and it found 2 errors in all those loops. So most times it did the whole loop fine, but sometimes it failed a single test with a single error.

That was enough to replace the RAM under warranty, and I got no more scrub errors on the next scrub. Problem solved.

Except... except. Any file written during that time was cached in that RAM first. And if the parity checks that ZFS does are done on the RAM copy of the data with a bad bit - say, a single bit in a single byte that sometimes comes up 1 when it should be 0 - the checksum data is done on bad data. So ZFS preserves that bad data with checksum integrity.

A cosmic ray flip at just the wrong time would be a single file in your hoard - maybe you'd never notice. The statistical analysis at the start of this post is true.

But a subtly bad stick of RAM? It might sit in your system for years - two in my case - and any file written in those two years might now be suspect.

And any file with a date later than that is also suspect, since it might have been written to, modified, copied, or touched from a file in your suspect date range.

I've found dozens of files with a single bad byte, based on the small percentage I've been able to compare against internet versions.

And the problem is not easy to sort out! I have backups of important stuff, sure - but I'm now looking at thirteen years of edits to possible bad files, to compare to backups. And I don't keep backup version history that old. And for Linux ISOs, while many files are easy to replace, replacing every file is a much bigger task.

So, TL;DR: Yes, folks, in my opinion you want ECC RAM on your storage machine(s.) Lest you wind up looking at every file written since the first Obama administration with suspicion, like I now do.

109
 
 
The original post: /r/datahoarder by /u/DryDealer3816 on 2025-04-02 14:14:20.

What would you do?

I thought of this because I'm currently downloading Professor Leonard's Calculus playlist because I don't want it to go anywhere before I have a chance to watch it 🥺. So if they announced YouTube is getting wiped in a year (and they didn't do anything to try and stop the obviously incoming download frenzy) what would you do?

I'm not sure if I'm allowed to make a post like this here, if I'm not, my apologies. I didn't see anything in the rules that would suggest this kind of post is forbidden.

110
 
 
The original post: /r/datahoarder by /u/Super5948 on 2025-04-02 13:10:31.

I'm looking at getting 2 new drives, 10-20TB each to keep the exact same data on (everything I have). But I'm struggling to make sense of how the different brands and models (such as WD colors) actually stack up against one another. The marketing lingo is extremely thick everywhere I look.

Previously I've owned a pair of smaller WD Blacks for the same purpose as they promised good reliability and read/write performance, but they were extremely expensive and seem even more so currently.

Right now I'm looking at two Toshiba MG10 Enterprise 20TB drives as they promise higher speeds than my blacks and much larger storage at a lower price. Is there a catch?

Any help or suggestions would be greatly appreciated.

111
 
 
The original post: /r/datahoarder by /u/Scoot_Scoot96 on 2025-04-02 12:51:32.

Hey all,

My Pixel 6 died last month, I managed to get the motherboard repaired to salvage the data, I got the phone back today. In an effort to stop this from happening in the future I want to setup an incremental backup system (I would've replaced the phone if I didn't loose all my data).

However, it seems that there is no real one stop shop for a backup solution on android.

I'm also not sure what data I shuold be saving (Messages, photos, videos, documents, list of installed apps??)

Does anyone know of any backup software for android, preferably suited for NAS?

What data does everyone backup?

I'm not against rooting the phone, but I would rather not.

112
 
 
The original post: /r/datahoarder by /u/Silent_Performer_123 on 2025-04-02 12:39:33.

I'm looking for alternatives to scrape public posts from Instagram profiles. I previously used Instaloader and occasionally Gallery-DL, but these tools are increasingly returning errors. Does anyone know of open-source tools that can effectively scrape public posts from specific profiles and export the metadata from each post in a separate JSON format?

113
 
 
The original post: /r/datahoarder by /u/acriax on 2025-04-02 11:13:29.

I love how quickly I can find the right files on my computer using this app. I've had it a long time, but I remember that when I first started using it, I could just bring it up and search for something and it would show it. Nowadays when I start it up and search, it shows the results, but then it clears out everything and a green loading bar in the bottom right corner goes up slowly, then after maybe 30 seconds it works as usual after that until I reboot the computer. I've scoured through the settings and asked ChatGPT for help on if there's any indexing setting I've missed, but from what I can tell it all looks correct. So now I'm wondering if this is normal for this app now? Thanks.

114
 
 
The original post: /r/datahoarder by /u/Red_Gyarados88 on 2025-04-02 10:55:38.

New to the sub, new to the urging feeling to back up my stuff. I'm looking to purchase or build backup / occasional use storage device for media. Would like it to be somewhat portable, not fixed on SSD or HDD and kind of like the idea of building something 1-2 tb so that in 3-5 yrs I can add it to a larger enclosure of some kind.

I know I'm overthinking this and could use some ideas and help. Does anyone have suggestions for a 100$, 1-2 tb storage reliable, durable external all in one or diy build?

Thank you!!

115
 
 
The original post: /r/datahoarder by /u/Methbot9000 on 2025-04-02 09:22:19.

This has happened to me a few times now. I try to rip a CD on my main machine (a Macbook Pro M1 Max) with an external DVD drive and there will be a certain track that it gets stuck on. I usually rip with iTunes/Music for simplicity, but when this happens I also try using XLD and it gets stuck on the same track that iTunes did. But then if I plug the exact same external DVD drive into my old 2015 Macbook Air (Intel i5) the CD rips with no issues at all.

Why would this happen? The drive is a random brand (Froibetz) usb drive from amazon.

As you can imagine the two machines are on different OS, but I don't think that's the issue because the Air is currently on Monterey, and I believe the MBP which is now on Sonoma was having the same issue back when it was running Monterey.

116
 
 
The original post: /r/datahoarder by /u/Dayyy021 on 2025-04-02 08:28:40.
117
 
 
The original post: /r/datahoarder by /u/tashlouise94 on 2025-04-02 08:25:08.
118
 
 
The original post: /r/datahoarder by /u/Technical-Formal-376 on 2025-04-02 06:27:57.

By that I mean a program to download from webnovel.con preferably mobile friendly

119
 
 
The original post: /r/datahoarder by /u/SchizophrenicScreams on 2025-04-02 03:38:49.

Been torrenting for a couple weeks now and finished downloading all seasons of aqua teen hunger force. All the episodes are 1080p HMAX but 2gbs or less per season. I just found another season pack for 20gb each. What exactly is the point of getting those for higher storage when these exist? Is there something I'm missing?

Edit: you reddit nerds need to stop bitching when someone wants to learn about your hobbies 😂 yall came out the womb knowing about webrips?

120
 
 
The original post: /r/datahoarder by /u/heritage95 on 2025-04-02 03:28:58.

I'm looking to streamline my htpc setup by moving some drives from an old pc server into an enclosure on my main pc and adding a drive.

The enclosure that just arrived this week is a Terramaster D4-320. I also just got a Seagate Expansion Desktop 24TB that (I presume) has a Barracuda drive in because it was manufactured in 2025. It is unopened so I can return it.

The main use case is an HTPC storage for myself. My main PC might be on for 12-16 hours a day but I will not be read/writing to the drives 24/7. I love getting stuff but as life gets busier, I don't have enough time to watch. I am hopeful that the all the reliability specs aren't as relevant because I'm not running a shared server.

My concerns are that my drives aren't "recommended" by Terramaster for use as DAS. The Barracuda 24TB is not whitelisted by Terramaster. Nor are the 2 drives in my old PC server. In fact, one of my old drives (St4000dm004 and) is "not recommended" (maybe bc it's 5400 rpm?) and the other (ST2000DM001) is not listed at all.

I'll be content if the HDDs work to their specs even if it's not as fast as the enclosure could operate. But if there are potential adverse consequences like not recognizing the drives or device failure, I'll have to rethink my combination of drives/enclosure.

What am I looking at if I place these three drives into the Terramaster enclosure?

121
 
 
The original post: /r/datahoarder by /u/ArcFarad on 2025-04-02 02:46:51.

(Apologies if this has been addressed before, but I tried searching the sub and didn't see anything)

I want all of my various machines to back up to my Synology sitting on my local network. I'm then going to back up the Synology to iDrive.

I can use any backup software from macOS/Windows/Ubuntu to the NAS, and then I'll be using iDrive from the NAS to the cloud.

My question is this: How do I handle versioning between these two backup steps? Almost all software (including iDrive) support incremental backups, old versions, etc.

It seems weird to have a database of multiple file versions running from each machine to the NAS, and then, what? Multiple versions of the multiple-versions-databases to iDrive? I feel like I'm doing something wrong.

Any suggestions are welcome!

122
 
 
The original post: /r/datahoarder by /u/Big-Consideration337 on 2025-04-02 02:26:42.

I just set up my bad and uploaded all my data, I’m using it as a home media server. What are some good ways I can back up all the data preferably online as buying the synology set me back a little and I’m not itching to buy more external storage devices at the moment

123
 
 
The original post: /r/datahoarder by /u/Tacitquazer on 2025-04-02 01:39:44.

Hello Reddit,

Recently an SSD of mine failed, and I lost a lot of media. I was able to take it into a shop, and they got a recovery of all the files, but stored in folders based on filetype, not relevancy. The problem with this is that it includes all of the system images, and icons that are found all around the computer. Is there any way I can do a bulk sorting and filtering of filesize, aspect ratio, or any other property?

I was looking for the best sub-reddit to post this in, and this seemed to be a decent option. If none of you guys are able to help, could I get some suggestions on where I could turn next? Thanks!

124
 
 
The original post: /r/datahoarder by /u/Live_Ostrich_6668 on 2025-04-02 00:56:51.

For those out of the loop, back in December 2020, three big academic publishers—Elsevier, Wiley, and the American Chemical Society—sued Sci-Hub and LibGen in the Delhi High Court (High courts are only second to the Supreme Court in India, which is the highest judicial authority in the country) by filing a 'copyright infringement' lawsuit against them. The court then asked it to pause any new uploads, and the website has been inactive since then. There have been multiple hearings since then, but the case keeps on dragging with hopes dwindling for a verdict anytime soon.

Here's the full timeline of the hearings that keeps getting postponed:

https://www.reddit.com/r/scihub/s/otkFGlQKqh

Now the question is, was this exactly the kind of outcome that the publishers were trying to achieve? If yes, then why did they selected India? Why not any other country?

Lastly and most importantly, why did Alexandra Elbakyan (the founder) even chose to comply?

From Sci-Hub's wiki page:

In order to have a better chance of winning a lawsuit presented against her and Sci-Hub by Elsevier in India, Elbakyan complied with a preliminary injunction issued by an Indian court, and suspended in 2021 upload of new publications, except for some batch releases of content.

https://en.m.wikipedia.org/wiki/Sci-Hub

So it seems like both the founder and the publishers thought they had a 'better chance of winning' in India, which is quite baffling and incomprehensible.

What are your thoughts on the whole issue, and what do you think is going to be most likely outcome in this battle?

125
 
 
The original post: /r/datahoarder by /u/No_Account_5605 on 2025-04-02 00:49:23.

I mean ones that go for £15 on amazon Also are they region locked Thanks

view more: ‹ prev next ›