this post was submitted on 18 Nov 2023
1 points (100.0% liked)

Data Hoarder

24 readers
1 users here now

We are digital librarians. Among us are represented the various reasons to keep data -- legal requirements, competitive requirements, uncertainty of permanence of cloud services, distaste for transmitting your data externally (e.g. government or corporate espionage), cultural and familial archivists, internet collapse preppers, and people who do it themselves so they're sure it's done right. Everyone has their reasons for curating the data they have decided to keep (either forever or For A Damn Long Time (tm) ). Along the way we have sought out like-minded individuals to exchange strategies, war stories, and cautionary tales of failures.

founded 10 months ago
MODERATORS
 

https://i.imgur.com/FbVi4YK.png

With everything going on over at YouTube, what is everyone using as their backup solution?

I'm using TubeArchivist to maintain this VERY large collection.

Discord Link

top 21 comments
sorted by: hot top controversial new old
[–] [email protected] 1 points 10 months ago

My immediate reaction is “why?!?”.

Followed almost instantly by “well, someone should probably be doing it”. Then “I’m really glad it isn’t me”. So I guess thanks for your service.

[–] [email protected] 1 points 10 months ago (2 children)

I've been pretty selective, mostly YouTube channels that do 'Tech Documentary Stuff' cause I enjoy re-watching that. LGR would be a prime example.

However, I integrate all of these in my Kodi infrastructure, so only large channels have metadata online in the TVDB or the like.

For those without, I'd love some tips on something to generate Kodi compatible series metadata directly from YouTube.

[–] [email protected] 1 points 10 months ago (1 children)

With TubeArchivist, all the metadata is stored in an ES database and can be queried via API. There's already a Plex and Jellyfin plugin folks are using.

[–] [email protected] 1 points 10 months ago

Can you link me on the Jellyfin fork??

[–] [email protected] 1 points 10 months ago (1 children)

Quasi-related - what other tech channels are you following? I like LGR quite a bit. Just found The Serial Port - loving that one, building a dial-up ISP.

[–] [email protected] 1 points 10 months ago

Techmoan is great. As is Technology Connections.

[–] [email protected] 1 points 10 months ago (1 children)

Are you willing to share your collection?

[–] [email protected] 1 points 10 months ago

If you have a specific request that i have or want to send me 164TB of drives, sure

Before you ask - No, I don't have a full export of what I have in inventory yet. I'm not too savvy with ElasticSearch (yet) and as far as I'm aware, no one else has built a tool for export yet

[–] [email protected] 1 points 10 months ago

I'm using TubeArchivist as well. My collection is slightly larger than the OP's at this point. I collect everything from personally-enjoyable and re-enjoyable content, random stuff at risk during policy changes, to stuff from my early internet years that seems to slowly rot away on the internet. I also like the new themes

[–] [email protected] 1 points 10 months ago (2 children)
[–] [email protected] 1 points 10 months ago

I believe Google is removing some inactive accounts by December of this year.

[–] [email protected] 1 points 10 months ago

Removing inactive accounts is one thing, but they’re also cracking down on ad blocking. Could be a first sign that they’re going to be working on stopping YT downloads

[–] [email protected] 1 points 10 months ago (3 children)

No offense, but I don’t get it.

One could safeguard thousands of movies, song, art of any kind, that might disappear because some of it becomes lost media. Instead some prefer to backup content made for an ad platform.

I genuinely want to know what these 1m are about

[–] [email protected] 1 points 10 months ago

Because some of it is really good?

[–] [email protected] 1 points 10 months ago (1 children)

I think independent media has the biggest potential of becoming “lost media”. With potential no other copies of this media accessible online, I find it increasingly important to have this backed up.

You’ll always be able to find a copy of Forrest Gump online, but go through your liked videos on YT and see how many are lost forever.

[–] [email protected] 1 points 10 months ago

This is a big part of it IMO. Even 'streaming exclusive' stuff that see's no physical release gets pirated enmass so many copies exist. Few people copy YouTube content however, since it's already online, free, and easily accessible. ...Until the day it's not accessible of course.

[–] [email protected] 1 points 10 months ago

One could safeguard thousands of movies, song, art of any kind, that might disappear because some of it becomes lost media. Instead some prefer to backup content made for an ad platform.

Let's be clear here, movies, songs, and most art are also vehicles to serve you ads or generate admission/purchase revenue off the consumer. Like you criticize YouTube for being an 'Ad Platform' but elevate 'Songs' as if you've never heard of a radio station?

Funny enough, most films, music and other content are much better protected from being lost. They are mass produced, mass released, and have many many many pirated copies not to mention retail physical copies. YouTube on the other hand? People take for granted that it's there, online, easy to access from anything, and much of it doesn't easily have redundant copies on the internet so it vanishes in a flash when removed from YouTube.

[–] [email protected] 1 points 10 months ago

Whats going on at Youtube?

[–] [email protected] 1 points 10 months ago

Technically impressive! That’s a HUGE collection!

I am curious, though, as to your reason for doing this?.. Storing and maintaining 150 TB of small video files is going to be costly and time consuming…. While I do understand that it may be a good idea to archive some specific content that you may fear will otherwise be lost over time, in this case the volume is huge that you can never even watch all of it — let alone know if the content is something worth keeping?

No offense, everyone is free to spend their own time and money on whatever. But I do get curious what the plan is behind all this?

[–] [email protected] 1 points 10 months ago

Can I just use this as a downloader and then move the collection to Jellyfin? Or is there a way to link this to Jellyfin?

[–] [email protected] 1 points 10 months ago

I've been using a python script I wrote years ago and a config file with channel links / folder directories. it looks like I need to check out TubeArchivist after reading this thread.