this post was submitted on 23 Nov 2023
1 points (100.0% liked)

Data Hoarder

24 readers
1 users here now

We are digital librarians. Among us are represented the various reasons to keep data -- legal requirements, competitive requirements, uncertainty of permanence of cloud services, distaste for transmitting your data externally (e.g. government or corporate espionage), cultural and familial archivists, internet collapse preppers, and people who do it themselves so they're sure it's done right. Everyone has their reasons for curating the data they have decided to keep (either forever or For A Damn Long Time (tm) ). Along the way we have sought out like-minded individuals to exchange strategies, war stories, and cautionary tales of failures.

founded 10 months ago
MODERATORS
 

So I'll fully admit this is probably user error and not due to any bugs with the software. But I've been on a bit of a tear lately utilizing YT-DLP to download videos from YouTube. Now I'm not a stranger to this process, I'd used Youtube-DL successfully for years now. But I did recently take advantage of using it in conjunction Aria2c with the understanding that it would give me a bit more of a boost . Well...

I don't know if it's YT-DLP or Aria or my NAS or my Internet connection, but I went through to organize these downloads and found TONS and TONS of video files ending with ".part-frag###." There are thousands and thousands, and upon a cursory glance, seems to affect hundreds of video downloads. Each folder has dozens of these files and if it doesn't, it has split mp4/m4a. Rarely does a folder seem to have one clean video file like I would expect. I skimmed through and there seems to be finished files amongst the mess that seem to play fine, I can't be 100% sure though these files are as complete as they should be (surely the part-frags exist for a reason).

So firstly, what program or setting is causing this plague of fragments? Secondly, is there anything I can do to resolve theses files at this point? Yes, I can delete all the frags, but that doesn't give me any comfort in the integrity of whatever video file remains (and it won't solve the problem for future downloads). And as I archive to a txt file, I can't just re-download as YT-DLP considers them downloaded (I've tried redownloading to see it forces a re-check and cleans things up, but no dice). Is it just refusing to merge or delete for some reason?

you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 1 points 10 months ago

it can speed up downloads on sites that limit per connection so downloading fragments 8 at a time speeds things up considerably

that's what I've used it for