Data Hoarder

24 readers

1 users here now

We are digital librarians. Among us are represented the various reasons to keep data -- legal requirements, competitive requirements, uncertainty of permanence of cloud services, distaste for transmitting your data externally (e.g. government or corporate espionage), cultural and familial archivists, internet collapse preppers, and people who do it themselves so they're sure it's done right. Everyone has their reasons for curating the data they have decided to keep (either forever or For A Damn Long Time (tm) ). Along the way we have sought out like-minded individuals to exchange strategies, war stories, and cautionary tales of failures.

founded 10 months ago

MODERATORS

[email protected]

Question about YT-DLP and a mess .part-frags (alien.top)

submitted 10 months ago by [email protected] to c/[email protected]

5 comments fedilink hide all child comments

So I'll fully admit this is probably user error and not due to any bugs with the software. But I've been on a bit of a tear lately utilizing YT-DLP to download videos from YouTube. Now I'm not a stranger to this process, I'd used Youtube-DL successfully for years now. But I did recently take advantage of using it in conjunction Aria2c with the understanding that it would give me a bit more of a boost . Well...

I don't know if it's YT-DLP or Aria or my NAS or my Internet connection, but I went through to organize these downloads and found TONS and TONS of video files ending with ".part-frag###." There are thousands and thousands, and upon a cursory glance, seems to affect hundreds of video downloads. Each folder has dozens of these files and if it doesn't, it has split mp4/m4a. Rarely does a folder seem to have one clean video file like I would expect. I skimmed through and there seems to be finished files amongst the mess that seem to play fine, I can't be 100% sure though these files are as complete as they should be (surely the part-frags exist for a reason).

So firstly, what program or setting is causing this plague of fragments? Secondly, is there anything I can do to resolve theses files at this point? Yes, I can delete all the frags, but that doesn't give me any comfort in the integrity of whatever video file remains (and it won't solve the problem for future downloads). And as I archive to a txt file, I can't just re-download as YT-DLP considers them downloaded (I've tried redownloading to see it forces a re-check and cleans things up, but no dice). Is it just refusing to merge or delete for some reason?

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 1 points 10 months ago

What I would do is create a new directory. Download a few videos within subdirectories within that new directory using only YT–DLP. Don’t use aria and upon completion compare the whole videos from the new download session using only YT-DLP with the previous one (aria). I would check by playing the files side by side if possible and check the videos’ duration. Check the file sizes and run diff in a terminal with both files provided as arguments. If diff complains of a mismatch between the binary files they indeed differ in one aspect.