this post was submitted on 03 Apr 2025
1 points (100.0% liked)

It's A Digital Disease!

20 readers
3 users here now

This is a sub that aims at bringing data hoarders together to share their passion with like minded people.

founded 2 years ago
MODERATORS
[email protected]
1
Getting all website content programatically (no deep search) (zerobytes.monster)
submitted 1 day ago by [email protected] to c/[email protected]
0 comments fedilink hide all child comments
 
The original post: /r/datahoarder by /u/silverhand31 on 2025-04-03 09:49:57.

Hi guys, im looking for a way to download the whole website (just homepage is fine) given url programmatically.

I know I can open website right click save page as, and everything gonna be store locally. But i want to do that with programming.

I dont need fancy speed, so if there is existing tool use with CLI, it would fine to me.

I was thinking about download it via web.archive.org too (i dont need that up-to-date content). I hope that there are tools for that?

Do you have any hunch how im going with this?

Thank.

(i have proxy/vpn to avoid blocking)

no comments (yet)
sorted by: hot top controversial new old
there doesn't seem to be anything here