Ask Lemmy
A Fediverse community for open-ended, thought provoking questions
Please don't post about US Politics. If you need to do this, try [email protected]
Rules: (interactive)
1) Be nice and; have fun
Doxxing, trolling, sealioning, racism, and toxicity are not welcomed in AskLemmy. Remember what your mother said: if you can't say something nice, don't say anything at all. In addition, the site-wide Lemmy.world terms of service also apply here. Please familiarize yourself with them
2) All posts must end with a '?'
This is sort of like Jeopardy. Please phrase all post titles in the form of a proper question ending with ?
3) No spam
Please do not flood the community with nonsense. Actual suspected spammers will be banned on site. No astroturfing.
4) NSFW is okay, within reason
Just remember to tag posts with either a content warning or a [NSFW] tag. Overtly sexual posts are not allowed, please direct them to either [email protected] or [email protected].
NSFW comments should be restricted to posts tagged [NSFW].
5) This is not a support community.
It is not a place for 'how do I?', type questions.
If you have any questions regarding the site itself or would like to report a community, please direct them to Lemmy.world Support or email [email protected]. For other questions check our partnered communities list, or use the search function.
Reminder: The terms of service apply here too.
Partnered Communities:
Logo design credit goes to: tubbadu
view the rest of the comments
Thanks, good ideas and recommendations!
The data is overshared, so several nodes store your data (as you store theirs. The storage requirements are roughly your data size times the number of over shares. The number of overshares is confugurable on a per data basis), this makes the availability high.
The data is encrypted with AES 256 (CTR) so nodes do not know what they are storing.
It's hard to invent new logos I guess, at least mine is blue and laying on the side...
It might feel like IPFS but the underlying tech is completely different, so it's not a DHT but uses dynamic links, which means that you can update your shared data without the need to re distribute the link file.
Ha ha point taken, I'll convert it to a better format.
i'm interested in the dynamic linking, what mechanism is used to stop situations like left-pad or the pypi incident where a file is removed replaced with a malicious alternative?
The idea is that nodes are trust-less, they do not know anything about the data.
An owner is authentified over an RSA handshake, so if the owner is not compromised, your request for updating a data will be rejected.
A malicous node though, must be both lucky (asked to share the data, so it can try to serve the malicious data) but also must have access to the link file so he can use the AES key to encrypt the bad data. This could happen if a malicious peron gets the hand on the link file, which sort of defeats the whole idea in the first place but it's an attack vector for sure.
So back to the drawing board again.
The public RSA key of the owner is already in the link file, I think I can use it to authenticate the data (say the original user uses his private RSA to sign a hash of the data and adds it to the payload).
Very good feedback, I thought I had it all covered. It seems like I can make a secure fix but I'll think a bit more about it first of course.
your reaction makes me more confident that this may turn into something interesting :)
i take it then that files must have some ownership information associated with them, to distinguish the author from a relay node? or is that just a private key.
Thanks!
The file does have an ownership information (the public RSA key of the owner), it's just that a malicious node doesn't show any proof of ownership of the data it's sending (which causes the problem). I think I have a straight forward solution but I'll ponder it a bit more before digging in :-) I'd like to have the possibility to download from several nodes at once which makes things a bit more interesting.
Would you mind if I copy paste this conversation over to [email protected] ? I guess here it will disappear in the abyss...
Cheers
Valmond
sure, do that. and good luck with this, i did something similar for a project once and as usual its those last 5% that are going to cost you 90% of the time.
Thank you!
Ya, there was a saying in video game dev circles, when you have finished 90% of a game, that's good, because now you only have the other 90% to do...
What was that project, if you'd like to share?
it was nothing official, just a toy file sharing thing using the chord dht algorithm. it consisted of a single binary that you fed with a directory and the IP of an extant member, and it would sync everything on the network into that directory.
no edge cases handled, so the 90% remained.
Interesting thing that Chord stuff.
Sounds like a good little tool to have there, it's always hell to share stuff among computers and phones and stuff! Guess overwriting from one side to the other (and vice versa) would be hell to fix in a user friendly manner though... If you don't go for a master/slave system, or git repo style...
yeah one thing we couldn't fix before losing interest was eventual consistency and authority. files would sort of flicker in and out while state was being propagated. i dread to think what kind of bandwidth use that thing would have when sharing large files.