this post was submitted on 02 Feb 2025
212 points (96.9% liked)
United States | News & Politics
2211 readers
992 users here now
Welcome to [email protected], where you can share and converse about the different things happening all over/about the United States.
If you’re interested in participating, please subscribe.
Rules
Be respectful and civil. No racism/bigotry/hateful speech.
Post anything related to the United States.
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
No worries. I've got some 20TB drives i can throw these on 😁
Now, running a model that large... Well, I'll just have to stick with the 8-13b params.
you can actually run the model, but it just goes very slowly… i run the 70b model on my m1 mbp, and it technically “requires” 128gb or VRAM - it still runs, just not super fast (though i’d say it’s useable in this case a about 1 word per ~300ms)