this post was submitted on 10 Jan 2025
12 points (92.9% liked)

LocalLLaMA

2551 readers
10 users here now

Community to discuss about LLaMA, the large language model created by Meta AI.

This is intended to be a replacement for r/LocalLLaMA on Reddit.

founded 2 years ago
MODERATORS
SkySyrup
pax
noneabove1182
12
How much gpu do i need to run a 90b model (lemm.ee)
submitted 1 month ago by [email protected] to c/localllama
16 comments fedilink hide all child comments
 

Do i need industry grade gpu's or can i scrape by getring decent tps with a consumer level gpu.

you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 3 points 1 month ago

Not exactly. Digits still uses a Blackwell GPU, only it uses unified RAM as virtual VRAM instead of actual VRAM. The GPU is probably a down clocked Blackwell. Speculation I've seen is that these are defective and repurposed Blackwells; good for us. By defective I mean they can't run at full speed or are projected to have the cracking die problem, etc.