this post was submitted on 27 Jan 2024
24 points (96.2% liked)
LocalLLaMA
2265 readers
10 users here now
Community to discuss about LLaMA, the large language model created by Meta AI.
This is intended to be a replacement for r/LocalLLaMA on Reddit.
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Local LLMs usually run using CPU only
You absolutely can use GPU. Try the auto installer and select AMD when the option shows up. The documentation should have more info.
https://github.com/oobabooga/text-generation-webui
Interesting. Thanks for sharing!
I want to use software like stable diffusion and oogabooga. Don't they utilize GPU ?
Ooga Booga is just a webui for local text gen LLMs, that usually only utilise CPU
SD, on the other hand, does utilise GPU. I haven't tried it but I don't see why it won't run with the open source AMD drivers, they are good. Just try it yourself, everything is free and you would only lose some time in the worst case scenario
All the ones I've used so far are able to use the GPU but it has to be enabled in the app settings. I mostly use LM Studio and it flies on my nvidia 3060. Doesn't seem to have options for AMD GPUs though, unless I'm mistaken.