LocalLLaMA

2265 readers

10 users here now

Community to discuss about LLaMA, the large language model created by Meta AI.

This is intended to be a replacement for r/LocalLLaMA on Reddit.

founded 1 year ago

MODERATORS

SkySyrup

pax

noneabove1182

A question about running LLMs with an AMD card (lemmings.world)

submitted 10 months ago by [email protected] to c/localllama

16 comments fedilink hide all child comments

On my machine I'm running opensuse tumbleweed and has the amdgpu driver installed. I use it for gaming and recently I've become interested in running LLMs. So I would like to keep a balance of both without compromising too much on performance.

I know that there are proprietary drivers for AMD cards but I'm hesitant to install it as I've heard that it performs less efficiently in games when compared to the open source driver.

I'm mainly confused about this ROCM thing. Is it not included with the opensource amdgpu drivers ? Or is it available as a separate package?

So what driver to use ?

Or perhaps, is it possible to run oogabooga or stable diffusion within a distrobox container (with the proprietary drivers) and still keep using the open source gpu drivers for the Host operating system.

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 5 points 10 months ago (2 children)

Has that changed recently? I've ran ROCm successfully on an RX6800. I seem to recall that was supported, the host OS (Arch) was not.

[–] taladar 2 points 10 months ago

When I tried it maybe a year or so ago there were four supported chipset in that version (5.4.2 I think) of rocm but I don't remember which card models those were since they were only specified in that internal chip name. Mine wasn't supported at the time (5700XT)

[–] [email protected] 2 points 10 months ago

No, GFX1030 is still supported.