23
Beginner questions thread (self.localllama)
submitted 9 months ago by noneabove1182 to c/localllama

Trying something new, going to pin this thread as a place for beginners to ask what may or may not be stupid questions, to encourage both the asking and answering.

Depending on activity level I'll either make a new one once in awhile or I'll just leave this one up forever to be a place to learn and ask.

When asking a question, try to make it clear what your current knowledge level is and where you may have gaps, should help people provide more useful concise answers!

you are viewing a single comment's thread
view the rest of the comments
[-] [email protected] 3 points 3 months ago

Late to the party, I never got FOSAI working until I found LMStudio, but I have 2 questions:

  1. Is there any way I could utilize my GPU, a Radeon RX6800M (12GB VRAM)? I got Mistral-7B doing 5 tokens/s but it's all running on the CPU.

  2. Is there any model specifically for programming questions? This could be of immense help to my projects without having to ask ChatGPT.

[-] [email protected] 3 points 3 months ago

Have you tried the guide on AMD's site? It looks like it's for Windows, and I don't know what you're running. Plus, I use Ollama, so I probably can't be of much help.

For programing, my favorite is Dolphin-Mixtral, but I've had good results with Dolphin-Mistral and Llama2.

this post was submitted on 02 Oct 2023
23 points (96.0% liked)

LocalLLaMA

2118 readers
1 users here now

Community to discuss about LLaMA, the large language model created by Meta AI.

This is intended to be a replacement for r/LocalLLaMA on Reddit.

founded 1 year ago
MODERATORS