LocalLLaMA

2268 readers

1 users here now

Community to discuss about LLaMA, the large language model created by Meta AI.

This is intended to be a replacement for r/LocalLLaMA on Reddit.

founded 1 year ago

MODERATORS

SkySyrup

pax

noneabove1182

Noob here, what's the best overall model for getting started with ? (lemmings.world)

submitted 10 months ago by [email protected] to c/localllama

3 comments fedilink hide all child comments

I have an rx 6600 and 16gb of ram and an i5 10400f

I am using oobabooga web-ui and I happened to have a gguf file of LLama2-13B-Tiefighter.Q4_K_S .

But it always says that the connection errored out when I load the model.

Anyway, please suggest any good model that I can get started with.

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 3 points 10 months ago

I am using Mistral 7b Instruct for text summary and some light "assistant" type chatting for the last several months. I have been pleased at how accurate it is for my needs, especially given it's size.

I recall alot of trial and error to find models that were compatible with the version of llama-cpp-python that oobabooga uses (at any given time). GGUF should have made the model format (and therefore model selection) more simple, but i imagine there are still nuances that make it more difficult than it should be to find a working model for a noob.

Best of luck, let us know how it goes