LocalLLaMA

2268 readers

1 users here now

Community to discuss about LLaMA, the large language model created by Meta AI.

This is intended to be a replacement for r/LocalLLaMA on Reddit.

founded 1 year ago

MODERATORS

SkySyrup

pax

noneabove1182

Noob here, what's the best overall model for getting started with ? (lemmings.world)

submitted 10 months ago by [email protected] to c/localllama

3 comments fedilink hide all child comments

I have an rx 6600 and 16gb of ram and an i5 10400f

I am using oobabooga web-ui and I happened to have a gguf file of LLama2-13B-Tiefighter.Q4_K_S .

But it always says that the connection errored out when I load the model.

Anyway, please suggest any good model that I can get started with.

top 3 comments

sorted by: hot top controversial new old

[–] [email protected] 3 points 10 months ago (1 children)

I'd suggest checking out WolframRavenwolf on raddit, he does regular LLM tests.

I'm looking at Beyonder 4x7B, Mistral Instruct 2x7B, Laser Dolphin 2x7B, and previously used Una Cybertron.

[–] [email protected] 3 points 10 months ago

Hey thanks ! I'll check these out.

[–] [email protected] 3 points 10 months ago

I am using Mistral 7b Instruct for text summary and some light "assistant" type chatting for the last several months. I have been pleased at how accurate it is for my needs, especially given it's size.

I recall alot of trial and error to find models that were compatible with the version of llama-cpp-python that oobabooga uses (at any given time). GGUF should have made the model format (and therefore model selection) more simple, but i imagine there are still nuances that make it more difficult than it should be to find a working model for a noob.

Best of luck, let us know how it goes