this post was submitted on 29 Oct 2023
16 points (100.0% liked)

LocalLLaMA

2179 readers
2 users here now

Community to discuss about LLaMA, the large language model created by Meta AI.

This is intended to be a replacement for r/LocalLLaMA on Reddit.

founded 1 year ago
MODERATORS
SkySyrup
pax
noneabove1182
16
I've started uploading quants of exllama v2 models, taking requests (huggingface.co)
submitted 10 months ago* (last edited 10 months ago) by noneabove1182 to c/localllama
0 comments fedilink hide all child comments
 

Finally got a nice script going that automates most of the process. Uploads will all be same format, with each bit per weight going into its own branch.

the first two I did don't have great READMEs but the rest will look like this one: https://huggingface.co/bartowski/Mistral-7B-claude-chat-exl2

Also taking recommendations on anything you want to see included in readme or quant levels

no comments (yet)
sorted by: hot top controversial new old
there doesn't seem to be anything here