this post was submitted on 12 Aug 2023
15 points (100.0% liked)

LocalLLaMA

2269 readers
4 users here now

Community to discuss about LLaMA, the large language model created by Meta AI.

This is intended to be a replacement for r/LocalLLaMA on Reddit.

founded 1 year ago
MODERATORS
SkySyrup
pax
noneabove1182
15
Oobabooga text-generation-webui now has ctransformers support! (github.com)
submitted 1 year ago by noneabove1182 to c/localllama
0 comments fedilink hide all child comments
 

PR 3313 has been merged with this commit

This is pretty great for those looking for performant llama.cpp/falcon/mpt offloading, and just in general a good CPU inference tool, wanting to use text-generation-webui

For those unaware, this is the ctransformers repo

And for anyone looking for an updated docker image, I have provided an image here on dockerhub

and as always my git repo with instructions can be found here on github

Happy inferencing :)

no comments (yet)
sorted by: hot top controversial new old
there doesn't seem to be anything here