If anyone else wonders how that compares to llama.cpp's "2bit" quantization, here is the in-depth discussion: https://github.com/ggerganov/llama.cpp/discussions/4327
this post was submitted on 09 Dec 2023
11 points (100.0% liked)
LocalLLaMA
2268 readers
1 users here now
Community to discuss about LLaMA, the large language model created by Meta AI.
This is intended to be a replacement for r/LocalLLaMA on Reddit.
founded 1 year ago
MODERATORS