this post was submitted on 14 Jul 2023
7 points (100.0% liked)

LocalLLaMA

2327 readers
1 users here now

Community to discuss about LLaMA, the large language model created by Meta AI.

This is intended to be a replacement for r/LocalLLaMA on Reddit.

founded 2 years ago
MODERATORS
 

Apologies for the basic question, but what's the difference between GGML and GPTQ? Do these just refer to different compression methods? Which would you choose if you're using a 3090ti GPU?

you are viewing a single comment's thread
view the rest of the comments
[–] markon@lemmy.world 1 points 1 year ago

Also llama.cpp offers very fast performance with the ggmls compared to using transformers, and sometimes faster than ExLlama.