this post was submitted on 05 Mar 2025
15 points (100.0% liked)

LocalLLaMA

2662 readers
2 users here now

Community to discuss about LLaMA, the large language model created by Meta AI.

This is intended to be a replacement for r/LocalLLaMA on Reddit.

founded 2 years ago
MODERATORS
SkySyrup
pax
noneabove1182
15
Qwen/QwQ-32B · Hugging Face (huggingface.co)
submitted 4 days ago by [email protected] to c/localllama
5 comments fedilink hide all child comments
all 6 comments
sorted by: hot top controversial new old
[–] [email protected] 2 points 4 days ago (1 children)

insane, absolutely insane

[–] [email protected] 1 points 4 days ago (1 children)

Why insane? For quality, speed, size? I find the coder 1.5b and 3b light and good

[–] [email protected] 3 points 4 days ago

It matches R1 in the given benchmarks. R1 has 671B params (36 activated) while this only has 32

[–] [email protected] 2 points 4 days ago (1 children)

GGUF quants are already out: https://huggingface.co/bartowski/Qwen_QwQ-32B-GGUF

[–] JamonBear 1 points 4 days ago

Yay! let's try

ollama run hf.co/bartowski/Qwen_QwQ-32B-GGUF:Q4_K_M

/set parameter num_ctx 32768