Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.
Get support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.
As ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.
Rules:
Rule 1 - No harassment or personal character attacks of community members. I.E no namecalling, no generalizing entire groups of people that make up our community, no baseless personal insults.
Rule 2 - No comparing artificial intelligence/machine learning models to cryptocurrency. I.E no comparing the usefulness of models to that of NFTs, no comparing the resource usage required to train a model is anything close to maintaining a blockchain/ mining for crypto, no implying its just a fad/bubble that will leave people with nothing of value when it burst.
Rule 3 - No comparing artificial intelligence/machine learning to simple text prediction algorithms. I.E statements such as "llms are basically just simple text predictions like what your phone keyboard autocorrect uses, and they're still using the same algorithms since <over 10 years ago>.
Rule 4 - No implying that models are devoid of purpose or potential for enriching peoples lives.
Heres a link to the papers, starting around the end of page 8 is revelant paragraph. Thank you hendrik! https://github.com/deepseek-ai/DeepSeek-R1/blob/main/DeepSeek_R1.pdf
You might want to explain the reference in case we didn't all learn the paper by heart...
https://github.com/deepseek-ai/DeepSeek-R1/blob/main/DeepSeek_R1.pdf
Table 3 on page 9. The "Aha" paragraph starts on the page before that.
And yes, I think I tried DeepSeek and it wrote things like that, I'm not sure if "Aha", but similar things. Also "Wait..." and that something didn't look right and it had to backtrack... It generally writes a lot of weird stuff. Sometimes it even writes wrong things and then doesn't listen to itself and it silently corrects itself in the process. Other models phrase things with a different tone. At least that's what I've seen. Some also repeat a lot of stuff. I'll write what I want, and then it goes on and on "the user wants me to ..." and repeats everything once again, just not in first person, but in third person. And then after some nonsense, it tries to dissect the problem.
Thanks for the suggestion and sharing your take :)
the 'okay...' 'hold on,' 'wait...' let me read over it again...' is part of deepseeks particular reasoning patterns, I think the aha moments are more of an emergent expression that it sometimes does for whatever reason than part of an intended step of the reasoning pattern but I could be mistaken. I find that lower parameter models will suffer from not being able to follow its own reasoning patterns and quickly confuses itself into wrong answers/hallucinations. Its a shame because 8B models are fast and able to fit on a lot of GPUS but they just cant handle keeping track of all the reasoning across many thousands of tokens.
The best luck ive had was with bigger models trained on the reasoning patterns that can also switch them on and off by toggling the tag. My go to model the past two months has been deephermes 22b which is toggleable deepseek reasoning baked into mistral 3 small, its able to properly bake answers to tough logical questions putting its reasoning together properly after thinking for many many tokens. Its not perfect but it gets things right more than it gets them wrong which is a huge step up.
Nice. I'll try it.
Yeah, I also think it's more an expression which somehow emerged, and it's not really a Eureka moment. They also seem to put it that way in the paper. The say it's an Aha for the scientists, and the whole reasoning process is an Aha, but they don't really write the model is having an Aha moment due to some insight it had.
I don't know if you watch Youtube videos, but Computerphile made a video about DeepSeek and an interesting video about Forbidden AI techniques a few days ago. That's also about the reasoning process and how LLMs can be lazy and take unwanted shortcuts, and hide information in the thinking step.
they don’t really write the model is having an Aha moment due to some insight it had.
Well, they really can't write it that way because it would imply the model is capable of insight which is a function of higher cognition. That path leads to questioning if machine learning neural networks are capable of any real sparks of sapience or sentience. Thats a 'UGI' conversation most people absolutely don't want to have at this point for various practical, philosophical, and religous/spiritual implications.
So you can't just outright say it, especially not in an academic STEM paper. Science academia has a hard bias against the implication of anything metaphysical or overly abstract at best they will say it 'simulates some cognative aspects of intelligence'.
In my own experience, the model at least says 'ah! aha! Right, right, right, so..` when it thinks it has had an insight of some kind. Whether or not models are truly capable of such thing or is merely some statistical text prediction artifact is a subjective discussion of philosophy kind of like a computer scientist nerds version of the deterministic philosophical zombie arguments.
Thanks for sharing the video! I havent seen computerphile in a while, will take a look especially with that title. Gotta learn about dat forbidden computation :)