this post was submitted on 18 Jun 2024
89 points (98.9% liked)
TechTakes
1225 readers
143 users here now
Big brain tech dude got yet another clueless take over at HackerNews etc? Here's the place to vent. Orange site, VC foolishness, all welcome.
For actually-good tech, you want our NotAwfulTech community
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
It's not a "fancy word" here, but a technical term. An AI making things up is actually called hallucination.
the technical term is either “confabulation” or “bullshit”; “hallucination” is a misleading label coined by the ai pushers.
It used to mean things like false positives in computer vision, where it is sort of appropriate: the AI is seeing something that's not there.
Then the machine translation people started misusing the term when their software mistranslated by adding something that was not present in the original text. They may have been already trying to be misleading with this term, because "hallucination" implies that the error happens when parsing the input text - which distracts from a very real concern about the possibility that what was added was being plagiarized from the training dataset (which carries risk of IP contamination).
Now, what's happening is that language models are very often a very wrong tool for the job. When you want to cite a court case as a precedent, you want a court case that actually existed - not a sample from the underlying probability distribution of possible court cases! LLM peddlers don't want to ever admit that an LLM is the wrong tool for that job, so instead they pretend that it is the right tool that, alas, sometimes "hallucinates".
I am saying that coining it as a term was stupid and intended to make it sound intelligent when it isn't.
oh definitely, it's fucking terrible question-begging. I'd like to know when it traces back to, and how good faith it was or wasn't
It originally comes from false positives in computer vision afaik, where it makes some sense as the model is "seeing" things that aren't in the image.
Of course is the term stupid. Neither is an LLM an AI, nor is any AI in the current state intelligent. In the end it all boils down to being answer machines. Complex ones, but still far away from anything even remotely being am AI.
Technical terms can still be, technically speaking, dumb as fuck.
Lmao
oh but you see, it's "hallucination" when LLM is wrong and it's hype cycle fuel when it's correct. no, LLMs don't "hallucinate", that implies that this state is peculiar, isolated, triggered by very specific circumstances. LLMs bullshit all the time, sometimes they are right, sometimes not, the process that produces both types of response is the same. pushing for "hallucination" tries to obscure that. use of "hallucination" also implies that LLMs know something, they don't, by design. it just so happens that if they "get" things right, it's because it appeared in training material enough times to make an impression in model.
Bullshitting to me is giving intentionally wrong statements. LLMs do not generate intentionally wrong statements. Saying they do, means that you imply intelligence.
LLMs know nothing nor are they intelligent. They also are not right or wrong, they generate output based on statistics.
"Hallucination" as a term for "AIs" making things up is used since the early 2000s (even if it's meaning has changed since then).
bullshitting as in when you give a confident answer without regard of actual reality. previously discussed there LLMs do exactly that: generate confidently, authoritatively sounding text without regard of facts, because these things do not know facts or anything for that matter.
maybe it's high time to change terms then
So you say there could be different meanings of the same word? Like “bullshitting” or “hallucination”?
mod post: please desist, it's just tiresome now
agreed
Absolutely.