Fuck AI

John Searle is a troll.

The Chinese Room proposes a true AGI - fully convincing for any purpose. That software speaks Chinese with human-level intelligence. But then he opens the computer, yanks out the CPU, and asks it what the hard drive was doing. The CPU doesn't know shit. It wouldn't know why it's adding 2+2. It blindly obeys instructions; that is its purpose. Replacing that with a guy gives you a bored guy who also doesn't know shit. But he's blindly obeying a book that speaks Chinese, with human-level intelligence. That mind is in the software.

LLMs do not have that intelligence - but you only know that by talking to them. If they were fully convincing, to the same degree as texting with any human person, you'd have to say the computer is intelligent or the person is not. They will have equally demonstrated the ability.

But John fucking Searle's stupid little metaphor has people ready to say, nuh-uh, a demonstrable machine intelligence would not count. No amount of visible creativity, reasoning, comprehension, or insight would matter, unless the idiot in the box was following the billions of instructions per word. As if the average person playing pen-and-paper processor could figure out they're squaring pi. As if their understanding is what makes the answer correct.

If this new idiot's accidental split personality was indistinguishable from an actual Chinese speaker, then it really would know Chinese fluently. That's what being fluent is. Whatever stupid rules the guy has memorized about these squiggles: if the output seems fully intelligent, then intelligence is contained in those rules. If you want to learn about the underlying nature of thought, talk to the guy in person. But if you want to know how this simulated mind understands things, you have to send him some squiggles.