As of late there’s been a lot of thinking engendered by Apple’s The Illusion of Thinking paper.

I propose we get ahead of this sequence of paper-rebuttal-paper and define the iterated function sequence \( \{ n \ge 1: \textsf{(The Illusion of)}^{n} \textsf{(Thinking)} \} \). Whether this series of thoughts will converge to any fixed point is left as a conjecture for the reader.

A new post by Sean Godecke, commenting on a recent publication Is chain-of-thought AI reasoning a mirage? includes this incisive critique:

“LLMs construct superficial chains of logic based on learned token associations, often failing on tasks that deviate from commonsense heuristics or familiar templates Models often incorporate … irrelevant details into their reasoning, revealing a lack of sensitivity to salient information models may overthink easy problems and give up on harder ones Together, these findings suggest that LLMs are not principled reasoners but rather sophisticated simulators of reasoning-like text”

I want to tear my hair out when I read quotes like these, because all of these statements are true of human reasoners. Humans rely on heuristics and templates, include irrelevant details, overthink easy problems, and give up on hard ones all the time! The big claim in the paper - that reasoning models struggle when they’re out of domain - is true about even the strongest human reasoners as well.

Well, I think there’s a strong point here. I’m not maintaining that humans and LLMs are equally powerful at arbitrary reasoning, but reasoning is an intrinsically complex task that we might sometimes forget humans grapple intensively with too.