Three Myths About Artificial Intelligence: Fluency, General Intelligence and Neutrality

When you ask a machine to write a poem and it does, has it thought, or merely imitated well those who thought before it? A simple question hides a deep problem, for we speak of artificial intelligence as if we knew what it is. Three myths muddle our understanding of it.

Myth One: Fluency Is Understanding

A model that writes you a persuasive letter does not "understand" what it writes as you do; it simulates understanding. In 1950 Alan Turing replaced the question "Can machines think?" with a practical test of observable behaviour. John Searle argued, with his "Chinese Room" thought experiment, that manipulating symbols skilfully is not enough on its own for understanding, an argument that is still debated.

We are inclined to attribute to machines a mind they do not have. When Joseph Weizenbaum released ELIZA in 1966, a conversational program with simple rules, he was surprised that people confided their secrets to it and believed it understood them, as he recounted in his 1976 book. This is the "ELIZA effect", and modern models are better at provoking it. The danger is not that the machine deceives us, but that we are eager to be deceived.

Myth Two: Narrow Intelligence Is General Intelligence

What we have is narrow intelligence that excels at a specific task. However widely modern models are used, it is not general in the way human flexibility is. Many assumed that Deep Blue "thought" when it beat Garry Kasparov in 1997, yet it was in essence a vast computational search with evaluation functions written by people.

A field experiment supports this, led by researchers from Harvard Business School and other universities in partnership with Boston Consulting Group (2023), with 758 consultants. The researchers called it the "jagged frontier": within the model's capability, performance improved noticeably, but outside it participants were about 19 percentage points less likely to reach the correct answer. A decision may worsen because the user trusts a persuasively worded answer; this is an interpretation, not a measured result. Leaping from narrow successes to an "imminent general intelligence" is a conclusion beyond what the evidence shows.

Myth Three: The Machine Is Neutral

A machine that learns from our records reproduces their patterns, biased ones included, under an appearance of objectivity. Mathematical neutrality does not mean neutral results, and talk of an "objective intelligence" free of human bias is a myth, even if some systems go beyond human examples through self-directed experimentation. This is covered in the article "Algorithmic Bias".

Three Practical Rules

1. Ask: is my task inside the range of what the tool does well, or outside it?
2. Beware the "ELIZA effect": behind every elegant answer lies a statistical probability, not conscious understanding.
3. Keep the human in the place of decision and accountability; moral responsibility cannot be carried by an algorithm.

The closest description of artificial intelligence, in our view, is that it is neither a mind we have built nor an inert mirror, but an active mirror that recomposes what it reflects, revealing more about us than about itself. So what does this mirror show about how we ourselves think? With that question we meet tomorrow with a critical eye, not a dazzled one.