Is ChatGPT really intelligent? Of course it is.

I realise I am wading into a contentious and in fact extremely momentous issue. Many of the points I will make are not provable, in the sense that their veracity can’t be strictly measured against unequivocal empirical evidence. They are more in the tradition of philosophical arguments, a framing of the issues under consideration, or maybe a particular intellectual “stance” towards the subject.

I argue that ChatGPT is definitely intelligent (ChatGPT is the new online bot, that has received worldwide publicity, which is able to answer questions and write essays and stories).

Dimmer Switch

It’s important to make the following point about this at the start. I am following the view of Dan Dennett, the philosophy professor and frequent author on consciousness and artificial intelligence. He argues that we need a new model for consciousness (and I’m taking intelligence as a close proxy to this): it’s the “dimmer switch” model. In other words, there’s no one, clear cut-off point between something being conscious and not conscious. A very basic organism can have a very limited degree of consciousness (and intelligence I argue): ie a very simple organism that moves toward light in order to survive. It acts intelligently to secure its survival at a very rudimentary level. We can then ratchet up the level of consciousness through more sophisticated animals, up to human beings. Likewise, in parallel, so our argument runs, we can assign different levels of consciousness (and intelligence) to machines – from basic machines such as thermostats up to the most sophisticated computers doing trillions of calculations.

So an animal – or a machine – can have only a very limited level of consciousness or intelligence, but even at that stage I believe it’s warranted to say that they have those properties. I then argue that it becomes more evidently plausible to assign intelligence to a machine as we move up the chain and see what machines are now capable of.

That’s a lot to chew on if you are not predisposed to thinking of machines as being intelligent or conscious, but I’ll proceed with some more framings.

The Turing Test

One of the most famous (and one of the earliest in modern times) framings of machine intelligence is that made by Alan Turing in 1950: the celebrated Turing Test. One person sits in a room connected to a second a room which contains a person and a computer. The person cannot see or hear the participants in the second room, but only communicates electronically. The person asks questions of the others. If the person can’t distinguish between the answers of the person and the computer, Turing says we can conclude that the computer is intelligent. (Yes, there would have to be sufficient time taken for the test etc etc and a sufficient number of challenging questions and so on, which would need to be stipulated.)

Quite a number of philosophers have quibbled and argued with Turing’s assertion. Is he really testing intelligence? What is intelligence anyway? Couldn’t you be fooled or misled etc etc? Yes, it’s always possible that someone could be mistaken, but Turing was a pragmatist. I read him as saying, “if it sounds like intelligence, communicates like intelligence and smells like intelligence, it probably is intelligence.” Turing was always looking at pragmatic real-world answers to questions – including this one: could we demonstrate, in a practical way, that a machine has intelligence? Yes we could, he said. I agree with Turing.

Turing Test revisited

These arguments continue to this day – and vociferously so. In a recent Guardian article on ChatGPT, I commented on the article along the lines above. There were lots of comments on the thread I started, and I quote a large chunk of one response, since it’s really germane to the central arguments about this issue and sums up many of the counter-arguments to my opinion:

That its (ChatGPT’s) output can appear like a reasoned argument is not because it has actually reasoned, it’s just that it has been trained on a lot of ‘reasoned’ arguments and knows what a reasoned argument looks like statistically.

So when it comes to predicting the next word, it really is only statistically predicting the next word – it isn’t actually performing any kind of reasoning, it’s not considering the concepts involved, it’s not generating an internal mental model and then manipulating it in the manner that humans do when they reason about something – even when apparently generating a ‘reasoned’ argument.

The difference might sound subtle, but it is a huge difference, and the author of the article, to his credit, seems to have recognised very well the difference.

ChatGPT isn’t doing any sort of reasoning – even when it generates an apparently ‘reasoned’, novel response in response to a seemingly novel question.

That said, it really is very, very impressive just how adaptive and broad the resultant generative model of ChatGPT is, and it clearly will have many uses… but it is only as good as the input it has been trained on.

In other words, it has to see a statistically significant number of examples of the sort of thing you want it to generate in order to be able to generate equivalent things itself.

…in essence it can only learn novel structures from (a lot of) examples, not from understanding (novel) grammatical rules. Effectively the grammatical rules are ‘learned’ through statistics, rather than understanding… it is totally useless when it comes to inventing genuinely new ideas, or anything that does require genuine reasoning (where such reasoning has not already been performed before for it a learn from).

I note a number of things.

1. This comment is essentially repudiating Turing’s argument, since crucially, internally ChatGPT is not doing the same thing as human beings. Humans generate an ”internal mental model” and then manipulate it. Whereas., ChatGPT trains itself on a vast number of reasoned arguments (for example culled from examples on the web).

Turing’s answer to this (and mine) is – so what? Who cares what the internal process is like, if the output and behaviour is intelligent? Machine intelligence does not have to mimic human inelligence – why should it? It simply needs to behave intelligently, and it can do this in any way it likes. Taking this more broadly, there is a kind of “human chauvinism” implied here. Why shouldn’t there be other routes to intelligence? Are we going to argue that human (and animal) intelligence is the only way to achieve intelligence in the universe? That seems to me like an extraordinarily bad bet. (And additionally, the “internal mental model“ idea raises all sorts of questions: who or what has the internal mental model? And who exactly manipulates it? There’s a danger here of a regress into an infinite series of homunculi (little men) one inside the other, each having their own mental model.)

2. So the commentator argues that ChatGPT isn’t really doing any reasoning (and therefore isn’t really intelligent) since it bases its responses on a statistical analysis of lots of examples. The commentator accepts that the bot learns grammatical rules, but only through “statistics rather than understanding.”

So stepping back a bit here – if you look at ChatGPT, it’s capable of writing (I think most would accept) a reasonably good essay on pretty much any question you ask it – from philosophy, to advanced physics, to Marxism, to art and literature. I’ve tried it – if you haven’t I encourage you to try it yourself. ChatGPT responds appropriately to quite complex questions and requests for essays, lays out logical arguments,  supported by appropriate evidence. It uses grammar and vocabulary correctly, and clearly picks up the intent of questions and instructions put to it. It can also write creative, fictional stories.

So it can use grammar and language correctly, write essays and reasoned arguments, and make up engaging fictional stories. Firstly, no single human being could possibly do this range of activities. But more importantly, if you look at the bot’s individual answers and essays – at a conservative estimate, I would say they are better than 60% of humanity could currently achieve (I think it’s almost certainly higher than that, but let’s be conservative).

ChatGTP has also passed the following:

• Stanford Medical School final exam in clinical reasoning
• Four law school courses at the University of Minnesota
•  United States Medical Licensing Exam — a three part exam that aspiring doctors take between medical school and residency
• Wharton School final exam in operations management

But despite this, the commentator still maintains that ChatGPT is not really intelligent.

The Chinese Room

A lot of the arguments around this are essentially a continuation of the philosopher John Searle’s Chinese Room argument. Searle argued, in a thought experiment, that you could have a room that produced correct Chinese language, but still have no understanding or concept of Chinese. Here is Stanford Philosophy’s summary:

The argument and thought-experiment now generally known as the Chinese Room Argument was first published in a 1980 article by American philosopher John Searle (1932– ). It has become one of the best-known arguments in recent philosophy. Searle imagines himself alone in a room following a computer program for responding to Chinese characters slipped under the door. Searle understands nothing of Chinese, and yet, by following the program for manipulating symbols and numerals just as a computer does, he sends appropriate strings of Chinese characters back out under the door, and this leads those outside to mistakenly suppose there is a Chinese speaker in the room.

The narrow conclusion of the argument is that programming a digital computer may make it appear to understand language but could not produce real understanding. Hence the “Turing Test” is inadequate. Searle argues that the thought experiment underscores the fact that computers merely use syntactic rules to manipulate symbol strings, but have no understanding of meaning or semantics.

The main reply to Searle is the “Systems Reply”. It basically says, that yes, if you looked at each element in the room individually and followed the programme for manipulating elements of the system, you won’t find intelligence or understanding in any of those elements, but, if you take the system as a whole, intelligence “emerges” out of the system functioning together. Understanding is an “emergent property” of the whole apparatus. So, as a whole, the system does understand Chinese, contrary to Searle’s argument. And in answer to the commentator above, concepts emerge out of the machine system – the computer as a whole system does have a concept of Chinese and how to use it.

Searle argues that he could internalize the whole system in his head and he still wouldn’t understand Chinese, even if he could speak it. The system still isn’t really intelligent, according to him.

But I agree with the systems answer.

Human Intelligence

This invites another turn in the argument. How exactly do humans generate intelligence? I have a thought experiment in which John Searle (as in the sci-fi movie Fantastic Voyage) is reduced down to a tiny size and is able to inject himself into the human brain and walk around inspecting it. Searle will find various structures in the brain – with electro-chemical circuits whizzing between them, governed by some system – but he’ll fail to find consciousness or intelligence in any of the organic apparatus manipulated in its system, and concludes that there is no understanding in any of this – and he argues, as above, that it’s in vain to appeal to the idea of intelligence emerging out of the whole system. You can’t see understanding anywhere so there is no understanding. So therefore, his argument would run, humans are not intelligent.

As Dan Dennett argues, the human brain and consciousness are made up of extremely numerous, blind, automatic elements that function together, and as a whole produce intelligence. Human intelligence does not come out of nowhere. It is not produced by magic, or something metaphysical. It emerges out of physical structures, brain tissue, neurons and synapses. None of those things are conscious or intelligent in themselves – and nor is the programme that runs them – but they combine to produce those properties.

Nothing New under the Sun

Returning to another argument from the commentator on the Guardian article:

(ChatGPT) it is only as good as the input it has been trained on.… it is totally useless when it comes to inventing genuinely new ideas,

So this is another frequently aired criticism of AI intelligence. Alan Turing anticipated this early on. “There’s nothing new under the sun, as the saying goes. But who can be certain that anything you have done or thought wasn’t simply the growth of a seed that was planted in you by teaching, or the effect of following well-known general principles?”

Turing said, even with the very rudimentary early computers of his day (which of course he helped invent), that he was often surprised by outputs from computers. He did not always predict what they would do. And as he implied, novelty may surprise us, but any idea that appears new to us needs to be developed by the machine out of a basis of existing knowledge and ideas.

So how does a human being come up with a new idea – or a new fictional story? Not out of nothing. The human being must draw from the immense and complex inputs she has had during her life, and then recombines them in what appears as a new way – something that was capable of being done, given all those preconditions. Yes, humans have unique experience, and can therefore produce something unique out of those experiences. This is what human originality emerges from. It is not, to my mind, some mysterious, ineffable process. Originality comes out of knowledge, experience and inputs and combinations of those elements.

And, circling back, all of this is completely possible for a machine. The machine, fundamentally I argue, follows the same process. It accumulates knowledge and inputs, and then manipulates and combines them. And this is just what humans have to do as well. And if a random or different unique element is needed, say to create a completely unique machine, it’s not a problem to generate a pseudo-random element to input into a machine.

So I don’t think the, “but machines can’t be original,” holds water. If that were the case, I don’t think human beings could be original either. But this also takes an adjustment of what “original” actually means in practice.

In Brief

There’s this argument: “Ah, but what about self-awareness?”  I’ll leave Steven Pinker (cognitive scientist) to answer that one:

“Self-knowledge, including the ability to use a mirror, is no more mysterious than any other topic in perception and memory. If I have a mental database for people, what’s to prevent it from containing an entry for myself? If I can learn to raise my arm and crane my neck to sight a hidden spot on my back, why couldn’t I learn to raise a mirror and look up at it to sight a hidden spot on my forehead? And access to information about the self is perfectly easy to model. Any beginning programmer can write a short piece of software that examines, reports on, and even modifies itself. A robot that could recognize itself in a mirror would not be much more difficult to build than a robot that could recognize anything at all. There are, to be sure, good questions to ask about the evolution of self-knowledge, its development in children, and its advantages (and, more interesting, disadvantages, as we shall see in Chapter 6). But self-knowledge is an everyday topic in cognitive science, not the paradox of water becoming wine.” Pinker, Steven. How the Mind Works (pp. 134-135). W. W. Norton & Company. Kindle Edition.

And then there’s the, “ah, but what about emotions?” argument. Not surprisingly, I don’t think emotions are a magical, mysterious process created by the ether. They are a combination of physical sensations and mental thoughts. I don’t see any insuperable barrier to creating that in machines, if we – or they – wish to do so: some kind of reward system and a generated response to rewards. Whether that is worth doing is another question, and as Turing said, we shouldn’t expect machine intelligence to be exactly like human intelligence. Why should it be? It might do its own thing and have its own equivalent of emotions.

Final thoughts

So, I argue, the properties of consciousness and intelligence, are emergent properties that emerge out of material processes –  brains, machines and the acquisition of knowledge and experiences.

My contention, along with Dan Dennett, is that there is no fundamental, non-arbitrary difference between human and machine intelligence. Both spheres are capable of it. And as computers and AI advance, they do more and more things that were once just the preserve of humans: diagnosing diseases, making complex mathematical calculations, playing chess, or writing reasoned essays and creative stories. I don’t see that process slowing down.

Am I worried about the advance of AI? I do find it somewhat scary and disturbing – especially as someone who is interested in writing fiction. How long do I have, as a fiction writer, before AI starts producing fiction that is far better than any human can achieve?

PS.

There has just been much publicity about a rival AI bot to ChatGPT – from Google, which made a mistake in one of its references. This has led to lots of mirth and comments on AI. What if an AI programme makes a mistake? – comes the question. Gotcha! Once again, Alan Turing anticipates this. For a start, we don’t expect humans never to make mistakes – and we certainly ascribe intelligence to them. Why would we be more forgiving of a human than a machine? But Turing went further than that – he said that mistakes could be, and are often, associated with intelligence:

The argument from Godel’s and other theorems ( objections to machine intelligence) rests essentially on the condition that the machine must not make mistakes. But this is not a requirement for intelligence. It is related that the infant Gauss was asked at school to do the addition 15+ 18+21 + .. , +54 (or something of the kind) and that he immediately wrote down 483, presumably having calculated it as (15+54)(54-12)/2.3. One can imagine circumstances where a foolish master told the child that he ought instead to have added 18 to 15 obtaining 33, then added 21, etc, From some points of view this would be a ‘mistake’, in spite of the obvious intelligence involved. One can also imagine a situation where the children were given a number of additions to do, of which the first 5 were all arithmetic progressions, but the 6th was say 23+34+45+ …  +100+112+122+ …  +199.  Gauss  might  have given the answer to this as if it were an arithmetic progression, not having noticed that the 9th term was 112 instead of 111. This would be a definite mistake, which the less intelligent children would not have been likely to make.

PPS

To answer another common objection, which goes as follows: “But everything a chat bot or a computer does is derivative – it all depends on human intelligence, and was sourced by human intelligence. They can’t claim credit for intelligence themselves.”

So, yes, granted that the computer and the chat bot wouldn’t exist without human intelligence and ingenuity. It was necessary for humans to create them. But that isn’t the end of the story.

One starkly put response is as follows: humans are dependent, through evolution, on single cell organisms which were some of the earliest forms of life. Single cell organisms were absolutely essential for the development and existence of human intelligence, since without that evolutionary link, it wouldn’t exist. But no one would argue that single cell organisms “own” human intelligence or should take credit for it. We could move up evolutionary time a little – does a common ancestor primate own human intelligence? Is our intelligence merely derivative of a less fully developed humanoid? Or, going in the other evolutionary direction – does inanimate matter own human intelligence?

My point is that machine intelligence is, indeed, an outgrowth of human intelligence – but that needn’t limit it. From this perspective, machine intelligence is a new stage in evolution, proceeding on from the human. And machine intelligence is already, unarguably, capable of doing things that no human brain can do – analysis of vast amounts of data for example – and is certainly capable of coming up with answers and insights that humans didn’t predict.

Once again, Alan Turing anticipated this objection – and here is his typically homely (homey in US English) analogy:

The view (d) that intelligence in machinery is merely a reflection of that of its creator is rather similar to the view that the credit for the discoveries of a pupil should be given to his teacher. In such a case the teacher would be pleased with the success of his methods of education, but would not claim the results themselves unless he had actually communicated them to his pupil. He would certainly have envisaged in very broad outline the sort of thing his pupil might be expected to do, but would not expect to foresee any sort of detail. It is already possible to produce machines where this sort of situation arises in a small degree.