This post is going to be a little different because it strays a bit out of computer science and into two interesting philosophical thought experiments related to it: namely the Turing Test and the Chinese Room. They often come up in the context of modern computing – especially the profound advances that have taken place in machine learning – and I wanted to explore them some.

Thought experiments about thoughts

The Turing Test, as you may know, is a thought experiment about artificial intelligence that the great computer scientist Alan Turing thought up. The idea is to have text chats open with two entities. One is a computer that is programmed to try and act human in all respects. Specifically it receives questions over text and has to answer them like a human would. The other chat is with a human who’s trying to show their own humanity. A set of judges chats with both and then votes on who is the human and who is the computer. If at least 30% of the judges believe that the computer is human, the test is passed.

Interestingly there have been many occasions where computer programs have passed situations that are at least superficially similar to the test. The earliest I know of is ELIZA, a sort of virtual therapist created in the 60s at MIT. Due to the time period in which it was written, it was of course very simple and could only do some basic pattern matching and filling in of templates to generate its responses to the user. Despite that, some of its users credited it with showing real insight. This trend has continued as AI has advanced, with several recent AI systems claimed to truly have passed the test. Not exactly. The domains that they operate in remain very restricted and don’t hold up to the sort of focused inquiry that would terminally confuse them but be fine for humans.

So what would happen if something that truly passes that test comes about? Many people use “passing the Turing Test” as a synonym for a truly thinking computer, or even more optimistically, one that could continually improve itself and bring in a supposed technological singularity. One reason to be skeptical at least of the “thinking” part is the Chinese Room thought experiment, devised by John Searle. He supposes someone locked in a room with a large database of Chinese characters along with a giant book of instructions that manipulates the symbols to eventually create an output. The person in the room occasionally receives a question written in Chinese that’s passed in the room, follows the instructions in the book to manipulate symbols, and passes the Chinese output back outside of the room. Someone on the outside who understands Chinese thinks that the room is answering questions put towards it, but to the person in the room all the symbols are meaningless. They are not answering any questions at all, just following the book’s instructions. The parallel between this and systems like ELIZA and modern chatbots is clear. We attribute thoughts to them where none supposedly exist.

Seeing how to see

Over the years since this thought experiment was first proposed there’s been large amounts of discussion with many replies and counter-replies, but I wanted to focus on one interesting issue that differentiates the ways on which current AI can do things like “seeing” from humans that ties back into how future AI may behave.

diagram showing the layers of neurons in a convolutional neural network
Neurons, neurons, and more neurons.
Image credit: https://commons.wikimedia.org/wiki/File:Typical_cnn.png

The current state-of-the-art in machine image recognition uses a convolutional neural network. This uses a large network of artificial “neurons” that behave somewhat similarly to human ones. The network takes as input the image, with each neuron receiving and processing input on a few adjacent pixels (called the convolution) on the image. The neurons feed into further layers of neurons, and eventually a final layer of output neurons choose which of many things the image could be. The whole thing is inspired very loosely by the visual cortex in brains.1

While there may be some structural similarities between the carbon and silicon implementations of seeing and categorizing, there is a very important behavioral difference between current image recognition systems and how we humans see things. The artificial network always makes the same choice. It may output probabilities of different images, but once it makes a decision that decision does not change.

silhouette of a woman spinning that can be perceived as either clockwise or counter-clockwise
Which way?
Image credit: https://commons.wikimedia.org/wiki/File:Spinning_Dancer.gif

Humans, on the other hand, certainly do not see things this way. A great example of this is the famed Spinning Dancer “optical illusion”. It’s a simple GIF loop of a silhouette of a woman spinning. The part where the illusion comes in is that it can be seen as either spinning clockwise or counter-clockwise, but never both at the same time. When I see it spinning in a certain direction, the direction generally remains undisturbed. With time and effort I can get the spinning to change directions and see it a different way, but I can’t see both directions at the same time.

Could computers also perceive a change like humans can? Perhaps, but not the current machine learning networks being used to detect images. The network would have to be stateful and feed its output back into its input, along with some sort of perturbation mechanism so that the perception of spinning direction can be changed. While some networks used to analyze things like sounds and videos have limited statefulness and feedback, the mechanisms used are unlikely to work in this case, nor are there serious proposals for a network that could. The reason is that the training mechanism used to teach the network to match images to categories breaks down with that level of feedback. I don’t think that’s a coincidence.

Machine willing

At this point you may be asking what a silly optical illusion has to do with the thought experiments I was talking about. I think that the Chinese Room’s strongest argument is that in the experiment the computer is not really doing things, just simulating them. For example if someone asks me the question “What’s your favorite kind of wine?” and I answer “Sauvignon Blanc”, I’ll have truthfully answered a question that’s based on my own experience. But if I had never drunk wine and answered the question, say, while working as an actress, I would simply be following a script like in the Chinese Room experiment. I think that in order for a computer to truly think, it has to act. And this is where the neural network feedback comes in.

Everything we humans do as voluntary actions involves some sort of feedback mechanism. I intend to type out this post, see that I’ve typed it and that words have appeared on my laptop screen, and read them back and think about them. I intend to drink some wine, experience the taste, and consider what I think of the taste. And so on.

How could this sort of action-taking be modeled in silicon? Honestly I haven’t the slightest idea. Perhaps it can be done, but we will need to advance our knowledge about AI still farther before we get there.


  1. The way in which the network is trained to recognize different categories is certainly not, which will become important.