Consciousness as simulation

Consciousness as simulation of behaviour and perception: Lecture notes

Germund Hesslow
Dpt of Physiological Sciences
Lund University

Summary of lecture given in
Lund 1996-03-19
Skövde 1999-11-05
Göteborg 1999-11-18
Karolinska institutet 1999-03-10

A The problems of consciousness

The problem of consciousness actually consists of several problems. For many neuroscientists as well as for philosophers and laymen, the following are usually considered to be among the most important problems:

How does the inner world arise?

The central problem for most people. How can we see things which have no material existence?

What are mental objects?

What is the object we see with our "inner eye" or think about if it is not a material object?

What is the function of the inner world?

Could evolution have produced an organism like a human being and responding just like a human to any given certain input, but with no inner world? If so, why didn’t it?

Do animals have an inner world?

Can machines have an inner world?

B The central message of the lecture

These questions can be given plausible answers with the help of ideas which have been around for a long time, mainly in associationistic psychology and the behaviourist tradition. The central idea is that the brain can generate its own input. I will describe a physiological mechanism which can briefly be described as simulation of behaviour and perception. It has three components:

1) Simulation of behaviour.

Thinking that one is doing something is similar to actually doing it. It is covert behaviour.

2) Simulation of perception.

Imagining that one is seeing (hearing, feeling) something is very similar to actually seeing (hearing, feeling) it and activates the same brain structures.

3) Anticipation.

We assume an associative mechanism which enables simulation of both behaviour and perception to elicit other perceptual activity. An important consequence of this is that simulation of behaviour can elicit perceptual activity which resembles the activity which probably would have occurred if the the simulated actions had actually been performed.

C Simulation of behaviour

Behaviour is generated in a hierarchical fashion in the frontal lobes.

Activity in sensory cortex is signalled to the frontal lobe. The main flow of activity then goes caudally through supplementary and premotor cortex to primary motor cortex and from there via descending tracts to the brain stem and spinal motor neurones. Single muscles and elementary movements are controlled by neurones in the primary motor cortex in the pre-central gyrus. More complex movements are generated by more rostrally located neurones.

In covert or simulated movement, activity in the frontal lobes can occur just as in actual movement, but with no or only weak activation of the primary motor cortex.

The assumption that behaviour can be simulated in this context means that the frontal lobe activity occurring just before a movement can sometimes occur without the final parts of the sequence. Thus, the signal flow from the prefrontal cortex via the activity in the pre-motor areas may occur even if the flow of neural activity is interrupted before it activates the primary motor cortex and results in overt behaviour.

There is substantial evidence for the existence of covert behaviour

Psychological experiments reveal parallelism between imagined and actual movements. The time taken to perform imagined movements corresponds to the time the corresponding overt movements take.

Physiological experiments such as blood flow measurements provide direct evidence for a similarity between imagined and overt behaviour.

D Simulation of perception

Perceptual activity can be elicited in the nervous system in the absence of external stimuli.

Example: If I cut my finger, nerve fibres in the skin will send impulses to the brain where they cause a perception of the damage. If the peripheral nerve fibres could be activated artificially, somewhere along their pathways, a similar perception would occur in the brain. Something like this sometimes occurs in patients with amputated limbs. Nerve fibres which would normally transmit impulses from the skin can continue to send impulses to the brain and give rise to what is known as ‘phantom’ pain. A trivial but important implication is that we may perceive a peripheral damage even if no damage exists.

Similarly, we may have visual perceptions which are not caused by external events. If we activate the visual pathways, such as may occur if we apply a mild pressure to the eyeball, we may ’see’ stars or flashes, because we simulate the activity in the optic nerve which normally occurs when we actually see stars or flashes.

Thus, we can perceive things or events which do not exist

It is not necessary to assume the existence of mental limbs or mental stars in order to understand how such perceptual activity can occur.

No assumption is being made here about the nature of perceptual activity. We have not assumed that the brain constructs an ‘image’ or a ‘representation’ which is being watched by a homunculus in the brain.

There is abundant evidence that imagining something activates parts of the visual cortex in a way which resembles the activity which is elicited when the imagined object is actually observed.

Cognitive psychological experiments reveal a high degree of parallelism between visual imaging and visual observation. Physiological experiments, blood flow measurements etc during imaging show activity in the same areas which are active during observation.

E How can perceptual simulation be elicited?

Perceptual activity can be elicited by

a) exteroceptive and interoceptive stimuli (normal perception)

b) other perceptual activity

c) covert behaviour

Images can be elicited by other sensory stimuli

We all know this from personal experience. For instance, hearing a work may trigger an associated visual experience. The scientific understanding of the underlying processes is poor, however. It has been assumed in british associationistic psychology and in the behaviourist tradition that such associations can be accounted for by simple learning mechanisms. E.g., B. F. Skinner has written about "conditioned seeing". A plausible example is sensory preconditioning.

Consider the simple organism in fig. 1, which reacts to a stimulus S with a certain behavioural response R. S will cause a series of neuronal events which are designated by s. These will cause other neuronal activity r which in turn will cause the overt response R.

Figure 1: simple organism

Think of s as activity in the sensory cortex and of r as ‘preparatory’ or ‘incipient’ behaviour, i.e. neural activity in the pre-motor and supplementary motor areas of the frontal lobe.

Evidence that perceptual activity can be elicited by other perceptual activity is sensory pre-conditioning. A stimulus S₁ is first repeatedly presented before a second stimulus S₂. S₁ may be said to ‘predict’ S₂. We then establish a conditioned response R to S₂. If we now present S₁, it will elicit R. In the first demonstration of this phenomenon, Brogden presented dogs with repeated paired tone and light stimuli. He then paired the tone with a foot shock which elicited leg flexion until the tone elicited a conditioned flexion response. When he finally presented the light alone, it too elicited a conditioned response. Since light had not been paired with the foot shock, the simplest explanation for this phenomenon is that the light elicited some activity which was similar to that normally elicited by a tone, say a "tone-perception", and that this became a link in the chain from light to leg flexion.

Figure 2: Sensory preconditioning

Thus, one way in which a simulated perception of S might be triggered is simply by some other sensory activity, which has been associated with S. For instance, hearing a car, a bird or a musical instrument might activate visual cortical areas and elicit simulated perceptions of the corresponding objects. A special, but important, case would be the sound of a word.

Images can be elicited by incipient and simulated actions

It was also assumed in older behavioural literature that a covert action could function as a stimulus for or simulated perception such as "conditioned seeing".

The sensory consequences of behaviour are predictable. Walking into walls usually give rise to similar stimuli. Suppose the organism O which performs the response R in the situation S and that this results in the new situation S. S' is potentially harmful and O has learned to perform an important defensive response R'. If this sequence of events occurs regularly, it would be useful for O if the predictable stimulus S' could be simulated in response to the preparatory or incipient behaviour r, preceding R, because it would enable O to make the defensive response R' faster. We will assume an associative mechanism called ‘anticipation’ which consists in the ability of early stages in a behavioural response being able to elicit sensory activity which resembles that activity which would normally be elicited by the corresponding overt behaviour.

Figure 3: Anticipation

When a response has been performed, the situation for the animal will change from S into, say, S' which causes a new behaviour, R' etc. In this way, a single initial stimulus can trigger a long sequence of responses. The successive situations and responses are illustrated in fig. 3. If there is a consistent relation between stimuli and responses, there will also, as a consequence, be a correlation between types of internal events. Thus when a response R performed in situation S causes the situation to change into S', the ‘preparatory’ response r', performed in the presence of s, will be followed by the internal stimulus s'. We will assume that this regularity can be learned, so that performing the neural activity r in the presence of s, by some associative mechanism elicits the ‘expected’ activity s' (fig. 3B).This may have effects on the animal’s behaviour which are similar to the effects of actually perceiving S', i.e. the animal may perform r'.

Such a mechanism could have a great survival value, because it would enable an organism to ‘anticipate’ the consequences of a response and to emit behaviour adequate to the new situation with greater speed or to suppress behaviour which has negative consequences.

Example: In a classic experiment Tolman let a rat explore a T maze terminating with one black and one white chamber, both of which contain food. The rat is then placed in a black chamber and given painful stimulation. When the rat is later placed in the T maze it does not go the left at the branching point, even though it has never been punished for this behaviour. It is as if it ‘foresees’ the consequence of going left.

Figure 4: Tolman's experiment

A plausible interpretation of this experiment is that the rat at the branching point prepares walking to the left, that this (via the anticipation mechanism) elicits a perception of the black chamber and that this (via a classical conditioning mechanism) elicits anxiety. In a sense, the rat really does ‘foresee’ the consequence of walking left.

F Simulating chains of behaviour

A stimulus S elicits a response R which causes a new stimulus S' etc. producing a chain of stimuli and responses.

Figure 5: Behavioural chain

Once the mechanism of anticipation is in place, it will enable the organism to simulate a behavioural chain by performing a series of covert responses and corresponding simulated perceptions. S is followed by s and the incipient response r. This will elicit the normal consequence of R, s', even if R is suppressed. s'can then elicit a new incipient response r' etc.

Figure 6: Simulation of behavioural chain

Example: Maze

Suppose that a simple organism, starting at the bottom of the maze below can perform two responses, moving forward to the left (L) and forward to the right (R). It has a visual system which is capable of generating a unique activity pattern for each letter in the maze and for each of these activity patterns form a connection with one of the two possible responses. Suppose also that it has previously moved through the maze and learned a number of associations such as the following:

preparing to move left when seeing B is followed by d, i.e. seeing D
preparing to move right when seeing D is followed by g, i.e. seeing G

Figure 7: Maze

Even if the organism had never walked from B to G via D, it could simulate this response chain. Performing the covert left move when seeing B, would trigger ‘seeing’ D, which would trigger a new movement.

The animal may have been subjected to operant conditioning which had made the response l in d slightly stronger than r. If F was not a reinforcing position to be in, the relative strength of r in D would increase and the next time the organism started out from B it would emit r, thus finding itself ‘seeing’ G.

Example: Simulating a conversation

Consider two persons carrying on a conversation. A person utters something that is a stimulus for B to respond, thereby providing a stimulus for a new utterance from A and so on. The brain mechanism would not be different if A were to talk to himself, listening and responding to his own utterances. If activity in Brocas area on its way to the motor cortex could be fed directly into the auditory cortex, A could simulate the conversation without producing any audible utterance at all, yet utilising the same mechanisms as when taking part in an overt conversation.

An interesting consequence of the anticipation mechanism is that we will "hear" a verbal response twice, first when the preparatory response activates the auditory cortex and then when we actually hear the sound produced by the overt utterance. This may be the explanation for the idea that there is a mental event, a ‘thought’, which precedes and causes the utterance.

G The problems of consciousness again

How does the inner world arise?

The inner world is an obvious and unavoidable consequence of the simulation process. If covert behaviour can generate perceptual activity which resembles the activity generated by perception of the external world, then one will perceive something similar to external world, even if it does not exist.

Notice that this conclusion is independent of the mechanisms of perception. In particular, we do not have to make any assumptions at all about the existence "images" or "representations". If we can simulate the perception of x, then it will seem to us that we are observing x, regardless of how the brain normally goes about it to make us see x, that is regardless of whether it uses "images" or "representations".

What are mental objects?

The inner world does not presuppose the existence of mental objects.

The ‘zombie’ problem: why is the inner world necessary?

The answer to this question is simply that an inner world is an unavoidable consequence of simulation. Once we can perform covert movements and these are accompanied with activity in sensory cortex which mimicks perception, the inner world is there.

Do animals have an inner world?

Yes, if they can generate their own sensory input via either a sensory conditioning mechanism or via an anticipation mechanism

Can machines have an inner world?

Same answer as for animals.

I An additional advantage of the simulation hypothesis

There is growing evidence that the cerebellum and basal ganglia, usually regarded as strictly motor structures, participate in cognitive functions. This has seemed puzzling to many but is a logical consequence of the view presented in this lecture. If movement simulation is movement preparation, it must involve the same brain structures. This also explains the cognitive symptoms of cerebellar or basal ganglia disease.

References

Previous versions of the simulation hypothesis were presented in
Hesslow G. (1994) Will neuroscience explain consciousness? Journal of Theoretical Biology. 171:29-39.
Hesslow, G. (1996) Hjärnan och själen: kan fysiologin förklara den inre världen. I Widén, L. (red) En bok om hjärnan. Sockholm, Tiden.

Many of the critical ideas can be found in the behaviourist literature, for instance
Skinner BF (1953) Science and Human Behavior. Macmillan, New York
Skinner BF (1974) About Behaviorism. Knopf, New York

Much of the empirical evidence for covert behaviour cited in the lecture can be found in
Jeannerod M (1994) The representing brain: Neural correlates of motor intention and imagery. Behavioral.and.Brain Sciences. 17: 187-245

Evidence for simulation of perception is reviewed in
Kosslyn SM (1994) Image and Brain: The Resolution of the Imagery Debate. MIT Press, Cambridge