Is Perception the 'Leading Edge' of Memory?

Daniel C. Dennett
Center for Cognitive Studies
Tufts University
Medford, MA 02155
USA
cogstud@pearl.tufts.edu

FINAL DRAFT
Locarno Conference
MEMORY AND OBLIVION
October 21, 1992

Consciousness appears to us to consist of a sequence of contentful items, arranged in a sequence, the so-called "stream of consciousness," in which each item in turn bursts quite suddenly into consciousness and thereby enters memory, perhaps only briefly to be remembered, and then forgotten. I think that hidden in this comfortable and largely innocent picture of consciousness is a deep and seductive mistake. I intend to expose and elucidate that mistake, and describe an alternative vision.

To begin, I want you to recall an occasion on which you have seen fireworks. Perhaps as a child you were startled to realize that a distant flash and a somewhat later boom were caused by the same explosion in the sky. Let's call that the fireworks effect. No doubt some adult explained to you that the reason you had that conscious experience was that the light, traveling much faster than the sound, arrived at you before the sound. You, the observer, are located at a point in space, and when light and sound (and aromas and heat and so forth) reach that point you become conscious of them. This introduces the idea that there is a sort of finish line somewhere in your brain; crossing this line marks the onset of consciousness of any item or content. This finish line can also be seen as the front door of memory, and the idea has been very nicely expressed by the Oxford philosopher, Michael Lockwood. In a paper criticizing my view (Lockwood, forthcoming), he says--as if it were obvious: "consciousness is the leading edge of perceptual memory." (As you see, my memory played a trick on me: I didn't remember his line exactly when I gave the organizers of this conference my title. I remembered him as saying, "perception is the leading edge of memory," but what he said was equally apt: "consciousness is the leading edge of perceptual memory.") And this is an idea which strikes many people--not least Lockwood himself--as so obvious as to need no serious discussion. It is my difficult job to explain to you why this is actually not only not obvious, but rather a subtle mistake.

The idea is certainly appealing. If we think about the fireworks, we can see just how appealing it is. Imagine watching "in slow motion" as a little girl experiences the fireworks effect. We see the light start to spread from the explosion (at the speed of light, of course), and soon, when it hits the retina, you say, "Well, she's not conscious of it yet--not quite yet!" After all, merely arriving at her retina isn't enough. You watch as the neural signal from the retina slowly travels up the optic nerve to a relay in the lateral geniculate nucleus, and then on to area V-1 in the cortex, and you say, "Well, still not conscious yet." It is tempting to suppose that somewhere slightly deeper, and at a time slightly later, something special must happen. At that instant--and not before then--the little girl becomes conscious of the light. Then at some much later instant the sound arrives at her ear and works its way slowly from the eardrum on up through the brain, until it too arrives at the imagined finished line at some still later time. This is the idea I want to examine. What it suggests, I think, is that consciousness is a very special sort of--shall we say--turning of the corner of an event in the world.

I want to contrast it with some other varieties of turning the corner. We can begin with two very simple cases: reflection, as in a mirror, and refraction, as through a lens.

[figure 1 about here]

Notice that in these cases the photons literally turn the corner, and these events can be very precisely located in time and space. Another variety of corner turning is known as transduction.

[figure 2 about here]

Two different kinds of transduction are shown here, artificial and biological: in one, we have a photo-cell, a mechanical device designed to "transduce" the light: the light enters the device and that triggers a switch in medium, from light rays to sound, in this case: the transducer goes "beep!" whenever the light hits it. In the biological case, we have a cone cell in the retina; when the light strikes the cell, that's the end of the line for the light wave. What happens thereafter is an electrochemical burst of activity in the nervous system.

Now what about (conscious) observation? Is the entry into consciousness just another sort--but a very special sort--of transduction?

[figure 3 about here]

The red light strikes the conscious observer's eye and . . . eventually the person says, "red light!" or presses a button, or steps on the brakes of his car, or performs some other act that manifests that the corner has been turned. But now, what can that very special kind of corner turning be? That is the question that faces us.

If we look in the history of science and philosophy for answers to that question, perhaps the most famous, but also the answers to that question, perhaps the most famous, but also the Descartes. Here is Descartes's own diagram:

[figure 4 about here]

According to Descartes, the light from the arrow on the right fell on the eye and caused a mechanical action in the retina-- that's transduction into a different mechanical medium--and this caused further mechanical effects (waves) in the cerebro-spinal fluid (which he called "animal spirits"), which in turn arrived at yet another sort of transducer, right in the middle of the brain: the pineal gland. Now, finally, we get transduction into a new medium, the medium of consciousness itself. Some sort of miraculous change of medium occurred, turning the mere vibration of the pineal gland- -a mechano-physical action--into something mysterious. This was truly inexplicable, according to Descartes, but mysterious or not, it was a sudden transition into a new medium, the immaterial or non-physical medium of consciousness. Then, if the person (the person's immaterial mind) decides to point at the arrow, there must be a reverse transduction: a decision, which is immaterial, must somehow set in motion the animal spirits, which pump the muscles up, which cause the hand to go up.

Now today, and actually for quite some time, Descartes's model has been in disrepute. This miraculous transduction into a completely different medium has been rejected. Most modern theorists are materialists; they've thrown away the Cartesian "dualist" idea of an immaterial medium, but I claim that they haven't thrown away quite enough. They've left behind the idea that consciousness is transduction into a very special medium in the brain. This imagined medium is a physical medium, but it is unique in some way that is yet to be discovered. And whatever the medium turns out to be, the passage into the state of consciousness is very precisely timeable and locatable as a sort of transduction. I call this idea Cartesian materialism, and it's simply wrong. In fact, neuroscientists have known it is wrong for quite a long time, but one does not have to go back very far to find explicit endorsements of the idea. C. S. Sherrington was a great neuroscientist earlier in the century, and one could hardly improve on his expression of the Cartesian view:

The mental action lies buried in the brain, and in that part most deeply recessed from outside world that is furthest from input and output. (Sherrington, 1934)

Neuroscientists today know, and have known for some time, that there is no place in the brain--such as the pineal gland--where everything funnels down together to a point or portal where this "special transduction" occurs. We now have elaborate maps of functional neuroanatomy, and the detail in them is being added to at a great rate, and there is no one place on these maps "where consciousness happens." It is commonly accepted that consciousness is somehow distributed in space and time throughout the brain, but many neuroscientists--not all of them--have difficulty seeing the implications of this. Recently, working with the neuroscientist, Marcel Kinsbourne, I published an article about this issue in the journal Behavioral and Brain Sciences (Dennett and Kinsbourne, 1992). One of the virtues of this interdisciplinary journal is that every article that appears in it is accompanied by several dozen commentaries by other experts in the relevant fields, together with a response by the author or authors of the "target article". It was fascinating to me to see that the scientists who submitted commentaries on this article were divided roughly into three groups: those who denied that anyone in the field was so foolish as to maintain Cartesian materialism, those who applauded our attack on what they claimed was indeed a pernicious confusion in the field, and those who were impervious to our attack, and staunchly defended Cartesian materialism! So this is today a controversial issue in the neurosciences.

Here is a diagram inspired by Lockwood's remark about the leading edge.

[figure 5 about here]

I take it to represent the covert, or tacit, or underground image that haunts the thinking of many people working on the mind today. We can read the diagram from left to right, following the sequence in time of events involved in conscious experience of external events. (It does not matter that the event experienced be an external, perceived event rather than an internal, introspected event, but for simplicity I will concentrate on perceptual cases, and more particularly on visual perception.) Light from events reaches the eye, and this is followed by processing in various parts of the brain that is all unconscious processing. Or perhaps we should call it preconscious processing. It is in the medium of neuronal activity in various tracts--we need not be more specific for our gene

One of the virtues of this diagram, apparently, is that it does justice to a list of truisms:

  1. Light must strike the retina before processing can begin.

  2. Processing must happen before consciousness can happen.
    (We're not directly aware of the light that falls on our retinas.)

  3. Consciousness must happen before you can remember an experience.
    (That, I would suppose, is a tautology.)

  4. A memory must be laid down before you can report it.
So this model seems at first simply to illustrate some undeniable truisms about the nature of consciousness. But now I will show you an example of a phenomenon that exposes the difficulty with this model. (I hope it's apparent that what I'm doing, unlike the other speakers at this conference, is focussing in on a very microscopic issue of time and space, and the precise nature of the way in which our memories are loaded with the experiences that we have. On any large scale there is no denying that consciousness is the leading edge of memory. The mistake that I'm exposing is the mistake of supposing that what works so well on the large scale of seconds or minutes, hours or days or years, breaks down when you try to apply it on the microscopic scale of fractions of a second.)

My illustration is the phenomenon known as meta-contrast, a visual illusion that has been much studied by experimental psychologists. If you were a subject in a meta-contrast experiment, you would sit watching a screen, on which shapes would be very briefly flashed. Suppose for instance that a colored disk is flashed briefly in the center of the screen. You would have no difficulty at all seeing the disk--this is not a "subliminal perception" experiment. You would be able to say "a blue disk/ a red disk/ a green disk," with never an error. The flash is long enough and bright enough so that anybody can see it. However, if they were then to follow the flash of the disk by flashing a ring in such a way as to surround the space where the disk was, what you would see--or report--would depend on how long the delay was before the second stimulus, the ring, was flashed.

[figure 6 about here]

If the interval of time between the two stimuli is made very small- -a few milliseconds, a few thousandths of a second--a remarkable thing happens. All you see (well, all you say that you see) is just the second stimulus. You don't see the disk at all, you see only the ring. It is a stunning effect. (I have a videotape of the phenomenon I like to show to skeptics. There seems to be a disk flashing on the left and a ring flashing, slightly later, on the right. In fact, there are two disks flashing every time, with the same duration and intensity.)

Now when researchers first started theorizing about this, they were tempted to tell a story like this:

[figure 7 about here]

The disk arrives at the eyeball first, of course, and it starts getting processed on its way up, up, up through the nervous system, and the ring arrives at the eyeball slightly later. Somehow it overtakes the disk! It intercepts it, and ambushes it on its way to the theater of consciousness, so that the only thing that enters the theater is just the second stimulus. Theorists who thought this way then wondered how the ring message came to be accelerated through the system. How could it "catch up" and "pass" or "intercept" the disk message?

Before we try to answer that question, notice that there's another story that could be told about what happens in meta- contrast: the disk does make it all the way to the theater all right; it has its brief moment in the limelight as it crosses the stage; it's only afterwards, in memory, that there's dirty work done. The tampering happens after the disk has come in the front door of memory. The memory of the second stimulus, the ring, erases the memory of the conscious experience of the disk.

[figure 8 about here]

These are apparently two different theories of the same phenomenon. I want to give them simple, memorable names, because the contrast they exhibit arises again and again. Figure 8, in which the dirty work happens after the presentation event in consciousness, depicts a contamination of memory, so I'll call it the Orwellian theory, recalling George Orwell's novel 1984. You will recall, in the Ministry of Truth, the evil historians who rewrote the state archives after the fact, concealing from all future investigators what really happened. So figure 8 is the Orwellian theory of meta-contrast. To echo an expression that Dr. Petella uses in his contribution to this symposium, the Orwellian model of meta-contrast diagnoses it as a hallucination of memory; you simply fail to remember something that you really experienced, and remember something different instead.

The earlier theory exhibited in figure 7, which says that the dirty work happens before consciousness and is then accurately recorded thereafter, I shall call Stalinesque, because it reminds us of Stalin's "show trials" in the 30s, where elaborately staged counterfeit events were presented in a show trial, and then accurately recorded for the archives. There was no tampering with the archives; the tampering came before the show.

So now we seem to have a question for scientists to answer: Is the truth about meta-contrast Stalinesque or Orwellian? But notice that the only difference between the two hypotheses is whether the dirty work is taken to happen before or after the postulated "leading edge." It may seem that it just has to be one way or the other, that there's no third alternative. I would like to show you that there is. There's nothing necessary about the particular model of consciousness shown in figure 5. Consider a different model. (I trust you can see that I've simply turned most of the first model on its side.)

[figure 9 about here]

In this model we have processing continuing along in time and memory continuing along simultaneously, in parallel. And we've lost the vertical line in the middle; we've lost the "leading edge." Look what happens when we superimpose our question about meta- contrast on the new diagram. There may be some real uncertainty or ignorance about just when and where in the brain the interference happens between the effects of the first stimulus and the effects of the second stimulus. Eventually, we can resolve this ignorance by further scientific investigation, and in the meantime we can represent all the possible alternatives by sliding the diagram of the interference from left to right across the diagram. Does the interference happen relatively early

[figure 10 about here]

or relatively late?

[figure 11 about here]

Locate the interference wherever in time you like; this will not answer the question: Orwellian or Stalinesque?--because the defining feature distinguishing the two apparent possibilities is no longer in the model. There is no finish line!

[figure 12--first elab-- about here]

Let me elaborate on this alternative model. According to figure 12, your visual system decomposes its tasks into separate transductions--separate corner turnings--which determine various visual properties in different places in your brain. Shape, color, motion, location are fixed in different places and at different times, for even a single event, such as the flashing of a colored disk. These properties, once transduced, are then available for influencing later transductions, later bindings, revisions, erasures. Your perceptual judgments evolve gradually, but since they continually replace their predecessors, your brain normally keeps no record of before and after, and hence you are unable to detect this revision process--though its traces can be uncovered by subtle experiments. In the case of meta-contrast, if you are shown sometimes a single stimulus--the ring--and sometimes both stimuli, you will claim in each case to have seen just a single stimulus, the ring, but if you are required to guess each time whether it was preceded by a disk, your guesses will be substantially better than chance, which shows that some residual effects are still in your brain.

You may well wonder what the horizontal line dividing processing from memory signifies in figure 12. What work is that line doing? The answer is: it isn't doing any work; it is, in fact, simply a vestigial trace of the bad model in figure 5. On the alternative model, there isn't any real boundary in time or space separating processing from memory. Now this is really not so surprising, in the light of the other presentations at this colloquium, for a common theme in them has been that both memory and perception are constructive processes evolving their constructions in time, revising, embellishing, dissolving, changing. The mistake lies in supposing that in addition to these editing processes, there is a privileged process that amounts to the "official" presentation of a canonical version (rather like the frames of a film being illuminated in turn by a sort of Cartesian cinema projector.

This alternative model, which I call the Multiple Drafts Model, fits the neuroscientific facts better, much better, than the model in figure 5, the Cartesian Theater. In the case of meta- contrast, for instance, here is what happens if you are shown just a single stimulus, the disk or "first" stimulus:

[figure 13 about here]

The first thing that your brain decides is simply that something has happened--you don't yet know what. If you give the brain enough time, it will go on to determine that what happened was, lets say, on the left, and that it was a circle, and then that there was some blue, and finally these contents get bound together to create the discriminated content: there was a blue disk. What is going to be the future of that blue-disk-content bound together? It may almost immediately deteriorate and have no more effects at all. Or if the green ring doesn't come along, it may not only hang around, but be recapitulated, reflected upon, and each time that happens this may further consolidate it so that even years later you will remember that blue disk.

[figure 14 about here]

But notice that however many times this happens, there is not a first instant at which you are conscious of a blue disk. The content blue is actually determined by your brain after the content circle, and the content that these two contents go together is something that comes along fractionally later still. But of course your experience is not one of first realizing that something's happened, then realizing that there's a circle shape, then realizing that it's blue or that there is blue in the world, then and finally realizing that the blue goes with the circle; you can't distinguish that sequence, but in fact, that sequence is occurring in your brain.

This temporal freedom provided by the Multiple Drafts Model permits us to explain other initially puzzling, even apparently paradoxical, phenomena, and in the time remaining,I will briefly present one example. For almost a hundred years, psychologists have studied phenomena of apparent motion, known as phi phenomena. We are all familiar with phi phenomena; they are the basis for motion pictures and television. The rapid succession of stationary shapes slightly displaced creates the illusion of motion. In the simplest cases (which are the best cases for psychological research), single spots of colored light are the stimuli. If a little red light is flashed on a screen in front of you, and then another little red light is flashed on the screen slightly to one side or the other, you will see what appears to be a single moving spot of red light.

The philosopher Nelson Goodman once asked the psychologist Paul Kolers some years ago, what happens if the lights are different colors? (Goodman, 19xx[add to bib]) For instance, what if you flash a red light, and then you flash a green light? Will there be apparent motion? Kolers and von Grnau ran the appropriate experiments, and the answer is: yes, there is motion. Now you may well wonder: what happens to the color of the "single" light that you see? It starts off red and then there is an abrupt mid-trajectory change to green. But this is an illusory trajectory, of course, not a real trajectory, and this presents a puzzle.

[figure 15 about here]

Your brain cannot create the content of a mid-trajectory color change--it cannot create frames C and D in the metaphorical diagram--until it has received and analyzed the second stimulus (as represented by frame B in the diagram). It has to "know" that there's a second light, and it has to know what color it is, before it can start creating the illusion that we observe in this case. A Stalinesque theory to "solve" this problem would be to suppose that there's something like a "delay loop" in the brain: that A and B arrive in sequence at some antechamber, some editing studio, somewhere between the eyeball and the theater of consciousness. And in that studio, during that brief delay but after B has arrived and been recognized, frames C and D are rapidly constructed or confabulated, and inserted into the film that is then sent up to the theater for viewing in a Stalinesque show trial.

But apparently there's another, Orwellian, theory which also could explain the illusion. According to it, you're conscious of A, and then you're conscious of B, and then your memory plays a trick on you. Frames C and D are spuriously inserted in memory by the Orwellian historians after the fact (of consciousness). Almost immediately you seem to remember having seen motion occurring between A and B, but this illusion, represented by frames C and D, is simply a contamination of memory.

Now which theory might be the truth? The Multiple Drafts Model declares that neither one is the truth. The truth is that the brain is quite capable of putting retrospective content elements into it's narrative stream. It can decide there's a circle on the left and it's red and there's a circle on the right and it's green, and that there must have been change in between them. But this natural but mistaken conclusion is "pre-dated": it is given a "postmark" which places it at an earlier time in the sequence in your own stream of consciousness. Now this is an idea that many people find extremely hard to accept because it suggests to them that there must be some sort of backwards causation in time, or the "projection" backwards in time of a later event. For instance, the Oxford physicist Roger Penrose, in his recent book The Emperor's New Mind (1989), suggests that we have to have a revolution in physics in order to explain these effects.

What such phenomena actually show is indeed that the subjective sequence of conscious experience does not always line up with the objective sequence of the events in your brain that determine your subjective experience. Graphically, experienced time can have backwards kinks in it when we map it onto objective time.

[figure 16 about here]

The order in which events seem to happen to you in your stream of consciousness is not the same as the order of events occurring in your brain which are the very vehicles of those contents in your experience. I want to show you, however, that this idea is not as strange and revolutionary as it first appears. In fact, we are already quite comfortable with it when it is applied to a different dimension: space. Consider this diagram of a periscope.

[figure 17 about here]

You know that when you use a periscope, you experience a rather striking effect: the light bounces off the mirrors into your eyes, and this has the effect of shifting your point of view, almost miraculously, up to where the top mirror is. That is where your eyes seem to be, that is where you seem to be when you use a periscope. The actual events in your brain that accomplish vision are happening down in the brain, but where you seem to be is translated up by a couple of mirrors which preserve the content of vision as it would be in the higher location. This does not involve any mysterious projection in space of some ghostly or immaterial eye or mind; it is merely a logical projection. It is a projection that is embedded in the content of your vision, not a property of the vehicles of that content. In an exactly parallel way, phenomena such as the phi phenomenon (and other, more complicated phenomena discussed in Dennett 1991 and Dennett and Kinsbourne, 1992), show that the brain can create what we might call temporal periscopes, curious occasions when time itself is apparently bent by the way the brain deals with the events falling on it.

This has some rather striking implications. What we learn from the periscope is that the idea of here--the observer's spatial location--is fixed by the content, not by the physical location of the brain events the neural events that are its vehicles. It is also true, I am claiming, that the subjective sense of now--the observer's temporal location--is fixed by the content of those brain events, not by their temporal locations. That is, the temporal sequence of subjective experience is not fixed by the sequence in which the relevant events actually happen in in the brain, but by the sequence that they represent. In other words, for the same reason that subjective location is not to be equated with some location of transduction, temporal location is not to be equated with some time of transduction. Notice that the apparent location of your eye when you use a periscope (see figure 17) is not due to a special transduction event. Reflection in a mirror is not transduction at all--there is no change in medium. The transduction of the light actually happens in your eye, and it is followed by later transductions and other operations in the brain, but the apparent or subjective location of the observer--of you--is determined by the content (not by the vehicle), which is fixed by the structure of the light at that point. By the same token, subjective timing--subjective sequence and subjective simultaneity, which constitute the order in which your stream of consciousness unfolds--is not actually determined by the order of the contentful events that occur in your brain, but rather by the content: by the sense that your brain makes of all of those contents.

[figure 18 about here]

One last little story will illustrate my point. Figure 18 is in fact an early diagram of the brain by Vesalius. Right in the middle, marked "L" is the pineal gland. But I want to make opportunistic use of his diagram to illustrate my main point on a different scale of space and time. Let's pretend that this is a map of the Earth. "L" can stand for London. "G" can stand for Ghent. What is known in American history as the War of 1812 was fought between the British and the Americans, and on Christmas Eve, 1814, in Ghent, the two opposing nations signed a peace treaty. The news of that signing thereupon began to travel out around the globe in all directions at a rather slow pace. It arrived in London, no doubt, within a few hours, at most a day, after the signing of the treaty in Ghent. It arrived in New Orleans too late to prevent a battle, the notorious Battle of New Orleans, which was fought two weeks after the treaty was signed. Over a thousand British troops were killed.

Now suppose we were to ask the following somewhat bizarre question: when did the British Empire learn about the signing of the truce? The ambassador in Ghent learned about the signing of the truce instantaneously; he watched his own hand sign the treaty. The members of Parliament, and the King, and the other officials in London learned it some time later. The poor commander of the British forces near New Orleans learned it only too late, alas, several weeks after the event. Suppose we knew to the day, to the minute, to the second, when each element, each agent, of the British Empire learned of the signing of the truce. This still wouldn't tell us when "the British Empire" learned of the signing of the truce, because no one of those agents counts as the place where the British Empire resides.

You might be tempted to say this is false: what matters is when the King learns. As Louis XIV said, "L'etat, c'est moi!" But in this instance, the King was George III, and it really didn't make much difference when he learned things! He was not really in charge. So the best we can do, in answering the question about just when the British Empire learned of the signing of the truce, is to say something along the lines of "late 1814 to early 1815." In exactly the same way, since you are not located in any one place in your brain, but are rather distributed around throughout your brain--since Descartes was wrong about there being a point in the brain "where it all comes together"--if you ask yourself the question, "When did I become conscious of some particular event?" that question can have only a vague answer, not a precise answer. It could have a precise answer only if we could locate you at some point in your brain. Since the transmission of information around in the brain is relatively slow, the dating of events in consciousness--the dating for you--has to be smeared over maybe as much as 200 msec, a fifth of a second.

And so, in conclusion, we see that the time of becoming conscious cannot be precisely defined, and it follows from that that although consciousness is, as tradition would insist, the door into memory, it is not a sharp edge. There isn't any such moment as the instant of onset of consciousness.

REFERENCES

Dennett, D., 1991, Consciousness Explained, Boston: Little Brown.

Dennett, D. and Kinsbourne, 1992, M, "Time and the Observer: the Where and When of Consciousness in the Brain," Behavioral and Brain Sciences, vol. 15, pp183-247.

Lockwood, M., forthcoming, "Dennett's Mind," in Inquiry, March 1993.

Penrose, R., 1989, The Emperor's New Mind, Oxford: Oxford University Press.

ington, C. S., 1934, The Brain and its Mechanism, Cambridge University Press.