University of Southampton

2nd Year Practical: Categorical Perception

To get to the subject-coded version of the reading list or to the full reading list with abstracts click here.

Instructor: Stevan Harnad

After you have learned something -- anything at all -- you are no longer the same person, in the sense that you now know (consciously or unconsciously) something you did not know before. Usually you can also do something after learning that you could not do before (if only to say "oh yes, I know that," whereas before, you did not). It also often means that you do not see things quite the way you did before: A world in which I know what are and are not "sheep" is different, even if ever so subtly, from a world in which I do not know what sheep are, even if I had seen plenty of sheep before learning what they were.

This Practical will consist of projects that investigate the nature of this all-important change that occurs whenever we learn a new category. The category learning can be anything, from the most trivial ("Henceforth Tuesdays will be called "Wizzentoffals") to the most profound (learning to identify new, unfamiliar faces, cancerous and noncancerous cells, the works of a particular composer or artist, or artistic style, the pecking order of a social group, the personality types one encounters, cultural traits, prejudices, etc.). And the category learning can be direct from trial and error experience or from explicit verbal instruction. The method will be simple. You will be given several ways to measure how things look to your experimental subjects before category learning; then they will learn to categorise in a certain way; then how things look to them will be measured again afterward. We will try to draw conclusions about the nature of the change in our internal "representations" as we learn.

The theme of the Practical is "Categorical Perception" (CP). CP occurs when we categorise things, and categorising things is what most of cognition is about. You're categorising when you eat some things and don't eat others, sit on some things and not on others, call some things by one name and not others. Most of our lives we are sorting things out, and what we are sorting them into is categories.

"Don't categorise! Don't generalise!" We're often warned that we shouldn't pigeon-hole or stereotype people or things. But aren't I "pigeon-holing" whenever I call a person by his name every time I see him? After all, people never look exactly the same, every time you see them, so there are a lot of changing, irrelevant features (like circles under the eyes from sleeplessness) that I must be ignoring, and other, relevant, unchanging features I must be paying special attention to every time I successfully identify a person with his proper name. Indeed, when I correctly identify someone, I'm making the generalisation that anyone who has exactly those relevant, unchanging features is him.

The same is true if I identify people by surname, categorising them as members of a family, or a football team, or a neighborhood, a tribe, a nation. Yet, surprising as it may sound, no one yet has any idea how people manage to do all these everyday things! No one today could make a machine that was able to sort and identify people and things as we can.

Stereotyping and pigeon-holing go wrong when unwarranted generalisations are made from the categorisation. A good generalisation to make from the fact that someone plays for Manchester United would be that it's a good investment to bet money on his team's winning their next game. A bad generalisation would be that their opponents are a bunch of vile yobs whose supporters need to get their heads bashed in. The first kind of categorisation and generalisation is what brought us most of what is good about human culture and civilisation (language, art, science, technology, commerce); the second is what brought us the rest (authoritarianism, bigotry, racism). In studying CP you are studying both.

Being able to categorise things correctly (as in betting on Manchester United) is clearly useful, but there is evidence that the capacity is always purchased at a price: Although it may be subtle, your view of the world is biassed every time you learn to categorise things in a certain way. Things no longer look quite the same to you any more. This bias, called the CP (categorical perception) effect is what we will study experimentally in this Practical.

The most familiar example of categorical perception (CP), is unfortunately associated with an ethnic stereotype: It has been called (by Americans) the "Chinese-Waiter" effect, but of course it might just as well have been called by the Chinese the "American-Waiter" effect! It refers to the fact that, to Americans living in America, all Chinese waiters seem to look alike, whereas American waiters all look individual and distinct. A similar well-known example of CP (though it is actually not valid, see Pullum 1989) is that of Eskimo words for snow: Allegedly, Eskimos see many different categories of snow where we just see that one, uniform, white stuff. Another celebrated example is color categories: Which shades will look to you like shades of the same color (like scarlet and vermillion), and which shades will look like qualitatively different colors (like red and green) depends on which culture you happen to be in and which language you happen to speak. A last example is language itself: Native speakers of a language can hear and produce differences in sound categories (phonemes) that to a non-native sound alike (and are unpronouncable). Speakers of Chinese and Japanese can hardly hear, and usually cannot pronounce, the difference between an r and an l; English speakers have the same kind of difficulty hearing or pronouncing the difference between b and bh in Hindi. All of these are examples of CP: perception has been biassed by categorisation.

This CP phenomenon -- the compression and expansion of similarities and differences between things, depending on whether they belong to the same category (in which case they look similar) or to different categories (in which case they look different) -- has an interesting history, occurs in many very different domains (sight, hearing, language, social perception), and is now coming to be understood better as its relation to the categories we are born with and the ones we learn is experimentally studied in people and animals, modelled on computers, and investigated in the brain.

General reference (be sure to read the first and the last chapters of this book -- you can get them by clicking here):

Harnad, Stevan (1987) (Ed.) Categorical Perception: The Groundwork of Cognition. Cambridge University Press.

There are 8 copies of this book on reserve in the library and 2 more copies for longer term loan. I've also made the abstracts of hundreds of papers and the full texts of several papers on categorisation and CP available on Mosaic World-Wide-Web, including the first and last chapters of the CP book and all papers by me (everyone in the class should log on and use the Web!):

Whorf, B. L. (1964) Language, thought and reality. Cambridge MA: MIT Press

Pullum, G. K. (1989) The great eskimo vocabulary hoax. Natural Language and Linguistic Theory 7: 275-281.


Here is the basic experimental paradigm. It always involves 3 simple steps (plus room for lots of variations):

You start with a large number of stimuli -- many hundreds of them. Your subjects will be taught to sort them into 2, 3 or more categories. For this example, I will use lots of photos of one pair of identical (or nearly identical) twins (and I mean lots of photos of them: the best way to get huge numbers of photos is to taking frames from a video). This is the stimulus set; each photo is a different stimulus

I. MEASURING PRE-CATEGORISATION SIMILARITY: The first thing you need is a measure of how similar the photos are to each other before any category learning takes place. One of several ways to do this is to show each subject two photos and ask him to indicate on a scale from 1 - 9 how similar he finds them (1 = least similar, 9 = most similar). A large number of pairs of photos has to be pretested in this way.

There are other methods of measuring similarity too, some of them more appropriate for variants of the basic CP paradigm:

(i) ABX discrimination is one in which a pair of stimuli is presented, one after the other, and then a third stimulus is presented, which is either the same is the first one or the second one, and the subject has to say which. In CP, ABX discrimination may get better between categories and/or worse within categories

(ii) Signal detection analysis can be used to calculate d', which is a measure of the detectability of small differences between pairs of stimuli. d' should grow between categories and/or shrink within.

II. CATEGORISATION TRAINING: Next, the subjects are trained to categorise, by trial and error, with corrective feedback: They have to say for each photo whether they think it's twin X or twin Y, and after each response they get feedback telling them which twin it actually was.

III. MEASURING POST-CATEGORISATION SIMILARITY: Once your subjects' accuracy reaches a high level (say, 95% correct for a run of 100 photos in a row), they are again given the similarity judgment task, using the same pairs of photos as before.

A variant is to use a control group to measure pre-categorisation similarity, instead of using the same subjects both pre and post. This variant will be useful for some of the more complicated CP experiments below. (With the control group, you will have to think about ways that you could make them see the photos just as often as the experimental group does, but without receiving any category training. Another task, such as judging whether or not they have seen a given photo before, without feedback, might be a good control task, but you can think up your own.)

If there is a CP effect, the similarity ratings for X-X pairs and for Y-Y pairs should increase and the similarity ratings for X-Y pairs should decrease, when you compare them between pre-categorisation post-postcategorisation (or control vs. categorisation groups).

There is a statistical technique, called multidimensional scaling, that can analyse the pairwise similarity ratings in such a way as to locate each photo in a multidimensional space. The CP effect can then be seen as changes in each photo's "location" in that multidimensional space, with X's moving closer to X's, Y's moving closer to Y's, and X's moving away from Y's.

Now the twin/photo study is just one example of the basic paradigm. Here are ways in which that paradigm can be varied to design many different studies, each informative on some slightly different aspect of CP.

Harnad, S. (1987) Psychophysical and cognitive aspects of categorical perception: A critical overview. Chapter 1 of: Harnad, S. (ed.) (1987) Categorical Perception: The Groundwork of Cognition. New York: Cambridge University Press.

To get to the subject-coded version of the reading list or to the full reading list with abstracts click here.

1. Goldstone, Robert. Influences of categorization on perceptual discrimination. Journal of Experimental Psychology: General, 1994 Jun, v123 (n2):178-200.

Homa, Donald; Rhoads, Deborah; Chambliss, Daniel. Evolution of conceptual structure. Journal of Experimental Psychology: Human Learning & Memory, 1979 Jan, v5 (n1):11-23.

Harnad, S. (1992) Connecting Object to Symbol in Modeling Cognition. In: A. Clarke and R. Lutz (Eds) Connectionism in Context Springer Verlag.


NUMBERS OF CATEGORIES: The number of categories can be increased from 2 to 3, 4 or many more. Instead of many photos of two twins, you could use many photos of 3 or more same-sex siblings that look very similar, perhaps across several years as they get older. As you use more and more categories, it will become less important that they should be very similar. The task of learning the names of a dozen or more unrelated people of the same age, sex and race is already a sufficiently difficult one. All you need is many non-identical photos of each of the people for your large stimulus set.

LEVELS OF CATEGORIES: The categories need not be individuals either; the task could be to sort photos into families: You could have many photos of individual members of 2, 3, 4 or many more families, and the subjects have to learn the surname of the family each individual belongs to.

An interesting two-stage experiment would be to start with an individual categorisation task (say, with many photos of 12 individuals -- this time they need not all be very similar). Once the subjects have learned to name the 12 accurately, you could then train them on a higher-order category: that the individuals are actually members of, say, 4 families. In this experiment you would measure similarity three times, not just twice: Once before and once after they had learned to name the 12 individuals, and then once again after they learn to sort them into their 4 respective families.

An interesting variant of this experiment would be to do it in reverse order: Train the family categorisation first, then the individual categorisation. (In real life, it sometimes happens one way, sometimes another; it is not known how the pattern of CP effects -- or even the ease of learning -- varies depending on whether learning goes from the general to the particular or from the particular to the general.)

(In this and other experiments it may be easier to have control groups at each stage, rather than measuring similarity more than once for the same subjects.)

TYPES OF STIMULI: The kind of stimulus can be varied: There are plenty of other kinds of categories besides faces: You could use natural categories, like plants (individual species, or higher-order families), mushrooms (individual types, or higher-order categories like amanitas and non-amanitas, or even "edible" versus "inedible"). Many hard natural categorisation tasks can be found in biology, for example, categorising slides of tissue as cancerous vs. noncancersous. Or you could choose human categories, such as identifying paintings by 2 or more different painters, paintings in two or more different styles or periods.

To get to the subject-coded version of the reading list or to the full reading list with abstracts click here.

21. Etcoff, Nancy L.; Magee, John J. Categorical perception of facial expressions. Cognition, 1992 Sep, v44 (n3):227-240.

36. Lynch, Michael P.; Eilers, Rebecca E.; Bornstein, Marc H. Speech, vision, and music perception: Windows on the ontogeny of mind. Special Issue: Child development and music. Psychology of Music, 1992, v20 (n1):3-14.

77. Ludemann, Pamela M.; Nelson, Charles A. Categorical representation of facial expressions by 7-month-old infants. Developmental Psychology, 1988 Jul, v24 (n4):492-501.

291. Siegel, Jane A.; Siegel, William. Categorical perception of tonal intervals: Musicians can't tell sharp from flat. Perception & Psychophysics, 1977 May, v21 (n5):399-407.

ARTIFICIAL CATEGORIES: Or you can use a computer or hand-drawing to design artificial "categories" of your own. You can invent ways of varying patterns, and decide which variations will "matter" (i.e., which will be features that separate categories) and which variations will not matter. The advantage of artificial stimuli is that you can control for the degree of difficulty of the category learning, as well as for the degree of similarity between the stimuli. Some of the questions that could be better addressed with artificial stimuli than natural ones include: What is the difference between easy and hard categorisation tasks (besides that the latter take longer to learn)? Does CP occur only with hard tasks? Do CP effects get bigger, the harder the task? What's the difference between a category learning task that is based on a "natural" rule and one that is based on a random one? (An example of a "natural" rule might be that a mushroom is edible if it is small and light brown with a fringe; an unnatural rule would be that it is edible if it is small but not brown or fringed, or fringed but not small or brown, or brown but not small or fringed.)

AUDITORY STIMULI: You can also use sounds instead of visual stimuli: natural calls of different species of birds, sounds of different categories of musical instruments, different rock groups (but you must be careful to control for prior experience in that case, otherwise differences in how familiar the music is to different subjects may eclipse the experimental effects), different rock styles, or synthetic sounds you generate yourself, by audio or with a computer. All you need is a large number of them, and some basis for categorising them, and some factor or hypothesis about categorisation or CP that you wish to test.

SOCIAL CATEGORISATION: You can also investigate social categorisation. One interesting question concerns "fellow traveller" features: Supposing you teach subjects to sort photos of people into two categories: "successful" and "unsuccessful." The real rule on which you base the feedback might be something very complicated (based on features like the quality of the clothes the person in the photo is wearing, their age, their facial expressions, their hair styles, etc.). But correlated with the real rule, which might be complicated and somewhat arbitrary, there would be an incidental feature, such as whether or not there is a birthmark on the face. There would be both successful and unsuccessful faces with birthmarks, but there would be noticeably more successful faces than unsuccessful ones with birthmarks. But the subjects are being trained to categorise the faces as successful and unsuccessful, not as having or not having birthmarks. The test then becomes: (1) Will there be a CP effect based on the birthmarks alone (i.e., compared to pretest, will there be compression/separation based on birthmarks, rather than just success)? (2) Will there be a greater tendency to categorise new, untested faces as successful if they have birthmarks than one would predict from the actual size of the correlation between success and birthmarks? Is this bias stronger with social categories than with neutral sensory ones? (This topic can be related to matching/maximising in reinforcement theory and to biases in human judgment, for those who are interested.)

Krueger, Joachim; Clement, Russell W. Memory-based judgments about multiple categories: A revision and extension of Tajfel's accentuation theory. Journal of Personality & Social Psychology, 1994 Jul, v67 (n1):35-47.

To get to the subject-coded version of the reading list or to the full reading list with abstracts click here.

107. BOOK, EDITED Caverni, Jean-Paul, ed.; Fabre, Jean-Marc, ed.; Gonzalez, Michel, ed. Cognitive biases. North-Holland; Amsterdam, Netherlands, 1990. Series title: Advances in psychology, 68.

114. BOOK CHAPTER Feldman, Jack. Objects in categories and objects as categories. IN: A dual process model of impression formation. Advances in social cognition, Vol. 1.; Thomas K. Srull, Robert S. Wyer Jr., Eds. Lawrence Erlbaum Associates, Inc, Hillsdale, NJ, US. 1988. p. 53-63.

115. BOOK CHAPTER Rothbart, Myron. Categorization and impression formation: Capturing the mind's flexibility. IN: A dual process model of impression formation. Advances in social cognition, Vol. 1.; Thomas K. Srull, Robert S. Wyer Jr., Eds. Lawrence Erlbaum Associates, Inc, Hillsdale, NJ, US. 1988. p. 139-144.

CATEGORIES YOU CAN BOTH PERCEIVE AND PRODUCE: A special kind of category is one in which you can not only passively perceive it, but you can also produce it. The categories of speech phonemes are examples, but these have already been closely investigated. Music, less fully investigated, is another.

One kind of study here would be to take advantage of the fact that there is an analog production dimension and use it, instead of similarity judgment, as the measure for CP: For example, you might use a computer to generate a large number of squiggles (either using a mouse [computer-control] or using a graphics programme). Pre-categorisation (or control) subjects would then be shown the squiggles briefly and asked to copy them from memory using the mouse. A computer analysis could measure their accuracy. This could be compared to their accuracy after they have been trained to sort the squiggles into categories (as in the other experiments described earlier). The reason CP effects would be of special interest here is because they would show that it is not only how things look to your senses that changes when you learn to categorise them in a particular way, but also how you operate on them with your motor system.

Another variant along these lines would be to train the categorisation backwards, using production instead of category naming: The subject is first told the name of the squiggle category to produce, then asked to produce a member of that category, and if he is wrong, he is shown a correct member of the category as an example. In this variant the measure of similarity should be the standard one, similarity judgment, rather than imitation. The question is whether this kind of training produces a "motor bias" that makes squiggles produced by variants of the same basic movement pattern look more similar than squiggles produced by different movement patterns.

Perception/production experiments can also be done with sound patterns rather than squiggles, or any other stimulus that we can both perceive and imitate. In music, this variant can be used to investigate pitch perception/production (perfect pitch, relative pitch) and rhythm perception/production. Dance movements can be studied too.

To get to the subject-coded version of the reading list or to the full reading list with abstracts click here.

Lane, H. (1965) The motor theory of speech perception: A critical review. Psychological Review 72: 275 - 309.

18. Howard, David M.; Rosen, Stuart; Broad, Victoria. Major/minor triad identification and discrimination by musically trained and untrained listeners. Music Perception, 1992 Winter, v10 (n2):205-220.

62. Schulze, Hans-Henning. Categorical perception of rhythmic patterns. Special Issue: Rhythm perception, rhythm production, and timing. Psychological Research, 1989 Jun, v51 (n1):10-15.

SPATIAL CATEGORIES: Space is special because it is continuous rather than being divided into discrete categories. Nevertheless, we can and do categorise things spatially (for example, using X/Y coordinate systems, or the points of a compas). There is reason to expect that CP effects will be different or weaker with spatial stimuli. Try it out. Make your categories points in 2-dimensional space, and see whether training generates CP.

TEMPORAL CATEGORIES: Although time, like space, is continuous, we have a much greater tendency to divide it into discrete categories. An interesting effect to look at in the time-domain is "chunking": George Miller (1956) noted that although the magical number 7 +/- 2 is the limit on the number of "chunks" we remember, it can be increased by RECODING. If a person hears a and must repeat a series of short and long blips (as in Morse code), if the series is longer than about 7 he makes more and mistakes. If he learns Morse code, however (in which a is "short-long" and s is "short-short-short") then the number of blips he can remember becomes about three times as big: he can now remember strings of over 20 blips, but this is because he does not hear them as blips, but as letters in Morse code (and he can remember about 7 letters). The same is true for strings of zeros and ones, depending on whether or not you know the binary code. This recoding into bigger "chunks" is of course a form of CP, because in learning a code, you are learning to categorise (in Morse code short-long = "a"; in binary, 101 = "5"). The imitation method can be used to apply the CP paradigm to a temporal categorisation task like this.

Miller, G. A. (1956) The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review 63: 81 - 97.

LEARNING CATEGORIES BY VERBAL INSTRUCTION INSTEAD OF FEEDBACK FROM TRIAL AND ERROR: The reason language is so important and powerful is that it can spare us a lot of trial and error learning. If I simply tell you "It's safe to eat the small, brown, fringed mushrooms" then I have saved you a lot of time and trouble that would have been spent on trying (and perhaps getting a little sick from) a lot of mushrooms (provided you knew what "small," brown" and "fringed" means -- i.e., provided you had already learned those subcategories previously). Does verbal information produce CP effects? You can use the CP paradigm that has already been described to compare two category learning groups: one that learns by trial and error, the hard way, and one that is given verbal instructions. Both groups will need SOME examples, but the instruction group should reach the 95% accuracy level more quickly. Do both groups show the same amount of CP?

CATEGORICAL PERCEPTION IN CHILDREN: Adolescents and adults already have most of their basic categories already established and are probably getting most of their newer caegories by verbal instruction. Children (especially at the age of the "vocabulary spurt" when they learn as much as 50 new words per week) are in some ways much more dynamic subjects for the investigation of category learning. Do they produce bigger CP effects? Do they get them faster? Are they less able to form categories from instruction than from trial and error feedback? Most of the variants on the CP paradigm that have been decsribed here can be tried with children, and the results may well be surprising and interesting.

CATEGORICAL PERCEPTION AND LEFT-RIGHT DIFFERENCES IN THE BRAIN: The left hemisphere is specialised for language, the right hemisphere for visuospatial processing. The words of a language are the names of our categories.

It is always interesting to find tasks in which categories are NOT helpful, and perhaps even harmful. One example is mental rotation: Subjects are shown pairs of two-dimensional projections of three-dimensional objects. They are either two different objects, or the same object, but one in its standard position and the other rotated to varying degrees. Subjects are asked to say whether they are the same or different. It has been found that the subjects' reaction time is correlated with how big the rotation is, as if they were mentally rotating them to see whether or not they matched.

An experiment would be to train subjects to name the objects and then to present them in either the left or the right half-field of vision (which goes more quickly to the opposite side of the brain) for either (1) naming or (2) same/different judgments. The prediction is that performance will be better and/or faster in the right half-field (left hemishere) for the naming task and the left half-field (right hemishere) for the same/different task.

To get to the subject-coded version of the reading list or to the full reading list with abstracts click here.

Shepard, R. N. & Cooper, L. A. (1982) Mental images and their transformations. Cambridge: MIT Press/Bradford.

28. Rybash, John M.; Hoyer, William J. Hemispheric specialization for categorical and coordinate spatial representations: A reappraisal. Memory & Cognition, 1992 May, v20 (n3):271-276.

56. Kosslyn, Stephen M.; Koenig, Olivier; Barrett, Anna; Cave, Carolyn B.; and others. Evidence for two types of spatial representations: Hemispheric specialization for categorical and coordinate relations. Journal of Experimental Psychology: Human Perception & Performance, 1989 Nov, v15 (n4):723-735.

NEURAL NET MODELS OF CP: For those of you with some computer programming interest and experience, it may also be possible to do some modeling of category learning by neural nets.

Harnad, S., Hanson, S.J. & Lubin, J. (1991) Categorical Perception and the Evolution of Supervised Learning in Neural Nets. In: Working Papers of the AAAI Spring Symposium on Machine Learning of Natural Language and Ontology (DW Powers & L Reeker, Eds.) pp. 65-74.

Harnad, S. Hanson, S.J. & Lubin, J. (1994) Learned Categorical Perception in Neural Nets: Implications for Symbol Grounding. In: V. Honavar & L. Uhr (eds) Symbol Processors and Connectionist Network Models in Artificial Intelligence and Cognitive Modelling: Steps Toward Principled Integration. pp. 191-206. Acadamic Press.

To get to the subject-coded version of the reading list or to the full reading list with abstracts click here.