CATEGORISATION CAPACITY IN HUMANS AND MACHINES
Research Programme of the Cognitive Psychology Laboratory

Stevan Harnad
Professor of Psychology
Director, Cognitive Sciences Centre
University of Southampton
Highfield, Southampton
SO17 1BJ UNITED KINGDOM

phone: +44 703 592582
fax: +44 703 594597

A new Research Interest Group has been established in Cognitive Psychology within the Department, a Cognitive Psychology Laboratory has been created, and in addition, an interdisciplinary Cognitive Sciences Centre has been established with its base primarily in the Psychology Department and the Faculty of Social Sciences, but with very strong collaborative research links with the Department of Electronics and Computer Sciences in the Engineering Faculty, as well as Clinical Neurological Sciences in the Faculty of Medicine, and Philosophy, Archaeology and Linguistics in the Faculty of Arts.

Categorisation and Cognition.

Our capacity to categorise is at the heart of all of our cognitive capacity. People can sort and label the objects and events they see and hear with a proficiency that still far exceeds that of our most powerful machines. How do we manage to do it? The answer will not only tell us more about ourselves but it will allow us to apply our findings to enhancing our proficiency, both in the learning of categories and in our use of machines to extend our capacities.

Category Learning.

Category learning is the most general form of cognition. Animals learn categories when they learn what is and is not safe to eat, where it is safe to forage, who is friend and who is foe. Children learn the same kinds of categories, but they eventually go on to the much more powerful and uniquely human strategy of learning categories by name, rather then by performing some instrumental response on them, such as eating or fleeing. Whether they categorise by instrumental response or by name, however, children must still have direct experience with the objects they are categorising, and some sort of corrective feedback from the consequences of MIScategorising them. Eventually, however, categories can be learned from strings of symbols alone, with most of those symbols being themselves the names of categories. This is the most remarkable of our cognitive capacities, language, but language and cognition cannot be understood unless we analyse how they are grounded in categorisation capacity (Harnad 1990). This is theme of the present research programme.

The Behavioral, Computational and Neural Approaches.

There are three empirical ways to investigate the functional basis of our categorisation capacity. The first way is to (i) analyse our categorisation performance itself experimentally, particularly how we LEARN to categorise. The second way is to (ii) model our categorisation capacity with computers that must learn the same categories that we do, on the basis of the same input and corrective feedback that we get. The third way is to (iii) monitor brain function while we are learning categories, to determine what neural properties change during the course of learning, and to relate them to the performance changes during learning, as well as to the internal functioning of the machine models performing the same task.

Overall Behavioral Paradigm.

These three converging lines of investigation are the ones to be pursued in the Cognitive Psychology Laboratory, but in the initial year for which this proposal is submitted, we will pursue only (i) and (ii). The general experimental paradigm consists of presenting to human subjects and machines a large number of objects. In the first stage, the similarity of pairs of objects is measured, for both human subjects and machines. From the pairwise similarities, the position of each of the objects in a multidimensional similarity space is calculated. Then the subjects (or machines) are trained to sort the objects into categories by trial and error, with corrective feedback signalling them when they get it right and when they get it wrong. Once they have learned to categorise correctly, the pairwise similarities are again measured. Prior studies have shown that these similarities change as a consequence of category learning: Objects in the same category become more similar and objects in different categories become more different (Harnad 1987).

Corresponding Computational Paradigm.

The same thing happens with the machine models (Harnad et al. 1991, 1994), except that there we have the advantage of knowing HOW the model managed to learn the categorisation, and hence what functional role the within-category compression and the between-category separation in similarity space is playing in the successful learning of the category. Prior results indicate that internal representations of the objects move during the learning, as the similarity space is partitioned so as to carve out the new category. Boundaries are formed, separating what is on either side of them, and compressing what is within them. In perception, this effect is called "Categorical Perception" (CP), and it is known to underlie our color categories and speech-sound categories (Rosen & Howell 19987). Our current studies are showing that CP may underlie all of our categorisation (Andrews et al. in prep.).

The hypothesis is that this compression of the internal representations of members of the same category generates a portable "chunk" that allows further categories to be created out of higher-order recombinations of these chunks, each of them by strings of symbols, most of which are themselves the names of categories; these are exactly the same means as those being used in this very sentence to convey this theory to the reader (i.e., by a string of symbols grounded in our capacity to discriminate and categorise the objects in the world that the symbols in the string refer to; Harnad 1994).

Overview of Studies to be Conducted in the Laboratory:

Discrimination - Category Learning - Discrimination Paradigm.

The human experiments have the following pattern. A set of stimuli (visual or auditory) is computer-presented to a sample of subjects in pairs. The subject's task, depending on the experiment, is to discriminate the stimuli, either by (1) signalling whether they are the same or different, (2) indicating which of two pairs is more similar, or (3) indicating the degree of similarity on a 9-point scale. The results are analysed (a) using signal detection theory to determine the detectability of small differences between stimuli and (b) using multidimensional scaling and cluster analysis to determine the relative locations of all the stimuli in a multidimensional similarity space.

Next, subjects are trained to categorise the stimuli. The stimuli and categories are in some studies natural ones (as in classifying edible and inedible mushrooms, cancerous and noncancerous cells, male and female newborn chicks, photos of identical twins, etc.) and in other studies they are computer-generated artificial stimuli (patterns of varying degrees of complexity and interconfusability).

After pre-experimental pairwise similarity is measured, the subjects perform the experimental task, which consists of categorisation training. Samples of the stimuli are presented to the subjects for them to categorise, with corrective feedback provided after every attempt until a criterion for successful performance (e.g., an error-free series of 50 new stimuli) is attained.

After the categorisation training, the stimuli are again presented for pairwise comparison,to measure whether there has been within-category compression and/or between category separation.

Corresponding Neural Net Paradigm.

The human performance data are compared to those for neural nets that learn to categorise the same stimuli as the human subjects. For this there must be a measure of pairwise similarity of the stimuli as they appear to the net. I will illustrate how this can be done with one particular kind of neural net (delta learning rule with backpropagation of error-corrective feedback). The net is first trained to perform auto-association, that is, to match the stimulus that it gets as input with an identical output. In learning this, the "hidden units" mediating between input and output learn to change their strength of interconnectivity and hence their activations in response to any given input. In a trained autoassociative net, the activations of each hidden unit for each input can be treated as the components of a vector; interstimulus distance is then the Euclidean distance in that activation space.

After the net has mastered autoassociation, the pairwise distances between the hidden-unit representations of all the inputs are measured and then the net is trained to categorise. After the net has mastered categorisation, the pairwise distances are again measured and compared to their values before categorisation, as with the human subjects. CP occurs when there is within-category compression and/or between-category separation. Strong CP effects have been found with backpropagation nets (Harnad et al. 1991, 1994).

Experimental Questions to Be Addressed.

(1) What is the role of stimulus dimensionality in CP? The dimensionality of the categorisation task, the number of categories, and the degree of difficulty in finding the features that will partition the category will be systematically varied to see what role these play in category learnability and the compression/separation effect in similarity space (Staddon & Ying 1991). The hypothesis is that the magnitude of CP will vary directly with the degree of difficulty of the categorisation.

(2) Are there differences between adults and children in category learning and category learnability? Category learning will be investigated in adults and in children. The hypothesis is that children will exhibit much stronger CP effects, particularly during the critical years when the "vocabulary spurt" occurs (Garfield & Reznick 1990, Harnad 1994).

(3) What is the difference between categories learned from experience and categories learned from instruction? A new category can be learned on the basis of corrective feedback from trial and error experience, or else it can be taught by verbal description. The CP paradigm will be used to investigate the difference in terms of learning rate, learnability, and compression/separation effects in similarity space. The hypothesis is that direct experience will generate substantially stronger CP effects, but that verbal description will be able to generate them too, and that their magnitude will be proportional to (a) the magnitude of the CP effects for the subcategories of which the new category is verbally composed and (b) the degree to which the new category requires redrawing of the boundaries among subordinate categories or coordinate categories.

(4) How general an effect is CP among machine learning algorithms, and what is its function?

The generality of the strong CP effects that have been found with backpropagation nets will be tested with other kinds of nets such as Artmap (Carpenter et al. 1991). The hypothesis is that CP will occur with any learning rule in which stimulus proximity and discriminability is represented. The role of the factors investigated in (1) above will also be investigated in the computational models. The functional basis of the patterns found in humans will be sought in computational learning models, and the regularities found in the models will be tested in humans, using the same inputs where possible.

(5) Are hybrid symbolic/neural-net systems viable models for the "language of thought"?

Because of the symbol grounding problem (pure symbol systems are meaningless unless mediated by the mind of an interpreter, hence thinking cannot be just some computation; Harnad 1990), pure symbolic AI is not a viable model for cognition. The hypothesis is that hybrid symbolic/connectionist/analog models are better candidates. Both empirical and formal work will be done on the categorisation capacities of hybrid models, comparing performance in tasks of learning from anolog experience vs symbolic description, as in (3). The hypothesis is that hybrid systems will scale up to human categorisation capacity more readily than purely symbolic models or purely neurodynamic ones (Harnad 1992, 1993; Szepesvari & Lorincz 1993).

In connection with the third, neural component of the Laboratory's Reearch (iii) mentioned above we will also do joint work with the human electrophysiology Laboratory of Professor Sedgewick on the evoked potential correlates of category learning as well as the sensorimotor dimensions of categorisation, with support to be sought from the MRC as well as private biomedical companies. The paradigm will be the behavioral one, but this time neural nets will be used, not to model category learning, but to analyse the Event-Related Potentials, classifying them in terms of the category the subject is learning that the stimulus that eveokes them belongs to (Kloppel 1994). This provides a neural measure of pairwise distance between stimuli that is unobtrusive, involuntary, not subject to response bias, and measurable throughout the course of learning.

The interdepartmental, interfaculty collaborative dimension of this research within this university will be organised largely through the Cognitive Sciences Centre, of which I will be the Director, with the collaboration of Professors Tony Hey and Chris Harris of Electronics and Computer Science, Professor Wendy Hall, Media Lab, Professor Michael Sedgewick, Clinical Neurological Sciences, and Professor Bob Remington (ex officio), Head of the Psychology Department. The Cognitive Sciences are highly interdisciplinary, including Psychology, Neuroscience, Behavioral Biology, Artificial Intelligence, Robotics, Machine Vision/Speech, Neurocomputation, Linguistics, and subareas of Sociology, Archaeology and Philosophy (these are the departments at the University of Southampton with strengths in cognitive science; at different Cognitive Sciences Centres around the world the composition varies according to local strengths and representation). The common element uniting the Cognitive Sciences is a research interest in how the mind, and systems that can do what the mind can do (cognitive systems) work. There is a great deal of research activity in this interdisciplinary area. As the Laboratory and Centre become operational, substantial collaborative interdisciplinary grant-supported research will be conducted. Interdepartmental collaborative research applications, including joint bids to industrial sources, will be made by combining resources with the complementary expertise of the computational modelers, roboticists and vision/speech researchers in the research groups of Professors Hey and Harris.

References

Andrews, J., Livingston K. & Harnad, S. (in preparation) Categorical Perception Effects Induced by Category Learning. (Journal of Experimental Psychology, General, to be submitted).

Biederman, I. & Shiffrar, M. M. (1987) Sexing day-old chicks: A case study and expert systems analysis of a difficult perceptual-learning task. Journal of Experimental Psychology: Learning, Memory, & Cognition 13: 640 - 645.

Brown M. and C.J. Harris (1994) Neurofuzzy Adaptive Modelling and Control, Prentice Hall, Hemel Hempstead

Carpenter, Gail A.; Grossberg, Stephen; Reynolds, John H. (1991) ARTMAP: Supervised real-time learning and classification of nonstationary data by a self-organizing neural network. IN: Pattern recognition by self-organizing neural networks.; Gail A. Carpenter, Stephen Grossberg, Eds. MIT Press, Cambridge, MA, US. p. 503-544.

Goldfield, Beverly A.; Reznick, J. Steven. (1990) Early lexical acquisition: Rate, content, and the vocabulary spurt. Journal of Child Language, 1990 Feb, v17 (n1):171-183.

Harnad, S. (ed.) (1987) Categorical Perception: The Groundwork of Cognition. New York: Cambridge University Press.

Harnad, S. (1987a) Psychophysical and cognitive aspects of categorical perception: A critical overview. In: Harnad 1987.

Harnad, S. (1987b) The induction and representation of categories. In: Harnad 1987.

Harnad, S. (1990) The Symbol Grounding Problem. Physica D 42: 335-346.

Harnad, S., Hanson, S.J. & Lubin, J. (1991) Categorical Perception and the Evolution of Supervised Learning in Neural Nets. In: Working Papers of the AAAI Spring Symposium on Machine Learning of Natural Language and Ontology (DW Powers & L Reeker, Eds.) pp. 65-74.

Harnad, S. (1992) Connecting Object to Symbol in Modeling Cognition. In: A. Clarke and R. Lutz (Eds) Connectionism in Context Springer Verlag.

Harnad, S. (1993) Grounding Symbolic Capacity in Robotic Capacity. In: Steels, L. and R. Brooks (eds.) The "artificial life" route to "artificial intelligence." Building Situated Embodied Agents. New Haven: Lawrence Erlbaum

Harnad, S. (1994) The Origin of Words: A Psychophysical Hypothesis In Durham, W & Velichkovsky B (Eds.) "Naturally Human: Origins and Destiny of Language." Muenster: Nodus Pub.

Harnad, S. Hanson, S.J. & Lubin, J. (1994) Learned Categorical Perception in Neural Nets: Implications for Symbol Grounding. In: V. Honavar & L. Uhr (eds) Symbol Processors and Connectionist Network Models in Artificial Intelligence and Cognitive Modelling: Steps Toward Principled Integration. pp. 191-206. Acadamic Press.

Harnad, S. (1994) Computation Is Just Interpretable Symbol Manipulation: Cognition Isn't. Special Issue on "What Is Computation" Minds and Machines

Kloppel, B. (1994) Classification by neural networks of evoked potentials: A first case study. Neuropsychobiology 29: 47-52.

Staddon, John E. R.; Zhang, Ying. (1991) On the assignment-of-credit problem in operant learning. IN: Neural network models of conditioning and action. Quantitative analyses of behavior series.; Michael L. Commons, Stephen Grossberg, John E. R. Staddon, Eds. Lawrence Erlbaum Associates, Inc, Hillsdale, NJ, US. p. 279-293.

Szepesvari, Csaba; Lorincz, Andras. (1993) Behavior of an adaptive self-organizing autonomous agent working with cues and competing concepts. Adaptive Behavior, v2 (n2):131-160.

Rosen, S. & Howell, P. (1987) Auditory, Articulatory and Learning Explanations of Categorical Perception in Speech. In S. Harnad (Ed.) Categorical perception: The groundwork of Cognition. NY: Cambridge Univerity Press

Click here to return to Harnad/Soton Home Page
Sign up here if you wish to be a participant in the psychological experiments in the Cognitive Psychology Laboratory.