On Language and Connectionism:
Analysis of a Parallel Distributed Processing Model of Language Acquisition

This candidate-hypothesization module can only be part of the mechanism that acquires the past tense system. Other mechanisms or principles, such as those discussed in Pinker (1984), must evaluate the rule candidates and eliminate the incorrect ones, such as those that simply characterize lists of similar strong forms, and must retain any genuine rules in a general paradigm. As noted in Section (), regular rules are distinguished by applying in the default or "elsewhere" case. One can imagine the following learning strategy, which can be called the "Nonexceptional Exceptions to Exceptions Strategy", that would discover regular rules using this criterion. Comparing the acquired stem-past pairs whose first member contains eep, the child would notice that there are many exceptions to the tentative eep --> ept rule candidate and that most of the exceptions to it themselves follow the pattern holding of verbs whose present forms do not contain eep (seeped, peeped, steeped, beeped, etc.). Furthermore, exceptions to other subregularities such as bend/bent - lend/lent will also largely obey the pattern holding of verbs lacking end (end/ended, fend/fended, mend/mended, etc.). Thus, within the child's lexicon one regularity, the addition of /d/, knows no phonological bounds, and can potentially apply to any base form, whereas this is not true of any other regularity. In this way, some regularities can be enshrined as permanent productive rules whereas others can be discarded or treated differently.

Other constraints contributed by other principles and components of grammar would also influence the extraction and sorting of putative rules. For example, the syntax and lexicon would segregate derived forms out of these calculations, and the phonology would subtract out modifications abstracted from consonant clusters of simple words and perhaps from sets of morphologically unrelated rules. Finally, the general regular paradigm would be used when needed to fill out empty cells of word-specific paradigms with a unique entry, while following the constraint that irregular forms in memory block the product of the regular rule, and only a single form can be generated for a specific stem when more than one productive rule applies to it (multiple entries can exist only when the irregular form is too weakly represented, or when both multiple forms are witnessed in the input; see Pinker, 1984).

Though both our candidate-hypothesization module and the RM model share certain properties, let us be clear about the differences. The RM model is designed to account for the entire process that maps stems to past tense forms, with no interpretable subcomponents, and few constraints on the regularities that can be recorded. The candidate-hypothesization module, on the other hand, is meant to be a part of a larger system, and its outputs, namely rule candidates, are symbolic structures that can be examined, modified, or filtered out by other components of grammar. For example, the phonological acquisition mechanism can note the similarities between t/d/@o[i-]d and s/z/@o[i-]z and pull out the common phonological regularities, which would be impossible if those allomorphic regularities were distributed across a set of connection weights onto which countless other regularities were superimposed.

It is also important to note that, as we have mentioned, the candidate-hypothesization module is motivated by a requirement of the learnability task facing the child. Specifically, the child at birth does not know whether English has a regular rule, or if it does, what it is or whether it has one or several. He or she must examine the input evidence, consisting of pairs of present and past forms acquired individually, to decide. But the evidence is locally ambiguous in that the nonproductive exceptions to the regular rule are not a random set but display some regularities for historical reasons (such as multiple borrowings from other languages or dialects, or rules that have ceased to be productive) and psychological reasons (easily-memorized forms fall into family resemblance structures). So the child must distinguish real from apparent regularities. Furthermore, there is the intermediate case presented by languages that have several productive rules applying to different classes of stems. The "learnability problem" for the child is to distinguish these cases. Before succeeding, the child must entertain a number of candidates for the regular rule or rules, because it is only by examining large sets of present-past pairs that the spurious regularities can be ruled out and the partially-productive ones assigned to their proper domains; small samples are always ambiguous in this regard. Thus a child who has not yet solved the problem of distinguishing general productive rules from restricted productive rules from accidental patterns will have a number of candidate regularities still open as hypotheses. At this stage there will be competing options for the past tense form of a given verb. The child who has not yet figured out the distinction between regular, subregular, and idiosyncratic cases will display behavior that is similar to a system that is incapable of making the distinction -- the RM model.

In sum, any adequate rule-based theory will have to contain a module that extracts multiple regularities at several levels of generality, assign them strengths related to their frequency of exemplification by input verbs, and let them compete in generating a past tense form for a given verb. In addition, such a model can attain the adult state by feeding its candidates into paradigm-organization processes, which, following linguistic constraints, distinguish real generalizations from spurious ones. With this alternative model in mind, we can now examine which aspects of the developmental data are attributable to specific features of the RM model's parallel distributed processing architecture -- specifically, to its collapsing of linguistic distinctions -- and those which are attributable to its assumptions of graded strength, type-frequency sensitivity, and competition which it shares with symbolic alternatives.

Developmental Phenomena Claimed to Support the Rumelhart-McClelland Model

Developmental Sequence of Productive Inflection (The "U"-shaped curve)

A standard account of this sequence is that in the first stage, with no knowledge of the distinction between present and past forms, and no knowledge of what the regularities are in the adult language that relate them, the child is simply memorizing present and past tense forms directly from the input. He or she correctly uses irregular forms because the overregularized forms do not appear in the input and there is no productive rule yet. Regular past tenses are acquired in the same way, with no analysis of them into a stem plus an inflection. Using mechanisms such as those sketched in the preceding section, the child builds a productive rule and can apply it to any stem, including stems of irregular verbs. Because the child will have had the opportunity to memorize irregular pasts before relating stems to their corresponding pasts and before the evidence for the regular relationship between the two has accumulated across inputs, correct usage can in many cases precede overregularization. The adult state results from a realization, which may occur at different times for different verbs, that overregularized and irregular forms are both past tense versions of a given stem, and by the application of a Uniqueness principle that, roughly, allows the cells of an inflectional paradigm for a given verb to be filled by no more and no less than one entry, which is the entry witnessed in the input if there are competing nonwitnessed rule-generated forms and witnessed irregulars (see Pinker, 1984).

The RM model also has the ability to produce an arbitrary past tense form for a given present when they have been exemplified in the input, and to generate regular past tense forms for the same verbs by adding -ed. Of course, it does so without distinct mechanisms of rote and rule. In early stages, the links between the Wickelfeatures of a base irregular form and the Wickelfeatures of its past form are given higher weights. However, as a diverse set of regular forms begin to stream in, links are strengthened between a large set of input Wickelfeatures and the output Wickelfeatures containing features of the regular past morpheme, enough to make the regularized form a stronger output than the irregular form. During the overregularization stage, "the past tenses of similar verbs they are learning show such a consistent pattern that the generalization from these similar verbs outweighs the relatively small amount of learning that has occurred on the irregular verb in question" (PDPII, p. 268). The irregular form eventually returns as the strongest output because repeated presentations of it cause the network to tune the connection weights so that the Wickelfeatures that are specific to the irregular stem form (and to similar irregular forms manifesting the same kind of stem-past variation) are linked more and more strongly to the Wickelfeatures specific to their past forms, and develop strong negative weights to the Wickelfeatures corresponding to the regular morpheme. That is, the prevalence of a general pattern across a large set of verbs trades off against the repeated presentation of a single specific pattern of a single verb presented many times (with subregularities constituting an intermediate case). This gives the model the ability to be either conservative (correct for an irregular verb) or productive (overregularizing an irregular verb) for a given stem, depending on the mixture of inputs it has received up to a given point.

Since the model's tendency to generalize lies on a continuum, any sequence of stages of correct irregulars or overregularized irregulars is possible in principle, depending on the model's input history. How, then, is the specific shift shown by children, from correct irregular forms to a combination of overregularized and correct forms, mimicked by the model? Rumelhart and McClelland divide the training sequence presented to the model into two stages. In the first, they presented 10 high-frequency verbs to the model, 2 of them regular, 10 times each. In the second, they added 410 verbs to this sample, 334 of them regular, and presented the sample of 420 verbs 190 times. The beginning of the downward arm of the U-shaped plot of percent correct versus time, representing a worsening of performance for the irregular verbs, occurs exactly at the boundary between the first set of inputs and the second. The sudden influx of regular forms causes the links capturing the regular pattern to increase in strength; prior to this influx, the regular pattern was exemplified by only two input forms, not many more than those exemplifying any of the idiosyncratic or subregular patterns. The shift from the first to the second stage of the model's behavior, then, is a direct consequence of a shift in the input mixture from a heterogeneous collection of patterns to a collection in which the regular pattern occurs in the majority.

It is important to realize the theoretical claim inherent in this demonstration. The model's shift from correct to overregularized forms does not emerge from any endogenous process; it is driven directly by shifts in the environment. Given a different environment (say, one in which heterogeneous irregular forms suddenly start to outnumber regular forms), it appears that the model could just as easily go in the opposite direction, regularizing in its first stage and then becoming accurate with the irregular forms. In fact, since the model always has the potential to be conservative or rule-governed, and continuously tunes itself to the input, it appears that just about any shape of curve at all is possible, given the right shifts in the mixture of regular and irregular forms in the input.

Thus if the model is to serve as a theory of children's language acquisition, Rumelhart and McClelland must attribute children's transition between the first and second stage to a prior transition of the mixture of regular and irregular inputs from the external environment. They conjecture that such a transition might occur because irregular verbs tend to be high in frequency. "Our conception of the nature of [the child's] experience is simply that the child learns first about the present and past tenses of the highest frequency verbs; later on, learning occurs for a much larger ensemble of verbs, including a much larger proportion of regular forms" (p. 241). They concede that there is no abrupt shift in the input to the child, but suggest that children's acquisition of the present tense forms of verbs serves as a kind of filter for the past tense learning mechanism, and that this acquisition of base forms undergoes an explosive growth at a certain stage of development. Because the newly-acquired verbs are numerous and presumably lower in frequency than the small set of early-acquired verbs, it will include a much higher proportion of regular verbs. Thus the shift in the proportion of regular verbs in the input to the model comes about as a consequence of a shift from high frequency to medium frequency verbs; Rumelhart and McClelland do not have to adjust the leanness or richness of the input mixture by hand.

The shift in the model's input thus is not entirely ad hoc, but is it realistic? The use of frequency counts of verbs in written samples in order to model children's vocabulary development is, of course, tenuous. To determine whether the input to children's past tense learning shifts in the manner assumed by Rumelhart and McClelland, we examined Roger Brown's unpublished grammars summarizing samples of 713 utterances of the spontaneous speech of three children observed at five stages of development. The stages were defined in terms of equally spaced intervals of the children's Mean Length of Utterance (MLU). Each grammar includes an exhaustive list of the child's verbs in the sample, and an explicit discussion of whether the child was overregularizing the past tense rule.(Note 3) In addition, we examined the vocabulary of Lisa, the subject of a longitudinal language acquisition study at Brandeis University, in her one-word stage. Two of the children, Adam and Eve, began to overregularize in the Stage III sample; the third child, Sarah, began to overregularize only in the Stage V sample except for the single form heared appearing in Stage II which Brown noted might simply be one of Sarah's many cases of unusual pronunciations. We tabulated the size of each child's verb vocabulary and the proportion of verbs that were regular at each stage.(A verb was counted whether it appeared in the present, progressive, or past tense form, and was counted only once if it appeared in more than one form. Since most of the verbs were in the present, this is of little consequence. We counted a verb once across its appearances alone and with various particles since past tense inflection is independent of these differences. We excluded modal pairs such as can/could since they only occasionally encode a present/past contrast for adults. We excluded complement-taking verbs which encode tense and mood in English and hence which do not have obvious past tenses such as in going to, come on, and gimme.)

The results, shown in Table 1 and Figure 2, are revealing. The percentage of the children's verbs that are regular is remarkably stable across children and across stages, never veering very far from 50%. (This is also true in parental speech itself: Slobin, 1971, showed that the percentage of regular verbs in Eve's parents speech during the period in which she was overregularizing was 43%). In particular, there is no hint of a consistent increase in the proportion of regular verbs prior to or in the stage at which regularizations first occur. Note also that an explosive growth in vocabulary does not invariably precede the onset of regularization. This stands in stark contrast to the assumed input to the RM model, where the onset of overregularization occurs subsequent to a sudden shift in the proportion of regular forms in the input from 20% to 80%. Neither the extreme rarity of regular forms during the conservative stage, nor the extreme prevalence of regular forms during the overproductive stage, nor the sudden transition from one input mixture to another, can be seen in human children. The explanation for their developmental sequence must lie elsewhere.

We expect that this phenomenon is quite general. The plural in English, for example, is overwhelmingly regular even among high-frequency nouns(We are grateful to Maryellen McDonald for this point.): only 4 out of the 25 most frequent concrete count nouns in the Francis and Kucera (1982) corpus are irregular. Since there are so few irregular plurals, children are never in a stage in which irregulars strongly outnumber regulars in the input or in their vocabulary of noun stems. Nonetheless, the U-shaped developmental sequence can be observed in the development of plural inflection in the speech of the Brown children: for example, Adam said feet nine times in the samples starting at age 2;4 before he used foots for the first time at age 3;9; Sarah used feet 18 times starting at 2;9 before uttering foots at 5;1; Eve uttered feet a number of times but never foots.

Examining token frequencies only underlines the unnaturally favorable assumptions about the input used in the RM model's training run. Not only does the transition from conservatism to overregularization correspond to shift from a 20/80 to an 80/20 ratio of regulars to irregulars, but in the first, conservative phase, high-frequency irregular pairs such as go/went and make/made were only presented 10 times each, whereas in the overregularizing phase the hundreds of regular verbs were presented 190 times each. In contrast, irregular verbs are always much higher in token frequency in children's environment. Slobin (1971) performed an exhaustive analysis of the verbs heard by Eve in 49 hours of adults' speech during the phase in which she was overregularizing and found that the ratio of irregular to regular tokens was 3:1. Similarly, in Brown's smaller samples, the ratios were 2.5:1 for Adam's parents, 5:1 for Eve's parents, and 3.5:1 for Sarah's parents. One wonders whether presenting the RM model with 10 high-frequency verbs, say, 190 times each in the first phase could have burned in the 8 irregulars so strongly that they would never be overregularized in Phase 2.

If children's transition from the first to the second phase is not driven by a change in their environments or in their vocabularies, what causes it? One possibility is that a core assumption of the RM model, that there is no psychological reality to the distinction between rule-generated and memorized forms, is mistaken. Children might have the capacity to memorize independent present and past forms from the beginning, but a second mechanism that coins and applies rules might not go into operation until some maturational change put it into place, or until the number of verbs exemplifying a rule exceeded a threshold. Naturally, this is not the only possible explanation. An alternative is that the juxtaposition mechanism that relates each stem to its corresponding past tense form has not yet succeeded in pairing up memorized stems and past forms in the child's initial stage. No learning of the past tense regularities has begun because there are no stem-past input pairs that can be fed into the learning mechanism; individually acquired independent forms are the only possibility.

Some of the evidence supports this alternative. Brown notes in the grammars that children frequently used the present tense form in contexts that clearly called for the past, and in one instance did the reverse. As the children developed, past tense forms were used when called for more often, and evidence for an understanding of the function of the past tense form and the tendency to overregularize both increase. Kuczaj (1977) provides more precise evidence from a cross-sectional study of 14 children. He concluded that once children begin to regularize they rarely use a present tense form of an irregular verb in contexts where a past is called for.

The general point is that in either case the RM model does not explain children's developmental shift from conservatism to regularization. It attempts to do so only by making assumptions about extreme shifts in the input to rule learning that turn out to be false. Either rules and stored forms are distinct, or some process other than extraction of morphophonological regularity explains the developmental shift. The process of coming to recognize that two forms constitute the present and past tense variants of the same verb, that is, the juxtaposition process, seems to be the most likely candidate.

Little needs to be said about the shift from the second stage, in which regularization and overregularization occurs, to the third (adult) stage, in which application of the regular rule and storage of irregular pasts cooccur. Though the model does overcome its tendency to overregularize previously acquired irregular verbs, we have shown in a previous section that it never properly attains the third stage. This stage is attained, we suggest, not by incremental strength changes in a pattern-finding mechanism, but by a mechanism that makes categorical decisions about whether a hypothesized rule candidate is a genuine productive rule and about whether to apply it to a given verb.

On the psychological reality of the memorized/rule-generated distinction. In discussing the developmental shift to regularization, we have shown that there can be developmental consequences of the conclusion that was forced upon us by the linguistic data, namely that rule-learning and memorization of individual forms are separate mechanisms. (In particular, we pointed out that one might mature before the other, or one requires prior learning -- juxtaposing stems and past forms -- and the other does not). This illustrates a more general point: the psychological reality of the memorized/rule-generated distinction predicts the possibility of finding dissociations between the two processes, whereas a theory such as Rumelhart and McClelland's that denies that reality predicts that such dissociations should not be found. The developmental facts are clearly on the side of there being such a distinction.

First of all, children's behavior with irregular past forms during the first, pre-regularization phase bears all the signs of rote memorization, rather than a tentatively overspecific mapping from a specific set of stem features to a specific set of past features. Brown notes, for example, that Adam used fell-down ten times in the Stage II sample without ever using fall or falling, so his production of fell-down cannot be attributed to any sort of mapping at all from stem to past. Moreover there is no hint in this phase of any interaction or transfer of learning across phonetically similar individual irregular forms: for example, in Sarah's speech, break/broke coexisted with make/made and neither had any influence on take, which lacked a past form of any sort in her speech over several stages. Similar patterns can be found in the other children's speech.

A clear example of a dissociation between rote and rule over a span in which they coexist comes from Kuczaj (1977), who showed that children's mastery of irregular past tense forms was best predicted by their chronological age, but their mastery of regular past tense forms was best predicted by their Mean Length of Utterance. Brown (1973) showed that MLU correlates highly with a variety of measures of grammatical sophistication in children acquiring English. Kuczaj's logic was that irregular pasts are simply memorized, so the sheer number of exposures, which increases as the child lives longer, is the crucial factor, whereas regular pasts can be formed by the application of a rule, which must be induced as part of the child's developing grammar, so overall grammatical development is a better predictor. Thus the linguistic distinction between lists of exceptions and rule-generated forms (see Section ) is paralleled by a developmental distinction between opportunities for list-learning and sophistication of a rule system.

Another possible dissociation might be found in individual differences. A number of investigators of child language have noted that some children are conservative producers of memorized forms whereas others are far more willing to generalize productively. For example, Cazden (1968) notes that "Adam was more prone to overgeneralizations than Eve and Sarah" (p. 447), an observation also made by Brown in his unpublished grammars. More specifically, Table 1 shows that Sarah began to regularize the past tense two stages later than the other two children despite comparable verb vocabularies. Maratsos (1987) documented many individual differences in the willingness of children to overgeneralize the causative alternation. If such differences do not reflect differences in the children's environments or vocabularies (they don't in the case of the past tense), presumably they result from the generalizing mechanism of some children being stronger or more developed than that of others, without comparable differences in their ability to record forms directly from the input. The RM model cannot easily account for any of these dissociations (other than by attributing crucial aspects of the generalization phenomena to mechanisms entirely outside their model), because memorized forms and generalizations are handled by a single mechanism -- recall that the identity map in the network must be learned by adjusting a large set of connection weights, just like any of the stem alterations; it is not there at the outset, and is not intrinsically easy to learn.

The question is not closed, but the point is that the different theories can in principle be submitted to decisive empirical tests. It is such tests that should be the basis for debate on the psychological issue at hand. Simply demonstrating that there exist contrived environments in which a network model can be made to be mimic some data, especially in the absence of comparisons to alternative models, tells us nothing about the psychology of the child.

Performance with No-Change Verbs

In support of this hypothesis, Bybee and Slobin found in an elicitation experiment that for verbs ending in t or d, children were more likely to produce a past tense form identical to the present than a regularized form, whereas for verbs not ending in a t or d, they were more likely to produce a regularized form than an unchanged form. In addition, Kuczaj (1978) found in a judgment task that children were more likely to accept correct no-change forms for nonchanging verbs than correct past tense forms for other irregular verbs such as break or send, and less likely to accept overregularized versions of no-change verbs than overregularized versions of other irregular verbs. Thus not only do children learn that verbs ending in t/d are likely to be unchanged, but this subregularity is easier for them to acquire than the kinds of changes such as the vowel alternations found in other classes of irregular verbs.

Unlike the three-stage developmental sequence for regularization, children's sensitivity to the no-change subregularity for verbs ending in t/d played no role in the design of the RM model or of its simulation run. Nonetheless, Rumelhart and McClelland point out that during the phase in which the model was overregularizing, it produced stronger regularized past tense candidates for verbs not ending in t/d than for verbs ending in t/d, and stronger unchanged past candidates for verbs ending in t/d than for verbs not ending in t/d. This was true not only across the board, but also within the class of regular verbs, and within the classes of irregular verbs that do change in the past tense, for which no-change responses are incorrect. Furthermore, when Rumelhart and McClelland examined the total past tense response of the network (that is, the set of Wickelfeatures activated in the response pool) for verbs in the different irregular subclasses, they found that the no-change verbs resulted in fewer incorrectly activated Wickelfeatures than the other classes of irregulars. Thus both aspects of the acquisition of the no-change pattern fall out of the model with no extra assumptions.

Why does the model display this behavior? Because the results of its learning are distributed over hundreds of thousands of connection weights, it is hard to tell, and Rumelhart and McClelland do not try to tease apart the various possible causal factors. Misperception cannot be the explanation because the model always received correct stem-past pairs. There are two other possibilities. One is that connections from many Wickelfeatures to the Wickelfeatures for word-final t, and the thresholds for those Wickelfeatures, have been affected by the many regular stem-past pairs fed into to the model. The response of the model is a blend of the operation of all the learned subregularities, so there might be some transfer from regular learning in this case. For example, the final Wickelphone in the correct past tense form of hit, namely it#, shares many of its Wickelfeatures with those of the regular past tense allomorphs such as @o[i-d#]. Let us call this effect between-class transfer.

It is important to note that much of the between-class transfer effect may be a consequence -- perhaps even an artifact -- of the Wickelfeature representation and one of the measures defined over it, namely percentage of incorrect Wickelfeatures activated in the output. Imagine that the model's learning component actually treated no-change verbs and other kinds of verbs identically, generating Wickelfeature sets of equal strength for cutted and taked. Necessarily, taked must contain more incorrect Wickelfeatures than cutted: most of the Wickelfeatures that one would regard as "incorrect" for cutted, such as those that correspond to the Wickelphone t@o[i-d] and @o[i-d#], happen to characterize the stem perfectly (StopVowelStop, InterruptedFrontInterrupted, etc.), because cut and ted are featurally very similar. On the other hand, the incorrect Wickelfeatures for taked (those corresponding to Wickelphones Akt and kt#) will not characterize the correct output form took. This effect is exaggerated further by the fact that there are many more Wickelfeatures representing word boundaries than representing the same phonemes string-internally, as Lachter and Bever (1987) point out (recall that the Wickelfeature set was trimmed so as to exclude those whose two context phonemes belonged to different phonological dimensions -- since the word-boundary feature # has no phonological properties, such a criterion will leave all Wickelfeatures of the form XY# intact). This difference is then carried over to the current implementation of the response-generation component, which puts response candidates at a disadvantage if they do not account for activated Wickelfeatures. The entire effect (a consequence of the fact that the model does not keep track of which features go in which positions) can be viewed either as a bug or a feature. On the one hand, it is one way of generating the (empirically correct) phenomenon that no-change responses are more common when stems have the same endings as the affixes that would be attached to them. On the other hand, it is part of a family of phonological confusions that result from the Wickelphone/Wickelfeature representations in general (see the section on Wickelphonology) and that hobble the model's ability even to reproduce strings verbatim. If the stem-affix feature confusions really are at the heart of the model's no-change responses, then it should also have recurring problems, unrelated to learning, in generating forms such as pitted or pocketed where the same Wickelfeatures occur in the stem and affix or even twice in the same stem but they must be kept distinct. Indeed, the model really does seems prone to make these undesirable errors, such as generating a single CVC sequence when two are necessary, as in the no-change responses for hug, smoke, and brown, or the converse, in overmarking errors such as typeded and steppeded.

A third possible reason that no-change responses are easy for t/d-final stems is that unlike other classes of irregulars in English, the no-change class has a single kind of change (that is, no change at all), and all its members have a phonological property in common: ending with a t or d. It is also the largest irregular subclass. The model has been given relatively consistent evidence of the contingency that verbs ending in t or d tend to have unchanged past tense forms, and it has encoded that contingency, presumably in large part by strengthening links between input Wickelfeatures representing word-final t/ds and identical corresponding output Wickelfeatures. Basically, the model is potentially sensitive to any statistical correlation between input and output feature sets, and it has picked up that one. That is, the acquisition of the simple contingency "end in t/d --> no change" presumably makes the model mimic children. We can call this the within-class uniformity effect. As we have mentioned, the simplified rule-hypothesization mechanism presented in a previous section can acquire the same contingency (add a null affix for verbs ending in a nonsonorant noncontinuant coronal), and strengthen it with every no-change pair in the input. If, as we have argued, a rule-learning model considered many rules exemplified by input pairs before being able to determine which of them was the correct productive rule or rules for the language, this rule would exist in the child's grammar and would compete with the regular d rule and with other rules, just as competing outputs are computed in the RM model.

Finally, there is a fourth mechanism that was mentioned in our discussion of the strong verb system. Addition of the regular suffix d to a form ending in t or d produces a phonologically-illicit consonant cluster: td or dd. For regular verbs, the phonological rule of vowel insertion places an @o[i-] between the two consonants. Interestingly, no irregular past ends in @o[-id], though some add a t or d. Thus we find tell/told and leave/left, but we fail to find bleed/bledded or get/gotted. A possible explanation is that a phonological rule, degemination, removes an affix after it is added as an alternative means of avoiding adjacent coronals in the strong class. The no-change verbs would then just be a special case of this generalization, where the vowel doesn't change either. Basically, the child would capitalize on a phonological rule acquired elsewhere in the system, and might overgeneralize by failing to restrict the degemination rule to the strong verbs.

Thus we have an overlapping set of explanations for the early acquisition and overgeneralization of the no-change contingency. Bybee and Slobin cite misperception, Rumelhart and McClelland cite between-class transfer and within-class uniformity, and rule-based theories can cite within-class uniformity or overgeneralized phonology. What is the evidence concerning the reasons that children are so sensitive to this contingency?

Unfortunately, a number of confounds in English make the theories difficult to distinguish. No-change verbs have a diagnostic phonological property in common with one another. They also share a phonological property with regular inflected past tense forms. Unfortunately, they are the same property: ending with t/d. And it is the sharing of that phonological property that triggers the putative phonological rule. So this massive confound prevents one from clearly distinguishing the accounts using the English past tense rule; one cannot say that the Rumelhart-McClelland model receives clear support from its ability to mimic children in this case.

In principle, a number of more diagnostic tests are possible. First, one must explain why the no-change class is confounded. The within-class uniformity account, which is one of the factors behind the RM model's success, cannot do this: if it were the key factor, we would surmise that English could just as easily have contained a no-change class defined by any easily-characterized within-class property (e.g. begin with j, end with s). Bybee and Slobin note that across languages, it is very common for no-change stems to contain the very ending that a rule would add. While ruling out within-class uniformity as the only explanation, this still leaves misperception, transfer, and phonology as possibilities, all of which foster learning of no-change forms for stems resembling the relevant affix.

Second, one can look at cases where possessing the features of the regular ending is not confounded with the characteristics of the no-change class. For example, the nouns that do not change when pluralized in English such as sheep and cod do not in general end in an s or z sound. If children nonetheless avoid pluralizing nouns like ax or lens or sneeze, it would support one or more of the accounts based on stem-affix similarity. Similarly, we might expect children to be reluctant to add -ing to form verbs like ring or hamstring or rethink.

If such effects were found, differences among verbs all of which resemble the affix in question could discriminate the various accounts that exploit the stem-affix similarity effect in different ways. Transfer, which is exploited by the RM model, would, all other things being equal, lead to equally likely no-change responses for all stems with a given degree of similarity to the affix. Phonology would predict that transfer would occur only when the result of adding an affix led to adjacent similar segments; thus it would predict more no-change responses for the plural of ax than the progressive of sting, which is phonologically acceptable without the intervention of any further rule. Depending on how detailed the schema for past tense forms is thought to be, the misperception hypothesis might predict more no-change responses when a form resembled the recurrent patterns for regular pasts more closely. Thus there should be more no-change responses for thrust and bend, which match the schemas [... unvoiced consonant - t#] and [... voiced consonant -d#], than for bid or put or beat or seat, which would not match any schema exemplified by existing regular verbs since such verbs never contain a vowel-t sequence. rather than two schemas [... unvoiced consonant - t#] and [... voiced consonant -d#], which would not match hit, cut, and so on.]

Returning now to a possible unconfounded test of the within-class uniformity effect (implicated by Rumelhart and McClelland and by the rule-hypothesization module), one could look for some phonological property in common among a set of no-change stems that was independent of the phonological property of the relevant affix and see whether children were more likely to yield both correct and incorrect no-change responses when a stem had that property. As we have pointed out, monosyllabicity is a property holding of the irregular verbs in general, and of the no-change verbs in particular; presumably it is for this reason that the RM model, it turns out, is particularly susceptible to leaving regular verbs ending in t/d erroneously unchanged when they are monosyllabic. As Rumelhart and McClelland point out, if children are less likely to leave verbs such as decide or devote unchanged than verbs such as cede or raid it would constitute a test of this aspect of their theory; this test is not confounded by effects of across-class transfer.<What Rumelhart and McClelland don't point out, however, is that monosyllabicity and irregularity are not independent: in English, monosyllabicity is an important feature in defining the domain of many morphological and syntactic rules (e.g. nicer/*intelligenter, give/*donate the museum a painting; see Pinker, 1984), presumably because in English a monosyllable constitutes the minimal or basic word (McCarthy and Prince, 1987). As we have pointed out, all the irregular verbs in English are monosyllables or are contain monosyllabic roots, (likewise for nouns), a fact related in some way to irregularity being restricted to roots and monosyllables being prototypical English roots. So if children know that only roots can be irregular and that roots are monosyllables, (see Gordon, 1986, for evidence that children are sensitive to the interaction between roothood and morphology, and Gropen & Pinker, 1986, for evidence that they are sensitive to monosyllabicity), they may restrict their tendency to no-change responses to monosyllables even if it is not the product of their detecting the first-order correlation between monosyllabicity and unchanged pasts. Thus the ideal test would have to be done for some other language, in which a no-change class had a common phonological property independent of the definition of a basic root in the language, and independent of the phonology of the regular affix.>

A possible test of the misperception hypothesis is to look for other kinds of evidence that children misperceive certain stems as falling into a morphological category that is characteristically inflected. If so, then once the regular rule is acquired it could be applied in reverse to such misperceived forms, resulting in back-formations. For no-change verbs, this would result in errors such as bea or blas for beat or blast. We know of no reports of such errors among past tense forms but have observed in Lisa's speech mik for mix, and in her noun system clo (thes), len (s), sentent (cf. sentence), Santa Clau (s), upstair (s), downstair (s), bok (cf. box), trappy (cf. trapeze), and brefek (cf. brefeks = `breakfast')].

Finally, the process by which Rumelhart and McClelland exploit stem-affix similarity, namely transfer of the strength of the output features involved in regular pairs to the no-change stems, can be tested by looking at examples of blends of regular and subregular alternations that involve classes of verbs other than the no-change class. One must determine whether children produce such blends, and whether it is a good thing or a bad thing for the RM theory that their model does so. We examine this issue in the next two sections.

In sum, the class of English verbs that do not change in the past tense involves a massive confound of within-class phonological uniformity and stem-affix similarity, leading to a complex nexus of predictions as to why children are so sensitive to the properties of the class. The relations between different models of past tense acquisition, predictions of which linguistic variables should have an effect on languages and on children, and the classes of verbs instantiating those variables, is many-to-many-to-many. Painstaking testing of the individual predictions using unconfounded sets of items in a variety of inflectional classes in English and other languages could tease the effects apart. At present, however, a full range of possibilities are all consistent with the data, ranging from the RM model explaining much of the phenomenon to its being entirely dispensable. The model's ability to duplicate children's performance, in and of itself, tells us relatively little.

Frequency of Overregularizing Irregular Verbs in Different Vowel-Change Subclasses

• Class III. Verbs that undergo an internal vowel change and also add a final /t/ or /d/, e.g. feel/felt, lose/lost, say/said, tell/told.

• Class IV. Verbs that undergo an internal vowel change, delete a final consonant, and add a final /t/ or /d/, e.g. bring/brought, catch/caught. [Bybee and Slobin include in this class the pair buy/bought even though it does not involve a deleted consonant. Make/made and have/had were also included even though they do not involve a vowel change.]

• Class V. Verbs that undergo an internal vowel change whose stems end in a dental, e.g. bite/bit, find/found, ride/rode.

• Class VI. Verbs that undergo a vowel change of /I/ to /a@kern[3 points]e/ or /^/, e.g. sing/sang, sting/stung.

• Class VII. All other verbs that undergo an internal vowel change, e.g. give/gave, break/broke.

• Class VIII. All verbs that undergo a vowel change and that end in a diphthongal sequence, e.g. blow/blew, fly/flew. [Go/went is also included in this class.]

Rumelhart and McClelland suggest, as in their discussion of no-change verbs, that their model as it stands can reproduce the developmental phenomenon. Since the Bybee and Slobin subjects range from 1 1/2 to 5 years, it is not clear which stage of performance of the model should be compared with that of the children, so Rumelhart and McClelland examined the output of the model at several stages. These stages corresponded to the model's first five trials with the set of medium-frequency, predominantly regular verbs, the next five trials, the next ten trials, and an average over those first twenty trials (these intervals constitute the period in which the tendency of the model to overregularize was highest). The average strength of the overregularized forms within each class was calculated for each of these 4 intervals.

The fit between model and data is good for the interval comprising the first five trials, which Rumelhart and McClelland concentrate on. We calculate the rank-order correlation between degree of overregularization by children and model across classes as .77 in that first interval; however it then declines to .31 and .14 in the next two intervals and is .31 for the average response over all three intervals. The fact that the model is only successful at accounting for Bybee and Slobin's data for one brief interval (less than 3% of the training run) selected post hoc, whereas the data themselves are an average over a span of development of 3 1/2 years, should be kept in mind in evaluating the degree of empirical confirmation this study gives the model. Nonetheless, the tendency of Class VIII verbs (fly/flew) to be most often regularized, and for Class III verbs (feel/felt) to be among those least often regularized, persists across all three intervals.

The model, of course is insensitive to any factor uniquely affecting the juxtaposition of present and past forms because such juxtaposition is accomplished by the "teacher" in the simulation run. Instead, its fidelity to children's overregularization patterns at the very beginning of its own overregularization stage must be attributed to some other factor. Rumelhart and McClelland point to differences among the classes in the frequency with which their characteristic vowel changes are exemplified by the verb corpus as a whole. Class VIII verbs have vowel shifts that are relatively idiosyncratic to the individual verbs in the class; the vowel shifts of other classes, on the other hand, might be exemplified by many verbs in many classes. Furthermore, Class III and IV verbs, which require the addition of a final t/d, can benefit from the fact that the connections in the network that effect the addition of a final t/d have been strengthened by the large number of regular verbs. The model creates past tense forms piecemeal, by links between stem and past Wickelfeatures, and with no record of the structure of the individual words that contributed to the strengths of those links. Thus vowel shifts and consonant shifts that have been exemplified by large numbers of verbs can be applied to different parts of a base form even if the exact combination of such shifts exemplified by that base form is not especially frequent.

How well could the simplified rule-finding module account for the data? Like the RM model, it would record various subregular rules as candidates for a regular past tense rule. Assuming it is sensitive to type frequency, the rule candidates for more-frequently exemplified subregularities would be stronger. And the stronger an applicable subregular rule candidate is, the less is the tendency for its output to lose the competition with the overregularized form contributed by the regular rule. Thus if Rumelhart and McClelland's explanation of their model's fit to the data is correct, a rule-finding model sensitive to type-frequency presumably would fit the data as well.

This conjecture is hard to test because Bybee and Slobin's data are tabulated in some inconvenient ways. Each class is heterogeneous, containing verbs governed by a variety of vowel-shifts and varying widely as to the number of such shifts in the class and the number of verbs exemplifying them within the class and across the classes. Furthermore, there are some quirks in the classification. Go/went, the most irregular main verb in English, is assigned to Class VIII, which by itself could contribute to the poor performance of children and the RM model on that class. Conversely, have and make, which involve no vowel shift at all, are included in Class IV, possibly contributing to good average performance for the class by children and the model. (See the Appendix for an alternative classification.)

It would be helpful to get an estimate as to how much of the RM model's empirical success here might be due to the different frequencies of exemplification of the vowel-shift subregularities within each class, because such an effect carries over to a symbolic rule-finding alternative. To get such an estimate, we considered each vowel shift (e.g. i --> a@kern[2 pointse]) as a separate candidate rule, strengthened by a unit amount with each presentation of a verb that exemplifies it in the Rumelhart-McClelland corpus of high- and medium-frequency verbs. To allow have and make to benefit from the prevalence of other verbs whose vowels do not change, we pooled the different vowel no-change rules (a --> a; i --> i, etc.) into a single rule (the RM model gets a similar benefit by using Wickelfeatures, which can code for the presence of vowels, rather than Wickelphones) whose strength was determined by the number of no-vowel-change verbs in Classes I and II.(In a sense, it would have been more accurate to calculate the strength of the no-vowel-change rule on the basis of all the verbs in the corpus, regular and irregular, rather than just the irregular verbs. But with our overly simple strength function, this would have greatly stacked the deck in favor of correctly predicting low regularization rates for Class IV verbs and so we only counted the exemplifications of no-vowel-change within the irregular verbs.) Then we averaged the strengths of all the subregular rules included within each of Bybee and Slobin's classes. These averages allow a prediction of the ordering of overregularization probabilities for the different subclasses, based solely on the number of irregular verbs in the corpus exemplifying the specific vowel alternations among the verbs in the class. Though the method of prediction is crude, it is just about as good at predicting the data as the output of the RM model during the interval at which it did best and much better than the RM model during the other intervals examined. Specifically, the rank-order correlation between number of verbs in the corpus exemplifying the vowel shifts in a class and the frequency of children's regularization of verbs in the class is .71. The data, predictions of the RM model, and predictions from our simple tabulations are summarized in Table 2.

The Question of Blended Responses. An interesting issue arises, however, when we consider the possible effects of the addition of t/d in combination with the effects of a common vowel shift. Recall that the RM model generates its output piecemeal. Thus strong regularities pertaining to different parts of a word can affect the word simultaneously, producing a chimerical output that need not correspond in its entirety to previous frequent patterns. To take a simplified example, after the model encounters pairs such as meet/met it has strong links between i and @symbol[e]; after it encounters pairs such as play/played it has strong links between final vowels and final vowel-d sequences; when presented with flee it could then generate fled by combining the two regularities, even if it never encountered an ee/ed alternation before. What is interesting is that this blending phenomenon is the direct result of the RM model's lack of word structure. In an alternative rule-finding account, there would be an i --> @symbol[e] rule candidate, and there would be a d-affixation rule candidate, but they would generate two distinct competing outputs, not a single blended output. (It is possible in principle that some of the subregular strong verbs such as told and sent involve the superposition of independent subregular rules, especially in the history of the language, but in modern English one cannot simply heap the effect of the regular rule on top of any subregular alternation, as the RM model is prone to do). Thus it is not really fair for us to claim that a rule-hypothesization model can account for good performance with Class III and IV verbs because they involve frequently-exemplified vowel alternations; such alternations only result in correct outputs if they are blended with the addition of a t/d to the end of the word. In principle, this could give us a critical test between the network model and a rule-hypothesization model: unlike the ability to soak up frequent alternations, the automatic superposition of any set of them into a single output is (under the simplest assumptions) unique to the network model.

This leads to two questions: Is there independent evidence that children blend subregularities? And does the RM model itself really blend subregularities? We will defer answering the first question until the next section, where it arises again. As for the second, it might seem that the question of whether response blending occurs is perfectly straightforward, but in fact it is not. Say the model's active output Wickelfeatures in response to flee include those for medial @symbol[e] and those for word-final d. Is the overt response of the model fled, a correct blend, or does it set up a competition between [flid] and [fl@symbol[e]], choosing one of them, as the rule-hypothesization model would? In principle, either outcome is possible, but we are never given the opportunity to find out. Rumelhart and McClelland do not test their model against the Bybee and Slobin data by letting it output its favored response. Rather, they externally assemble alternatives corresponding to the overregularized and correct forms, and assess the relative strengths of those alternatives by observing the outcome of the competition in the restricted-choice whole-string binding network (recall that the output of the associative network, a set of activated Wickelfeatures, is the input to the whole-string binding network). These strengths are determined by the number of activated Wickelfeatures that each is consistent with. The result is that correct alternatives that also happen to resemble blends of independent subregularities are often the response chosen. But we do not know whether the model, if left to its own devices, would produce a blend as its top-ranked response.

Rumelhart and McClelland did not perform this test because it would have been too computationally intensive given the available hardware: recall that the only way to get the model to produce a complete response form on its own is by giving it (roughly) all possible output strings (that is, all permutations of segments) and having them compete against each other for active Wickelfeatures in an enormous "unconstrained whole-string binding network". This is an admitted kluge designed to give approximate predictions of the strengths of responses that a more realistic output mechanism would construct. Rumelhart and Mcclelland only ran the unconstrained whole-string binding network on a small set of new low-frequency verbs in a transfer test involving no further learning. It is hard to predict what will happen when this network operates because it involves a "rich-get-richer" scheme in the competition among whole strings, by which a string that can uniquely account for some Wickelfeatures (including Wickelfeatures incorrectly turned on as part of the noisy output function) gets disproportionate credit for the features that it and its competitors account for equally well. This could lead to a snowballing effect occasionally resulting in unpredictable winners. In fact, the whole-string mechanism does yield blends such as slip/slept. But as mentioned, these blends are also occasionally bizarre, such as mailed/membled or tour/toureder. And this is why the question of overt blended outputs is foggy: it is unclear whether tuning the whole-string binding network, or a more reasonable output construction mechanism, so that the bizarre blends were eliminated, would also eliminate the blends that perhaps turn out to be the correct outputs for Class III and IV.[To complicate matters even further, even outright blends are possible in principle within the rule-based model. For example, children might have two subregular rules that are superimposed, as might have been appropriate in an earlier stage of English. Or, there may be a response buffer that receives the output of the competition process, and occasionally two candidates of approximately equal strength slip out of the competition mechanism and are blended in the response buffer. The result would be a blended speech error from the point of view of the "design" of the rule-application module but possibly an adventitious correct response. Though this account may not seem as natural as the blending inherent in the network model, the notion of a serially ordered response buffer distinct from a representation of target segments is part of standard explanations for anticipatory and perseverative speech errors (e.g., Shattuck-Hufnagel, 1979).]

In sum, the relative tendencies of children to overregularize different classes of vowel-change irregular verbs does not favor the RM model. The model for one brief stage selected post hoc shows a moderately high correlation with data on children's behavior in the strength it assigns to overregularized forms. Much of this correlation is simply due to the frequencies with which the vowel alternations in a given class have been exemplified by verbs in the corpus as a whole. A rule-hypothesization model would also be sensitive to these frequencies under even the simplest of assumptions. But the ability of the network model to blend independent subregularities into a single response follows naturally from its lack of word structure and could lead to tests distinguishing the models. Unfortunately, whether the network model would actually output blended responses in its best incarnation is unknown; whether children output blended responses is a question we will turn to shortly.

"Eated" versus "Ated" Errors

What causes past + ed errors? There are two possibilities. One is that the child sometimes fails to realize that the irregular past is the past tense form of some base form. Thinking it is a base form itself, he or she feeds it into the past tense formation mechanism and gets a doubly-marked error. This cannot be the explanation for the model's behavior because correct present/past pairs are always provided to it. The alternative is that the two different changes are applied to the correct base form and the results are blended to yield the double-marking. This is similar to one of the explanations for the model's relatively infrequent overregularization of Class III and IV verbs discussed in the previous section, and to one of the explanations for the model's tendency to leave t/d-final stems unchanged discussed in the section before that. As in the previous discussions, the lack of a realistic response production mechanism makes it unclear whether the model would ever actually produce past + ed blends when it is forced to utter a response on its own, or whether the phenomenon is confined to such forms simply increasing in strength in the three-alternative forced-choice experiment because only the past + ed form by definition contains three sets of features all of them strengthened in the course of learning (its idiosyncratic features, the features output by subregularities, and the features of regularized forms). In Rumelhart and McClelland's transfer test on new verbs, they chose a minimum strength value of .2 as a criterion for when a form should be considered as being a likely overt response of the model. By this criterion, the model should be seen as rarely outputting past + ed forms, since such forms on the average never exceed a strength of .15. But let us assume for now that such forms would be output, and that blending is their source.

At first one might think that the model had an advantage in that it is consistent with the fact that ated errors increase relative to the eated errors in later stages, a phenomenon not obviously predicted by the misconstrued stem account. However, many of the phenomena we discuss below that favor the misconstrued-stem account also appear relatively late in the development of the past-tense system, so lateness itself does not distinguish the accounts. Moreover, Kuczaj (1977) warns that the late-ated effect is not very robust and is subject to individual differences. In a later study (Kuczaj, 1978), he eliminated these sampling problems by using an experimental task in which children judged whether various version of past tense forms sounded "silly". He found in two separate experiments that while children's acceptance of the eated forms declined monotonically their acceptance of ated forms showed an inverted U-shaped function, first increasing but then decreasing relative to eated-type errors. Since in the RM model the strengths of both forms monotonically approach an asymptote near zero, with the curves crossing only once, the model demonstrates no special ability to track the temporal dynamics of the two kinds of errors. In the discussion below we will concentrate on the reasons that such errors occur in the first place.

Once again, a confound in the materials provided by the English language confounds Rumelhart and McClelland's conclusion that their model accounts children's ated-type errors. Irregular past tense forms appear in the child's input and hence can be misconstrued as base forms. They also are part of the model's output for irregular base forms and hence can be blended with the regularized response. Until forms which have one of these properties and not the other are examined, the two accounts are at a stalemate.

Fortunately, the two properties can be unconfounded. Though the correct irregular past will usually be the strongest non-regularized response of the network, it is also sensitive to subregularities among vowel changes and hence one might expect blends consisting of a frequent and consistent but incorrect vowel change plus the regular ed ending. In fact the model does produce such errors for regular verbs it has not been trained on, such as shape/shipped, sip/sepped, slip/slept, and brown/brawned. Since the stems of these responses are either not English verbs or have no semantic relationship to the correct verb, such responses can never be the result of mistakenly feeding the wrong base form of the verb into the past tense formation process. Thus if the blending assumed in the Rumelhart and McClelland model is the correct explanation for children's past + ed overregularizations, we should see children making these and other kinds of blend errors. We might also expect errors consisting of a blend of a correct irregular alteration of a verb plus a frequent subregular alteration, such as send/soant (a blend of the d --> t and @symbol[e --> o] subregularities) or think/that (a blend of the ing --> ang and final consonant cluster --> t subregularities). (As mentioned, though, these last errors are not ruled out in principle in all rule-based models, since superposition may have had a role in the creation of several of the strong past forms in the history of English, but indiscriminately adding the regular affix onto strong pasts is ruled out by most theories of morphology).

Conversely, if the phenomenon is due to incorrect base input forms, we might expect to see other inflection processes applied to the irregular past, resulting in errors such as wenting and broking or wents and brokes. Since mechanisms for progressive or present indicative inflection would never be exposed to the idiosyncrasies or subregularities of irregular past tense forms under Rumelhart and McClelland's assumptions, such errors could not result from blending of outputs. Similarly, irregular pasts should appear in syntactic contexts calling for bare stems if children misconstrue irregular pasts as stems. In addition, we might expect to find cases where ed is added to incorrect base forms that are plausible confusions of the correct base form but implausible results of the mixing of subregularities.

Finally, we might expect that if children are put in a situation in which the correct stem of a verb is provided for them, they would not generate past + ed errors, since the source of such errors would be eliminated.

All five predictions work against the RM model and in favor of the explanation based on incorrect inputs. Kuczaj (1977) reports that his transcripts contained no examples where the child overapplied any subregularity, let alone a blend of two of them or of a subregularity plus the regular ending. Bybee and Slobin do not report any such errors in children's speech, though they do report them as adult slips of the tongue in a time-pressured speaking task designed to elicit errors. We examined the full set of transcripts of Adam, Eve, Sarah, and Lisa for words ending in -ed. We found 13 examples of irregular past + ed or en errors in past and passive constructions:

Adam:    ranned
         tooked
         stoled (twice)
         broked (participle)
         felled

Eve:     tored

Sarah:   flewed (twice)
         caughted
         stucked (participle)

Lisa:    torned  (participle)
         tooken (twice) (participle)
         sawn (participle)

What about errors consisting of a subregular vowel alternation plus the addition of ed? The only examples where an incorrect vowel other than that of the irregular form appeared with ed are the following:

Adam:    I think it's not fulled up to de top.
         I think my pockets gonna be all fulled up.
         I'm gonna ask Mommy if she has any more grain ... more stuff that she
           needs grained. [He has been grinding crackers in a meat grinder
           producing what he calls "grain".]

Sarah:   Oo, he hahted.

Lisa:    I brekked your work.

This conclusion is strengthened when we note that children do make errors such as wents and wenting, which could only result from inflecting the wrong stem. Kuczaj (1981) reports frequent use of wenting, ating, and thoughting in the speech of his son, and we find in Eve's speech fells and wents and in Lisa's speech blow awayn, lefting, hidding (= hiding), stoling, to took, to shot, and might loss. These last three errors are examples of a common phenomenon sometimes called `overtensing', which because it occurs mostly with irregulars (Maratsos & Kuczaj, 1978), is evidence that irregulars are misconstrued as stems (identical to infinitives in English). Some examples from Pinker (1984) include Can you broke those, What are you did?, She gonna fell out, and I'm going to sit on him and made him broken. Note that since many of these forms occur at the same time as the ated errors, the relatively late appearance of ated forms may reflect the point at which stem extraction (and mis-extraction) in general is accomplished.

Finally, Kuczaj (1978) presents more direct evidence that past + ed errors are due to irregular pasts misconstrued as stems. In one of his tasks, he had children convert a future tense form (i.e. "will + ") into a past tense form (i.e. "I already "). Past + ed errors virtually vanished (in fact they completely vanished for two of the three age groups). Kuczaj argues that the crucial factor was that children were actually given the proper base forms. This shows that children's derivation must be from ate to ated, not, as it is in the RM model, from eat to ated.

Yet another test of the source of apparently blended errors is possible when we turn our attention to the regular system. If the child occasionally misanalyzes a past form as a stem, he or she should do so for regular inflected past forms and not just irregular ones, resulting in errors such as talkeded. The RM model also produces such errors as blends, but for reasons that Rumelhart and McClelland do not explain, all these errors involve regular verbs whose stems end in p or k: carpeded, drippeded, mappeded, smokeded, snappeded, steppeded, and typeded, but not browneded, warmeded, teareded or clingeded, nor, for that matter, irregular stems of any sort: the model did not output creepeded/crepted, weepeded/wepted, diggeded, or stickeded. We suggest the following explanation for this aspect of the model's behavior. The phonemes p and k share most of their features with t. Therefore on a Wickelfeature by Wickelfeature basis, learning that t and d give you @o[i-d] in the output transfers to p, b, g and k as well. So there will be a bias toward @o[i-d] responses after all stops. Since there is also a strong bias toward simply adding t, there will be a tendency to blend the `add t' and the 'add @o[i-d]' responses. Irregular verbs, as we have noted, never end in @o[i-d], so to the extent that the novel irregulars resemble trained ones (see Section ), the features of the novel irregulars will inhibit the response of the @o[i-d] Wickelfeatures and double-marking will be less common.

In any case, though Rumelhart and McClelland cannot explain their model's behavior in this case, they are willing to predict that children as well will double-mark more often for p- and k-final stems. In the absence of an explanation as to why the model behaved as it did, Rumelhart and McClelland should just as readily extrapolate the model's reluctance to double-mark irregular stems and test the prediction that children should double-mark only regular forms (if our hypothesis about the model's operation is correct, the two predictions stem from a common effect). Checking the transcripts, we did find ropeded and stoppeded (the latter uncertain in transcription) in Adam's speech, and likeded and pickeded in Sarah's, as Rumelhart and McClelland would predict. But Adam also said tieded and Sarah said buyded and makeded (an irregular). Thus the model's prediction that double-marking should be specific to stems ending with p and d, and then only when they are regular, is not borne out. In particular, note that buyded and tieded cannot be the result of a blend of subregularities, because there is no subregularity according to which buy or tie would tend to attract a @o[i-d] ending.

Finally, Slobin (1985) notes that Hebrew contains two quite pervasive rules for inflecting the present tense, the first involving a vowel change, the second a consonantal prefix and a different vowel change . Though Israeli children overextend the prefix to certain verbs belonging to the first class, they never blend this prefix of the second class with the vowel change of the first class. This may be part of a larger pattern that children seem to respect the integrity of the word as a cohesive unit, one that can have affixes added to it and that can be modified by general phonological processes, but that cannot simply be composed as a blend of bits and pieces contributed by various regular and irregular inflectional regularities. It is suggestive in this regard that Slobin (1985), in his crosslinguistic survey, lists examples from the speech of children learning Spanish, French, German, Hebrew, Russian, and Polish, where the language mandates a stem modification plus the addition of an affix and children err by only adding the affix.

Once again we see that the model does not receive empirical support from its ability to mimic a pattern of developmental data. The materials that Rumelhart and McClelland looked at are again confounded in a way that leaves their explanation and the standard one focusing on the juxtaposition problem equally plausible given only the fact of ated errors. One can do better than that. By looking at unconfounded cases, contrasting predictions leading to critical tests are possible. In this case, six different empirical tests all go against the explanation inherent in the Rumelhart and McClelland model: absence of errors due to blending of subregularities, presence of wenting-type errors, presence of errors where irregular pasts are used in nonpast contexts, presence of errors where the regular past ending is mistakenly applied to non-verb stems, drastic reduction of ated-errors when the correct stem is supplied to the child, and presence of errors where the regular ending is applied twice to stems that are irregular or that end in a vowel. These tests show that errors such as ated are the result of the child incorrectly feeding ate as a base form into the past tense inflection mechanism, and not the result of blending components of ate and eated outputs:

Summary of How Well the Model Fares Against the Facts of Children's Development

To begin with, one must reject the premise that is implicit in Rumelhart and McClelland's arguments, namely that if their model can duplicate a phenomenon, the traditional explanation of that phenomenon can be rejected. For one thing, there is no magic in the RM model duplicating correlations in language systems: the model can extract any combination of over 200,000 atomic regularities, and many regularities that are in fact the consequences of an interaction among principles in several grammatical components will be detectable by the model as first-order correlations because they fall into that huge set. As we argued in Section 4, this leaves the structure and constraints on the phenomena unexplained. But in addition, it leaves many of the simple goodness-of-fit tests critically confounded. When the requirements of a learning system designed to attain the adult state are examined, and when unconfounded tests are sought, the picture changes.

First, some of the developmental phenomena can be accounted for by any mechanism that keeps records of regularities at several levels of generality, assigns strengths to them based on type-frequency of exemplification, and lets them compete in producing past tense candidate forms. These phenomena include children's shifts or waffling between irregular and overregularized past tense forms, their tendency not to change verbs ending in t/d, and their tendency to overregularize verbs with some kinds of vowel alternations less than others. Since there are good reasons why rule-hypothesization models should be built in this way, these phenomena do not support the RM model as a whole or in contrast with rule-based models in general, though they do support the more general (and uncontroversial) assumption of competition among multiple regularities of graded strength during acquisition.

Second, the lack of structures corresponding to distinct words in the model, one of its characteristic features in contrast with rule-based models, might be related to the phenomenon of blended outputs incorporating independent subregularities. However, there is no good evidence that children's correct responses are ever the products of such blends, and there is extensive evidence from a variety of sources that their ated-type errors are not the products of such blends. Furthermore, given that many blends are undesirable, it is not clear that the model should be allowed to output them when a realistic model of its output process is constructed.

Third, the three-stage or U-shaped course of development for regular and irregular past tense forms in no way supports the RM model. In fact, the model provides the wrong explanation for it, making predictions about changes in the mixture of irregular and regular forms in children's vocabularies that are completely off the mark.

This means that in the two hypotheses for which unconfounded tests are available (the cause of the U-shaped overregularization curve, and the genesis of ated-errors), both of the processes needed by the RM model to account for developmental phenomena -- frequency-sensitivity and blending -- have been shown to play no important role, and in each case, processes appealing to rules -- to the child's initial hypothesization of a rule in one case, and to the child's misapplication of it to incorrect inputs in a second -- have received independent support. And since the model's explanations in the two confounded cases (performance with no-change verbs, and order of acquisition of subclasses) appeal in part to the blending process, the evidence against blending in our discussion of the ated errors taints these accounts as well. We conclude that the developmental facts discussed in this section and the linguistic facts discussed in Section 4 converge on the conclusion that knowledge of language involves the acquisition and use of symbolic rules.

General Discussion

In this concluding section we do four things: we briefly evaluate Rumelhart and McClelland's strong claims about language; we evaluate the general claims about the differences between connectionist and symbolic theories of cognition that the RM model has been taken to illustrate; we examine some of the ways that the problems of the RM model problems are inherently due to its PDP architecture, and hence ways in which our criticisms implicitly extend to certain kinds of PDP models in general; and we consider whether the model could be salvaged by using more sophisticated connectionist mechanisms.

On Rumelhart and McClelland's Strong Claims about Language

There is also no basis for Rumelhart and McClelland's claim that in their network model, as opposed to traditional accounts, "there is no induction problem". The induction problem in language acquisition consists, among other things, of finding sets of inputs that embody generalizations, extracting the right kinds of generalizations from them, and deciding which generalizations can be extended to new cases. The model does not deal at all with the first problem, which involves recognizing that a given word encodes the past tense and that it constitutes the past tense version of another word. This juxtaposition problem is relegated to the model's environment (its "teacher"), or more realistically, some unspecified prior process; such a division of labor would be unproblematic if it were not for the fact that many of the developmental phenomena that Rumelhart and McClelland marshall in support of their model may be intertwined with the juxtaposition process (the onset of overregularization, and the source of ated errors, most notably). The second part of the induction problem is dealt with in the theory the old-fashioned way: by providing it with an innate feature space that is supposed to be appropriate for the regularities in that domain. In this case, it is the distinctive features of familiar phonological theories, which are incorporated into the model's Wickelfeature representations (see also Lachter & Bever, 1987). Aspects in which the RM model differs from traditional accounts in how it uses distinctive features, such as representing words as unordered pools of feature trigrams, do not clearly work to the advantage of the model, to put it mildly. Finally, the theory deals very poorly with the crucial third aspect of the induction problem, when to generalize to new items. It cannot make proper phonological generalizations or respect the morphosyntactic constraints on the domain of application of the regular rule, and in its actual performance it errs in two ways, both overestimating the significance of the irregular subgeneralizations and underestimating the generality of the regular rule.

The third claim, that the success of their model calls for a revised understanding of language and language acquisition, is hardly warranted in light of the problems we have discussed. To give credit where it is due, we do not wish to deny the extent to which Rumelhart and McClelland's work has increased our understanding of language acquisition. The model has raised intriguing questions about the role of the family resemblance structure of subregularities and of their frequency of exemplification in overregularization, the blending of independent subregularities in generating overt outputs, effects of the mixture of regular and irregular forms in the input on the tradeoffs between rote and generalization, and the causes of transitions between developmental stages, in particular, the relative roles of the present-past juxtaposition process and the pattern-extraction process. But the model does not give superior or radically new answers for the questions it raises.

Implications for the Metatheory and Methodology of Connectionism

Subsymbolic models accurately describe the microstructure of cognition, while symbolic models provide an approximate description of the macrostructure. (Smolensky, in press, p. 21.)

We view macrotheories as approximations to the underlying microstructure which the distributed model presented in our paper attempts to capture. As approximations they are often useful, but in some situations it will turn out that an examination of the microstructure may bring much deeper insight. (Rumelhart & McClelland, PDPI, p. 125).

...these [macro-level] models are approximations and should not be pushed too far. (Rumelhart & McClelland, p. 126; bracketed material ours here and elsewhere).

One of the reasons that connectionist theorists tend to reserve no role for higher-level theories as anything but approximations is that they create a dichotomy that, we think, is misleading. They associate the systematic, rule-based analysis of linguistic knowledge with what they call the "explicit inaccessible rule" view of psychology, which

... holds that the rules of language are stored in explicit form as propositions, and are used by language production, comprehension, and judgment mechanisms. These propositions cannot be described verbally [by the untutored native speaker]. (Rumelhart and McClelland, PDPII, p. 217).

In fact, there is no necessary link between realistic interpretation of rule theories and the "explicit inaccessible" view. Rules could be explicitly inscribed and accessed, but they also could be implemented in hardware in such a way that every consequence of the rule-system holds. If the latter turns out to be the case in a cognitive domain, there is a clear sense in which the rule-theory is validated -- it is exactly true -- rather than faced with a competing alternative or relegated to the status of an approximate convenience.(Note as well that many of the examples offered to give common-sense support to the desirability of eliminating rules are seriously misleading because they appeal to a confusion between attributing a rule-system to an entity and attributing the wrong rule-system to an entity. An example that Rumelhart and McClelland cite, in which it is noted that bees can create hexagonal cells in their hive with no knowledge of the rules of geometry, gains its intuitive force because of this confusion.)

Consider pattern-associators like Rumelhart and McClelland's, which gives symbolic output from symbolic input. Under a variety of conditions, it will function as a rule-implementer. To take only the simplest, suppose that all connection weights are 0 except those from the input node for feature f to the i output node for f , which are set to 1. Then the network will implement the i identity map. There is no read-head, write-head, or executive overseeing the operation, yet it is legitimate and even enlightening to speak of it in terms of rules manipulating symbols.

More realistically, one can abstract from the RM pattern associator an implicit theory implicating a "representation" consisting of a set of unordered Wickelfeatures and a list of "rules" replacing Wickelfeatures with other Wickelfeatures. Examining the properties of such rules and representations is quite revealing. We can find out what it takes to add /d/ to a stem; what it takes to reverse the order of phonemes in an input; whether simple local modifications of a string are more easily handled than complex global ones; and so on. The results we obtain carry over without modification to the actual pattern associator, where much more complex conditions prevail. The deficiencies of Wickelphone/Wickelfeature transformation are as untouched by the addition of thresholds, logistic probability functions, temperatures, and parameters of that ilk as they are by whether the program implementing the model is written in Fortran or C.

An important role of higher-level theory, as Marr for one has made clear, is to delineate the basic assumptions that lower level models must inevitably be built on. From this perspective, the high-level theory is not some approximation whose behavior offers a gross but useful guide to reality. Rather, the relation is one of embodiment: the lower-level theory embodies the higher level theory, and it does so with exactitude. The RM model has a theory of linguistic knowledge associated with it; it is just that the theory is so unorthodox that one has to look with some care to find it. But if we want to understand the model, dealing with the embodied theory is not a convenience, but a necessity, and it should be pushed as far as possible.

When Does a Network Implement a Rule?

In a radical or eliminative connectionist model, the overall properties of the rule-theory of a domain are not only caused by the mechanisms of the micro-theory (that is, the stipulated properties of the units and connections) but follow in a natural way from micro-assumptions that are well-motivated on grounds that have nothing to do with the structure of the domain under macro-scrutiny. The rule-theory would have second-class status because its assumptions would be epiphenomena: if you really want to understand why things take the shape they do, you must turn not to the axioms of a rule-theory but to the micro-ecology that they follow from. The intuition behind the symbolic paradigm is quite different: here rule-theory drives micro-theory; we expect to find many characteristics of the micro-level which make no micro-sense, do not derive from natural micro-assumptions or interactions, and can only be understood in terms of the higher-level system being implemented.

The RM pattern associator again provides us with some specific examples. As noted, it is surely significant that the regular past-tense morphology leaves the stem completely unaltered. Suppose we attempt to encode this in the pattern associator by pre-setting it for the identity map; then for the vast majority of items (perhaps more than 95% on the whole vocabulary), most connections will not have to be changed at all. In this way, we might be able to make the learner pay (in learning time) for divergences from identity. But such a setting has no justification from the micro-level perspective, which conduces only to some sort of uniformity (all weights 0, for example, or all random); the labels that we use from our perspective as theorists are invisible to the units themselves, and the connections implementing the identity map are indistinguishable at the micro-level from any other connections. Wiring it in is an implementational strategy driven by outside considerations, a fingerprint of the macro-theory.

An actual example in the RM model as it stands is the selective blurring of Wickelfeature representations. When the Wickelfeature ABC is part of an input stem, extra wickelfeatures XBC and ABY are also turned on, but AXC is not: as we noted above (see also Lachter & Bever, 1987), this is motivated by the macro-principles that individual phonemes are the significant units of analysis and that phonological interactions when they occur generally involve adjacent pairs of segments. It is not motivated by any principle of micro-level connectionism.

Even the basic organization of the RM model, simple though it is, comes from motives external to the micro-level. Why should it be that the stem is mapped to the past tense, that the past tense arises from a modification of the stem? Because a sort of intuitive proto-linguistics tells us so. It is easy to set up a network in which stem and past tense are represented only in terms of their semantic features, so that generalization gradients are defined over semantic similarity (e.g. hit and strike would be subject to similar changes in the past tense), with the unwelcome consequence that no phonological relations will `emerge'. Indeed, the telling argument against the RM pattern associator as a model of linguistic knowledge is that its very design forces it to blunder past the major generalizations of the English system. It is not unthinkable that many of the design flaws could be overcome, resulting in a connectionist network that learns more insightfully. But subsymbolism or eliminative connectionism, as a radical metatheory of cognitive science, will not be vindicated if the principal structures of such hypothetical improved models turn out to be dictated by higher-level theory rather than by micro-necessities. To the extent that connectionist models are not mere isotropic node tangles, they will themselves have properties that call out for explanation. We expect that in many cases, these explanations will constitute the macro-theory of the rules that the system would be said to implement.

Here we see, too, why radical connectionism is so closely wedded to the notion of blank slates, simple learning mechanisms, and vectors of "teaching" inputs juxtaposed unit-by-unit with the networks' output vectors. If you really want a network not to implement any rules at all, the properties of the units and connections at the micro-level must suffice to organize the network into something that behaves intelligently. Since these units are too simple and too oblivious to the requirements of the computational problem that the entire network will be required to solve to do the job, the complexity of the system must derive from the complexity of the set of environmental inputs causing the units to execute their simple learning functions. One explains the organization of the system, then, only in terms of the structure of the environment, the simple activation and learning abilities of the units, and the tools and language of those aspects of statistical mechanics apropos to the aggregate behavior of the units as they respond to environmental contingencies (as in Smolensky, 1986; Hinton and Sejnowski, 1986) -- the rules genuinely would have no role to play.

As it turns out, the RM model requires both kinds of explanation -- implemented macrotheory and massive supervised learning -- in accounting for its asymptotic organization. Rumelhart and McClelland made up for the model's lack of proper rule-motivated structure by putting it into a teaching environment that was unrealistically tailored to produce much of the behavior they wanted to see. In the absence of macro-organization the environment must bear a very heavy burden.

Rumelhart and McClelland (1986a) recognize this implication clearly and unflinchingly in the two paragraphs they devote in their volumes to answering the question "Why are People Smarter than Rats?":

Given all of the above [the claim that human cognition and the behavior of lower animals can be explained in terms of PDP networks], the question does seem a bit puzzling. ... People have much more cortex than rats do or even than other primates do; in particular they have very much more ... brain structure not dedicated to input/output -- and presumably, this extra cortex is strategically placed in the brain to subserve just those functions that differentiate people from rats or even apes. ... But there must be another aspect to the difference between rats and people as well. This is that the human environment includes other people and the cultural devices that they have developed to organize their thinking processes. (p. 143).

On the Properties of Parallel Distributed Processing Models

Distributed Representations

Hinton, et al (1986) point to a number of useful characteristics of distributed representations. They provide a kind of content-addressable memory, from which individual entities may be called up through their properties. They provide for automatic generalization: things true of individual X can be inherited by individual Y inasmuch as the representation of Y overlaps that of X (i.e. inasmuch as Y shares properties with X) and activation of the overlapping portion during learning has been correlated with generalizable properties. And they allow for the formation of new concepts in a system via new combinations of properties that the system already represents.

It is often asserted that distributed representation using features is uniquely available to PDP models, and stands as the hallmark of a new paradigm of cognitive science, one that calculates not with symbols but with what Smolensky (1987) has dubbed `subsymbols' (basically, what Rumelhart, McClelland, and Hinton call `microfeatures'). Smolensky puts it this way:

(18) Symbols and Context Dependence. In the symbolic paradigm, the context of a symbol is manifest around it, and consists of other symbols; in the subsymbolic paradigm, the context of a symbol is manifest inside it, and consists of subsymbols.

...the relation between a type and an instance can be implemented by the relationship between a set of units [features] and a larger set [of features] that includes it. Notice that the more general the type the smaller the set of units [features] used to encode it. As the number of terms in an intensional [featural] description gets smaller, the corresponding extensional set [of individuals] gets larger. (p. 84)

Of course, distributed representation in PDP models implies more than just featural decomposition: an entity is represented as nothing but the features it is composed of. Concatenative structure, constituency, variables, and their binding -- in short, syntagmatic organization -- are virtually abandoned. This is where the RM model and similar PDP efforts really depart from previous work, and also where they fail most dramatically.

A crucial problem is the difficulty PDP models have in representing individuals and variables (this criticism is also made by Norman (1986) in his generally favorable appraisal of PDP models). The models represent individual objects as sets of their features. Nothing, however, represents the fact that a collection of features corresponds to an existing individual: that it is distinct from a twin that might share all its features, or that an object similar to a previously viewed one is a single individual that has undergone a change as opposed to two individual objects that happen to resemble one another, or that a situation has undergone a change if two identical objects have switched positions.(We thank David Kirsch for these examples). In the RM model, for example, this problem manifests itself in the inability to supply different past tenses for homophonous verbs such as wring and ring, or to enforce a categorical distinction between morphologically disparate verbs that are given similar featural representations such as become and succumb, to mention just two of the examples discussed in Section .

As we have mentioned, an seemingly obvious way to handle this problem -- just increase the size of the feature set so that more distinctions can be encoded -- will not do. For one thing, the obvious kinds of features to add, such as semantic features to distinguish homophones, gives the model too much power, as we have mentioned: it could use any semantic property or combination of semantic and phonological properties to distinguish inflectional rules, whereas in fact only a relatively small set of features are ever encoded inflectionally in the world's languages (Talmy, 1985; Bybee, 1985). Furthermore, the crucial properties governing choice of inflection are not semantic at all but refer to abstract morphological entities such as basic lexical itemhood or roothood. Finally, this move would commit one to the prediction that semantically-related words are likely to have similar past tenses, which is just not true (compare, e.g. [hit/hit] versus [strike/struck] versus [slap/slapped] (similar meanings, different kinds of past tenses) or [stand/stood] versus [understand/understood] versus [stand out/stood out] (different meanings, same kind of past tense). Basically, increasing the feature set is only an approximate way to handle the problem of representing individuals; by making finer distinctions it makes it less likely that individuals will be confused but it still does not encode individuals as individuals. The relevant difference between wring and ring as far as the past tense is concerned is that they are different words, pure and simple.(Of course, another problem with merely increasing the feature set, especially if the features are conjunctive, is that the network can easily grow too large very quickly. Recall that Wickelphones, which in principle can make finer distinctions than Wickelfeatures, would have required a network with more than two billion connections.)

A second way of handling the problem is to add arbitrary features that simply distinguish words. In the extreme case, there could be a set of n features over which n orthogonal patterns of activation stand in one-to-one correspondence with n lexical items. This won't work, either. The basic problem is that distributed representations, when they are the only representations of objects, face the conflicting demands of keeping individuals distinct and providing the basis for generalization. As it stands, Rumelhart and McClelland must walk a fine line between keeping similar words distinct and getting the model to generalize to new inputs -- witness their use of Wickelfeatures over Wickelphones, their decision to encode a certain proportion of incorrect Wickelfeatures, their use of a noisy output function for the past tense units, all designed to blur distinctions and foster generalization (as mentioned, the effort was only partially successful, as the model failed to generalize properly to many unfamiliar stems). Dedicating some units to representing wordhood would be a big leap in the direction of nongeneralizability. With orthogonal patterns representing words, in the extreme case, word-specific output features could be activated accurately in every case and the discrepancy between computed-output and teacher-supplied-input needed to strengthen connections from the relevant stem features would never occur. Intermediate solutions, such as having a relatively small set of word-distinguishing features available to distinguish homophones with distinct endings, might help. But given the extremely delicate balance between discriminability and generalizability, one won't know until it is tried, and in any case, it would at best be a hack that did not tackle the basic problem at hand: individuating individuals, and associating them with the abstract predicates that govern the permissible generalizations in the system.

The lack of a mechanism to bind sets of features together as individuals causes problems at the output end, too. A general problem for coarse-coded distributed representations is that when two individuals are simultaneously represented, the system can lose track of which feature goes with which individual -- leading to "illusory conjunctions" where, say, an observer may be unable to say whether he or she is seeing a blue circle and a red triangle or a red triangle and a blue circle (see Treisman and Schmidt, 1982; Hinton, et al, 1986). The RM model simultaneously computes past tense output features corresponding to independent subregularities which it is then unable to keep separate, resulting in incorrect blends such as slept as the past tense of slip -- a kind of self-generated phonological illusory conjunction. The current substitute for a realistic binding mechanism, namely the "whole-string binding network", does not do the job, and we are given no reason to believe that a more realistic and successful model is around the corner. The basic point is that the binding problem is a core deficiency of this kind of distributed representation, not a minor detail whose solution can be postponed to some later date.

The other main problem with features-only distributed representations is that they do not easily provide variables that stand for sets of individuals regardless of their featural decomposition, and over which quantified generalizations can be made. This dogs the RM model in many places. For example, there is the inability to represent certain reduplicative words, in which the distinction between a feature occurring once versus occurring twice is crucial, or in learning the general nature of the rule of reduplication, where a morpheme must be simply copied: one needs a variable standing for an occurrence of a morpheme independent of the particular features it is composed of. In fact, even the English regular rule of adding /d/ is never properly learned (that is, the model does not generalize it properly to many words), because in essence the real rule causes an affix to be added to a "word", which is a variable standing for any admissible phone sequence, whereas the model associates the family of /d/ features with a list of particular phone sequences it has encountered instead. Many of the other problems we have pointed out can also be traced to the lack of variables.

We predict that the kind of distributed representation used in the two layer pattern-associators like the one in the RM model will cause similar problems anywhere they are used in modeling middle- to high-level cognitive processes.(Within linguistic semantics, for example, a well-known problem is that if semantic representation is a set of features, how are propositional connectives defined over such feature sets? If P is a set of features, what function of connectionist representation will give the set for -P?) Hinton, McClelland, and Rumelhart themselves provide an example that (perhaps inadvertently) illustrates the general problem:

People are good at generalizing newly acquired knowledge. ... If, for example, you learn that chimpanzees like onions you will probably raise your estimate of the probability that gorillas like onions. In a network that uses distributed representations, this kind of generalization is automatic. The new knowledge about chimpanzees is incorporated by modifying some of the connection strengths so as to alter the causal effects of the distributed pattern of activity that represents chimpanzees. The modification automatically change the causal effects of all similar activity patterns. So if the representation of gorillas is a similar activity pattern over the same set of units, its causal effects will be changed in a similar way. (p. 82)

The point is that people's inductive inferences depend on variables assigned to sets of individuals that pick out some properties and completely ignore others, differently on different occasions, depending in knowledge-specific ways on the nature of the inductive inference to be made on that occasion. Furthermore the knowledge that can totally alter or reverse an inductive inference is not just another pattern of trained feature correlations, but depends crucially on the structured propositional content of the knowledge: learning that all gorillas are exclusively carnivorous will lead to a different generalization about their taste for onions than learning that some or many are exclusively carnivorous or that it is not the case that all gorillas are exclusively carnivorous, and learning that a particular gorilla who happens to have a broken leg does not like onions will not necessarily lead to any tendency to project that distaste onto other injured gorillas and chimpanzees. Though similarity surely plays a role in domains of which people are entirely unfamiliar, or perhaps in initial gut reactions, full-scale intuitive inference is not a mere reflection of patterns of featural similarity that have been intercorrelated in the past. Therefore one would not want to use the automatic-generalization properties of distributed representations to provide an account of human inductive inference in general. This is analogous to the fact we have been stressing throughout, namely that the past tense inflectional system is not a slave to similarity but it is driven in precise ways by speakers' implicit "theories" of linguistic organization.

In sum, featural decomposition is an essential feature of standard symbolic models of language and cognition, and many of the successes of PDP models simply inherit these advantages. However what is unique about the RM model and other two-layer pattern associators is the claim that individuals and types are represented as nothing but activated subsets of features. This impoverished mechanism is viable neither in language nor in cognition in general. The featural decomposition of an object must be available to certain processes, but can only be one of the records associated with the object and need not enter into all the processes referring to the object. Some symbol referring to the object qua object, and some variable types referring to task-relevant classes of objects that cut across featural similarity, are required.

Distinctions among Subcomponents and Abstract Internal Representations

Why do Rumelhart and McClelland have to obliterate the traditional decomposition to begin with? The principal reason is that when one breaks a system down into components, the components must communicate by passing information -- internal representations -- among themselves. But because these are internal representations the environment cannot "see" them and so cannot adjust them during learning via the perceptron convergence procedure used in the RM model. Furthermore, the internal representations do not correspond directly to environmental inputs and outputs and so the criteria for matches and mismatches necessary to drive the convergence procedure are not defined. In other words the representations used in decomposed, modular systems are abstract, and many aspects of their organization cannot be learned in any obvious way. (Chomsky, 1981, calls this the argument from "poverty of the stimulus"). Sequences of morphemes resulting from factoring out phonological changes are one kind of abstract representation used in rule systems; lexical entries distinct from phonetic representations are another; morphological roots are a third. The RM model thus is composed of a single module mapping from input directly to output in part because there is no realistic way for their convergence procedure to learn the internal representations of a modular account properly.

A very general point we hope to have made in this paper is that symbolic models of language were not designed for arbitrary reasons and preserved as quaint traditions; the distinctions they make are substantive claims motivated by empirical facts and cannot be obliterated unless a new model provides equally compelling accounts of those facts. Designing a model that can record hundreds of thousands of first-order correlations can simulate some but not all of this structure and is unable to explain it or to account for the structures that do not occur across languages. Similar conclusions, we predict, will emerge from other cognitive domains that are rich in data and theory. It is unlikely that any model will be able to obliterate distinctions among subcomponents and their corresponding forms of abstract internal representations that have been independently motivated by detailed study of a domain of cognition. This alone will sharply brake any headlong movement away from the kinds of theories that have been constructed within the symbolic framework.

Discrete, Categorical Rules

Unconstrained Correlation Extraction

Can the Model be Recast Using More Powerful PDP Mechanisms?

In particular, two interesting kinds of networks, the Boltzmann Machine (Hinton & Sejnowski, 1986) and the Back-Propagation scheme (Rumelhart, et al, 1986) have been developed recently that have "hidden units" or intermediate layers between input and output. These hidden units function as internal representations and as a result such networks are capable of computing functions that are uncomputable in two-layer pattern associators of the RM variety. Furthermore, in many interesting cases the models have been able to "learn internal representations". For example the Rumelhart et al model changes not only the weights of the connections to its output units in response to an error with respect to the teaching input, but it propagates the error signal backwards to the intermediate units and changes their weights in the direction that alters their aggregate effect on the output in the right direction. Perhaps, then, a multilayered PDP network with back-propagation learning could avoid the problems of the RM model.

There are three reasons why such speculations are basically irrelevant to the points we have been making.

First, there is the gap between revolutionary manifestoes and actual accomplishments. Rumelhart and McClelland's surprising claims -- that language can be described only approximately by rules, that there is no induction problem in their account, and that we must revise our understanding of linguistic information processing -- are based on the putative success of their existing model. Given that their existing model does not do the job it is said to do, the claims must be rejected. If a PDP advocate were to eschew the existing RM model and appeal to more powerful mechanisms, the only claim that could be made is that there may exist a model of unspecified design that may or may not account for past tense acquisition without the use of rules and that if it did, we should revise our understanding of language, treat rules as mere approximations, and so on. Such an assertion, of course, would have as little claim to our attention as any other claim about the hypothetical consequences of a nonexistent model.

Second, a successful PDP model of more complex design may be nothing more than an implementation of a symbolic rule-based account. The advantage of a multilayered model is precisely that it is free from the constraints that so sharply differentiated the RM model from standard ones, namely, the lack of internal representations and subcomponents. Multilayered networks, and other sophisticated models such as those that have one network that can gate the connections between two others or networks that can simulate semantic networks, production systems, or LISP primitive operations (Hinton, 1981; Touretzky, 1986; Touretzky & Hinton, 1985) are appealing because they have the ability to mimic or implement the standard operations and representations needed in traditional symbolic accounts (though perhaps with some twists). We do not doubt that it would be possible to implement a rule system in networks with multiple layers: after all, it has been known for over 45 years that nonlinear neuron-like elements can function as logic gates and that hence that networks consisting of interconnected layers of such elements can compute propositions (McCulloch and Pitts, 1943). Furthermore, given what we know about neural information processing and plasticity it seems likely that the elementary operations of symbolic processing will have to be implemented in a system consisting of massively interconnected parallel stochastic units in which the effects of learning are manifest in changes in connections. These uncontroversial facts have always been at the very foundations of the realist interpretation of symbolic models of cognition; they do not signal a departure of any sort from standard symbolic accounts. Perhaps a multilayered or gated multinetwork system could solve the tasks of inflection acquisition without simply implementing standard grammars intact (for example, they might behave discrepantly from a set of rules in a way that mimicked people's systematic divergence from that set of rules, or their intermediate layers might be totally opaque in terms of what they represent), and thus would call for a revised understanding of language, but there is at present no reason to believe that this will be true.

As we mentioned in a previous section, the really radical claim is that there are models that can learn their internal organization through a process that can be exhaustively described as an interaction between the correlational structure of environmental inputs and the aggregate behavior of the units as they execute their simple learning and activation functions in response to those inputs. But there is at present no reason to believe the general version of this claim. An important technical problem is that when intermediate layers of more complex networks have to learn anything in the local unconstrained manner characteristic of PDP models, they are one or more layers removed from the output layer at which discrepancies between actual and desired outputs are recorded. Their inputs and outputs no longer correspond in any direct way to overt stimuli and responses, and the steps needed to modify their weights are no longer transparent. Since differences in the setting of each tunable component of the intermediate layers have consequences that are less dramatic at the comparison stage (their effects combine in complex ways with the effects of weight changes of other units before affecting the output layer), it is harder to ensure that the intermediate layers will be properly tuned by local adjustments propagating backwards. Rumelhart, et al (1986) have dealt with this problem in clever ways with some interesting successes in simple domains such as learning to add two-digit numbers, detecting symmetry, or learning the exclusive-`or' operator. But there is always the danger in such systems of converging on incorrect solutions defined by local minima of the "energy landscape" defined over the space of possible weights, and such factors as the starting configuration, the order of inputs, several parameters of the learning function, the number of hidden units, and the innate topology of the network (such as whether all input units are connected to all intermediate units, and whether they are connected to all output units via direct paths or only through intervening links) can all influence whether the models will properly converge even in some of the simple cases. There is no reason to predict with certainty that these models will fail to acquire complex abilities such as mastery of the past tense system without wiring in traditional theories by hand -- but there is also no reason to predict that they will.

These problems are exactly that, problems. They do not demonstrate that interesting PDP models of language are impossible in principle. At the same time, they show that there is no basis for the belief that connectionism will dissolve the difficult puzzles of language, or even provide radically new solutions to them. As for the present, we have shown that the paradigm example of a PDP model of language can claim nothing more than a superficial fidelity to some first-order regularities of language. More is known than just the first-order regularities, and when the deeper and more diagnostic patterns are examined with care, one sees not only that the PDP model is not a viable alternative to symbolic theories, but that the symbolic account is supported in virtually every aspect. Principled symbolic theories of language have achieved success with a broad spectrum of empirical generalizations, some of considerable depth, ranging from properties of linguistic structure to patterns of development in children. It is only such success that can warrant confidence in the reality and exactitude of our claims to understanding.

References

Anderson, J. R. (1976) Language, memory, and thought. Hillsdale, NJ: Erlbaum.

Anderson, J. R. (1983) The architecture of cognition. Cambridge, MA: Harvard University Press.

Armstrong, S. L., Gleitman, L. R., & Gleitman, H. (1983) What some concepts might not be. Cognition, 13, 263-308.

Aronoff, M. (1976) Word formation in generative grammar. Cambridge, MA: MIT Press.

Berko, J. (1958) The child's learning of English morphology. Word, 14, 150-177.

Bloch, B. (1947) English verb inflection. Language 23, 399-418.

Bowerman, M. (1987) Discussion: Mechanisms of language acquisition. In B. MacWhinney (Ed.), Mechanisms of language acquisition. Hillsdale, NJ: Erlbaum.

Brown, R. (1973) A first language: The early stages. Cambridge, MA: Harvard University Press.

Bybee, J. L. (1985) Morphology: a study of the relation between meaning and form. Philadelphia: Benjamins.

Bybee, J. L. & Slobin, D. I. (1982) Rules and schemes in the development and use of the English past tense. Language, 58, 265-289.

Carey, S. (1985) Conceptual change in childhood. Cambridge, MA: Bradford Books/MIT Press.

Cazden, C. B. (1968) The acquisition of noun and verb inflections. Child Development, 39, 433-448.

Chomsky, N. (1957) Syntactic structures. The Hague: Mouton.

Chomsky, N. (1965) Aspects of the theory of syntax. Cambridge, MA: MIT Press.

Chomsky, N. (1981) Lectures on government and binding. Dordrecht, Netherlands: Foris.

Chomsky, N. & Halle, M. (1968) The Sound Pattern of English. New York: Harper and Row.

Curme, G. (1935) A Grammar of the English Language II. Boston: Barnes & Noble.

de Jong, G. F. & Mooney, R. J. (1986) Explanation-based learning: An alternative view. Machine Learning, 1, 145-176.

Ervin, S. (1964) Imitation and structural change in children's language. In E. Lenneberg (Ed.), New directions in the study of language. Cambridge, MA: MIT Press.

Feldman, J. A. & Ballard, D. H. (1982) Connectionist models and their properties. Cognitive Science, 6, 205-254.

Fodor, J. A. (1968) Psychological explanation. New York, Random House.

Fodor, J. A. (1975) The language of thought. New York, T.Y. Crowell.

Francis, N. & Kucera, H. (1982) Frequency analysis of English usage: Lexicon and grammar. Boston: Houghton Mifflin.

Fries, C. (1940) American English grammar. New York: Appleton-Century.

Gleitman, L. R. & Wanner, E. (1982) Language acquisition: The state of the state of the art. In E. Wanner and L. R. Gleitman (Eds.), Language acquisition: The state of the art. New York: Cambridge University Press.

Gordon, P. (1986) Level-ordering in lexical development. Cognition, 21, 73-93.

Gropen, J. & Pinker, S. (1986) Constrained productivity in the acquisition of the dative alternation. Paper presented at the 11th Annual Boston University Conference on Language Development, October.

Halle, M. (1957) In defense of the Number Two. In Studies Presented to J. Whatmough. Mouton: The Hague.

Halle, M. (1962) Phonology in generative grammar. Word, 18, 54-72.

Halwes, T. and Jenkins, J. J. (1971) Problem of serial behavior is not resolved by context-sensitive memory models. Psychological Review, 78, 122-29.

Hinton, G. E. (1981) Implementing semantic networks in parallel hardware. In G. E. Hinton & J. A. Anderson (Eds.), Parallel models of associative memory. Hillsdale, NJ, Erlbaum.

Hinton, G. E., McClelland, J. L., & Rumelhart, D. E. (1986) Distributed representations. In D. E. Rumelhart, J. L. McClelland, and the PDP Research Group, Parallel distributed processing: Explorations in the microstructure of cognition. Volume 1: Foundations. Cambridge, MA: Bradford Books/MIT Press.

Hinton, G. E. & Sejnowski, T. J. (1986) Learning and relearning in Boltzmann machines. In D. E. Rumelhart, J. L. McClelland, and the PDP Research Group, Parallel distributed processing: Explorations in the microstructure of cognition. Volume 1: Foundations. Cambridge, MA: Bradford Books/MIT Press.

Hoard, J. & C. Sloat (1973) English irregular verbs. Language, 49, 107-20.

Hockett, C. (1942) English verb inflection. Studies in Linguistics, 1.2.

Jespersen, O. (1942) A modern English grammar on historical principles, VI. Reprinted 1961: London: George Allen & Unwin Ltd.

Keil, F. C. (1986) The acquisition of natural kinds and artifact terms. In W. Demopoulos and A. Marras (Ed.), Language learning and concept acquisition: Foundational issues. Norwood, NJ: Ablex.

Kiparsky, P. (1982a) From cyclical to lexical phonology. In H. van der Hulst, & N. Smith (Eds.), The structure of phonological representations. Dordrecht, Netherlands: Foris.

Kiparsky, P. (1982b) Lexical phonology and morphology. In I. S. Yang (Ed.), Linguistics in the morning calm. Seoul: Hansin, pp. 3-91.

Kucera, H. and N. Francis (1967) Computational analysis of present-day American English. Providence: Brown University Press.

Kuczaj, S. A. (1976) Arguments against Hurford's auxiliary copying rule. Journal of Child Language, 3, 423-427.

Kuczaj, S. A. (1977) The acquisition of regular and irregular past tense forms. Journal of Verbal Learning and Verbal Behavior, 16, 589-600.

Kuczaj, S. A. (1978) Children's judgments of grammatical and ungrammatical irregular past tense verbs. Child Development, 49, 319-326.

Kuczaj, S. A. (1981) More on children's initial failure to relate specific acquisitions. Journal of Child Language, 8, 485-487.

Lachter, J. & Bever, T. G. (1987) The relation between linguistic structure and associative theories of language learning -- A constructive critique of some connectionist learning models. Unpublished manuscript, University of Rochester.

Lakoff, G. (1987) Connectionist explanations in linguistics: Some thoughts on recent anti-connectionist papers. Unpublished electronic manuscript, ARPAnet.

Levy, Y. (1983) The use of nonce word tests in assessing children's verbal knowledge. Paper presented at the 8th Annual Boston University Conference on Language Development, October, 1983.

Liberman, M. & Pierrehumbert, J. (1984) Intonational invariance under changes in pitch range and length. In M. Aronoff & R. Oehrle (Eds.), Language sound structure. Cambridge, MA: MIT Press.

MacWhinney, B. & Snow, C. E. (1985) The Child Language Data Exchange System. Journal of Child Language, 12, 271-296.

MacWhinney, B. & Sokolov, J. L. (1987) The competition model of the acquisition of syntax. In B. MacWhinney (Ed.), Mechanisms of language acquisition. Hillsdale, NJ: Erlbaum.

Maratsos, M. (1987) A study in novel word learning: The productivity of the causative. In B. MacWhinney (Ed.), Mechanisms of language acquisition. Hillsdale, NJ: Erlbaum.

Maratsos, M. & Kuczaj, S. A. (1978) Against the transformationalist account: A simpler analysis of auxiliary overmarkings. Journal of Child Language, 5, 337-345.

Marr, D. (1982) Vision, San Francisco, Freeman.

McCarthy, J. & Prince, A. (forthcoming) Prosodic morphology.

McClelland, J. L. & Rumelhart, D. E. (1985) Distributed memory and the representation of general and specific information. Journal of Experimental Psychology: General, 114, 159-188.

McClelland, J. L., Rumelhart, D. E., & Hinton, G. E. (1986) The appeal of parallel distributed processing. In D. E. Rumelhart, J. L. McClelland, and the PDP Research Group, Parallel distributed processing: Explorations in the microstructure of cognition. Volume 1: Foundations. Cambridge, MA: Bradford Books/MIT Press.

McClelland, J. L., Rumelhart, D. E., & The PDP Research Group. (1986) Parallel distributed processing: Explorations in the microstructure of cognition. Volume 2: Psychological and biological models. Cambridge, MA: Bradford Books/MIT Press.

McCulloch, W. S. & Pitts, W. (1943) A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysics, 5, 115-133.

Mencken, H. (1936) The American Language. New York: Knopf.

Minsky, M. (1963) Steps toward artificial intelligence. In E. A. Feigenbaum and J. Feldman (Eds.), Computers and thought. New York: McGraw-Hill.

Newell, A. & Simon, H. (1961) Computer simulation of human thinking. Science, 134, 2011-2017.

Newell, A. & Simon, H. (1972) Human problem solving. Englewood Cliffs, NJ: Prentice-Hall.

Norman, D. A. (1986) Reflections on cognition and parallel distributed processing. In J. L. McClelland, D. E. Rumelhart, & The PDP Research Group, Parallel distributed processing: Explorations in the microstructure of cognition. Volume 2: Psychological and biological models. Cambridge, MA: Bradford Books/MIT Press.

Osherson, D. N., Smith, E. E., & Shafir, E. (1986) Some origins of belief. Cognition, 24, 197-224.

Palmer, H. (1930) A grammar of spoken English on a strictly phonetic basis. Cambridge: W. Heffer.

Pazzani, M. (1987) Explanation-based learning for knowledge-based systems. International Journal of Man-Machine Studies.

Pazzani, M. & Dyer, M. (1987) A comparison of concept identification in human learning and network learning with the generalized delta rule. Unpublished manuscript, UCLA.

Pierrehumbert, J. & Beckman, M. (1986) Japanese tone structure. Unpublished manuscript, AT&T Bell Laboratories, Murray Hill, NJ.

Pinker, S. (1979) Formal models of language learning. Cognition, 7, 217-283.

Pinker, S. (1984) Language learnability and language development. Cambridge, MA: Harvard University Press.

Pinker, S., Lebeaux, D. S., & Frost, L. A. (1987) Productivity and conservatism in the acquisition of the passive. Cognition, in press.

Putnam, H. (1960) Minds and machines. In S. Hook (Ed.), Dimensions of mind: A symposium. New York: NYU Press.

Pylyshyn, Z. W. (1984) Computation and cognition: Toward a foundation for cognitive science. Cambridge, MA: Bradford Books/MIT Press.

Rosch, E. & Mervis, C. B. (1975) Family resemblances: Studies in the internal representation of categories. Cognitive Psychology, 7, 573-605.

Ross, J. R. (1975) Wording up. Unpublished manuscript, MIT.

Rosenblatt, F. (1962) Principles of neurodynamics. New York: Spartan.

Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) Learning internal representations by error propagation. In D. E. Rumelhart, J. L. McClelland, and the PDP Research Group, Parallel distributed processing: Explorations in the microstructure of cognition. Volume 1: Foundations. Cambridge, MA: Bradford Books/MIT Press.

Rumelhart, D. E. & McClelland, J. L. (1986a) PDP models and general issues in cognitive science. In D. E. Rumelhart, J. L. McClelland, and the PDP Research Group, Parallel distributed processing: Explorations in the microstructure of cognition. Volume 1: Foundations. Cambridge, MA: Bradford Books/MIT Press.

Rumelhart, D. E. & McClelland, J. L. (1986b) On learning the past tenses of English verbs. In J. L. McClelland, D. E. Rumelhart, & The PDP Research Group, Parallel distributed processing: Explorations in the microstructure of cognition. Volume 2: Psychological and biological models. Cambridge, MA: Bradford Books/MIT Press.

Rumelhart, D. E. & McClelland, J. L. (1987) Learning the past tenses of English verbs: Implicit rules or parallel distributed processing? In B. MacWhinney (Ed.), Mechanisms of language acquisition. Hillsdale, NJ: Erlbaum.

Rumelhart, D. E., McClelland, J. L., & the PDP Research Group. (1986a) Parallel distributed processing: Explorations in the microstructure of cognition. Volume 1: Foundations. Cambridge, MA: Bradford Books/MIT Press.

Sampson, G. (1987) A turning point in linguistics. Times Literary Supplement, June 12, 1987, 643.

Savin, H. & Bever, T. G. The nonperceptual reality of the phoneme. Journal of Verbal Learning and Verbal Behavior, 9, 295-302.

Shattuck-Hufnagel, S. (1979) Speech errors as evidence for a serial-ordering mechanism in sentence production. In W. E. Cooper and E. C. T. Walker (Eds.), Sentence Processing: Psycholinguistic studies presented to Merrill Garrett. Hillsdale, NJ: Erlbaum.

Sietsema, B. (1987) Theoretical commitments underlying Wickelphonology. Unpublished manuscript, MIT.

Sloat, C. and J. Hoard (1971) The inflectional morphology of English. Glossa, 5, 47-56.

Slobin, D. I. (1971) On the learning of morphological rules: A reply to Palermo and Eberhart. In D. I. Slobin (Ed.), The ontogenesis of grammar: A theoretical symposium. New York: Academic Press.

Slobin, D. I. (1985) Crosslinguistic evidence for the language-making capacity. In D. I. Slobin (Ed.), The crosslinguistic study of language acquisition. Volume II: Theoretical issues. Hillsdale, NJ: Erlbaum.

Smolensky, P. (1986) Information processing in dynamical systems: Foundations of harmony theory. In D. E. Rumelhart, J. L. McClelland, and the PDP Research Group, Parallel distributed processing: Explorations in the microstructure of cognition. Volume 1: Foundations. Cambridge, MA: Bradford Books/MIT Press.

Smolensky, P. (in press) The proper treatment of connectionism. Behavioral and Brain Sciences.

Somers, F. (1977) Oykangand grammar.

Sweet, H. (1892) A new English grammar, logical and historical. Oxford: Clarendon.

Talmy, L. (1985) Lexicalization patterns: Semantic structure in lexical forms. In T. Shopen (Ed.), Language typology and semantic description, Vol. 3: Grammatical categories and the lexicon. New York: Cambridge University Press.

Touretzky, D. (1986) BoltzCONS: Reconciling connectionism with the recursive nature of stacks and trees. Proceedings of the Eighth Annual Conference of the Cognitive Science Society.

Touretzky, D., & Hinton, G. E. (1985) Symbols among the neurons: Details of a connectionist inference architecture. Proceedings of the Ninth International Joint Conference on Artificial Intelligence.

Treisman, A. & Schmidt, H. (1982) Illusory conjunctions in the perception of objects. Cognitive Psychology, 14, 107-141.

van der Hulst, H. & Smith, N. (Eds.) (1982) The structure of phonological representations. Dordrecht, Netherlands: Foris.

Wexler, K. & Culicover, P. (1980) Formal principles of language acquisition. Cambridge, MA: MIT Press.

Wickelgren, W. A. (1969) Context-sensitive coding, associative memory, and serial order in (speech) behavior. Psychological Review, 76, 1-15.

Williams, E. (1981) On the notions "lexically related" and "head of a word". Linguistic Inquiry, 12, 245-274.

                                   Table 1:

       Proportion of Children's Verbs that Have Regular Past Tense Forms

                                    Stage
                __________________________________________________________
                1-Word        I        II        III        IV        V
                __________________________________________________________

Adam              ---      .45(31)   .43(44)   .55(83)   .46(83)   .54(78)
                                        *

Eve               ---      .55(31)   .51(49)   .45(53)   .48(58)   .44(45)
                                        *

Sarah             ---      .61(18)   .37(49)   .52(44)   .43(58)   .51(84)
                                                                      *

Lisa             .53(53)     ---       ---       ---       ---       ---
__________________________________________________________________________

Mean for
Adam, Eve, & Sarah           .54       .44       .51       .46       .50


Size of verb vocabulary is listed in parentheses.
An asterisk indicates the stage at which the child began overregularizing.
                                   Table 2:

 Ranks of Tendencies to Overregularize Irregular Verbs involving Vowel Shifts

                  Children     RM        RM        RM        RM      Avg Freq o
                      *      1st set   2nd set   3rd set   Average   Vowel Shif
                 ______________________________________________________________

Verb Subclass
_____________

VIII   blow/blew     1 (.80)    1         1         1        1        1 (1.6)

VI     sing/sang     2 (.55)    4         4         4        4        3 (2.7)

V      bite/bit      3 (.34)    2         3         6        3        5 (3.9)

VII    break/broke   4 (.32)    3         6         3        6        2 (2.1)

III    feel/felt     5 (.13)    6         5         5        5        4 (3.8)

IV     seek/sought   6 (.10)    5         2         2        2        6 (4.5)
_______________________________________________________________________________

Rank Order Correlation
With Children's
Proportions                    .77       .31       .14      .31      .71




* Actual proportions of regularizations by children are in parentheses.

** Mean number of verbs in the irregular corpus exemplifying the vowel shifts
   within a class are indicated in parentheses.

Appendix: English Strong Verbs

Prefixed forms are listed when the prefix-root combination is not semantically transparent.

The term 'laxing' refers to the replacement of a tense vowel or diphthong by its lax counterpart. In English, due to the Great Vowel Shift, the notion 'lax counterpart' is slightly odd: the tense-lax alternations are not i-I, e-@symbol[e, u-U], and so on, but rather ay-I, i-@symbol[e, e-a@kern[2 points]e, o-O/a, u-O/a]. The term 'ablaut' refers to all other vowel changes.

I. T/D Superclass

hit, slit, split, quit, ?knit(+),(He knit a sweater is possible, not ??He knit
bid(As in poker, bridge, or defense contracts.), rid, ?forbid
shed, spread, wed(+)(The adjective is only wedded.)
let, set, beset(Mainly an adjective.), upset, wet(+)
cut, shut
put
burst, cast, cost, thrust (+)
hurt

2. T/D with Laxing Class

bleed, breed, feed, lead, mislead, read, speed(+), ?plead(+)
meet
hide (en), slide
bite (en), light(+), alight(+!)
shoot

3. Overt -T Ending

3a. Suffix -t

burn, ??learn, ?dwell, ??spell, ???smell
?spill, ??spoil

3b. Devoicing.

bend, send, spend, ?lend, ?rend
build

3c. -t with Laxing

lose
deal, feel, ?kneel(+)
mean
?dream
creep, keep, leap(+), sleep, sweep(+), weep
leave

3d. x - ought - ought

buy, bring, catch, fight, seek, teach, think

4. Overt -D Ending

4a. Satellitic laxing (cf. bleed group)

flee
say
hear

4b. Drop stem consonant

have
make

4c. With Ablaut [@symbol[e - o - o]]

sell, tell, foretell

4d. With unique vowel change and +n participle:

do

II. E-0 ablaut class

freeze, speak, ??bespeak, steal, weave(+)(Only in reference to carpets, etc. is get, forget, ??beget

??tread(Though trod is common in British English, it is at best quaint in American English.)

swear, tear, wear, ?bear, ??forbear, ??forswear

2. Satellitic x - o - o+n

awake, wake, break
choose

III. I - a@kern[2 pointse/^ - ^ Group]

ring, sing, spring
drink, shrink, sink, stink
swim
begin

2. I - ^ - ^

cling, ?fling, sling, sting, string, swing, wring
stick
dig
win, spin
?stink, ?slink

3. Satellites x - a@kern[2 pointse/^ - ^]

run (cf. I - a@kern[2 points]e - ^)
hang, strike(Stricken as participle as in 'from the record', otherwise as an ad
?sneak (cf. I - ^ - ^)

IV. Residual Clusters

1. x - u - x/o+n

blow, grow, know, throw draw, withdraw fly ?slay

2. e - U - e+n

take, mistake, forsake, shake, partake

3. ay - aw - aw

bind, find, grind, wind

4. ay - o - X

4a. ay - o - I+n

rise, arise write, ??smite ride drive, ?strive

4b. ay - o - ?

dive, shine(Typically intransitive: *He shone his shoes.)
?stride
??thrive V. Miscellaneous

1. Pure Suppletion

be
go, forgo, undergo

2. Backwards Ablaut

fall, befall (cf. get-got)
hold, behold (cf. tell-told)
come, become

3. x - Y - x+n

eat
beat
see (possibly satellite of blow-class)
give, forgive
forbid, ??bid(As in 'ask or command to'. The past bade is very peculiar, bidded is impossible, and the past participle is obscure, though certainly not bidden.)

4. Miscellaneous

sit, spit
stand, understand, withstand (possibly satellite of I - ^ - ^ class)
lie

5. Regular but for past participle

a. Add -n to stem (all allow -ed in participle)

sow, show, sew, prove, shear, strew

b. Add -n to ablauted stem

swell

(2)Compare in this regard Ross's (1975) study of productive affixation, which uncovers an actual constraint involving the central/extended distinction. Ross finds that prefixes like re-, un-, mis-, which affect meaning, are sensitive in various ways to the meaning of the base they attach to. He amply documents the fact that such prefixes reject metaphorically extended bases. Thus: "Horace Silver (*re-)cut Liberace" (cut = 'played better than'), "Larry (*mis-)fed Dennis" (fed = 'passed the basketball to'). Examination of Ross's numerous examples shows not one where metaphorical extension affects irregularity. The contrast could not be starker. Notions like 'past tense' have no sensitivity to the lexical semantics of the base.

(3)For details of the study, see Brown, 1973; for descriptions of the unpublished grammars, see Brown, 1973, and Pinker, 1984. Verification of some of the details reported in the grammars, and additional analyses of children's speech to be reported in this paper, were based on on-line transcripts of the speech of the Brown children included in the Child Language Data Exchange System; MacWhinney & Snow, 1985.