Throughout my professional career, two themes have been driving my work: time and control.
With respect to time, two publications from the late nineteenth en early twentieth century have thoroughly influenced my work. The first is the book by the Swiss neurologist Constantin von Monakow (1853-1930) on brain localization and brain dysfunction.
When we observe, Von Monakow says, the sometimes severe effects of brain damage on behavior, it is tempting to conclude that the patient no longer knows ‘how to move’ or ‘how to speak’. That is, we want to entertain the hypothesis that representations underlying such behaviors are lost. However, a representation is not static piece of knowledge, but a process. He compares representations to melodies. In the case of a music box, we cannot ask ourselves: where is the melody ‘localized’? The melody comes into existence only when it is ‘played’, that is when the various parts of the music box are set into movement in the proper way. The representations the brain is able to sustain are called kinetic melodies. They are realized by activating various parts of the cortex in the proper order and the proper rate. This description seems particularly appropriate with respect to language and movement, both sequential types of behavior.
Brain damage affects this process of temporal integration of information within a certain brain region. This can occur when the region itself is damaged, but also when areas connected to this region are damaged. This is von Monakow’s famous concept of diaschisis: brain damage can have nonlocal effects. For a long time, this has remained speculation, but with the advent of modern brain imaging, it became possible to confirm this hypothesis (Price et al., 2001).
In the spirit of the concept of kinetic melodies, Henk Haarmann and I designed a computer model, called SYNCHRON - in which syntactic trees underlying particular sentence types were built-up over time (Haarmann and Kolk, 1991a; see My publications). Lexical and phrasal nodes were ‘retrieved’ within a certain amount of time and when a node was retrieved, it remained available a certain amount of time. Retrieval time and memory time together, determined the temporal window, within which not only the tree had to be constructed, but semantic interpretation dependent upon this tree had to be completed as well. Aphasic behavior was simulated by slowing down retrieval or by shortening the memory time. With this ‘chronogenetic’ model, we were able to simulate aphasic data from two different studies. It is noteworthy that, at the time of this publication, there were no models that actually simulated data from brain-damaged patients. In later studies, we have found support for the reduced temporal window hypothesis (Haarmann and Kolk 1991b, Haarmann and Kolk, 1994; Hartsuiker and Kolk, 1998; Hartsuiker, Kolk and Huinck, 1999; Kok et al., 2007).
From Haarmann and Kolk (1991b). Shown is the dynamic build-up of the verb phrase. Retrieval and memory time are as supposedly present in unimpaired individuals.
The second publication that has shaped my thinking about time is the paper by the German neurologist Hubert Grashey (1939-1914) on the relation between aphasia and perception.
Grashey was testing an aphasic patient with severe word-finding difficulties, using pictures the patient had to name. More or less by accident, he observed that, when a particular picture was removed, the patient almost immediately forgot what it was. When the same picture was presented again together with other pictures, the patient could not tell which of the pictures he had seen just a moment ago. Grashey argued that this particular difficulty with keeping the concept of the picture in mind and his problems with naming were related. The name of the picture will take some time, to retrieve, however short, and during this period, the concept of the picture has to remain available. So, even for naming a picture, there is a temporal window in which the name must be retrieved, while the picture concept is still available.
A modern model that embodies Grashey’s hypothesis, is the spreading activation model of Gary Dell (Dell, 1986). In this model, speaking a word requires activation of the different types of information connected to that word, e.g. semantic, lexical and phonological. These types of knowledge are represented by nodes in a network. When a picture has to be named, activation starts in the semantic nodes and than spreads to the lexical and the phonological nodes. It takes a certain amount of time for activation to reach another node, and when this node is activated, it starts to loose its activation immediately. Both normal and aphasic naming error patterns can be simulated by assuming that temporal integration fails, in that either information decays too rapidly or is retrieved too slowly. Obviously, these temporal failures are more severe for the aphasic than for the non-aphasic speakers (Dell et al., 1997).
We have extended this analysis to stuttering as a developmental disorder. We have proposed that slow retrieval at the phonological level is responsible for the speech deficit in stutterers (Kolk and Postma, 1997).
Perhaps
the most important insight I got in my whole career was when I –
finally – understood the paper by the German neurologist and
psychiatrist Max Isserlin (1979-1941). Isserlin wrote about
agrammatism, a deficit that leads the patient to leave out
grammatical morphology, leading to the 
so-called
telegraphic
style. So a patient with such a disorder says things
like 'home all day, wife gone, watching
television'.
Such an utterance looks so much defective, that it took me a long time
to understand what Isserlin meant when he said that this sentence, from
its very beginning, was construed this way. What was incomplete, was
intended to be incomplete. Why would a patient intentionally produce
such incomplete sentences? The patient Isserlin reported upon in his
1922 paper, explained this in the following way: 'Sprechen, keine Zeit,
Telegammstil' - 'speaking no time, telegraphic style'. This account
nicely connects to the notions on time, decribed above. A reduced
temporal window for sentence processing, will typically not allow the
construction of a complete sentence, with its attendant grammatical
morphology, but will accomodate to the construction of a telegraphic
utterance. Assuming that the patients intends to be incomplete, implies
that such telegraphic construction are part of the normal language
repertoire. For an overview of our research supporting this theory, see
Kolk (2006), which contains references to more specific studies.
Isserlin positions his hypothesis within the theoretical framework of the Würzburg school of thought, headed by Oswald Külpe (1962-1915). In one of the central Würzburg studies, subjects were presented with questions like ‘to what category does the word ‘rose’ belong? The reaction times were compared to the ones obtained in free association tasks: “what do you think of first if you read the word ‘rose’”? To their amazement, the
Würzburgers found that the answers in the first task could be as fast, sometimes even faster, than in the free association task. The free associations should come up automatically and therefore delay the production of the right answer in the categorization task. There apparently is some internal mechanism that selects just the relevant set if items, over the ones that come up automatically. They called this mechanism Einstellung. In the English literature, it has become known as mental set. According to Isserlin, telegraphic speech is a phenomenon of mental set. In ordinary conversations, the automatic response is to produce a full sentence; the aphasic however develops a mental set for incomplete sentence forms.
In nowadays terminology, the selection of task-specific over automatic responses is referred to as executive control. An important way to study this mechanism is by asking the subject to name the color a word is printed in, say ‘green’, when the word itself refers to another color, say red. The automatic response in this Stroop task is of course ‘red’, whereas the task-specific response is ‘green’. Executive control is needed to select the task-specific response over the automatic one. If Isserlin’s claim is right, than aphasics should have executive control of their language system, because they are able to select an non-automatic way of formulating the sentence – the telegraphic way - rather than the automatic one, and this should hold for unimpaired language users as well. Although this may sound like a truism, current thinking is that language is a fully automatic process, not in need of executive control. Evidence, that there is such control comes from studies that demonstrate that unimpaired speakers can repair their own speech errors (e.g. Hartsuiker and Kolk, 2001). The process responsible for this repair behavior is called monitoring. We also have hypothesized that the stuttering symptoms are due to repair of errors in the planning of speech (see Kolk and Postma, 1997).
Most recently, we have provided
evidence that there is also
monitoring in language perception. In addition to speech errors, there
are many instances of misreading and mishearing. These errors are
documented in the literature, but the question of how these errors are
detected and repaired so far has not systematically been addressed. We
became first aware of the possibility of monitoring in perception,
when, in an ERP study, we observed P600 effects after semantic
anomalies. This finding required us to rethink the functional
significance of the P600 effect, which so far had been interpreted as
signaling syntactic processing per se. Our new interpretation was that
a P600 reflects reprocessing to check for a possible perception error.
This reprocessing we suppose, is triggered when, , what we hear or what
we read, conflicts strongly with what we expect. This theory has been
tested in a series of ERP studies ( Kolk et al., 2003; Van Herten et
al., 2005;2006; Vissers et al.,2006a,b; Vissers et al 2008; Van de
Meerendonk et al. 2010), Again, it
appears that there is much more executive control of language than is
commonly thought. We have recently started to investigate the
hymodynamic correlates of monitoring. One first result was that
spelling violations and grammatical violations activated the same area
in the left inferior frontal cortex (Broca's area; Van de Meerendonk,
Indefrey, P., Chwilla, and Kolk, in press).
Dell, G.S. (1986). A spreading activation theory of retrieval in
sentence production. Psychological
Review, 93, 283-321
Dell,
G.S., Schwartz, M.F., Martin, N., Saffran, E.M., & Gagnon., D.
A.
(1997). Lexical access in aphasic and non-aphasic speakers. Psychological Review, 104,
801-838.
Grashey, H. (1885). Über Aphasie und ihre Beziehung zur
Wahrnehmung. Archiv fur Psychiatrie
und
Nervenkrankheiten, 16, 654-688
(translation: R.de Bleser. On aphasia and its relation to perception. Cognitive Neuropsychology, 6,515-546,
1985)
Isserlin M. (1922) Uber agrammatismus. Zeitschrift für die gesamte
Neurologie und Psychiatrie, 75, 322–416. (partial translation:
Droller, H., Howard, D.,&
Campbell R. On agrammatism. Cognitive
Neuropsychology,
2, 303-345, 1985)
Van de Meerendonk, N., Kolk, H.H.J., Vissers, C.T.W.M., Chwilla,
D.J., 2010. Monitoring in language perception: Mild and strong
conflicts elicit different ERP patterns. Journal of Cognitive
Neuroscience. 22, 67–82.
Van de Meerendonk, N., Indefrey, P., Chwilla, D.J., and Kolk, H.H.J.
(in press) Monitoring in language perception: Electrophysiological and
hemodynamic responses to spelling violations. Neuroimage.
Von Monakow, C. (1914). Die
lokalisation im Grosshirn und der Abbau de Funktion durch kortikale
Herde. Wiesbaden: Bergman
(partial translations: Harris, G., & Pribram K.H. (transl/ed). Moods, states and mind.
London, England: Pinguinbooks,1969;Von Bonin (ed). Some papers on the cerebral
cortex. Springfield (Ill): Charles C. Thomas. 1960)
Price, C.J., Warburton, E.A., Moore, C.J., Frackowiak, R.S.J., & Friston, K.J. (2001). Dynamic diaschisis: Anatomically remote and context-sensitive brain lesions. Journal of Cognitive Neuroscience, 13, 419-429.