Show Less
Open access

Developing and Assessing Academic and Professional Writing Skills

Series:

Edited By Susanne Göpferich and Imke Neumann

Academic literacy used to be considered a complex set of skills that develop automatically as a by-product of academic socialization. Since the Bologna Reform with its shorter degree programmes, however, it has been realized that these skills need to be fostered actively. Simultaneously, writing skills development at all levels of education has been faced with the challenge of increasingly multilingual and multicultural groups of pupils and students. This book addresses the questions of how both academic and professional writing skills can be fostered under these conditions and how the development of writing skills can be measured.
Show Summary Details
Open access

Writing competence profiles as an assessment grid? – Students’ L1 and L2 writing competences and their development after one semester of instruction

Susanne Göpferich, Imke Neumann Justus Liebig University Giessen, Germany

Writing competence profiles as an assessment grid? – Students’ L1 and L2 writing competences and their development after one semester of instruction

Abstract

English: For the design of compensatory writing courses and determining the progress that students make after one semester of writing instruction, the L1 (German) and L2 (English) writing skills that German students come with when entering university were assessed using argumentative essays they had to compose at the beginning of writing courses and at their end respectively in either their L1 (n=26) or their L2 (n=35) depending on the type of writing course (German or English) they attended. The essays were evaluated by three raters in a discursive consensual assessment procedure using a (text-)linguistic error classification scheme as well as a holistic evaluation of their argumentative rigour. The application of the error classification scheme yielded assessments of the texts on four levels (formal, lexical, grammatical and textlinguistic correctness), whereas the holistic evaluation yielded an assessment of the texts’ argumentative rigour (stringency). A comparison of the results obtained at the beginning and at the end of the courses showed significant improvements at all levels for the English texts (L2) but only at the lexical level and the text level for the German texts. Furthermore, significant correlations were found between the lexical and grammatical sub-competencies, between the lexical and textlinguistic sub-competencies and between the grammatical and textlinguistic sub-competencies within each language at least at the beginning of the courses. Comparisons between languages both at the beginning and at the end of the courses revealed that L2 text production competence significantly lagged behind L1 text production competence at all levels except for the text level at the beginning of the writing courses. These findings are interpreted against the background of the cognitive capacity theory of writing and dynamic systems theory. The visualization of the results in area graphs representing writing competence profiles revealed characteristic patterns which suggest that such area charts, when available for a larger spectrum of writing competences, may be usable as grids for the assessment of writing competence. ← 103 | 104 →

German: Zur Konzeption kompensatorischer Schreibkurse für die L1 Deutsch und die L2 Englisch sowie zur Ermittlung der Schreibkompetenzentwicklung nach einem Semester Unterricht wurden die L1- und die L2-Schreibkompetenz von deutschen Studierenden anhand von argumentativen Texten erhoben, die diese zu Semesterbeginn und am Semesterende in Abhängigkeit vom besuchten Schreibkurstyp (Deutsch oder Englisch) in ihrer L1 (n=26) bzw. ihrer L2 (n=35) verfassen mussten. Die Texte wurden von drei Lehrkräften in einem diskursiven Prozess analytisch mit Hilfe eines (text-)linguistischen Fehlerklassifizierungsrasters sowie holistisch bewertet. Die Anwendung des Fehlerklassifizierungsrasters ergab eine Bewertung auf vier Ebenen (formale, lexikalische, grammatikalische und textlinguistische Korrektheit), während die holistische Evaluation zur Beurteilung der argumentativen Stringenz genutzt wurde. Der Vergleich der zu Kursbeginn und am Kursende erzielten Resultate ergab eine signifikante Verbesserung in allen Bereichen bei den englischen Texten (L2), jedoch nur eine Verbesserung im lexikalischen und textlinguistischen Bereich bei den deutschen Texten (L1). Darüber hinaus fanden sich innerhalb der Sprachen zumindest zu Kursbeginn signifikante Korrelationen zwischen der lexikalischen und grammatikalischen Teilkompetenz, zwischen der lexikalischen und der textlinguistischen Teilkompetenz sowie zwischen der grammatikalischen und der textlinguistischen Teilkompetenz. Der Vergleich der Ergebnisse in beiden Sprachen zeigte sowohl in der Erhebung zu Kursbeginn als auch am Kursende, dass die Textproduktionskompetenz in der L2 in allen Bereichen geringer ausfiel als in der L1; eine Ausnahme bildete lediglich zu Beginn der Kurse die textlinguistische Teilkompetenz. Die Ergebnisse werden vor dem Hintergrund der Cognitive Capacity-Theorie und der Dynamic Systems-Theorie interpretiert und in Flächendiagrammen, die Schreibkompetenzprofile darstellen, visualisiert. Diese Visualisierung lässt charakteristische Muster erkennen, die nahelegen, dass solche Flächendiagramme als Raster zur Bewertung von Schreibkompetenz genutzt werden könnten, wenn sie für ein größeres Spektrum von Schreibkompetenzprofilen vorliegen.

1    The context and rationale of the study

In the wake of the Bologna reform, German universities have realized that more time and effort need to be devoted to the development of students’ literacy or text competence, encompassing text reception competence, text production competence and the competence to learn from texts (Portmann-Tselikas/Schmölzer-Eibinger 2008: 5). At least three reasons have led to this insight: First, the development of these competencies is still in progress when students enter university and needs to be fostered actively in its dis ← 104 | 105 → cipline- and task-specific continuation (see, e.g., Beaufort 2007). Second, shorter university degree programmes give students less time to develop these competencies as a ‘by-product’ of their academic socialization. And third, an increasing student intake at universities (more than 50% of an age cohort in Germany in 2012) has led to more heterogeneous entrance qualifications among students due to more diverse educational biographies and thus the need for compensatory courses including compensatory writing courses (see also Göpferich 2015).

In order to gear its compensatory writing courses to the actual needs of the students who attend them, a research project was launched in 2012 at the newly established writing centre of the University of Giessen (Germany) in which students’ writing skills in their L1 (German) and their L2 (English) were assessed in areas that have frequently been assumed as mastered at university entrance level. Based on the findings of this assessment, L1 and L2 compensatory writing course syllabi were designed which address the needs detected, and students’ progress was assessed after one semester of instruction (30 contact hours and 30 hours of homework) in such L1 and L2 compensatory writing courses.

The assessment results were visualized in area graphs, so-called writing competence profiles, which set the individual writing sub-competencies analysed in the study in relation to each other. A comparison of these area graphs from the beginning and from the end of the writing courses and between languages illustrates the progress students made in the course of one semester of instruction and shows the developmental lag between the L1 and the L2 writing skills.

The compensatory writing courses focused on in this study were introduced against the background that basic linguistic skills related to writing, such as writing in a formally, lexically and grammatically correct and coherent manner (for details of the construct of writing competence measured, see Section 3.1), cannot be remedied in discipline-specific writing courses and therefore should be addressed in introductory general writing courses preceding discipline-specific courses for the following three reasons. First, dealing with such linguistic problems in discipline-specific writing courses would offer no challenge for those students who have already acquired the relevant competencies; second, teachers of discipline-specific writing-intensive seminars are subject-domain specialists and cannot be expected ← 105 | 106 → to have the specific linguistic and didactical knowledge to address general language-related issues; and third, discipline-specific writing courses ideally should be devoted to the advanced stages of academic writing socialization. In this manner, domain-specific literacy development and the development of domain-specific knowledge and competences can benefit from each other (cf. Ulmi et al. 2014: 7, 226).

2    Theoretical background and previous studies

This section introduces the models of writing competence development against which the findings from the present study will be interpreted, and sets this study in relation to previous ones comparing L1 and L2 writing skills.

2.1   Modelling writing competence development

Many models and theories of writing competence development that have been suggested to date have their origins in cognitive psychology. They explain processes of writing competence development with reference to automatization or routinization of sub-processes, or lower-order processes, of writing, which is assumed to release working memory capacity with the ensuing possibility of a reallocation of these cognitive resources to higher-order processes. Among these theories are McCutchen’s (1996) cognitive capacity theory of writing and Kellogg’s (2008) macro-stages of writing skills development, which are compatible with each other. According to Kellogg’s macro-stage model of writing skills development, the cognitive capacity that becomes available through increasing automatization of lower-order processes in accordance with the cognitive capacity theory can be used by authors, for example, to take an increasing number of perspectives on their texts into account. Whereas in the early stage of knowledge-telling, authors are limited to their author’s perspective (i.e., what the author intends to express, the author meaning), the stage of knowledge-transforming is characterized by authors being able to keep both the author meaning and the actual text meaning in mind and compare them for discrepancies, which can then be corrected. Kellogg’s highest stage of writing skills development, knowledge-crafting, then allows authors to keep both their author meaning and the text meaning in mind and, in addition, take into account whether ← 106 | 107 → their prospective readers will be able to derive the intended meaning from the text. This allows them to take a more reader-centred perspective.

Steinhoff (2007) and Pohl (2007) conducted corpus-linguistic studies to analyse the development of students’ academic writing competence. Steinhoff, for example, compared academic textual routines or procedures1 used in German term papers composed by university students at different stages of their academic socialization with their equivalents in articles composed by scholars and functionally comparable routines and formulations in a corpus of journalistic texts. For the interpretation of his data, he drew on Piaget’s theory of cognitive development. According to this theory, the driving force in the development of intelligence is the reestablishment of an equilibrium, whenever this has been destroyed, by processes of assimilation, i.e., the adaptation of the environment to the learner’s cognitive schemata, or accommodation, i.e., the adaptation of the learner’s cognitive schemata to the environment (Piaget 1972: 191; cf. Ortner 1995: 336). With regard to academic writing competence development, the search for this equilibrium means developing writing behaviour that is adequate in academic contexts out of writing behaviour that was acquired for other contexts and is found to no longer fulfil its function in a satisfactory manner in the new context. To determine students’ university entrance qualifications in writing, it is exactly the writing behaviour that students acquired for other contexts which needs to be focused on and thus is the focal point of the present study.

In his corpus analyses, Steinhoff (2007: 419) found that the development of students’ academic writing competence follows an inter-individual pattern. At the early stages of their academic writing skills development, assimilation processes dominate, i.e., processes in which known behaviour is used to solve new problems, whereas at more advanced stages, accommodation processes take over, i.e., processes in which new strategies are made use of. At the initial stage, two types of problem solving were found to dominate: transposition and imitation. Transposition is an assimilating strategy, in which writers try to solve their problems of academic writing by resorting to means and measures that are untypical of academic writing and ← 107 | 108 → occur in genres they are more familiar with, for example, in journalistic (or popular-science) texts, in genres that students were required to compose at school, and in oral communication. Imitation is an accommodating strategy in which students, unsuccessfully or only partially successfully, try to imitate academic language. Formulations that result from this strategy are characterized by, for example, extreme complexity, excessive nominalizations and the use of an extremely formal style, which often results in a lack of comprehensibility (Steinhoff 2007: 143 ff., 423). The stages of transposition and imitation are followed by a stage of transformation, in which, in an accommodating manner, the repertoire of academic formulations that students have at their disposal is expanded. Students begin to learn typical formulations of academic writing and the communicative functions for which they are used in academic contexts. The fact that, at this stage, they have not yet fully reached expert competence, shows in, for example, wrong collocations and an excessive and repetitive use of certain domain-specific formulations. Another characteristic of this stage is that students begin to recognize that certain general-language expressions are used for more specific purposes in academic texts (Steinhoff 2007: 146 ff., 423 f.). Students’ writing competence development reaches its (preliminary) final stage, the stage of contextual adequacy, when they start to use academic language following domain-specific conventions and employing a differentiated academic vocabulary (Steinhoff 2007: 424, cf. 148 f.).

Another paradigm in which writing competence development can be modelled is dynamic systems theory (DST; Thelen/Smith 1994; Van Gelder 1989).2 DST has found its way from mathematics into various other fields of research, such as developmental psychology (see Thelen/Smith 1994), second-language acquisition (e.g., de Bot/Lowie/Verspoor 2007; Verspoor/Lowie/Van Dijk 2008), translation studies (Göpferich 2013) and also L2 writing skills development (e.g., Verspoor/de Bot/Lowie 2004; Verspoor/Schmid/Xu 2012; Nitta/Baba 2014). In DST, competences or skills are envisaged as dynamic systems, i.e., sets of variables that are interconnected and thus interact over time (de Bot/Lowie/Verspoor 2007: 8). With regard to writing competence, these variables can be regarded as variables for writ ← 108 | 109 → ing sub-competencies that, in their entirety, make up writing competence. These individual sub-competencies may not develop at the same pace, nor will they always develop in a linear manner. Some may stagnate while others continue to develop. Certain sub-systems (i.e., sub-competencies or clusters of sub-competencies) may be precursors of other sub-systems in the developmental process. For a specific sub-competence to start developing, it may be necessary for other sub-competencies to have exceeded a certain threshold value. If we start from the cognitive capacity theory of writing (McCutchen 1996), which assumes limited working memory capacity, it appears plausible to assume that the value one variable takes, i.e., the cognitive effort required for a specific sub-process of writing, such as lexical choices, has an effect on all the other variables. For example, if one sub-competence reaches an advanced level, which may result in automatized performance of the respective tasks, working memory capacity is released. This capacity then becomes available for the application of other sub-competencies and their development, such as the capacity to take larger stretches of a text into account to create coherence, for which there may not have been enough cognitive resources left as long as other sub-processes still needed cognitive effort.

DST further assumes that, in dynamic systems, repeller states and attractor states exist. These can best be explained by means of the metaphor of a plane with holes and bumps in it on which a ball rolls. The ball and its trajectory on the plane represent a specific competence and its development path. On its ways over the plane, the ball is attracted by the holes (attractor states) and repelled by the bumps (repeller states). To get it out of a hole, much energy is needed. Such holes can explain fossilization of certain errors or stages of development, as observed in second-language acquisition, which can only be overcome by intensive training. The development that a particular person undergoes, i.e., the trajectory of the ball, may be highly individual, yet attractor states account for the fact that certain sub-competencies seem to occur in a specific order or at a specific stage of competence development because the corresponding attractor states, in an evolutionary perspective, possess qualities that make the application of cognitive resources settle into an equilibrium at these stages. In DST, the fact that a certain competence stage has been achieved becomes visible by a relative stagnation in the development of the values the various variables ← 109 | 110 → that make up the system take at this stage, whereas a move out of the hole, a new competence development burst, shows in a large variability in the set of variables which form the competence system (Thelen/Smith 1994: 97).

What complicates the analysis of writing skills development in a DST approach is that dynamic systems are nested, i.e., every system is always part of a larger system. Accordingly, writing competence forms a sub-system of the larger system of communicative competence among others. The aspects of writing competence focused on in this article (for details, see Section 3.1) again form only one component of the larger academic writing competence system analysed by Steinhoff and Pohl. In their investigations, however, they concentrated on the developmental stages following a writing competence level that students are generally assumed to have acquired already at school, which, as the findings from the present study will show, is not always the case.

An even more encompassing concept of writing competence can be found in recent publications on writing competence assessment that draw on Bronfenbrenner & Morris’ (2006) bioecological model of human development. These publications (e.g., Slomp 2012; Driscoll/Wells 2012; Wardle/Roozen 2012) plead for analyses of writing competence development which do not only take into account contextual factors such as the type of writing instruction students were exposed to, but also characteristics of the individual, dispositions, such as value, self-efficacy, attribution and self-regulation (Driscoll/Wells 2012: 7). This nesting or embeddedness of competences makes it difficult to draw a border around the system an investigation focuses on, and drawing such a border always means ignoring factors beyond the border that may have effects relevant to the system in question.

2.2   Language (in-)dependence of writing skills

The main difference that has been found between L1 and L2 text production is the higher cognitive demand involved in choices of lexis and grammar in the L2 as compared with the L1. In accordance with the cognitive capacity theory of writing, however, higher cognitive demands at these lower levels of text production have been found to have negative effects on higher-level decisions (Silva 1992; Devine/Railey/Bischoff 1993; Whalen/Menard 1995; Schoonen et al. 2003; Roca de Larios/Manchon/Murphy 2006; see also the ← 110 | 111 → literature review by Cumming 2001). Such findings provide support for the assumption that the epistemic benefits of writing are less pronounced when this writing takes place in the L2. They also warrant the assumption that the epistemic function of writing can only be fully exploited in the L1 if students have achieved a certain minimum fluency with regard to lower-order processes, e.g., at the lexical and grammatical levels, when writing in this language. This underlines the importance of offering compensatory writing courses which address these issues for preparing students to derive the maximum benefit from discipline-specific writing courses where writing is also used as a means of more profound reflection.

3    Methodology

3.1   The construct assessed and the assignments

The construct of writing competence assessed in the present study was the students’ ability to express themselves in a formally and linguistically correct, cohesive, coherent and well-reasoned manner in a genre they were familiar with from their secondary education (for the reasons, see Section 2.1). Requiring them to compose texts of unfamiliar genres could have led to a lack of motivation and, according to the cognitive capacity theory of writing (McCutchen 1996), to cognitive capacity being absorbed by reflections concerning the unfamiliar genre which could then have led to poorer performance on lower levels, such as the levels of formal, lexical and grammatical correctness, that would otherwise have received more attention. In such cases, performance on these lower levels might not have reflected students’ real competence at these levels, although their competence was more relevant for the purposes of this study than their performance. Testing students’ competencies at these levels in tests which did not require them to compose a text but to fill in gaps or complete sentences was not an option either because in that case students could have fully concentrated on the level in focus, which is an unrealistic situation because communication usually takes the form of texts, and composing texts involves more complex cognitive processes which – at least to some extent – have to be juggled in parallel. Furthermore, assignments had to be selected that did not require specialized prior content knowledge that would have led to failure if students had not possessed it. This was a requirement because neither the ← 111 | 112 → students’ prior content knowledge nor their ability to do literature research were part of the construct for which the students were tested, and failure to complete the assignment because of a lack of prior knowledge would not have allowed to elicit data on those aspects of writing skills that were in the focus of this study. In addition, the texts to be composed had to address a readership whose prior knowledge and interests could be judged realistically by the students.

For these reasons, the particular type of assignment selected was writing an argumentative article for a student magazine. This genre could be expected to be familiar to students because the composition of argumentative texts forms part of the German standard school curriculum. Furthermore, the ability to compose a logical argumentation is a central academic skill. The manner in which students connect statements indicates whether they are able to keep their argumentative goal in mind and come to a logical conclusion. To ensure that students did not lack the prior knowledge relevant for completing the assignment, topics were selected about which all students could be expected to have the relevant prior knowledge required. Offering them different topics to choose from was an additional measure to make sure that the topic did not present an obstacle for students.

3.2   Data collection

In their second weekly 90-minutes writing course sessions (out of 15), the participants of general writing courses that were offered at the writing centre of Justus Liebig University in Giessen in the winter semester 2012/13, the summer semester 2013 and the winter semester 2013/14 were required to write an argumentative text of approximately 250 to 350 words on one of three topics they could choose from for a student magazine (see the assignments in Appendix A). Students who attended writing courses for German had to compose their texts in their L1 German; students who attended writing courses for English had to do so in their L2 English. They were not allowed to use external resources. In the last session of their writing courses, students were required to compose another text of the same genre and in the same language on one of three other topics they could again choose from. The texts had to be completed within the 90-minutes sessions. The students wrote the texts in a computer lab using Microsoft Word as the word processor they were familiar with. ← 112 | 113 →

3.3   Research population

The population of the study presented in this article was composed of two groups: The first group comprised 49 students who attended writing courses in their L1 German (German group). The second group comprised 51 other students who attended writing courses in their L2 English (English group). Out of the 49 students of the German group, only 26 completed their writing assignments at the end of the German writing courses; out of the 51 students of the English group, only 35 did so in the English courses. Only the data obtained from students who completed the writing assignments at both the beginning and the end of their writing courses could be taken into account in the comparisons of their performances at the beginning and at the end of the courses.

The 49 participants who attended the German writing courses came from all faculties. 35 of them were enrolled in BA programmes, 6 students in MA programmes, 6 students in teacher training programmes, 1 student in an 8-semester university diploma programme and another student in a programme for veterinary medicine. They were in their 1st to 22nd semesters (median semester: 3; average semester: 4).

Out of the 51 participants of the English writing courses, 24 were enrolled in study programmes with English language and literature as one of their subjects (21 students in BA programmes, 1 student in an MA programme and 2 students in a teacher training programme). The remaining 27 students came from all faculties. 27 out of the 51 students were in their 2nd semester, 3 in their 4th semester, 4 in their 5th semester, 4 in their 6th semester, and 13 in a higher semester (median semester: 2; average semester: 3.4).

3.4   Treatment

In the German writing courses, the following topics, which address both the weaknesses detected in the texts composed at the beginning of the courses and additional topics, were covered: planning writing projects, time management; finding a topic and specifying a research question; literature research and reading strategies; strategies for overcoming writer’s block; writing an outline; text organization; planning one’s line of argumentation; specificities of academic writing (in contrast to non-academic writing); prototypical text macrostructures and typical formulations for specific sections ← 113 | 114 → (scaffolding); text coherence and cohesion, advance organizers and logical connectors; register, readability and comprehensibility; systematic revision, peer feedback; grammar: subjunctive, active and passive voice; punctuation rules; and correcting spelling and punctuation errors.

For English, two types of writing courses were offered. One type followed an ‘English only’ approach, the other one, a contrastive German-English approach. In the latter type of course, students were not only taught, and had to complete assignments, in English but were also confronted with genres in German which they had to compare with their English counterparts and to translate into English. Both types of courses covered the following topics: brainstorming; specificities of academic writing in English, useful vocabulary and formulations (scaffolding); Fillmore’s scenes-and-frames semantics and its relevance for academic writing; finding a topic and specifying a research question; planning and writing argumentative texts with a focus on their macrostructure; how to quote and avoid plagiarism; paragraph structure (topic sentence, introduction, supporting details, conclusion/transitions); text coherence and cohesion with a focus on logical connectors, parallelism; introduction to different genres: reading and composing book reviews, instructions (‘how to’-texts) and popular-science texts; taking one’s audience into account; punctuation rules; refreshing grammatical knowledge: use of articles, prepositions, syntax, functional sentence perspective; editing. In addition, the contrastive German-English course also introduced students to Fillmore’s scenes-and-frames semantics and its relevance for translation; criteria for translation quality assessment; using monolingual and bilingual dictionaries; and translating book reviews.

The diagrams that follow indicate for all students whether they were students with English as a major or minor in their degree programme (“Eng”) or not (“non_Eng”). For those who had English as a major or minor in their degree programmes, they also indicate whether they attended a course that was taught in English only (“Eng_mono”) or a contrastive German-English course (“Eng_Ger”).

3.5   Data analysis

The texts produced at the beginning and at the end of the courses were subjected to in-depth analyses, which combine a detailed error analysis with a holistic assessment (cf. Weigle 2002). For the error analysis, the primarily (text-)linguistic error classification scheme in Table 1 was used. Earlier ← 114 | 115 → versions of this error classification scheme, which were slightly adapted for the present study, had been developed on the basis of linguistic errors that occurred in German popular-science articles produced by students of technical communication (cf. Göpferich 2002: 286 ff.) and translation errors made by translation students when translating from their L2 (English) into their L1 (German) (Göpferich 2010: 54 f.).

Table 1:   Error classification scheme

Error category    Description/Example
Formal errors
punctuationmissing or wrong punctuation mark; if both a comma at the beginning and at the end of an insertion are missing, this is counted as only one error; repeated comma errors are counted as repetition errors only if they have the same cause
spellingspelling mistake which is not an obvious typo
formattingline break where there should be none (the participants were not required to do any other formatting in the text); italics where there should be none and vice versa
Lexical errors
semanticuse of words and phrases which do not express the intended meaning either denotatively (semantic: denotation) or connotatively (semantic: connotation). The category “semantic: connotation” includes the use of a wrong register at word level.

use of expressions which do not exist; use of word forms from another language which cannot be expected to be understood in the target culture, etc. Note: This only refers to words and phrases, not to longer stretches of text that have been translated literally and are unidiomatic in the target language. See: “idiomaticity/genre”.

Note: Blendings and wrong illocutionary indicators are classified as separate categories. See also “collocation” and “modality/illocution”. ← 115 | 116 →
collocationwrong collocation (e.g., fast speed instead of high speed). This category only includes cases in which the meanings of the words used are appropriate but in which these words cannot be combined for other reasons, such as idiomaticity or convention.
blendingerror caused by melding together parts of linguistic units or constructions which enter working memory simultaneously
prepositionuse of a wrong preposition (see also “other grammar”)
modality/illocutionwrong illocutionary indicator, such as sollte (recommendation) instead of muss (instruction)
redundancysuperfluous repetition of meaning components, ideas, statements or words including tautologies (see also “text coherence”)
Grammatical errors
tenseuse of wrong tense
aspectuse of wrong aspect
case, number, agreementuse of wrong case or grammatical number, mostly after prepositions or in appositions; agreement error
moodwrong mood, e.g., in indirect speech
voiceactive voice instead of passive voice and vice versa; e.g. Die Truppenstärken erhöhten sich. (which is wrong because they cannot do that by themselves) instead of Die Truppenstärken wurden erhöht.
word formmorphologically wrong word form, such as adjectives instead of adverbs and vice versa; wrong formation of past tense forms.

Where in doubt whether “word form” or “semantic: denotation” should be used, “semantic: denotation” was used if the error could be assumed to be due to deficiencies in the lexicon, and “word form”, if the error was rather caused by a lack of grammatical competence. ← 116 | 117 →
syntaxsyntactic error; participle instead of infinitive and vice versa; missing participle in a construction where German just uses a prepositional phrase without participle; long prenominal modifiers in English (interference from German) which should be transformed into a participle construction after the noun; constructions which are hard to understand due to their length, long parentheses, etc.; utterances which are grammatically correct but which would only make sense if additional lexical elements were inserted, e.g., The few utterances could not be categorized as either one of the two models. Comment: For this utterance to make sense it would have to be reformulated into: The few utterances could not be categorized as belonging to either one of the two models.

lack of parallelism at the syntactical level
valencymissing actant; use of a lexical element that requires further specification; applicable also to nominalised verbs, e.g., Aufenthalt (from sich aufhalten in), where the indication of a place is required
specifier (article or determiner)use of a determiner, e.g., an article, where there should be none; use of a definite article where an indefinite article should be used; etc.
infinitive/participlegrammatically wrong use of an infinitive or participle construction (e.g., Das Wetter war zu schlecht, um schwimmen zu gehen.)
secondary subjectivizationuse of verbs expressing human actions in connection with non-human subjects (possible in English but rare in German); e.g., This book describes (correct) vs. Dieses Buch beschreibt (wrong).
other grammarother grammatical errors, such as the use of a prepositional phrase instead of a genitive and vice versa (e.g., von seinem Vater instead of seines Vaters) ← 117 | 118 →
Text-level (textlinguistic) errors
text coherencencoherent text segments, e.g., logically wrong connection of clauses and sentences by the use of semantically inappropriate conjunctions; use of wrong pronouns; sentence not related to its context; referent unclear; missing second part of correlative (two-part) conjunctions; repetition of a noun phrase where a pronoun would be sufficient
senseincomprehensible or nonsensical section longer than a phrase (otherwise it is counted as a semantic error), contradictory statements
implicitnesstoo much information left implicit, e.g., author does not express something to which a conjunction, etc. refers (e.g., There are three types of birch trees. Therefore, I will describe only one. Here, therefore refers to a sentence that was left implicit, i.e., I cannot cover them all.)
functional sentence perspective (FSP)wrong topic-comment structure (theme/rheme)
rhetoricloss of communicative emphasis or effect (e.g., replacing a poem by a mere description of its content); literal repetitions (see, however, “redundancy” and “text coherence”)
Other
idiomaticity/genreunidiomatic expression which does not lead to a change of meaning but may make the text hard to understand and betray that it is a translation in a negative sense; use of expressions which do not conform to genre conventions (e.g., Das Bild ist kein Zufallstreffer. instead of Das Bild ist kein Schnappschuss. and Anfangend mit Namen as a title.)

Note: If this refers to lexical units and not to longer stretches of text, “semantic: denotation” was used.
cultural specificitymissing adaptation to the target culture or missing cultural neutralisation ← 118 | 119 →

Every error identified in the texts was highlighted and annotated with a corresponding error tag (see two examples of annotated texts in Appendix B). To reduce subjectivity, each text was marked by three raters in a discursive process in which discrepancies among the raters were discussed and reconciled and the error classification scheme with its explanations specified to ensure consistent interpretation where necessary. In order to be able to compare the numbers of errors in the texts, which differed in length, the number of errors per 100 words (error ratio) was calculated for each text.

Since text quality cannot be assessed exhaustively on the basis of error ratios alone, each text was additionally subject to a holistic assessment in which its argumentative rigour (henceforth called stringency) was determined by the same three raters who performed the error analysis, again in a process of discursive consensual assessment. Stringency is considered a central quality criterion for argumentative texts. Stringency assessments focus on whether the line of argumentation in a text is logical, whether the text has a logical conclusion and whether the line of argumentation leads to this conclusion without deviations. For stringency, the texts were rated on a scale from 0 to 3 points. Three points were awarded for a high level of stringency without deviations and with a logical conclusion; 2 points, for a line of argumentation with few or minor deviations and a logical conclusion; 1 point, for a text with frequent or substantial deviations but a conclusion; and 0 points, for an unorganized text and/or a text without any clear conclusion. The two texts in Appendix B are examples of texts with minimal stringency (0 points) and high stringency (3 points), respectively.

In order to obtain a single overall text quality score for each text, the error ratios (errors per 100 words) that occurred in each rubric of the error classification scheme were converted into points, too, again on a rating scale from 0 to 3 for each rubric so that an overall score could be calculated by adding the scores in each of the rubrics. Three points were awarded for extremely low error ratios in the respective rubric, 0 points, for extremely high error ratios. Errors of the rubric “Other” occurred only extremely rarely, so this rubric could be ignored, leaving the four rubrics “formal correctness”, “lexical correctness”, “grammatical correctness” and “textlinguistic correctness”. The lattermost rubric represents the text-microstructural counterpart of the category “stringency”, which takes text-macrostructural aspects into account.

When converting the error ratios into points, potential outliers had to be excluded. This was achieved as follows: For the error ratios the students had in ← 119 | 120 → each of the rubrics indicated above, the medians and standard deviations were calculated. For each rubric, the sum of the median error ratio and the standard deviation (and not the highest error ratio that had occurred) was divided by four. The result indicates the length of each of the four stretches of equal length (quartiles) into which the range of error ratios from 0 to the sum of the median and the standard deviation can be subdivided. The error ratios in the lowest error ratio quartile were converted into 3 points, the error ratios in the second lowest error ratio quartile, into 2 points, etc. (see Fig. 1). In this manner, each text was assessed according to 5 criteria, for each of which it obtained 0 to 3 points. These could then be added to obtain a single quality score for each text.

img

Fig. 1:   Conversion of error ratios into points

In order to allow for a comparison of the results of the German and the English texts, the quartiles were calculated on the basis of the error ratios obtained in all German and English texts composed at the beginning of the semesters (n=100), i.e., the error quartiles were calculated on the basis of one median and one standard deviation for 100 texts (49 in German and 51 in English) in each of the four rubrics “formal errors”, “lexical errors”, “grammatical errors” and “text-level errors”. The significance level was set at 10% (p < 0.1).

4    Results

Fig. 2 provides an overview of the mean error ratios that occurred in each of the rubrics of the error classification scheme in Table 1 in the German (L1) and the English (L2) texts composed at the beginning (Pre) and at the end (Post) of the writing courses. It also provides insight into the types of errors that occurred most frequently in the two languages and thus into the topics that were to be covered in the respective writing courses. ← 120 | 121 →

img

← 121 | 122 →

Fig. 2:   Error ratios at the beginning (Pre) and at the end (Post) of German and English writing courses compared

A comparison of the mean total error ratios in each of the five categories “formal errors”, “lexical errors”, “grammatical errors”, “textlinguistic errors” and “other errors” at the beginning of the writing courses with the respective values at the end of the writing courses reveals that, in both languages, these error ratios decreased in all five categories. For the decrease in the English writing courses, a Wilcoxon test yielded significant differences at a 10% level (p < 0.1) for all error categories, for the decrease in the German writing courses, only for the error categories “lexical errors”, “text-level errors” and “other errors” (see Table 2).

Table 2:   Statistical significance of decrease in error ratios from beginning (Pre) to end (Post) of writing courses (Wilcoxon test, p < 0.1)

img

*: = significant at 10% level

Furthermore, the following correlations between error ratios in different categories were found (see Table 3). The table only lists those error category pairs between which a significant correlation was found in at least one of the two languages at at least one point of measurement.

Table 3:   Correlation between error ratios within languages

img

*: = significant at 10% level

← 122 | 123 →

A comparison between languages shows that the mean error ratios in the students’ L1 texts (German) were lower in all five error categories (“formal”, “lexical”, “grammatical”, “text-level” and “other”) at both the beginning and the end of the writing courses than the corresponding mean error ratios in the students’ L2 texts (English). Whereas the mean error ratios in the L1 and the L2 differed by less than 1.25 errors per 100 words in the rubrics “formal errors” and “other” and even by less than a quarter of an error in the rubric “text-level errors”, the differences between the L1 and L2 were more substantial in the error categories “lexical errors” and “grammatical errors”. On average, the English texts composed at the beginning of the semester contained 2.61 more lexical errors and 2.53 more grammatical errors per 100 words than the German texts composed at the beginning of the semester. The English texts composed at the end of the semester on average contained 2.49 more lexical errors and 2.14 more grammatical errors per 100 words than the German texts composed at the end of the semester. A Wilcoxon-Mann-Whitney test yielded significant differences at a 10% level (α = 0.1) between the L1 and the L2 in all error categories except for text-level errors at the beginning of the semester and between all error categories without exception at the end of the semester (see Table 4).

Table 4:   Statistical significance of the quality lag in the L2 as compared with the L1 (Wilcoxon-Mann-Whitney test)

img

*: = significant at 10% level

← 123 | 124 →

With regard to stringency, the English texts obtained lower values than their German counterparts at the end of the courses (mean stringency of the German texts: 0.96 out of 3 at the beginning, 1.92 at the end; mean stringency of the English texts: 0.97 at the beginning, 1.07 at the end). The improvement from the beginning to the end of the semester turned out to be more considerable in German (+0.96) than in English (+0.1).

The observation of a decrease of error ratios from the beginning to the end of the writing courses for both languages and the lower values in German (L1) as compared with English (L2) can also be made with regard to the error ratios in most of the individual sub-categories. Exceptions could only be found in some of the sub-categories that occurred very rarely and therefore will not be further discussed in this article.

The significant improvement of the students’ results also becomes salient in the line charts in Figures 3 and 4, which represent the students’ overall text quality scores (in points) at the beginning and at the end of the German and the English writing courses respectively. The numbers in parentheses indicate the students’ semester of study at the time of data collection. The students are arranged according to the total scores they obtained at the end of the semester with the best students appearing on the right and the poorest on the left.

img

Fig. 3:   Students’ writing competence at the beginning and at the end of the German (Ger) writing courses represented in points (sum of the points for formal, lexical, grammatical and textlinguistic correctness as well as stringency)

← 124 | 125 →

img

Fig. 4:   Students’ writing competence at the beginning and at the end of the English writing courses represented in points (sum of formal, lexical, grammatical and textlinguistic correctness as well as stringency)

Figures 3 and 4 show that most students improved (black lines above grey ones), whereas only few obtained poorer results at the end than at the beginning of writing courses (grey lines above black ones). A comparison between languages shows that the overall scores students achieved in their L2 were lower than the scores they obtained in their L1. Except for the fourth position from the right, the fourteen best positions in Fig. 4 are all occupied by students who were taught following the English-only (“Eng_mono”) approach.

Figures 5 to 8 visualize the students’ individual scores in each of the five rubrics in relation to each other. In each figure, the students are again arranged according to the total scores they obtained at the respective point of measurement with the best students on the right and the poorest on the left. ← 125 | 126 →

img

Fig. 5:   Students’ scores in points for L1 writing (German) in each of the five rubrics at the beginning of the semester

img

Fig. 6:   Students’ scores in points for L1 writing (German) in each of the five rubrics at the end of the semester

← 126 | 127 →

img

Fig. 7:   Students’ scores in points for L2 writing (English) in each of the five rubrics at the beginning of the semester

img

Fig. 8:   Students’ scores in points for L2 writing (English) in each of the five rubrics at the end of the semester

The flatter graphs for the L2 (English) texts show again that the students’ writing competence in their L2 lags behind their writing competence in their L1 (German). In the German courses, the weakest students appear to have benefited from the courses more than the strongest students. In the English courses such a tendency cannot be observed as clearly. For both German ← 127 | 128 → and English, the relationships between the scores in the individual rubrics seem to become more harmonious with increasing overall results, i.e., the higher the students’ overall scores become, the less the scores in the five individual rubrics differ from each other and the more likely it becomes that the students score in all five rubrics.

With regard to the development of the students’ scores in each of the five rubrics, the hypothesis was set up in Göpferich (2014) that, if students had not yet achieved the minimum score of at least one point in each of the five sub-categories, their participation in writing courses would first lead to an improvement of the lower-level competencies as reflected by “formal correctness”, “lexical correctness” and “grammatical correctness”, before the higher-level competencies “textlinguistic correctness” and “stringency”, which require more working memory capacity, continue to develop. Whereas the data obtained for L1 writing competence development seem to support this hypothesis, if formal correctness is ignored3, the data obtained for L2 writing contradict it. In English, a number of students (the 6 weakest students at the end of the semester) scored in the rubrics “text­linguistic correctness” and/or “stringency” without obtaining any points in the lower-level categories. In contrast, it is interesting to note, however, that Ger_Stud_12, who took a German writing course both in his fourth and in his fifth semester mainly improved his writing skills over these two semesters with regard to text-level competence and stringency (see Fig. 5, where he occurs as Ger_Stud_12a and Ger_Stud_12b).

5    Discussion

The error analyses conducted in the present study have revealed that, upon entering university (and even in later semesters), many students have not yet reached a level of language competence, neither in their L1 nor their L2, that has just to be expanded for the specific purposes of academic writing, but that they also need instruction to close gaps in their general linguistic and ← 128 | 129 → text competence. The error analyses presented in this article have revealed areas in which this is the case.

A comparison of the qualities of the texts produced by the students at the beginning of writing courses with those produced by these same students at the end of these courses has furthermore shown, for both the German and the English courses, that these courses have led to significant improvements in the students’ text production competence. The course syllabi thus seem to have been effective. What we cannot tell from the data analysed, however, is whether modifications in the course syllabi or other teaching methods would lead to even better results and whether the effects observed will be lasting ones.

The findings presented in Figures 7 and 8 might suggest that the “English only” teaching method could be more effective than the contrastive German-English approach. This, however, would be a premature conclusion for the following reason. In order to benefit from a contrastive approach, students need to develop interference resistance. Interference resistance only comes with increasing translation competence and goes hand in hand with the ability to overcome fixedness on a source text and to translate more creatively (cf. Bayer-Hohenwarter 2012). Translation novices have been found to have an inclination to translate word by word and thus to be prone to interference, whereas the translation units tackled by expert translators tend to be larger and their strategies to be more creative (see, e.g., Gerloff 1988: 54 ff.; Dragsted 2005; Englund Dimitrova 2005: 96 ff., 140 f., 231; 2006). Against the background of these findings, it can be hypothesized that the extent to which students benefit from a contrastive approach increases with the level of translation competence they have acquired. The students under scrutiny in the present study were all translation novices, for whom a monolingual approach turned out to be the more effective one. From this finding, however, it cannot be concluded that this would also be the case for students with an advanced level of translation competence. There might be a break-even point with regard to translation competence at which a contrastive approach of teaching writing skills will start to yield better results than a monolingual one. A monolingual approach indeed seems to be the approach to be preferred for translation novices. Further research involving students of various translation competence levels will be needed in order to delve deeper into this topic. ← 129 | 130 →

Apart from writing course effectiveness, the present study has also shed some light on the development of writing competence and, more specifically, on the development of individual writing sub-competencies, such as formal, lexical, grammatical, textlinguistic and compositional (stringency) competencies, as components of the dynamic system ‘writing competence’ in relation to each other. The development of writing competence and its sub-competencies becomes visible in Figures 5 to 8. In these four figures, the patterns at the right ends of the diagrams are more parallel (harmonious) than the patterns at the left ends of the diagrams. Both for the L1 (German) and for the L2 (English), the stretches of the x-axes which show this more parallel pattern are larger for the results obtained at the end of the courses than for the results obtained at the beginning. This suggests that a higher level of writing competence is reflected not only by higher overall scores but also by a more harmonious relationship between the scores in each of the five rubrics, which are assumed to reflect relevant sub-competencies of writing competence. This finding is in line with dynamic systems theory where the reaching of a higher developmental stage has been found to coincide with more stable or harmonious patterns, whereas initial phases of new competence development bursts show in a large variability in the set of variables which form the competence system (Thelen/Smith 1994: 97).

The striking differences in the patterns in Figures 5 to 8 suggest that it might be worthwhile to investigate whether the patterns that occur in the area charts can be correlated with competence levels such as the levels A1 to C2 (at least with regard to the skill ‘writing’) in the Common European Framework of Reference for Languages (CEFR). A similar endeavour of correlating less advanced learner text characteristics with competence levels of the CEFR (maximum B1) has already been undertaken by Verspoor, Schmid & Xu (2012). The competence levels of the students who, in the study presented in this article, scored 13 or more points might reflect the level C1 or C2. Since students are required to have achieved level B2 of the CEFR in order to be admitted to a BA programme in English language and literature at Justus Liebig University, the average scores of the students who have English as a major or minor should reflect at least this level. The fact that only students who study English obtained more than 8 points might suggest that the scoring system used for the present analysis is sensitive enough to allow for a differentiation between at least the competence levels of B1 upwards with B1 being ← 130 | 131 → the level of English that all German students are expected to have mastered when leaving high school and B2 representing the entrance level required for an English programme. Figures 7 and 8 then also show, however, that not all students admitted to English programmes fulfil this requirement in practice.

In line with McCutchen’s (1996) capacity theory of writing, the hypothesis can be formulated that a high stringency score in combination with a high score for textlinguistic correctness can only be obtained when the sub-competencies at the lower levels have been automatized to a certain extent. This hypothesis is based on the assumption that to achieve textlinguistic correctness and stringency, an author has to be able to consider an amount of co- and context that is rather complex compared with the amount of co- and context that has to be considered in order to achieve correctness in the rubrics which represent less complex text-production sub-competencies. Taking into consideration this larger context requires considerable working memory capacity. This working memory capacity can again be assumed to be available only for authors who have acquired a certain amount of confidence and routine with regard to the less complex sub-competencies. Whereas the findings for L1 writing corroborate this hypothesis, the scores that students obtained at the end of their L2 writing courses (see Figure 8) indicate that even students whose lower-level sub-competencies were still underdeveloped managed to obtain relatively high scores for textlinguistic correctness and stringency. An explanation that reconciles this finding with the capacity theory of writing is that macrostructures of argumentative texts as well as the development of a line of argumentation were focused on in the writing courses that students had attended. This may have led to routines for the organization of such texts which then allowed the students to compose stringent and coherent texts without the high cognitive load that this would involve when authors were confronted with the composition of texts for which they did not have fixed genre patterns in their long-term memories to fall back on. If this explanation is correct, this will also mean, however, that the competences students have acquired specifically for writing argumentative texts may not be transferrable to composition tasks that involve different genres. Expert writers are flexible in adapting their higher-order sub-competencies, such as textlinguistic competency and stringency, to new genres, whereas students who have just acquired rou ← 131 | 132 → tine with regard to a very limited repertoire of textual patterns are not (cf. Steinhoff 2007: 330)4.

The fact that the lexical and grammatical error ratios were found to correlate at both the beginning and the end of the English courses and at the beginning of the German writing courses corroborates the finding from DST applied to second-language acquisition research that lexicon and grammatical repertoire are connected growers (de Bot/Lowie/Verspoor 2007: 19). No explanation can be provided for the fact that their correlation at the end of the German writing courses was non-significant.

The present study has also provided insight into the interrelationship between L1 and L2 writing skills. The finding that the error ratios at the lexical and grammatical levels were significantly higher in L2 composition than in L1 composition corroborates earlier findings (see Section 2.2). Against the background of the capacity theory of writing, the higher cognitive load involved in decision making at the lexical and grammatical levels in the L2 is a possible explanation for the lower stringency values in the L2 as compared with the L1.

6    Conclusion

The methods of data collection and analysis presented in this article have allowed us to take stock of the linguistic areas in which students commit errors when composing texts, especially argumentative ones, in their L1 and their L2. From the findings presented, conclusions can be drawn as to the topics that should be covered in compensatory writing courses at university entrance level. We do not claim, however, that the topics mentioned in this article are exhaustive.

The methods employed in the present study have also allowed us to ‘measure’ the impact that one semester of writing instruction has on the development of students’ writing skills in their L1 and L2. The study has also allowed us to glimpse into the relationships between writing sub-com ← 132 | 133 → petencies and their development in relation to each other over time as well as into the interdependence between writing skills in students’ L1 and L2, where findings of earlier studies could be replicated.

To gain a more complete picture of different stages of writing competence development, the corpus of writing data analysed for the present study needs to be extended to also include data of highly proficient writers on the one hand and writers with a lower competence level than the students under scrutiny in this article on the other hand. In order to investigate the influence of a contrastive or translation-oriented approach on the development of writing competence, the corpus should also comprise data from participants with different levels of translation competence. Ideally, data for both languages should be collected from the same writers composing in each of the two languages, whereas in the present study, the L1 and L2 writers were not identical.

A comparison of area graphs from populations with different levels of writing and translation competence could eventually allow us to address the question whether patterns in the area graphs can be correlated with competence levels, such as the ones of the CEFR. Such comparisons could then also give us more profound insights into the development of individual sub-components of writing competence in relation to each other over time.

Acknowledgements

For the research presented in this article funding is acknowledged from the German Federal Ministry of Education and Research (project code 01PL12035). Furthermore, we would like to thank Mareike Eckert, Carmen Neis and Bridgit Nelezen for their support in annotating the texts. Special thanks go to Jens Blank for sharing his expertise in statistics with us.

References

Bayer-Hohenwarter, Gerrit, 2012: Translatorische Kreativität: Definition – Messung – Entwicklung. (Translationswissenschaft 8). Tübingen: Narr.

Beaufort, Anne, 2007: College Writing and Beyond: A New Framework for University Writing Instruction. Logan (UT): Utah State Univ. Press.

de Bot, Kees / Lowie, Wander / Verspoor, Marjolijn, 2007: “A Dynamic Systems Theory approach to second language acquisition.” Bilingualism: Language and Cognition 10(1), 7–21. ← 133 | 134 →

Bronfenbrenner, Urie / Morris, Pamela A., 2006: “The bioecological model of human development.” In: Lerner, Richard M. / Damon, William (eds): Handbook of Child Psychology. Vol. 1. New York (NY): Wiley, 793–882.

Cumming, Alister, 2001: “Learning to write in a second language: Two decades of research.” International Journal of English Studies 1(2), 1–23.

Devine, Joanne / Railey, Kevin / Bischoff, Philip, 1993: “The implications of cognitive models in L1 and L2 writing.” Journal of Second Language Writing 2(3), 203–225.

Dragsted, Barbara, 2005: “Segmentation in translation: Differences across levels of expertise and difficulty.” Target. International Journal of Translation Studies 17(1), 49–70.

Driscoll, Dana Lynn / Wells, Jennifer, 2012: “Beyond knowledge and skills: Writing transfer and the role of student dispositions.” Composition Forum 26. http://compositionforum.com/issue/26/beyond-knowledge-skills.php (last accessed: 09 June, 2015).

Englund Dimitrova, Birgitta, 2005: Expertise and Explicitation in the Translation Process. (Benjamins Translation Library 64). Amsterdam, Philadelphia: John Benjamins.

Englund Dimitrova, Birgitta, 2006: “Segmentation of the writing process in translation: Experts vs. novices.” In: Sullivan, Kirk P. H. / Lindgren, Eva (eds): Computer Keystroke Logging and Writing: Methods and Applications. Oxford: Elsevier, 89–201.

Gerloff, Pamela, 1988: From French to English: A look at the translation process in students, bilinguals, and professional translators. (Unpublished PhD thesis) Cambridge (MA): Harvard University.

Göpferich, Susanne, 2002: Textproduktion im Zeitalter der Globalisierung: Entwicklung einer Didaktik des Wissenstransfers. (Studien zur Translation 15). Tübingen: Stauffenburg.

Göpferich, Susanne, 2010: “The translation of instructive texts from a cognitive perspective: novices and professionals compared.” In: Göpferich, Susanne / Alves, Fabio / Mees, Inger M. (eds): New Approaches in Translation Process Research. (Copenhagen Studies in Language 39). Copenhagen: Samfundslitteratur, 5–52.

Göpferich, Susanne, 2013: “Translation competence: Explaining development and stagnation from a dynamic systems perspective.” In: Eh ← 134 | 135 → rensberger-Dow, Maureen / Göpferich, Susanne / O’Brien, Sharon (eds): Interdisciplinarity in Translation and Interpreting Process Research. Special Issue of Target – International Journal of Translation Studies 25(1), 61–76.

Göpferich, Susanne, 2014: “Methods of measuring students’ text production competence and its development in writing courses.” In: Engberg, Jan / Heine, Carmen / Knorr, Dagmar (eds): Methods in Writing Process Research. Frankfurt/M.: Lang, 198–209.

Göpferich, Susanne, 2015: Text Competence and Academic Multiliteracy: From Text Linguistics to Literacy Development. (Europäische Studien zur Textlinguistik 16). Tübingen: Narr.

Jechle, Thomas, 1992: Kommunikatives Schreiben. Prozess und Entwicklung aus der Sicht kognitiver Schreibforschung. Tübingen: Narr.

Kellogg, Ronald T., 2008: “Training writing skills: A cognitive developmental perspective.” Journal of Writing Research 1(1), 1–26.

McCutchen, Deborah, 1996: “A capacity theory of writing: Working memory in composition.” Educational Psychology Review 8(3), 299–325.

Nitta, Ryo / Baba, Kyoko, 2014: “Task repetition and L2 writing development: A longitudinal study from a dynamic systems perspective.” In: Byrnes, Heidi / Manchón, Rosa (eds): Task-Based Language Learning – Insights from and for L2 Writing. Amsterdam: John Benjamins, 107–136.

Ortner, Hanspeter, 1995: „Die Sprache als Produktivkraft. Das (epistemisch-heuristische) Schreiben aus der Sicht der Piagetschen Kognitionspsychologie.“ In: Baurmann, Jürgen / Weingarten, Rüdiger (eds): Schreiben. Prozesse, Prozeduren und Produkte. Opladen: Westdeutscher Verlag.

Piaget, Jean, 1972: Theorien und Methoden der modernen Erziehung. Wien, München, Zürich: Molden.

Pohl, Thorsten, 2007: Studien zur Ontogenese des wissenschaftlichen Schreibens. Tübingen: Niemeyer.

Portmann-Tselikas, Paul R. / Schmölzer-Eibinger, Sabine, 2008: „Textkompetenz.“ Fremdsprache Deutsch (Themenheft Textkompetenz) 39, 5–16.

Roca de Larios, Julio / Manchon, Rosa M. / Murphy, Liz, 2006: “Generating text in native and foreign language writing: A temporal analysis of problem-solving formulation processes.” The Modern Language Journal 90, 100–114. ← 135 | 136 →

Schoonen, Rob / an Gelderen, Amos / de Glopper, Kees / Hulstijn, Jan / Simis, Annegien / Snellings, Patrick / Stevenson, Marie, 2003: “First language and second language writing: The role of linguistic knowledge, speed of processing, and metacognitive knowledge.” Language Learning 53(1), 165–202.

Silva, Tony, 1992: “L1 vs. L2 writing: ESL graduate students’ perceptions.” TESL Canada Journal 10(1), 27–47.

Slomp, David H., 2012: “Challenges in assessing the development of writing ability: Theories, Constructs and Methods.” Assessing Writing 17(2), 81–91.

Steinhoff, Torsten, 2007: Wissenschaftliche Textkompetenz. Sprachgebrauch und Schreibentwicklung in wissenschaftlichen Texten von Studenten und Experten. Tübingen: Niemeyer.

Thelen, Ester / Smith, Linda B., 1994: A Dynamic Systems Approach to the Development of Cognition and Action. Cambridge (Mass.), London: MIT Press.

Ulmi, Marianne / Bürki, Gisela / Verhein, Annette / Marti, Madeleine, 2014: Textdiagnose und Schreibberatung: Fach- und Qualifizierungsarbeiten begleiten. Opladen, Toronto: Barbara Budrich.

Van Gelder, Tim, 1998: “The dynamical hypothesis in cognitive science.” Behavioral and Brain Sciences 21, 615–656.

Verspoor, Marjolijn / de Bot, Kees / Lowie, Wander, 2004: “Dynamic Systems Theory and Variation: a Case Study in L2 Writing.” In: Aertsen, Hendrik (ed.): Words in their Places. Amsterdam: VU, 407–421.

Verspoor, Marjolijn / Lowie, Wander / van Dijk, Marijn, 2008: “Variability in Second Language Development from a Dynamic Systems Perspective.” The Modern Language Journal 92(2), 214–231.

Verspoor, Marjolijn / Schmid, Monika S. / Xu, Xiaoyan, 2012: “A dynamic usage based perspective on L2 writing.” Journal of Second Language Writing 21(3), 239–263.

Wardle, Elizabeth / Roozen, Kevin, 2012: “Addressing the complexity of writing development: Toward an ecological model of assessment.” Assessing Writing 17(2), 106–119.

Weigle, Sara Cushing, 2002: Assessing Writing. Cambridge: Cambridge Univ. Press. ← 136 | 137 →

Whalen, Karen / Menard, Nathan, 1995: “L1 and L2 writers’ strategic and linguistic knowledge: A model of multiple-level discourse processing.” Language Learning 45(3), 381–418.

Appendix A

Assignment:

Imagine you are writing for a student magazine that covers current controversial issues. For this magazine, compose a text (approx. 250 – max. 350 words in length) about one of the following topics. Your text should be comprehensible (understandable) to your fellow students, i.e., those who are in the same semester at university as you are and have little background knowledge about the topic.

1.    Free music for everyone! The Pirate Party in Germany would like to make all music legally available for free download on the internet. Should the copyright restrictions on music and other forms of intellectual property be removed?

2.    In an attempt to cut down on pollution in big cities, Germany has created so-called “Umweltzonen” (green zones) in cities such as Berlin, Frankfurt and Hamburg, where the driving of automobiles which do not meet certain environmental criteria is forbidden. Some even recommend banning the driving of cars altogether in big cities. Should there be a general ban on cars in big cities?

3.    After having been briefly introduced in 2007 and 2008, tuition fees for university students in Hesse have again been abolished. Should tuition fees be reinstated in the state of Hesse, or should universities charge no tuition to attend them? ← 137 | 138 →

Appendix B

Example of an annotated text with minimal stringency (0 points)

The utopia of getting to work in time

Who does not know the decision <semantic: denotation> between taking the bus or <semantic: denotation> driving with the <specifier> own car? There are enough aspects <semantic: denotation> speaking <syntax> for each of it <case, number, agreement>. In Germany, experts want to make this decision easier for travelers: they simply want to ban driving with <preposition> cars in big cities altogether. A brief overview of the advantages and disadvantages of prohibiting cars in German cities <syntax>. <formatting> 7 am. <spelling> <punctuation> and the traffic jam near Frankfurt seems endless. 30 minutes left until work starts. The same every day <genre/idiomaticity>: thousands of workers driving into the city every morning. The same number driving outside <semantic: denotation> every afternoon. The nerverending <spelling> traffic <collocation> is not just bad for the nerves of the poor drivers, it is even worse for the environment around <preposition> Frankfurt, Berlin, <syntax> Hamburg... <formatting> So why not taking <word form> the bus or <specifier> train? Especially with trains <punctuation> there is no risk of getting stuck in the rush hour. Big cities as the above <syntax> can be reached from nearly everywhere <punctuation> and the people who have to go there <text coherence> for work will <modality/illocution> not live too many kilometers away. Another plus: the infrastructure in Germany is highly developed and gets improved regularly. Waiting two hours for the next bus was a long time ago <idiomaticity/genre>. You pay for it <text coherence> and save lots of money you would <syntax> have <tense> to spend for parking lots <semantic: denotation> or tickets because, in a hurry, you passed <semantic: denotation> a red light. Avoiding traffic jams, <punctuation> help <word form> the environment and <punctuation> by <preposition> doing so <infinitive/participle>, save <repetition word form> a lot of money-<formatting> there aren’t any disadvantages of banning all cars <implicitness>, are there? <syntax> Of course this <text coherence> can <modality/illocution> just be an <specifier> utopia. <repetition formatting> Taking the bus or the train is a good thing to do <punctuation> but in times of decreasing <semantic: denotation> prices every year <syntax>, it became ← 138 | 139 → <tense> a luxory <spelling> and impossible for people with low wage <word form>. Workers who cannot afford the 30 Euros for the train from Gießen to Frankfurt every day will be excluded <semantic: denotation> from public transport completely. Furthermore, the train or the bus leaves. <punctuation> With <semantic: denotation> you <syntax> in it or not. Your car is always waiting <aspect> for you. A great plus for all vehicles who will bring you to work late just because of your own fault. <sense> Flexibility, independence and always on time <syntax> – advantages no German worker does not want to have. If there will not be <tense> a change in mind <semantic: denotation><formatting>- and in the price-policy-<repetition formatting> banning cars in big cities will stay what it is today: wishful thinking. (410 words)

Example of an annotated text with maximum stringency (3 points)

Warum das Autofahren in Großstädten generell untersagt werden sollte – ein Betrag <spelling> von [XY], ein umweltinteressierter Student <case, number, agreement> der JLU Gießen

In Freiburg, Deutschlands selbstproklamierter <word form> „GreenCity“ <punctuation> hat man nun schon seit einigen Jahren Erfahrungen mit einem nahezu vollständigen Verkehrsverbot in der Innenstadt sammeln können. Statt auf den PKW greifen Freiburger nun wie selbstverständlich auf das sehr gute innerstädtische ÖPNV Netz <spelling> zurück, welches durch eine Wiedereinführung des Tram-Netzes enorm an Attraktivität gewonnen hat. Dies erlaubt ein stressfreieres Vorankommen in der Innenstadt und trägt zur großen Akzeptanz des Verbots in der Bevölkerung bei. Einer Umfrage des Freiburger Geographischen Instituts zufolge, <punctuation> sind <case, number, agreement> die Mehrheit der Bürger, <punctuation> sehr zufrieden mit dem Verbot. Den weiteren Gründen für die breite Akzeptanz des Verkehrsverbots möchte ich im Folgenden nachgehen und aufzeigen, warum das Freiburger Modell auch in anderen deutschen Städten umgesetzt werden sollte.

Eine Auswertung der Umfrage der freiburger <spelling> Geographen ergab, dass ein wichtiger Grund für die breite Akzeptanz nicht etwa dem Ursprünglichen <spelling> Ziel, der Verbesserung der Atemluft, zugeschrieben werden muss, sondern dass Freiburgs Einzelhandel nun viel problem ← 139 | 140 → loser auch fußläufig erreicht werden kann. <punctuation> Ein positiver Umstand, der von allen Befragten hervorgehoben wurde. Nicht nur für den innerstädtischen Einzelhandel, auch für die Freiburger Gastronomie und Hotellerie hat sich die Abschaffung des innerstädtischen Verkehrs als wahrer Geldsegen erwiesen. <formatting> Freiburg, die Stadt der 1000 Bächlein, liegt in der Nähe des Drei-Länderecks <spelling> und zog schon immer Wochenendtouristen, vor Allem <spelling> aus der Schweiz, an. Seitdem sich die Stadt, Dank <spelling> des Verkehrsverbots, als „Shopping-Metropole“ des Südwestens fest etabliert hat, verzeichnet Sie <spelling> nun auch unter der Woche Gäste aus allen Nachbarländern. Vor Allem <repetition spelling> die Franzosen schätzen die ruhige Innenstadt mit den zahlreichen Möglichkeiten zum Flanieren und Einkehren und fallen auch unter der Woche regelrecht in der schönen Zähringerstadt ein. Die Freiburger selbst bewerten das neue Interesse an ihrer Stadt im Ausland als sehr positiv und sind stolz auf ihre grüne Stadt mit dem neuen internationalen Flair.

Das Beispiel Freiburg zeigt, <punctuation-> nach Meinung des Autors, <-punctuation> anschaulich, wie eine Regelung, welche ursprünglich der Verbesserung der Umwelt <collocation> dienen sollte, auch viele andere positive Effekte, vor Allem <repetition spelling> für den innerstädtischen Einzelhandel, haben kann. Insofern kann eine flächendeckende Einführung von Autoverboten in allen deutschen Innenstädten nur begrüßt und aktiv vorangetrieben werden. Eine wichtige Voraussetzung für das Gelingen und für die breite Akzeptanz dafür ist jedoch ein sehr gutes ÖPNV-Angebot in der Innenstadt und, damit einhergehend, die Schaffung von ausreichend Pendlerparkplätzen am Stadtrand.

Um dies zu erreichen, sollten zukünftige autofreie Städte als Vorbereitende <spelling> Maßnahme eine Wiedereinführung der Trambahnen überlegen <semantic: denotation>, welche wegen ihres sauberen und kosteneffizienten Elektroantriebs in Fachkreisen als idealer „grüner“ Autoersatz in Großstädten angesehen werden. Außerdem ist der Bau einer Tram-Infrastruktur kosteneffizienter als der Neubau eines U-Bahnnetzes, der besseren Alternative für finanzkräftigere Großstädte. (443 words) ← 140 | 141 →


1       The term textual routines/procedures (“wissenschaftliche Textprozeduren”) refers to cognitive acts or moves, which may be closely connected with specific formulations (cf. Steinhoff 2007: 431).

2       The outline of DST presented here is based on an earlier version in Göpferich (2013) introducing DST in the context of translation competence development.

3       This can be considered legitimate for two reasons: First, although punctuation rules were covered in the courses, they were not focused on. Second, students do not attribute much importance to formal correctness and therefore do not invest much cognitive capacity into this aspect of text quality, which, as a consequence, will then still be available for higher-level decisions.

4       With reference to Jechle (1992: 55), Steinhoff (2007: 330; our translation) states: “From a social-communicative perspective, highly developed writing skills cannot be assumed before writers are able to adapt to their readerships in a flexible manner, i.e., before they are able to appropriately address their readerships in various texts or genres.”