MAESTRO: Guiding Students to Skillful

Performance of the Writing Process ⁽¹⁾

Dr. Kurt Rowley

Command Technologies, Inc.

USAF Armstrong Laboratory / HRTI

Brooks AFB, Texas USA

Melinda Crevoisier

Command Technologies, Inc.

USAF Armstrong Laboratory / HRTI

Brooks AFB, Texas USA

Abstract: MAESTRO is an adaptive writing process tutor for high school level English instruction. The principal factors influencing the design of MAESTRO were research results from use of the R-WISE (Reading and Writing in a Supportive Environment) writing tutor, field input from high school English teachers who had used R-WISE with their classes, cognitive research into the writing process, and the cognitive apprenticeship instructional strategy. This paper summarizes each of these factors and describes how they influenced the design of MAESTRO. A brief discussion of the software architecture is also included. An ongoing study is measuring the instructional efficacy of MAESTRO.

The use of computers to teach fundamental skills associated with the production of written text has been the focus of numerous studies in educational technology and applied cognitive psychology over the past 20 years [Gregg & Steinberg, 1980; Britton & Glynn, 1989]. One common assumption in these and related explorations of the use of computers to teach and support the learning of writing skills is the belief that the computer, as a character-based information processing media, should be a useful partner in the teaching and learning of writing skills. In most of these approaches to supporting the development of writing skills with the computer, the general efficacy of the use of computer-based tools to improve student writing has been validated. For example, Zellermayer and his colleagues tested a software writing assistant designed to provide students with writing strategy ideas and facilitate metacognition about writing, two important writing process skills. They found evidence that by using this facilitative software students were able to internalize the guidance and produce improved writing products [Zellermayer, Salomon, Globerson & Givon, 1991]. The MAESTRO research program is an attempt to design effective facilitative software that guides students to skillful performance of the entire writing process. The MAESTRO research program has addressed this challenge by designing adaptive cognitive computer technologies to meet the requirements of classroom instruction. In order to accomplish this, MAESTRO's design has been informed by four primary factors: previous findings in the R-WISE research program, field input from high school English teachers who used R-WISE with their classes, cognitive research into the writing process, and the cognitive apprenticeship instructional strategy.

The first factor influencing the design of MAESTRO was previous research conducted through the U.S. Air Force's Fundamental Skills Training (FST) Program. FST is a multi-year research effort to transfer advanced, adaptive training technology capabilities from Air Force technical training research, to public education. The R-WISE software was designed under the FST Program as an adaptive, supportive learning environment for strengthening the critical thinking skills associated with several writing tasks. R-WISE utilized instruction that was generated on a just-in-time basis, along with an adaptive help system, in order to meet specific needs of individual students. R-WISE was tested and evaluated at 14 high schools in 5 different U.S. States over the period 1992-1996.

The results of research into the effectiveness of R-WISE over the four years of the research program were that students using R-WISE outperformed control-group students on overall measures of writing quality and analytical reasoning skills, showing performance improvement of between one and two letter-grades [Carlson & Crevoisier, 1994; Carlson & Miller, 1996; Rowley & Miller, in press; Rowley, Miller & Carlson, in press]. In order to better understand how R-WISE accomplished this performance improvement, to identify those design features most contributory to the success of R-WISE, and to inform the design of a second generation writing tutor (MAESTRO), input from English teachers who had used R-WISE with their classes was collected in the Fall of 1995.

The second major factor influencing the design of MAESTRO was field input from teachers who had used R-WISE with their classes. The input included interviews with 25 teachers and several computer lab supervisors. These interviews provided information about the perceived effects of various components of R-WISE on student writing abilities. The interviews also produced information regarding system capabilities considered by the teachers to be important in the design of a follow-on writing tutor for use in a classroom environment.

Based on the experience of these teachers using R-WISE, the following system capabilities were identified as important for the design of MAESTRO: (1) provide individualized coaching by automatically adapting instruction to student performance and student learning styles, (2) incorporate motivational strategies into the interface designs including the use of technologies capable of raising interest and sustaining the motivation of students to write, (3) support the teachers' integration of the tutor with their classes by allowing them to easily incorporate their classroom curriculum into the tutor, (4) address the overall writing process as well as individual writing sub-process skills, and (5) use a single, consistent interface that both tutors and supports the students, helping them improve their writing process while they complete writing assignments. Each of these factors as well as many minor points raised by the teachers, was addressed in the design of MAESTRO. The new writing tutor was designed to provide individualized learner guidance, utilize a motivational strategy throughout, and allow for a high degree of teacher control over the tutor's operation. The focus of the teachers on the entire writing process led to a review of cognitive research on the writing process. The name chosen for the new tutor, MAESTRO, reflects the notion that the tutor teaches by guiding students to skillful performance of the tasks associated with the writing process.

A third important factor in the design of MAESTRO, consideration of cognitive research into the writing process, is based on the teachers' observation that the tutor should teach the entire writing process. Smith and Lansman point out that writing tools are needed that are designed to "...closely match and augment the inherent cognitive processes human beings use to perform the complex, multifaceted task of writing" [Smith & Lansman, 1989, p.18]. The types of tools that Smith and Lansman describe would be designed around the nature of the writing process. In order to support the entire writing process, Kellogg points out that an automated environment should support the production of ideas, support the heavy cognitive load of learning the writing process, and support and sustain the interest of the student [Kellogg, 1989]. Based on years of cognitive research into the writing process, Bereiter and Scardemalia recommend a similar level of support of the writing process, noting that "The use of procedural facilitation--simplified routines and external supports--can help students through the initial stages of acquiring more complex executive processes..." [Bereiter & Scardemalia, 1987, p. 363]. In order to provide procedural facilitation for the student, it is important to have a concrete concept of the nature of the expert writing process. This expert writing process has been identified in some detail through a now classic protocol analysis of an expert writer [Flower & Hayes, 1980]. The major components of these writing tasks include goal-setting, generating ideas, developing a writing plan, translating ideas into text, and revising the text with regard to the original writing goals.

The cognitive view of the writing process provides an overall instructional objective for MAESTRO. All instruction, activities, and supporting advice in the tutor are focused on shaping the student's writing process toward an expert writing process. The work environment for MAESTRO was designed to reflect and directly support the development of the writing process inherent in the executive skills of a typical expert writer. Using MAESTRO, the student can navigate through, and perform work in 22 workspaces that simulate tasks performed by the expert writer, in order to develop a cognitive writing process typical of an expert writer [see Figs 1 & 2].

FIGURE 1: Typical MAESTRO Workspace - Notes

The MAESTRO student interface helps the student visualize the overall writing process skills involved in expert-level writing. MAESTRO monitors the student's use of the workspaces in order to determine the level of mastery of the writing process achieved by the student, and provides relevant coaching and advice in order to help the student learn and practice expert-level writing process skills.

Using a Cognitive Apprenticeship Instructional Strategy

The fourth major factor in the design of MAESTRO is an instructional strategy designed to teach cognitive skills associated with the writing process. Field input suggested that the tutor be able to help the students learn the writing process while working on regular assignments. This led to the consideration and selection of a cognitive apprenticeship instructional strategy for the tutor. MAESTRO was designed to draw on a cognitive apprenticeship instructional strategy in order to help the student through the challenge of developing writing process skills in the context of regular writing work. The strategies associated with a cognitive apprenticeship typically follow patterns identified initially by Alan Collins and his associates [Collins, 1988; Brown, Collins & Duguid, 1989]. These strategies include teaching the tactics and heuristics necessary for proper task accomplishment, sequencing instruction from simple to complex, situating the learning and practice in the context in which performance will be required, modeling proper behavior or cognition, and gradually fading the coaching or 'scaffolding' as the student gains competence. MAESTRO's instructional strategy follows this pattern.

FIGURE 2: Typical MAESTRO Workspace - Drafting

The initial tactics and heuristics associated with expert performance of the writing process are taught by MAESTRO through a series of instructional modules selected to be consistent with student interests. The instructional modules must be mastered by the student prior to the use of any workspaces. Once the student has learned the initial tactics of expert performance, tasks are sequenced from simple to complex. This is accomplished by the inference engine that drives MAESTRO as it selects appropriate instructional modules for the student, appropriate advice statements and workspace access, based on the progress the student is making toward the acquisition of expert writing process skills [see Figs. 3 & 4].

FIGURE 3: MAESTRO Instructional Module

FIGURE 4: MAESTRO Advice Statement

The assignments include real-life writing tasks, and use multimedia video segments, graphical images and readings that depict interesting and probable writing situations for the student. While the student works on these situated assignments, proper writing behavior is modeled in examples that accompany the instructional modules and workspaces, as well as through the menu system and coaching advice statements that help to shape the writing process of the student toward an expert writing process. As the student adopts expert strategies, the coaching and scaffolding fade and the student is given open workspaces without unsolicited advice.

Software Architecture: The Design of MAESTRO

MAESTRO was designed and developed as a writing process tutor based on the design factors described above. All writing in MAESTRO is accomplished through use of a series of student-controlled writing-process workspaces. During use of the writing-process workspaces, MAESTRO provides instruction on the writing process, and a series of assignments to be performed in the workspaces. As the student works on the assignments in the workspaces the tutor monitors the emerging writing process of the student and provides coaching and advice statements.

MAESTRO was written in Multimedia ToolBook 4.0 (Asymetrix) using the Paradox database (Borland). The software was designed using principles of object-oriented programming and an open-ended architecture. The object-oriented design makes the system generic in the sense that the controlling module does not reference specific workspaces by name, but accesses instructional resources based exclusively on rules. The rules allow the tutor to produce individualized instruction, by comparing the student operation of the workspaces with a model of the expert writing process. Because of the object-oriented design of the software, each module within the tutor operates completely independently from the others, sharing only high-level information related to the rules.

Because of the object-oriented design of the software, each module within the tutor operates completely independently from the others, sharing only high-level information. This design greatly simplifies software maintenance and enhancement [see Fig. 5]. These independent modules are: (1) controlling tutoring system engine; (2) expert model (writing process rules); (3) instructional model (instructional strategy rules); (4) student model (historical information, learning style information, performance mastery, student writing products); (5) instructional resources including writing process workspaces, adaptive advice, tailored instructional modules, and writing assignments; (6) the student interface; and (7) a teacher interface.

FIGURE 5: MAESTRO System Architecture

1. Controlling Tutoring System Engine

The session is monitored by a rule-based tutoring system to ensure that the instruction, assignments, and advice are appropriate for each individual student. The tutor manages the writing environment by (1) enabling and disabling portions of the writing process workspaces; (2) providing the student with appropriate instructional modules that train the student in the use of those workspaces; (3) monitoring the completion of the assignments; (4) providing advice statements when needed or requested; and (5) continuously updating the student models.

2. Expert Model

This consists of the expert writing process rules that govern whether a particular part of the process has been mastered. Each time a student satisfactorily completes a portion of the assignment, the historical student model is analyzed to determine which workspace should be enabled and at what level the student should work.

3. Instructional Model

As the student attempts to move to another level within a writing assignment, the system analyzes the student models, the assignment, the expert model, and the student's writing products to determine whether the student is progressing sufficiently as to be allowed to advance. The analysis uses interface management rules, learner guidance rules and writing product assessment rules. If the student is not allowed to continue, a piece of advice is chosen based on the results of the above analysis, and presented to the student.

4. Student Model

Three student models are maintained: (1) an historical model, which is a performance history of student actions within a particular assignment; (2) a record of Tailored Instructional Modules that the user has either attempted or mastered (including the scores from the skill mastery assessments); and (3) the learning style model, which includes information on the extent to which the user is a verbal, visual or kinesthetic learner, whether she is classified as a Knowledge Teller or Knowledge Transformer, and what type of motivation the user prefers (e.g., Music, Sports, Academic, Art, Hobbies).

5. Instructional Resources

Writing Process Workspaces

The 22 cognitive writing process workspaces directly simulate the stages of the expert writing process. They include the following areas:

• Goals. Goals screens cover the spectrum of goal-setting, from identifying topic, reader, and style, to reflecting on writer biases and experience, and analyzing thesis ideas.
• Ideas. Ideas screens include a sequence of visualizations of the idea-generation process of an expert writer. This includes a questioning technique to explore a topic (cube), note-taking (notes), organizing ideas into categories (cluster), and organizing categories into an outline format (outline).
• Plan. The Plan screens assist with the development of a formal writing plan based on information generated in the Ideas screens. This includes several split-screens in which the student copies ideas from the left half of the screen to a structured writing plan diagram on the right half.
• Draft. Drafting screens assist in the generation of sentences and paragraphs, based on the writing plan diagram. The student refers to the writing plan diagram on the left side of the screen, then translates those ideas into text by generating sentences and paragraphs on the right-hand side of the screen.
• Revise. Revision screens help the student select appropriate editing questions for their draft based on their writing goals, and then apply those questions in the final editing and review of their work.
• Publish. This is an open workspace for formatting, highlighting, proof-reading, spell-checking, and distributing a final writing product, including the ability to distribute student essays to the teacher or peers for review, to an acquaintance via email, or to post the writing to a world-wide web page.

Adaptive Advice

There are over 400 advice statements of three types: Writing Process, Level of Completion and TIM Summary. Both solicited and unsolicited advice is presented to the student.

Tailored Instructional Modules

The controlling tutoring system engine selects the most appropriate TIM to use for each stage of the writing process, based on individual student learning characteristics, student performance, student interests, and the demands of the current writing assignment. Several TIMs are available for each stage of the writing process, reflecting multiple approaches to teaching, multiple levels of student competence, and a variety of writing styles. Each TIM includes screens that present information using text and graphics, as well as assessment and practice screens. The unique way in which writing instruction is delivered guarantees that it is individualized due to the design of the TIMs, which are generated in real time based on the student's present needs, with each "page" and each item on each page being chosen independently.

Writing Assignments

The assignments include text, graphics and video segments, and are designed to represent writing situations that the students could face in real life. Students must submit writing products created in MAESTRO to resolve the needs expressed in the writing assignments. Multimedia video, graphics, readings and Internet search features are all available to assist the student in understanding and analyzing the topic for each assignment.

6. Student Interface

The workspaces have a unified design that makes consistent use of the same types of visual interface in different contexts, so that the user is not forced to learn multiple approaches to operating the workspaces. Student is given progressive control over the menu system as knowledge is demonstrated. Tailored Instructional Modules, Writing Progression Advice, Summary Advice for the Expert Workspaces, and Writing Process Advice are given to the student as she works.

7. Teacher Interface

The Teacher Authoring Tool allows the teacher to select and create Assignments based on the writing skill needs of students and on the teacher's curriculum, provides access to class rosters, creates reports of student writing products, and includes the teacher console--where teachers receive real-time feedback on each student's performance in the laboratory.

Ongoing Research: Testing and Evaluation

MAESTRO is being tested at 24 middle and high schools in Texas, New Mexico, Ohio, New York, and Pennsylvania during the 1996-97 school year. The current experiment will measure the effects of the tutor on individual student writing products. Ongoing research will identify organizational and implementation issues related to the adoption of the technology by public schools, and provide information on the utility of the design factors as implemented the classroom environment. Based on the success of the original R-WISE designs that contributed to the design of MAESTRO, and based on early teacher feedback, the MAESTRO evaluation will provide a useful test of the application of advanced training technologies to teaching the writing process.

Bereiter, C., & Scardemalia, M. (1987). The psychology of written composition. Hillsdale, NJ: Lawrence Erlbaum Associates.

Brown, J., Collins, A. & Duguid, P. (1989). Situational cognition and the culture of learning. Educational Researcher, 18, 32-42.

Britton, B. & Glynn, S., Eds. (1989). Computer writing environments: Theory, research, & design. Hillsdale, New Jersey: Lawrence Erlbaum Associates.

Carlson, P. & Crevoisier, M. (1994). R-WISE: A learning environment to teach prose composition. Technical Paper of the Human Resources Directorate, Technical Training Division, Brooks AFB, Texas.

Carlson, P. & Miller, T. (1996). Beyond word processing: Using an interactive learning environment to teach writing. Technical Paper of the Human Resources Directorate, Technical Training Division, Brooks AFB, Texas.

Collins, A. (1988). Cognitive apprenticeship and instructional technology. Cambridge, MA: BBN Laboratories, Inc.

Flower, L., & Hayes, J. R. (1980). The dynamics of composing: Making plans and juggling constraints. In L. Gregg, & E. Steinberg (Eds.), Cognitive processes in writing (31-50). Hillsdale, NJ: Lawrence Erlbaum Associates.

Gregg, L. & Steinberg, E., Eds. (1980). Cognitive processes in writing. Hillsdale, NJ: Lawrence Erlbaum Associates.

Kellogg, R. (1989). Idea processors: Computer aids for planning and composing text. In Britton, B. & Glynn, S. (Eds.), Computer writing environments: Theory, research, & design, pp. 57-92.

Rowley, K., Miller, T. & Carlson, P. (in press). Informing the design of a supportive writing environment: Assessing student performance by instructional styles, and by modes of student control. Technical Paper of the Human Resources Directorate, Technical Training Division, Brooks AFB, Texas.

Rowley, K. & Miller, T. (in press). Addressing replicability and measurement issues in studies of the R-WISE writing environment. Technical Paper of the Human Resources Directorate, Technical Training Division, Brooks AFB, Texas.

Smith, John B. & Lansman, Marcy. (1989). A cognitive basis for a computer writing environment. In Britton, Bruce K. & Glynn, Shawn M. (Eds.), Computer writing environments, theory, research, & design (pp. 17-56). Hillsdale, New Jersey: Lawrence Erlbaum Associates.

Zellermayer, M., Salomon, G., Globerson, T., & Givon, H. (1991). Enhancing writing-related metacognitions through a computerized writing partner. American educational research journal, 28(2), pp. 373-391.

Acknowledgments

The MAESTRO software was developed at the U.S. Air Force's Armstrong Laboratory, Brooks AFB, Texas, as part of the Fundamental Skills Training (FST) Project. Funding for this FST research was provided by the U.S. Air Force's Armstrong Laboratory, Phillips Laboratory, Rome Laboratory, and Wright Laboratory. Support was also provided by the University of Texas at San Antonio, under a cooperative research agreement.

1.

^* This paper is a composite of two papers presented at ED-MEDIA 97 in Calgary, June, 1997

© Copyright 2006, Kurt Rowley, Ph.D.