VP-SE Research Group
(C)
Centre Universitaire d'Informatique,
Université de Genève,
CH-1211 Genève 4, Suisse
If one examines however the production of technology-based learning material, such as that involving the computer, the videodisc, the CD-ROM drive, and other auxiliary devices, one sees a variety of methods and strategies in use. Furthermore one sees little full-scale high quality material, in spite of a long period of development. We regard the period so far as experimental. The issue was learning how to use the technology, and learning how to modify older production systems, such as those for textbooks, for the new medium.
Many developments are not adequate to a full production strategy. Although we will not argue the case, since we discuss a system we believe will be successful, the use of authoring languages and authoring systems is an unfortunate direction.
The development of the production system discussed in this paper was started over twenty-three years ago at the University of California, Irvine. About ten years ago we were joined by colleagues at the University of Geneva, and since then the two groups have been closely affiliated. Many aspects of the system, such as pedagogical design, have changed little over time, even as new media and new computer capabilities were added. Other aspects, such as the technical implementation, have changed; they are still under evolution. The system has produced commercially viable material in varying subject areas, and continues to be used in a wide range of current projects.
There is much to be learned from industry concerning management. The use of the standard project management techniques, many now available on computers, is essential for large projects. The computer is used in several of the developmental stages, easing the management tasks to some extent, as full use of the computer capability is considered.
We emphasize that the teachers must be familiar with the subject area, and see students daily in classes, in just the area we are working in. Thus if we are developing material in middle school science we expect that most of the good teachers involved in the design process will be middle school science teachers.
In addition to the good teachers, other types of individuals may be present. First there may be researchers in the learning areas involved. In some areas, such as science education, there is considerable research. In some areas there is little available. If something is known about how people learn in a particular area we want to make use of that, to the maximum extent possible. We also like to have people familiar with highly interactive learning material.
We do not involve programmers at this stage. We want the designers to feel free to make any choices they think pedagogically best, and we do not want to restrict them in these choices. Furthermore, we do not typically have screen designers involved. They will appear later in the production process.
Overall design starts with the broadest considerations of the units to be developed, including the general principles of this material. It does not get down to minute details of how individual modules behave when used by students; such details occur in detail design.
Overall design sessions produce an outline of the material to be produced and module descriptions, so that those who work on the detail design of each module will have guidance as to how to proceed, and will see how their module fits into the overall production program.
Designers in both overall design and detail design work in groups, generally with larger groups in the overall design activities. Overall design begins with the project proposal, a document already approved, that gives guidance to the project.
The first new activity in overall design is likely to be brainstorming, in the formal sense of the term. Since developers will probably not be familiar with brainstorming it is necessary to give them a brief description of the process including strong emphasis that in the first stages of brainstorming there is no criticism or discussion of the ideas presented. Developers are urged not to worry about the details of what they are proposing or whether they have already been suggested, but to generate many ideas quickly. A person experienced at brainstorming should run the session.
Brainstorming produces a variety of ideas that can be classified. Some represent pedagogical techniques. Others refer to subject areas to be covered. Some may be general philosophical principles. No attempt is made to separate these initially, but at a later stage one can begin to break them into categories, and use them as guidance for the more formal development of outline and module descriptions.
We have found it useful for overall designers to state what they regard as the major learning difficulties with each unit. These are not restrictive on the detail designers who follow, but give clues to them.
We have groups of about four teachers working together. A typical session is one week, full time. Design sessions that occupy only a part of the individual's time, such as two hours a day, are inefficient, because teachers do not remember all the minute details of the discussions that have ensued. Several groups will work together in a sizable project, meeting early in the morning and at meals, but maintaining their groups during the week or two in which they are working.
Many of our designers have never been involved in design of extensive curriculum material, and most are not familiar with interactive techniques, so the first morning is an introductory session. We do not expect designers to know anything about computers. The introduction must include information about group dynamics. Teachers are usually not familiar with working in small groups, although this activity is well known within industry. So they must be prepared for the often almost violent disagreements likely to occur in these groups and for the necessity of quickly reaching consensus.
Teachers learn about the script strategies for recording detail design decisions in the early hours of working on the design, although they have seen scripts in advance. The script has varied little in style in the twenty four years we have been using this process.
Scripts give all the details of what is to happen. They are a semi-formal visual programming language, oriented to the needs of learning. They show the text, the media to appear, the information for programmers, comments to help later designers, the tests to be performed on student performance or on user input, and the paths followed as the results of these tests. Examples of scripts are contained in our literature. The natural language components of the script formalism make it easy for newcomers to quickly gain speed in developing scripts.
We find it useful to have the media partially available before the final detail design session. It is difficult to design highly interactive material involving video unless the video is fully known to the designers, available at the design sessions. They need to examine it over and over to see just which portions can be used for what purposes. We will not discuss the questions of how media are selected and developed, but we have additional tentative information on these issues for those interested.
With regard to programming we discuss two possibilities. One represents our older strategy, involving direct programming of the modules. The second involves an automatic programming environment, now in working form through recent efforts at the University of Geneva.
We typically have used a higher level language, TurboPascal in recent efforts. But occasionally we have been forced to use other languages too.
Since the material will need to be frequently revised, particularly in the early stages and through formative evaluation, it is important to work in a structured language allowing easy revision. It is also very important to follow the structures of good modern software engineering. Programs must be carefully reviewed to see that these standards are met.
We have found it important to develop tools to assist programming activities. Most high level programming languages do not allow effective control of the screen. This is somewhat aided by modern windowing environments, but they do not provide the best capabilities. We have for over ten years had our own windowing environment, particularly oriented to the screens we wish to present in technology-based learning material.
We also have tools to separate everything dependent on natural language from the rest of the material so it would be relatively easy to move materials to different languages, important for future international development projects.
A full course development environment has been built, using the script as a semi-format visual specification for modules. This environment supports script entry, script update, version control, (partial) automatic program generation, computer supported editing of the hand-programmed parts, and computer supported translations of the computer dialogs.
We do not expect to write the full program automatically, but we estimate that we should write more than 80% in this fashion. Our current developments, programmed in Ada at the University of Geneva on Suns under X-11, do not yet attain this figure. But they are operational, and we have used them in several recent projects.
We estimate that the overall savings in cost will be about 30%, and the overall savings in time will be about 45%, when these facilities are complete. These figures are for the entire development process. Additional funding will be needed. Information about the automatic programming environment is available from the authors and in recent papers.
Generally we want the screen to have only the relevant information on it at any given time, as we want to focus students' attention on what is important at the moment, not on older material now only garbage from the standpoint of learning. Blank space is one of the most valuable commodities here. Unlike blank space in books, blank space is free on the screen.
We recommend two rounds of formative evaluation and improvement. That is, after the materials have been improved, they can be tested once again and improved a second time.
In addition to the usual purposes of formative evaluation, computers allow the possibility of determining whether the material is motivationally captivating for the users. It can be used in environments in which the student is under no pressure to continue to run the material, such as public library environments, where no exam is expected by the students. Then one can record where students frequently leave the program, indicating sections that do not hold the attention of the student. Then these sections can be reworked. Our previous experience with this type of formative evaluation indicates that a primary cause of students losing interest is a lack of interaction in passive material, with large amounts of reading or viewing video. Often it is possible to make dramatic improvements in motivation at such locations.
Summative evaluation is typically not part of the development process. The original design group should not be involved in summative evaluation.
Maintenance is now computer supported. When the specification (the script) is changed, the system indicates to the programmer the parts of the program that need to be updated. During this process, the programmer can see, at any moment, the part of the specification that corresponds to the relevant part of the program. Similarly, the system can show a translator changes in text since the last translation, so that the different translations can be brought up to date quickly, without having to compare the different language versions manually.
These costs are roughly in four equal parts, reflecting the four stages of the development process indicated in this paper, management, pedagogical design, technical implementation, and formative evaluation and improvement.
Development of a course will typically be a three year period, with the last year devoted to formative evaluation and improvement. Again this is similar to the Open University.