A Shell for Intelligent Tutoring Systems

Bettina Reinhardt1, Stefan Schewe2

1University of Wuerzburg 2Policlinic of the University of Munich
Allesgrundweg 12 Pettenkoferstraße 8a
97218 Gerbrunn 80336 München
Germany Germany
phone: 049 - 931 - 7056115
fax: 049 - 931 - 7056120
reinhard@informatik.uni-wuerzburg.de

Abstract:

Computer based learning gets more and more important in higher education. Most of Germans medical students see very few patients at Medical School. One way to alleviate this problem is to present full patients-data in a computer program. Not only in medicine but in many other domains it becomes necessary to offer students a flexible way of learning. We developed a shell for building intelligent tutoring systems to test and examine new cases in different expert system knowledge bases. Our system uses case data and problem solving knowledge to handle classification problems. The main method of the system is to present case data and to control the student's actions by comparing them to the underlying expert system. It should be used in combination with traditional learning methods like textbooks or classes and help to extend knowledge by revision and application to new cases. Additional there is a hypermedia component to help presenting the case data as close to reality as possible.

1 Introduction & Motivation

Computer based learning gets more and more important in higher education. 92% of all medical students in Germany do not get the chance in 7 years of education to look after more than 6 patients over their full time in hospital (Eysenbach, 1994). One way to handle this problem is to present full patient-data to the students in a computer program. Another reason for computer based learning is the flexible way it can be used. W. Wide (Wide, 1994) points out the advantages of flexible learning as follows: "In its broadest sense, flexible learning ... enables the learning to take place at the time, place and pace which suits the learner's own circumstances and needs". Learning with computers also helps students to be more free in their actions so that they do not feel the pressure to do well. They feel free to ask stupid questions and try stupid actions, from which they can learn in a Trial&Error manner.

A computer based system can help to learn, applicate and test knowledge without being controlled by a university teacher. It also can be used to applicate knowledge being learned in classes by using it in new cases, because to use known facts in new situations is an important part of learning.

There is more than one way to categorize learning software. It depends on the institutional technology, learning environment, human and institutional relationship and learning materials (Mapp, 1994). Some classification has been done in (Baumgartner&Payr, 1994) and (Eysenbach, 1994). The main categories are electronic textbooks, hypertext and presentation programs (Hermes (Pohl, 1993), Voxel-Man (Mueller, 1993)), drill&pratice programs (Number Crunchers (Baumgartner&Payr, 1994)),tutorial systems (ILIAD (Warner et al., 1988), GUIDON (Clancey, 1990)), simulation software (SimAnt (Bremer; 1991)) and microworlds and modeling programs (Interactive Physics (Baumgartner&Payr, 1994)).

What software to use is highly dependent on the task. If there is pure rote learning a drill&practice program may be the best. If the user needs to understand complex interrelations a tutorial system or simulation software is necessary. Another point in deciding for a special software is the didactic methods usually used in the domain. Over years different methods showed to be the best, so it would be much easier for the students to have at least a similar teaching method in the learning software.

More and more learning software in different quality is available in today's software shops. Most of the universities have the hardware to run most of the existing learning software. But however well designed this software is, the success is highly dependent on the acceptance of the program by teacher and students.

In the next section we go into Intelligent Tutoring Systems and classification problems. The system TRAINER will be explained in the third section and in the fourth section we show a small example how to use it. Before we get to the conclusion, some evaluation results of a medical application of TRAINER will be presented in section five.

2 Classification & Learning

Intelligent Tutoring Systems do not only describe knowledge, but they are also able to applicate and test learned knowledge. The figure 1 from (Puppe, 1992) illustrate the main components of an Intelligent Tutoring System (ITS):

Fig. 1: Main components of an ITS (Puppe, 1992)

Student modeling is used to derive an explanation for the student's actions. The most important models are: stereotypes, overlay models, enumerative theories of bugs, reconstruction of bugs, generation of bugs and combinations of the previous methods. The component Teaching Methods plans the dialogue with the student with didactic background. The user interface is important for the acceptance of the system, so it should be very good designed to keep the motivation high. Some ITS programs use Hypermedia to make them more attractive. Another important fact is the use of a knowledge base that makes the system able to follow the student in a very flexible way. More about ITS in (Burns et al., 1991; Clancey&Soloway, 1991; Goodyear, 1991).

Mostly ITSs work on procedural, for example mathematical domains, but a lot of work has been done in classification problem solving. Classification is defined in (Puppe, 1993) as follows:

"Classification (Diagnostics) is a problem solving type in which the solution is selected from a set of given solutions, possibly via a diagnostic intermediate structure, by a problem solving method. In diagnostic the problem features (observations) are also called symptoms, and the solutions diagnoses."

We will use the terms symptoms and diagnoses and TRAINER, the system introduced in section 3, will use them too.

Solving classification problems does not only require selection of one or more solutions for a given subset of observations, but also request of additional observations, that can improve the quality of the solutions.

Since expert system design for classification problem solving is a very well understood area the technique can be exploited for tutoring systems. There already exist some good knowledge bases, built for diagnostics, that can be used for teaching classification problem solving in the specific domain.

3 The case based system TRAINER

The biggest problem in building ITSs are the high costs. The solution for this problem is to build flexible shells, which make it possible to create new systems with little effort. TRAINER is such a shell based on the powerful expert system tool box D3 (Gappa et al., 1993; Puppe et al., 1994), that supports heuristic, case based, statistic and set covering classification. There already exist knowledge bases in D3 for medical domains, like rheumatology and neurology, as well as for technical problems, like error diagnostics in printing presses.

In this section we want to introduce our system TRAINER with a rheumatology knowledge base and in section four we show a small example of how it can look like, when a student works with the system.

TRAINER is the second approach to build a shell for a tutoring system to D3. The first system TUDIS is described in (Poeck&Tins, 1992). The difference between the two systems is mainly the interface. The evaluations of TUDIS showed that it was too hard to use for most of the students. Also the cases are so huge that a new way to present the case data had to be found. TRAINER is more variable in the interface and uses a dynamic hierarchy similar to the Macintosh finder, that helps the student to concentrate on the relevant information without scrolling.

One of the major problems in building a shell for Intelligent Tutoring Systems is to find a way to fit more than just one domain. Different domains use different teaching methods and a different linguistic usage, what makes it difficult to decide how the ITS interface has to look like.

In this respect our implementation is very flexible. There exists a configuration dialogue allowing concerning interface and learning strategy.

One of the main decisions is the level of interaction between the system and the student. In TRAINER there are two major ways to guide the student through a case. First the expert can divide the symptoms in groups, like history, examination, technical tests and laboratory tests in a medical knowledge base. The system presents each of these groups sequentially and before the student gets new information (the next group) he has to select a suspected diagnosis. The other way is to give him some start symptoms and to let the student decide what examination to do on the patient next. In this case the system can criticize not only the chosen examination, but also a justification given by the student. In this way the students can be taught not only test interpretation but also test selection (indication).

One additional advantage of the system is the ability to connect it to the hypertext-tool of D3, called D3-HITS (Reinhardt, 1993), what makes it possible to present case data in a more natural way. Symptoms can be presented by multimedia elements, for example the heart beat as a sound, or a walking disorder as a video clip. In a preliminary version this connection is implemented. This makes it possible not only to criticize the problem solving tasks of the student but also his performance in identification of symptoms given by pictures.

4 Example

For the RHEUMATOLOGY-TRAINER we decided to divide the symptoms in groups to facilitate use of the tutoring system. In the evaluation scenario, described in chapter 5, the students work with the systems just two times. That made it necessary to guide them more than students who spend more time with the system.

The symptoms are divided in four groups history, examination, technical tests and laboratory test and the system is guiding the student sequentially through the case. To start TRAINER, a case is chosen from a case base and the symptoms of the first class "Anamnese" (history) are shown in a dynamic hierarchy (figure 2). A normal case in this knowledge base contains about 250 symptoms, what makes it inconvenient to present them in a conventional hierarchy.

By clicking on a symptom the student gets a pop-up-menu including explanations to the question, all alternatives to answer the question and, if existing, a drawing or a picture (figure 3).

Fig. 2: In a dynamic hierarchy symptoms about the case history and the facts about the patient, like age and gender are presented. The dynamic hierarchy enables the student to see a lot information in less space and decide for himself what answers he wants to see. The student can open all symptoms by clicking on the button SHOW ALL and with ALL FACTS he can decide if he wants to see only the unnormal symptom. There is a HELP button with an explaining text behind it and the GO ON button, that calls the next group of symptoms. The symptoms marked with an apple are presented by a picture. A click on a symptoms will open a pop-up with additional information like explanations and all alternatives to answer the question. The picture to this symptom can be open through this pop-up.

Fig. 3: Some of the symptoms are shown in pictures; the pictures can be drawings or photos. In this drawing the affected joints are shown. The patient has problems mainly with his wrists and ankles.

After the examination of the given facts new symptoms are shown in the same window after the student chooses one or more diagnoses from a hierarchy. The diagnoses are presented to him in a dynamic hierarchy and he can select one or more diagnoses on different levels. Initially, Trainer provides mostly approving feedback if only the student's hypothesis is pointing in the right direction (figure 4).

Fig. 4: The student chooses one more diagnosis and gets a friendly feedback, because he had selected at least one diagnosis in the right category. The use of the dynamic hierarchy is necessary to choose from over 70 diagnoses.

Fig. 5: After dividing the selected diagnoses in confirmed and suggested diagnoses, there is a much more precise critic in the following levels.

TRAINER`s following comments are more precise. The student has to weight the chosen diagnoses (suggested and confirmed) and the diagnoses are also weighted in the system. After the comparison of the two sets an assessment is given (figure 5). In all of the following levels the critic component compares the given weights and prints out the differences. For every diagnosis the student can ask for a list of symptoms, that point to it in this case and one that shows all symptoms pointing to it in general. He can also request to give reasons by himself for one of his chosen diagnoses and gets these symptoms criticized.

This interplay between getting new information and choosing suspected diagnoses ends when all symptoms are shown to the student. Then he has to give a justification for one of his last chosen diagnoses. Therefore he has to choose one of his diagnoses and pick relevant symptoms from the last hierarchy (figure 6).

Fig. 6: Hierarchy with all given symptoms to select relevant data for the diagnosis.

Fig. 7: Feedback of the reason for the diagnosis Vaskulitis chosen by the student. The selected symptoms are compared to the ones found by the system. In this window all reasons are print out and the one the student selected are printed bold. So he can see the right answer and get a comparison of his answer right away.

TRAINER compares his symptoms with the symptoms given by the underlying expert system and shows the result in a feedback window (figure 7). This method is also used, when the student wants to give reasons for a special diagnosis during the dialogue. TRAINER`s comment of student's input is always based on the same (limited) data of the actual level.

Afterwards the student can see the correct diagnoses of the case and get an explanation for them. He can ask for the relevant symptoms of every diagnosis.

This example dialogue is very restricted in its possibilities. TRAINER can do much more than the chosen configuration let guess. For example the friendly critic the first time the student has to choose suggested diagnoses can be replaced by the critic used at the other occasions. When the student only gets some initial information and has to request information from the system, TRAINER can not only compare the selection with the information chosen by the system, but also can say how good or bad the selection of the student was and explain it to him. Then there are less important decisions like if not relevant data should be shown to the student. If requested, a protocol of the student's action to each case can be printed out for further studies by the teacher. Additional configurations make TRAINER flexible for use in different domains.

5 Evaluation

The tutorial system RHEUMATOLOGY-TRAINER was evaluated as follows: After a first assessment of their knowledge about a certain rheumatological illness, 24 students were confronted with an actual, real patient with this illness in the rheumatology department. The professor discussed the patient with the students, evaluated the historical data, the differential diagnostic possibilities and the therapeutic options. The rheumatic cases were presented in the same way as the ones in internal medicine. The actual patient was presented to a group of 2 or 3 students in the 4th year of medical education. A standardized medical history of the patient mentioned above was taken on the computer screen by a technical assistant. All available patient data were added to his history by the expert system RHEUMA. The case comparison unit of D3 (Goos&Schewe93) chose a new patient with similar medical histories but different final diagnoses out of a database of 1017 rheumatologic patients. Knowing the diagnosis of the actual patient the student had to make the right diagnosis for the similar patient with the tutorial program on the next day of the course. The students were assisted by a human tutor in order to become familiar with the computer program using the known patient data. After this the students had to fill in the same multiple choice test on the disease of the actual patient, the results were compared to the ones of the first test. They also had to fill in another questionnaire regarding their opinion about the computer program, their motivation to learn and their view of how such programs could be integrated in the future medical curriculum.

One week later the students were confronted with a new actual, real patient with another rheumatological disease. Again they had to fill in two multiple choice tests. As opposed to the former method they had to diagnose another similar patient found with D3 using a paper version of the patient data arranged in the same way as it was done by the computer the week before. The last exam in rheumatology was taken at the end of the term as a part of a multiple choice exam in the whole field of internal medicine. As a comparison case to a new unknown and difficult rheumatological patient, an easy cardiological case with angina pectoris was chosen. The historical data, clinical tests, x-ray films etc. collected by the students were counted and compared with the two cases.

Every student had to complete a 40 item questionnaire, the most relevant results are presented in the following. The mean standard deviation , minimum and maximum of the scores between 1 (no) and 10 (yes) are given:

* The program was able to motivate the students. Examples:

<<Does the use of computer based tutoring systems in the medical curriculum make sense to you?>>

8.32 +/- 1.06; min 6; max 10.

<<Do you think that you can efficiently study with this computer program?>>

6.95 +/- 2.14; min 4; max 10.

* The program is mostly considered a possibility to self-improvement and a useful addition to the regular practical training. Examples:

<<The program is able to:>>

<<replace a professor>> 1.71 +/- 1.58; min 1; max 8

<<support a professor>> 7.10 +/- 2.07; min 2; max 10

<<improve my diagnostic abilities>> 7.24 +/- 1.82; min 2; max 10

<<support me with specific diagnostic decisions>> 7.43 +/- 2.72; min 1; max 10

<<improve my medical skills>> 7.59 +/- 2.37; min 3; max 10

<<add to the regular practical training>> 7.55 +/- 2.69; min 1; max 10

<<I would learn best with the computer in which surrounding:>>

<<within a course at the university>> 6.59 +/- 2.53; min 2; max 10

<<self learning at home>> 7.09 +/- 2.59; min 1; max 10

<<learning with friends>> 3.73 +/- 2.65; min 1; max 10

<<replace of a practical course>> 1.23 +/- 0.60; min 1; max 3

* Some limitations of the program were seen and have to be improved in the future. Examples:

<<Are you satisfied with the computer program with regard to>>

<<graphical layout>> 5.61 +/- 2.38; min 2; max 10

<<explanations contained in the program>> 5.94 +/- 1.68; min 3; max 8.

The final exam results showed no significant difference between the number of right answers concerning the difficult rheumatological and easy cardiological case.

A future development of the tutorial system is planned especially in the field of multimedia presentation of case data.

6 Conclusion

TRAINER seems to be a good tool to build an ITS for existing knowledge bases. Some of the knowledge bases have to be changed a little bit to increase their value as a teaching tool. The feedback from experts, who built applications with TRAINER, as well as the students, who already used these systems, are very positive and optimistic.

Compared to the two good known systems ILIAD (Warner et al., 1988) and GUIDON (Clancey, 1990) it is a very flexible program. ILIAD is restricted to only one knowledge base and there is no way to weight diagnoses differently. The GUIDON Program has additionally a student model, what is not yet implemented in TRAINER. There is a diploma thesis on this topic and a Socratic dialogue will be integrated soon. One advantage of TRAINER is, that the problem solver of D3, the underlying expert system, uses hypothesize&test, what is more adequate to the human problem solving method in classification. Guidon is using backward chaining, what makes it more difficult to explain the solution to the student. Also GUIDON is not able to criticize the request of additional information as TRAINER is able to, because GUIDON can only compare the selected information with the information chosen by the Mycin expert system. In GUIDON also additional rules had to be written to use the knowledge base for teaching. In TRAINER/D3 no knowledge had to be added to use existing knowledge bases in the tutoring system.

Trainer is a very young program, that has a well tested expert system shell as a base. The evaluations in classes will bring out good suggestions. The addition of the Socratic dialog and a student model will make it more powerful and to apply more knowledge bases will make it more flexible.

At the moment the Rheumatology-Trainer is used the second time at the University of Munich and another TRAINER application in neurology is distributed over 500 times (MAC & WINDOWS versions).

References

Baumgartner P., Payr S. (1994) Learning with Software. Volume1 Digital         
Learning, Oesterreichischer StudienVerlag (in German)                          
Bremer M. (1991) SimAnt The electronic ant colony (Handbook), Ocean Software   
Limited 1991 (in German)                                                       
Burns H., Parlett J.W., Redfield C.L. (1991) Intelligent Tutoring Systems,     
New  Jersey LEA                                                                
Clancey W.J. (1990) Knowledge-Based Tutoring - The GUIDON Program,   MIT       
Press Cambridge                                                                
Clancey W.J., Soloway E. (1991) Artificial Intelligence and Learning           
Environments, MIT Press                                                        
Eysenbach (1994) Computer Manual for medicine and biological science,          
Urban&Schwarzenberg (in German)                                                
Gappa, U., Puppe, F., Schewe, S. (1993). Graphical knowledge acquisition for   
medical diagnostic expert systems. Artificial Intelligence in Medicine,Vol.    
5, 185-211                                                                     
Goodyear P. (1991) Teaching Knowledge and Intelligent Tutoring, Ablex          
Publishing Corporation                                                         
Goos K., Schewe S. (1993) Case based reasoning in clinic evaluation,           
Proceedings to: 4th European Conference on Artificial Intelligence in          
Medicine AIME93, Munich Germany                                                
Mapp L. (1994) Learning from learning technology: a framework of               
implementation issues, in: The CTISS File 17 (July 1994)                       
Mueller B. (1993) Na, was fehlt uns denn? in screen MULTIMEDIA 10/93 (in       
German)                                                                        
Poeck K., Tins M. (1992) An intelligent tutoring system for classification     
problem solving, Proceedings of GWAI-92, Springer, Lecture notes in            
artificial intelligence 671, 1993                                              
Pohl C. (1993) Hypermedia-Learning in Business, HERMES in: tekom Online.       
Manuskripte der 4. Fachtagung fuer Online-Dokumentation, Munich October 1993   
(in German)                                                                    
Puppe F. (1992) Intelligent Tutoring Systems, Informatik Spektrum, Band 15     
Heft 4  (in German)                                                            
Puppe F. (1990) Systematic Introduction to Expert Systems, Springer            
Puppe, F., Poeck, K., Gappa, U., Bamberger, S., Goos, K. (1994). Reusable      
components for a configurable classification tool box, KI/2, 13-18 (in         
German)                                                                        
Reinhardt B. (1994) Identification of symptoms in a multimedia                 
training-system, Proceedings of "Hypermedia in Aus- und Weiterbildung",        
Dresden 1994 (in German)                                                       
Warner HR, Haug P, Bouhaddou O, et al. ILIAD: en expert consultant to teach    
differential diagnosis. In: Proceedings SCAMC88. New York: IEEE Comp Soc       
Press, 1988 371-2                                                              
Wade W. (1994) Flexible learning and Flexibility in Course Provision, in:      
The CTISS File 17 (July 1994)                                                  

Acknowledgment

We would like to thank our colleagues Karsten Poeck and Frank Puppe for providing us with helpful comments for this draft and Thomas Quack, the teaching assistant at the University of Munich.