Abstract

Although studies have shown that expert safety-critical decision-makers base the majority of their incident command decisions on past experiences, there is not yet a decision support tool enabling them to access relevant experience during an incident. The Remote Intelligent Management Support and Training (RIMSAT) project aims to provide such a solution through the integration of model-based reasoning (MBR) and case-based reasoning (CBR) technologies. CBR allows Incident Commanders (ICs) to access lessons learned from previous, similar incidents and to compare their current situation with the previous one. MBR allows ICs to calculate resource needs and safety margins. Integration of MBR and CBR through a technique called context-sensitive feature weighting (CSFW) facilitates the use of the decision support tool and enhances the tool's ability to provide relevant lessons by emphasising the aspects of the incident which are most important in the specific context.

1. Introduction

As a fire fighter goes through his career, he gathers experience and knowledge through his attendance at incidents and through study of procedures and a few archetypal incidents. It is rare that he encounters the same incident twice, but often certain situations within incidents recur; if not in the experiences of one fire fighter, at least among a collection of fire fighters. Unfortunately, the same mistakes are repeated at these recurrences because there is no post-incident sharing of experience with fire fighters who did not participate in the incident. Each fire fighter represents a separate channel of experience and remains isolated with regard to the vast wealth of collective experience within the fire service.

Though it has been demonstrated that fire fighters base their decisions on past experiences, they are currently unable to benefit from the experiences of other fire fighters in any systematic way. The RIMSAT (Remote Intelligent Management Support and Training) project results suggest that a decision support tool using Model-Based Reasoning and Case-Based Reasoning could make relevant incident lessons, contacts, and solutions quickly available to Incident Commanders at the incident scene. These lessons, contacts, and solutions could have life-saving implications.

2. Current Situation

2.1 - Decision-making by Incident Commanders

Studies have shown that Incident Commanders (IC) in the fire service, as well as many other safety-critical decision-makers, base the majority of their incident management decisions on experience. Gary Klein, in his authoritative studies of decision-making by ICs, proposes the recognition-primed decision (RPD) model as most accurately reflecting the decision-making processes of the best safety-critical decision-makers. In RPD, ICs recognise a situation as similar to a previous one, and then take a course of action which experience has taught them to be correct. Of course, no incident is identical to a previous one, but ICs are able to tailor their strategy through an understanding of the goals, cues, and expectancies that they have acquired through experience of similar incidents. Klein's study concludes that 80% of non-routine decisions by ICs are made using RPD, with the assumption that routine incidents would show an even higher percentage of RPD [4].

Some incidents may indeed require comparison of options, particularly in the case of less experienced ICs, and completely unfamiliar incidents may require creative decision-making. Killion has suggested that the ideal decision-making method for military safety-critical incidents would incorporate both RPD and analytic decision-making, as the two methods complement each other. He suggests a method whereby an initial set of courses of action (COAs) are proposed according to RPD, then analysed and compared by a commander's staff who then propose a subset of the most relevant COAs to the Commander, who then makes a final decision based on his/her recognition-primed decision model [11].

2.2 - Decision support in the fire service

Incident Commanders on the fire ground currently have a variety of technologies and information resources at their disposal, but none that are capable of delivering relevant past experiences. They have command planning systems that enable them to get a physical layout of the incident ground, keep track of resources, and send brief situation reports to the brigade control centre. They also have databases that include information about hazardous materials, specific risk information according to the address of the incident, and standard operating procedures.

At major incidents, the Incident Commander will have an associate called a "facilitator", who will remind the IC about hazards that should be considered, or elements of standard operating procedure which should not be neglected. In addition, there is a monitoring officer, who generally has more experience than the IC, who is watching for potential dangers or neglected issues, and who will make suggestions, if necessary, to the IC.

3. Delivering highly relevant experience

The Incident Commander thus has a wealth of information about the premises and materials involved in the incident but, regarding the means of dealing with the present incident, he has only the monitoring officer's and his own personal experiences and generic recipes for generic incidents. Yet, there are millions of incidents each year being attended by hundreds of thousands of experienced ICs. RIMSAT's hypothesis is that it is possible to capture a large number of these experiences and render them rapidly exploitable through Case-Based Reasoning (CBR) and Model-Based Reasoning (MBR).

4. Technology Description

4.1 - Case-Based Reasoning (CBR)

CBR allows one to build a library of cases, or experiences, which are described according to a fixed set of descriptors called the domain model. The domain model has features (such as "Smoke Colour") and values (such as "black", "white", etc.). Later on, one can call up relevant cases by specifying a set of Feature-Values. For example, a simple query could be:

Cases will be sent to the user with a similarity rating which tells him what percentage of the case values correspond to the values of the query.

Thus, Case-Based Reasoning provides a way of archiving experiences and quickly retrieving appropriate experiences when needed. As one can emphasize the importance of certain values over others, the search queries can be extremely precise.

4.2 - CBR Advantages

In a decision support function for critical incidents, CBR has a number of advantages, including:

1. It can be used in highly complex, incompletely understood domains, as it creates a library of concrete experiences. The experience does not need to be completely understood, only recorded.
2. It can provide solutions where no algorithms exist to solve a problem.
3. Positive and negative experiences may be recorded.
4. Evolution of the system is simple as one only need add new experiences to the library.

4.3 - CBR Disadvantages

CBR also has some disadvantages, including:

1. It is necessary to build a library of cases before the system can be useful.
2. Some values may be more significant in some contexts than others. Unless the user specifies the particular importance of those values, they will be treated with the same importance as the rest of the values.
3. Cases require interpretation, as a CBR reasoner does not necessarily provide the "correct" solution for a problem; it merely provides suggestions of possible solutions.

4.4 - Model-Based Reasoning (MBR)

"The Model-Based Reasoning process itself can be viewed as the symbolic processing of an explicit representation of the internal workings of a system in order to predict, simulate, and/or explain the resultant behaviour of the system from the structure, causality, functional and behaviour of its components" [1]. Thus, the first step in MBR is building an accurate representation (model) of the system that you want to be able to reason about. Once there is a complete model and an algorithm which correctly reproduces the relationships and behaviour of the system, one can get complete, precise solutions based on a set of inputs.

4.5 - MBR Advantages

MBR has several important benefits:

1. Close affinity with the real system: In MBR the main patterns in the system structure are usually reflected in the model and so interpretation and explanation are much enhanced.
2. Principled approach: A domain theory with underlying principles provides the reference for model manipulation and reasoning. Thus it is contained and complete.
3. Completeness: The MBR approach provides for the generation or treatment of all cases within a well-defined framework.
4. Unexpected cases: It is not necessary to have prior experience of all types of behaviour. Completeness ensures that exceptions do not exist.

4.6 - MBR Disadvantages

Model-based reasoning is a relatively new technique and despite intense research activity, there are few full-scale applications in industry or commerce - and as mentioned above, most of them are focused on fault diagnosis. If MBR is to realise its full potential the following problems must be solved:

1. Modelling is difficult: Building principled models for a domain is difficult, time consuming and expensive. Unlike heuristic methods it is not incremental. Consequently, a decision to invest in the technology must often be made without the 'encouragement' of a small prototype or preliminary demonstration.
2. Lack of model-builders: Due to the amount of learning and effort involved, model designers would benefit greatly from any available toolsets - unfortunately, very few such toolsets exist.
3. Need for reusable libraries: component libraries are usually built up as a product of the model construction process. However, they should be designed for reuse in order to save expense and increase reliability. Work on ontologies and component catalogues will ease this problem.
4. Need for integration with other methods: When different tools are smoothly integrated into a modelling and simulation environment, then sufficient benefit would be gained to encourage a wider adoption of MBR as the basis for commercial solutions [1].

5. RIMSAT Challenges

To build a decision support tool for Incident Commanders incorporating MBR and CBR, several challenges related to the domain must be surmounted:

1. There are significant organisational barriers (such as liability) which prevent fire brigades from sharing experience.
2. ICs are very short on time when they are managing an incident. Therefore, RIMSAT users on the incident ground cannot spend significant time building search queries and reading incident reports.
3. Fire service personnel have little time for administrative tasks (such as building case bases and models).
4. Very few exploitable models exist in fire fighting command.
5. Incident information is not recorded in a way that it could be exploited for broad knowledge sharing. One needs to be familiar with the incident to understand what has been recorded.
6. Incident details are very inter-dependent with regard to their significance, e.g. water-pollutants are not important if there is no watercourse in the vicinity.

6. Developments

6.1 - Integrated CBR and MBR for decision support

While the organisational barriers to knowledge sharing must be overcome internally, the other five key challenges have been addressed in RIMSAT's integrated CBR/MBR decision support tool. The RIMSAT domain model is composed of the key features of industrial fires and hazardous material transportation incidents. These features have both symbolic and numerical values. All of the values can be used in the CBR query and the similarity function of the CBR engine takes into account closeness to matches (i.e. 20 is closer to 10 than 200, strong wind is closer to light wind than it is to no wind, etc.).

As the RIMSAT Operator is updating his query, he is also recording the incident details that can later be used to build a case. Furthermore, the numerical values are used as factors in the MBR computations of resource needs and safety margins while many of the symbolic values act to a greater or lesser degree as "aggravating factors" in the MBR computations. To summarize, in RIMSAT, the user is simultaneously and seamlessly recording incident details, making CBR queries, and inputing data for MBR use.

However, if not for an added functionality, the RIMSAT domain model, containing hundreds of features, would still be cumbersome to navigate in the heat of an incident, despite its logical taxonomic structure. It would also be impracticable for the ICs to decide the importance of each element of the CBR query repeatedly throughout the incident.

Therefore, RIMSAT employs a more subtle MBR mechanism called context-sensitive feature weighting (CSFW), which affects both the presentation of the features in the user interface and the weighting of the features in the CBR similarity calculation. CSFW is a formalism allowing one to specify that in context C0, Feature F0 has Relevance (weight) R0. Relevance is the generic term that RIMSAT uses for the degree of importance given to a feature with regard to a certain context. Relevance governs the priority given to the feature in the query interface and the weight given to a feature in the CBR similarity calculation. Contexts are composed of a set of values. The selection of these values is thus a trigger for the prioritisation of features in the interface and similarity calculation. The system automatically prioritises the elements of the current situation, according to the current situation.

In the above table, the features are listed in the order in which they would appear in the interface at the beginning of a user session, as the order of presentation is based on relevance. As no context has been specified at the beginning of a session, the presentation is based on the incontextual relevance.

Relevance also determines the weight that they would be given in the similarity calculation of the CBR engine. The weight, unless altered by the existence of a triggering context, will correspond to the incontextual relevance. One can see in the column "CSFW" how various combinations of values can lead to an increased relevance. For instance, water pollutants in the vicinity of a watercourse are given a higher weight than otherwise, as are strong or light winds when there are air pollutants. The water pollutants are more relevant to the IC's tasks when there is a watercourse in the vicinity than when there is not. The CBR search engine will emphasise the highly weighted features in its selection of similar units of experience (cases) to present to the user.

Meanwhile, in the RIMSAT interface, as values are specified, the potential relevance of features will increase, i.e. if water pollutants are selected by the user, the potential relevance of "Environment Around Site" is 4, as the selection of one of its values (a. watercourse) would cause its relevance to increase to 4. The interface organises features based on their highest potential relevance. Therefore, as soon as "water pollutants" were selected, the feature "Environment Around Site" would be promoted to a higher position in the interface.

RIMSAT has created a model-builder that allows one to ascribe context-sensitive feature weighting (CSFW) tags to each feature of the domain model. This model-builder is essential because without it, the excessive work necessary to build the model would have become a barrier to its construction. The CSFW tags use Booleans AND, OR, XOR, and NOT.

Thus, to recap, when filling in details about an incident in the RIMSAT tool, the user is simultaneously:

1. making CBR queries
2. inputing data for MBR use
3. recording incident details
4. activating triggers that facilitate navigation and remind him of important information needs
5. defining weights for the CBR query.

6.2 - The Incident Analysis Process and Unit of Experience

Following the incident, the incident details recorded in RIMSAT are used for incident debriefs, in which lessons are evoked. These lessons form the core of Units of Experience (UoEs), which are the cases in RIMSAT [13]. The UoEs are built using the Incident Analysis Process (IAP). The IAP consists of identifying lessons and then identifying the key context details which make the lesson relevant. With these context details, the UoE can be mapped to the domain model and thus entered into the RIMSAT experience base and found via the CBR querying process described earlier. The UoE is accompanied by a natural language explanation and administrative information for tracability. The IAP is shown in Figure 1.

It was an important conclusion of RIMSAT that the UoEs concern only a specific set of circumstances, not an entire incident. It was determined that an incident was too large and complex to be captured in a single UoE. A single incident may yield scores of Units of Experience [3].

6.3 - Training with RIMSAT

The fire service has acknowledged the training benefits of the RIMSAT system. Organisational learning should be able to make a giant leap forward with the RIMSAT system through the building of experience bases, discussion of units of experience, and definition of terminology, relationships and rules for CBR and MBR [10] [9]. However, this is not the focus of the current paper.

7. Results and Conclusion

Our conclusion is that CBR and MBR may be integrated through a common domain model to enable safety-critical decision-makers to quickly and simultaneously:

1. access and compare experiences relevant to the current incident
2. input data for MBR decision support
3. record incident details which can be used to create units of experience
4. activate triggers which will optimise their user interface
5. optimise the CBR search through context-sensitive feature weighting

Feedback from the fire service has been very positive. The criteria fixed at the beginning of the project for the knowledge management (KM) processes and models were:

• Comprehensibility and usability for trained fire service personnel.
• Feasibility for the CBR and MBR technology partners.
• Conformity to INRECA CBR methodology.
• Ability to evolve with the organisation (both in terms of knowledge and fundamental organisational change)

These criteria have been met. Importantly, the fire service trainees have reported that the RIMSAT UoE authoring procedure is both easy and significantly more valuable than the current incident review procedures. Nonetheless, it is clear that the integrity of the system is highly dependent on the proper training of users and the respect of the RIMSAT Knowledge Management procedures.

The RIMSAT prototype will undergo an extensive series of trials by the UK fire service in the period between September 2003 and March 2004. To render the system deployable, additional work (following the project end) will likely be necessary to complete the domain model for RIMSAT domains (industrial fires and HAZMAT transportation incidents) and/or for other types of incidents.

New models may also be added to the RIMSAT system as they are discovered. For instance, the National Oceanic and Atmospheric Administration has a variety of models for chemical spills which could be rather easily incorporated into the RIMSAT system [7]. Such additions would require adding the model features to the domain model and ascribing CSFW tags to those features.

Another promising related area of study is pattern matching between incidents in order to find new predictive models for an early warning system [8].

8. Business Benefits

RIMSAT takes a step towards bringing about the decision-making proposed by Killion [11] - a combination of the recognition-primed decision model and analysis. Integrated CBR/MBR decision support tools would be useful in a variety of safety-critical and mission-critical domains such as rescue, police investigations, transportation, military command, and the energy industry. CBR has already shown itself to be useful in safety-critical domains such as aviation search and rescue [6].

On a wider scale, integrated CBR/MBR decision support tools respond to many of the knowledge management requirements of the business world. Companies today are seeking new ways to retain experience and to make it quickly available in an easily exploitable form. While the stakes and time pressures may not be as significant as those present in fire fighting or military operations, the principles are the same: more efficient knowledge retention, sharing, and accessibility. Specific business domains include hi-tech customer service centres, engineering/development teams, and manufacturing.

9. References

About the Author

Andrew Lewis recently completed the RIMSAT project (Remote Intelligent Management Support and Training), funded by the European Commission. This project combined knowledge management techniques with case-based reasoning to build a decision and training support system for firefighters. As part of the RIMSAT project, Andrew Lewis developed a strategy for capturing, structuring, and reusing experience for emergency service personnel. He is the editor of the Knowledge Management and Critical Incident Management Special Interest Group on www.knowledgeboard.com. Among other projects, he has also defined knowledge management processes and system requirements for companies in the automotive and transport industries. He is editor of "Learning from Critical Incidents", a book about the findings of the RIMSAT project, to be published in April 2004 by Sapientia Editions. Andrew is a consultant with Nemesia. Contact Nemesia, 89-93, Avenue Paul Vaillant Couturier, Gentilly, 94250, France Tel: +33 1 41 24 27 51, Fax: +33 1 41 24 27 59, Email: Andrew.lewis@nemesia.com

Citation

Lewis, A., "RIMSAT DSS Project: Integrating Model-Based and Case-Based Reasoning", DSSResources.COM, 04/05/2004.

Andrew Lewis provided permission to feature this article and archive it at DSSResources.COM on December 03, 2003. This article was posted at DSSResources.COM on April 5, 2004.