WO2000041613A2 - Expert system for real-time decision support - Google Patents

Abstract

An expert system for real-time decision support, and more particularly a computerized system (124) for performing individual point of care diagnostics and treatment using expert decision making processes. The invention also includes an expert system (104) for data entry and processing that provides real-time decision support and an electronic record of an individual interview. The invention is particularly well suited to the medical industry.

Description

EXPERT SYSTEM FOR REAL-TIME DECISION SUPPORT

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priority under 35 U.S.C. §119(e) from Provisional Application Serial No. 60/115,948, filed January 14, 1999; and Provisional Application Serial No. Serial No. 60/115,914, filed January 14, 1999, the disclosures of which are incorporated herein by reference.

TECHNICAL FIELD

The invention relates generally to an expert system for real-time decision support, and more particularly to a computerized system for performing individual point of care diagnos- tics and treatment using expert decision making processes.

BACKGROUND

The healthcare industry is faced with the challenge of providing high quality healthcare at lower costs. The knowledge base in today's medical fields is growing rapidly, as are both the number and expense of diagnostic and therapeutic interventions. Health care providers can no longer be familiar with, let alone master, the knowledge domain, or be expected to keep up with all of the changes in medical decision making which are intended to improve patient outcome and reduce costs. Computer assisted healthcare systems have the potential of helping health care providers make cost effective medical decisions and simulta- neously keep them abreast of changes in their particular area of clinical decision making. Automation of diagnostic healthcare has been attempted in the past, but current systems fail to adequately provide real-time information at the point of care. Current systems generally are not adaptable to local practice guidelines or modifications to established protocols. As a result, these systems fail to adequately provide an environment in which established guidelines can be uniformly and consistently applied during the healthcare process.

Other decision support systems presently in use generally assist with diagnoses only, or provide alerts in the form of drug or laboratory information. Still other support systems simply provide required information in a useable but passive format, allowing a clinician to make a decision based on displayed information.

Accordingly, the inventors have determined that there is a need for an expert system for real-time decision support, and more particularly for a computerized system for perform- ing individual point of care diagnostics using expert decision making processes. Such a system should be acceptable to health care providers by making: 1) entry of clinical data to support the medical decision making easy; 2) providing guidelines acceptable to clinician users; and 3) provide an acceptable response time. The present invention meets these needs.

SUMMARY

The invention includes an expert system for real-time decision support, and more particularly a computerized system for performing individual point of care diagnostics and generation of treatment plans using expert decision making processes. The invention also includes an expert system for data entry and processing that provides real-time decision support and an electronic record of an individual interview. The invention is particularly well suited to the medical industry.

Knowledge-based expert systems are computer programs which process an expert knowledge base, are able to make rational decisions and recommendations by inferring from this knowledge, and can justify their decisions and recommendations. Thus, these types of programs can process and apply human knowledge, enabling that knowledge to be used flexibly, often mimicking the outcome of a human decision-making process.

In a preferred embodiment of a decision support system, the invention implements a paperless electronic medical record which defines the essential and desirable elements of a clinical encounter and provides real-time medical decision support. The invention allows for rapid data input and retrieval, promotes continuous quality assurance practices, provides real-time patient assessments, and is readily adaptable to changing guidelines, all essential in the health care delivery environment. The system stores patient records and medical history in a relational database. The system includes medical decision support rules to assist in medical decision making. These rules select patient assessment information stored in the database to define patient problems, desired outcomes, and recommended medical interven- tions to achieve those outcomes. Medical decision recommendations are generated in real time to provide immediate decision support at the point of care. The preferred system includes a graphical user interface, which allows the user to easily and efficiently enter data and simultaneously presents suggestions for care and treatment in the form of suggested tests, diagnoses, and treatments.

At each step of decision making, the user is asked to verify the decisions of the expert system and is given the opportunity to override the decisions. The system records all clinical exceptions, which can be used to systematically modify the rules either generically or for specific sites and users. The preferred embodiment also provides structuring of a huge medical knowledge base into sections which emulate a set of expert care givers in particular specialties, resulting in recommendations to a user in acceptably short response times.

More particularly, one aspect of the invention includes an expert system for real-time decision support, comprising: an assessment system for receiving input data about a patient's health status; a problem identification system for analyzing the input data against a problem identification healthcare knowledge base and associating a probable diagnosis based on the analysis of the input data; a desired outcome generation system for analyzing the probable diagnosis against a desired outcome healthcare knowledge base and associating a desired outcome based on the analysis of the probable diagnosis; an intervention generation system for analyzing the probable diagnosis and desired outcome against an intervention healthcare knowledge base and associating an intervention based on the analysis of the probable diagnosis and desired outcome; and a recording system for analyzing patient recovery progress relative to desired outcomes. The invention further includes related methods and computer programs.

By providing computer-driven practice guidelines, the invention enables health care providers to deliver more uniform and cost effective medial care. The use of practice guidelines has been demonstrated to improve health outcomes. Clinical guidelines have been promoted as a way of decreasing variance in practice patterns and improving the quality and cost effectiveness of medical care. While numerous organizations have demonstrated the feasibility of developing guidelines, there are few proven methods for effectively disseminat- ing and implementing them. The details of one or more embodiments of the invention are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the invention will be apparent from the description and drawings, and from the claims.

DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram of the care giving process in accordance with the invention. FIG. 2 is a schematic block diagram of a data processing system in which the system may be employed.

FIG. 3 A is a functional block diagram of the invention. FIG. 3B is a flowchart showing the steps of determining a protocol.

FIG. 3C is a table showing an example of indicators that define Stable Angina. FIG. 3D is a table showing an example of different intervention types and their associated rules.

FIG. 3E is a table showing an example of different problems and their associated outcomes.

FIG. 4 is a data flow diagram of the core computational modules of the invention. FIG. 5 A is a diagram of hierarchical structure levels of a preferred tree data structure for storing patient data.

FIG. 5B is a representative patient tree data structure root node showing only major container nodes.

FIG. 6 is a diagram of specific tree data structure for storing patient data. FIG. 7 A is a general flowchart of the process for entering individual information and opening an individual chart.

FIG. 7B is a depiction of a graphical user interface that may be used to implement part of the functions set forth in FIG. 7A.

FIG. 7C is a depiction of a graphical user interface that may be used to display a patient's chart information in accordance with one embodiment of the invention.

FIG. 8A is a general flowchart and data diagram of the process for entering rapid scan data, determining problems, determining recommended interventions, and determining outcomes. FIG. 8B is a depiction of a graphical user interface that may be used to implement part of the functions of the Vital Signs submodule 804.

FIG. 8C is a depiction of a graphical user interface that may be used to implement part of the functions of the Physical Exam submodule 806. FIG. 8D is a depiction of a graphical user interface that may be used to implement part of the functions of the Pain Profile submodule 808 for chest pain.

FIG. 8E is a depiction of a graphical user interface that may be used to implement part of the functions of the Risk Factors submodule 810 for cardiac risk.

FIG. 9 A is a general flowchart of the process for entering assessment data, determin- ing problems, determining recommended interventions, and determining outcomes.

FIG. 9B is a depiction of a graphical user interface that may be used to implement part of the functions of the Allergies submodule 902.

FIG. 9C is a depiction of a graphical user interface that may be used to implement part of the functions of the Drug Profile submodule 904. FIG. 9D is a depiction of a graphical user interface that may be used to implement part of the functions of the History submodule 906.

FIG. 9E is a depiction of a graphical user interface that may be used to implement part of the functions of the Psych/Social module 908.

FIG. 9F is a depiction of a graphical user interface that may be used to display the results of the Determine Problems submodule 812.

FIG. 9G is a depiction of a graphical user interface that may be used to display the results of the Determine Interventions submodule 814.

FIG. 9H is a depiction of a graphical user interface that may be used to display the results of the Determine Outcomes submodule 816. FIG. 10A is a flowchart of the process for evaluating assessment data to determine individual problems, determine recommended interventions, and determine outcomes.

FIG. 10B is a depiction of a graphical user interface that may be used to implement part of the functions of the Labs submodule 1002.

FIG. 10C is a depiction of a graphical user interface that may be used to implement part of the functions of the ECG submodule 1004. FIG. 10D is a depiction of a graphical user interface that may be used to implement part of the functions of the ECG monitor submodule 1006.

FIG. 10E is a depiction of a graphical user interface that may be used to implement part of the functions of the Dx Tests submodule. Like reference numbers and designations in the various drawings indicate like elements.

DETAILED DESCRIPTION

Overview By providing computer-driven practice guidelines, the invention enables health care providers to deliver more uniform and cost effective medical care. While numerous organizations have demonstrated the feasibility of developing guidelines, there are few proven methods for effectively disseminating and implementing them.

In particular, the preferred embodiment of the invention overcomes the limitations of previous automated guidelines by integrating practice guidelines into the routine flow of healthcare so that recommendations are automatically delivered to the physician at the time the physician is seeing a patient. Rather than offer guidelines as an "off-line" reference tool, there are embedded in the electronic medical record, automatically passing patient clinical data through guideline rules and presenting the physician with guideline conclusions. The physician can then follow or diverge from the guideline advice. The system handles every step in the care of the patient.

By embedding the guidelines in a comprehensive electronic charting system, accessing the guideline requires no extra action by the physician, decreasing the likelihood that it will be perceived as an extra, time-consuming step. Furthermore, by presenting the guidelines at every individual encounter, the system works as a continuous reminder, eliminating problems that arise when traditional educational measures are relied upon to foster and maintain new behaviors.

In view of the problems and limitations of previous decision support systems, it is an object of the invention to emulate real-time expert healthcare at the point of care. The invention is based upon the concept that the critical knowledge regarding medical manage- ment does not exist in textbooks or other standard reference documents, but is best embodied in practice guidelines which include essential clinical data elements, rules for clinical decision making, and outcomes measures. The embedded guidelines are based upon expert review of the medical literature regarding evidence based medicine and augmented by review of expert panels. The invention is a comprehensive decision support system providing support for individual assessment, identifying individual problems and desired outcomes, recommending interventions and appropriate diagnostic tests and treatment, and reassessing to determine actual outcomes. This comprehensive decision support information is provided in real-time, at the point of care. It is believed that these features are unique in the industry. Another objective of the invention is to enforce compliance with established practice guidelines for improved quality of healthcare. The invention incorporates rule based medical practice guidelines and emulates expert care givers at the time of individual care. The invention ties each step of the individual care process to a specialty knowledge base with specific rules for that process. The invention produces a recommended diagnosis using a set of sophisticated rules applied to the essential clinical data entered into the system. The rules also determine recommended immediate treatments and diagnostic tests that are necessary. The preferred embodiment of the invention prompts the physician to order these treatments and tests, then records the results of these tests and changes in clinical condition brought about by treatment, and then refines the diagnosis as a result of this additional information. Finally, the invention makes a tentative final recommended diagnosis and suggests the appropriate treatment, including patient disposition, medications, schedules, and follow-up care, and creates detailed patient specific aftercare instructions. All these items can be verified by the health care provider.

It is a further objective of the invention to concurrently create an electronic individual record and provide the care giver with suggested cost-effective therapeutic interventions. A clinician can chart activities by simply selecting an intervention item for a permanent record of the encounter. The preferred embodiment of the invention provides access to individual records and history as well as to the rules and recommendations provided in the knowledge base. The user may enter assessment notes and intervention evaluation data and the system responds with recommendations, advice and suggestions tuned to the specific circumstances representing the current state of the individual and to the history of the individual.

In view of the foregoing and other objects of the invention, there is provided a data entry and real-time medical decision support system and method for generating an electronic medical record. The invention is an expert real-time decision support and electronic record system. In a specific embodiment, the invention provides real-time medical decision support and an electronic medical record of clinical encounters with patients. The invention is designed to improve the quality of healthcare by providing consistent standards of care to the care giver at the point of care by automatically enforcing and utilizing established protocols. The invention utilizes expert system technology to provide learned knowledge and facilitate decision-making during assessment, intervention, and evaluation processes. The invention is designed to help a clinician anticipate problems, symptoms, or side effects given a patient's health history, diagnosis, surgical procedures, treatment and medications. The preferred system will help a clinician track a patient's recovery time and assist in early detection of problems to prevent worsening which could lengthen a hospital stay.

Care Giving Process

FIG. 1 is a block diagram of the care giving process in accordance with the invention. Patient data 100, such as demographic data (e.g., age, sex, etc.) and medical history and symptom data (e.g., vitals, allergies, etc.), are input into an assessment module 102 which preferably includes a graphical user interface (GUI) that guides a practitioner through a desired set of questions drawn from an assessment knowledge base 104. Such questions may vary, depending on the answers to earlier questions (e.g., the questions posed to male patients may be different from the questions posed to female patients). The data gathered by the assessment module 102 is then processed by a patient problem identification module 106, which provides an initial diagnosis based upon inferences drawn from the patient's responses as applied to a patient problem knowledge base 108. For example, one diagnosis might be "possible pneumonia" based on inputs that include the patient's complaints (e.g., coughing, chest pain, etc.), temperature, blood pressure, respiratory function, prior medical history, etc. The system may also prompt the user to order additional tests and/or to give immediate treatment.

Thereafter, a desired outcome module 110 processes the initial diagnosis against an outcome knowledge base 112 and determines what desired outcomes may be available (e.g., "arrest infection", "eliminate tobacco use", etc.). An intervention module 114 then applies the patient problem against an intervention knowledge base 116 and determines an intervention plan, which may include a recommendation of further diagnostic tests, patient education, follow up, and specific treatment.

At all times, the system modules display results to a practitioner console 118 and permit a practitioner to override a diagnosis, desired outcome recommendation, and intervention plan.

Finally, the system measures the actual outcome by guiding a practitioner through the process during recurring visits to the practitioner, with the actual outcome data possibly affecting the other steps in the process as the patient's condition changes. In the preferred embodiment, an actual outcome module 122 provides a GUI that guides the practitioner through a desired set of questions drawn from an evaluation knowledge base 124.

The system provides access to individual records and history as well as to the rules and recommendations provided in the knowledge base. The user may enter assessment notes and intervention evaluation data and the system responds with recommendations, advice and suggestions tuned to the specific circumstances representing the current state of the individual and to the history of the individual.

Computing Environment

The invention may be implemented in a client/server architecture that utilizes modern communication technology to allow multiple care givers at client stations to simultaneously interface with the medical guidelines embedded in a network server. FIG. 2 is a schematic block diagram of a networked data processing system 200 in which the inventive decision support system may be employed. The representative system includes at least one server platform 202 and one client station 204, and can be scaled to include an arbitrary number of M server platforms 202 and N client stations 202. In the illustrated embodiment, the server platforms 202 include database storage 206 for the system's decision support rules and storage 208 for individual patient data records. Each client station 204 includes at least a data entry and display element 212, such as a CRT, mouse, and keyboard; an interface control element 214 for controlling communications with the server platforms 202; a processor 216, and local memory 218 (e.g., RAM, disk drive, etc.). Each client station 204 may be, for example, a standard personal computer which may be running under a standard operating system, such as the Microsoft Windows 95™ or Windows NT™ operating systems.

User interaction takes place in the client stations 204, each running a client application. Individual patient data may be transmitted to a server platform 202 across a network 210 from the client stations 204. The client stations 204 may interface with a server platform 202 located at the same medical facility by means of a local area network or may interface with a remote server platform 202 over a wide area network. The system may also provide portable access through the use of wireless keyboard or pen based systems deployed as client platforms 204. In the illustrated configuration, both wide area and local area networks may be used to provide direct access at the point of care using Common Object Request Broker

Architecture (CORBA) Transmission Control Protocol/Internet Protocol (TCP/IP) connectivity. In the preferred embodiment, distributed object request brokers, application development, and run-time management environments provide the security, transaction management and administrative management infrastructure needed to successfully scale the system for wider application by additional users and to develop, deploy, manage, and maintain an expanded system throughout its life cycle.

Processing of patient data using the decision support rules is preferably done on the servers 202 by means of a processor element 220 executing a server application. The server application interacts with the databases and performs rule processing. (Such processing could be executed on the client platforms 202 but the arrangement illustrated is the preferred configuration to implement potential mass data storage and high speed processing requirements, thereby allowing the use of smaller and less expensive client platforms which are likely to be utilized in large numbers in a typical healthcare environment). The server application may also be executed on a standard personal computer running under a standard network operating system. The databases may be implemented utilizing a commercially available relational database management system, such as Microsoft SQL, Oracle, or Sybase.

The invention may be implemented in hardware or software, or a combination of both. However, preferably, the invention is implemented in computer programs executing on programmable computers each comprising at least one processor, at least one data storage system (including volatile and non-volatile memory and/or storage elements), at least one input device, and at least one output device. Program code is applied to input data to perform the functions described herein and generate output information. The output information is applied to one or more output devices, in known fashion. Each program is preferably implemented in a high level procedural or object oriented programming language to communicate with a computer system. However, the programs can be implemented in assembly or machine language, if desired. In any case, the language may be a compiled or interpreted language.

Each such computer program is preferably stored on a storage media or device (e.g., CDROM or magnetic diskette) readable by a general or special purpose programmable computer, for configuring and operating the computer when the storage media or device is read by the computer to perform the procedures described herein. The inventive system may also be considered to be implemented as a computer-readable storage medium, configured with a computer program, where the storage medium so configured causes a computer to operate in a specific and predefined manner to perform the functions described herein.

Knowledge Base Processing

The preferred embodiment of the invention employs autonomous forward and backward chaining in the inference process to generate problems, interventions and out- comes. Problems are derived by matching input data to the expert knowledge base rules to generate terminal nodes which describe the problems and the associated rationale used to arrive at conclusions. Interventions and outcomes are generated by a similar process which matches the derived problems and the expert knowledge base rules to generate additional terminal nodes. FIG. 3 A is a functional block diagram of one embodiment of the invention. This embodiment operates on a collection of input data 300 that is evaluated by an inference engine using a set of rules to generate an output. The input data 300 consists of a patient's current medical state and medical history, which preferably is entered into the system by a care giver. An input translator 302 maps the input data into a set of hierarchical symptoms 304 in order to apply the rules in an inference engine 306. A protocol building tool 308 known as the ROOL Tool™ may be used to build a protocol 310 based on: identifying patient problems and indicators (e.g., vital signs and test results) to build a set of inference rules; identifying interventions to address the patient problems and building rules for the interventions; and identifying a set of desired outcomes. However, other means, such as hand-coding, may be used to build the protocol 310.

The resulting protocol 310 is expressed as a collection of IF-THEN (or IF-THEN- ELSE) rules based on the results of the indicators and interventions. In the preferred embodiment, rules have the following format: IF (indicator, operator, value) THEN problem(likelihood, urgency). Two or more IF statements may be concatenated into a single rule by simply selecting an existing rule and selecting an additional indicator, operator, and value set. An example of a rule is:

IF ("Vitals" JLast'7'List'V'Left Systolic Blood Pressure"."Any"."*"."Last" >= 140) {"Problems^,'."Hypertension"."Likelihood"."Last" := "Confirmed";} where ">=" is the operator and "140" is the value of the IF statement, and the THEN clause is set forth between curly brackets. The example rule checks the most recent left systolic blood pressure entered into the system for this patient, independent of the patient's physical position at the time, and confirming a problem of hypertension if the data is greater than 140. The rule structure is better understood with an understanding of the patient data tree structure. The patient data tree is constructed as a number of containers which are nodes attached to a root node. The root node is a unique patient identifier. The containers attached to the root node are logical elements of the problem domain. Further discussion of the patient data tree structure is set forth below in discussing FIGS. 5A and 5B. The above example of a rule is explained by reference to the patient data tree structure. The inference engine 306 handles the way in which protocol rules are combined to generate patient problem, intervention, and outcome decision trees 318. An output translator 312 maps these trees into respective patient problem, intervention, and outcome sets 314. The hierarchical symptom structure 304 and output sets 314 provide the basis for electronic records 316.

Protocol Building

The preferred protocol building tool 308 allows a user to interactively add rules to the knowledge database without requiring a software engineer to write, debug, and integrate new code into a new system. Further description of the preferred protocol building tool 308 is set forth in co-pending U.S. Patent Application Serial No. , entitled "Protocol

Building Tool For Medical Decision Support Expert System", filed January 12, 2000, assigned to the assignee of the present invention and incorporated by reference. However, the general procedure for determining a protocol in accordance with the invention may be summarized by reference to FIG. 3B, which is a flowchart showing the steps of determining a protocol:

• Define the problem (e.g. , "determining Stable Angina") (STEP 320).

• Define indicators for the problem, such as "chest pain - intermittent" (STEP 322).

Indicators are generally empirically determined, such as by clinical experience and/or statistical analysis of patient population data. In the preferred embodiment, such indicators are built into a tree structure representing patient assessment data. Each indicator preferably is qualified by "Any", "All", "First", or "Last". These qualifiers are used to decide what data will be included in building the protocol rule.

• Build a Problem Rule (STEP 324). The indicators are used to define the problem, the urgency of the problem, and the likelihood that the problem exists in light of the indicator. FIG. 3C is a table showing an example of indicators that define Stable Angina. • Define the Interventions (STEP 326). Once a user has built the problem rules, a user must determine the recommended interventions. In the preferred embodiment, interventions are divided into four categories: diagnostics, treatment, education, and follow up. Interventions are generally empirically determined, such as by clinical experience.

• Build the Intervention Rules (STEP 328). In the preferred embodiment, the defined interventions are also given likelihoods and urgencies, and used to build a set of intervention rules. FIG. 3D is a table showing an example of different intervention types and their associated rules.

• Define Desired Outcomes (STEP 330). Each patient problem has one or more desired outcomes. Outcomes are generally determined as desired results of the application of interventions to the original problem.

• Build Outcome Rules (STEP 332). In the preferred embodiment, the defined desired outcomes are used to build a set of outcome rules. FIG. 3E is a table showing an example of different problems and their associated outcomes. In the preferred embodiment, some of the outcomes may have associated empirically determined indicators to measure whether an outcome has been met. For example, in FIG. 3E, the outcome "Effective breathing pattern and optimal gas exchange" has an asterisk, indicating that associated indicators exist. Such indicators might include the following:

Effective breathing pattern and optimal gas exchange = "O₂ Saturation >=92" + "no cyanosis" + "no shortness of breath with or without exertion" + "breath sounds clear" + "RR>=10"

Rule Processing Data Flow

FIG. 4 is a data flow diagram of the core computational modules of the invention. Information broker 400, rule server 402, and patient data 404 modules interact with one other to provide the core computational facilities of the preferred embodiment of the present invention. Through these modules, individual patient data is converted from a relational database 406 to a format usable by the client application, and vice-versa. Protocol rules from a knowledge base 408 are processed using current institution-specific protocols to generate patient problem, intervention, and outcome sets.

More particularly, the information broker module 400 encapsulates the methods which interact with a relational patient database 406. Such encapsulation provides a layer of abstraction between the database implementation and the processes which interact with the database. In so doing, the information broker module 400 provides the means to easily configure the system interface with any of several commercially available database management systems. The information broker module 400 also performs a translation of patient data files from the relational tables stored in the patient database 406 to a patient data tree structure utilized by the patient data module 404 to perform rule processing, and translates updated patient data trees back to the stored relational tables. The patient data module 404 encapsulates the data pertinent to each patient represented in the data base. The rule server module 402 utilizes protocols stored in the knowledge base to perform rule processing on the data encapsulated by the patient data module 404 and generates problem, intervention, and outcome sets which become a part of the patient record and are handed back to the patient data module 404 for storage in the patient database 406.

In general, when a patient's chart is opened, the preferred embodiment of the invention invokes an initialization process which provides access to the knowledge base 408 and patient database 406. Rule processing is performed on a patient data tree structure which is constructed based on patient input data and is maintained by the information broker 400. Rule processing is initiated by user request once the appropriate data has been entered into the system. The rule server 402 builds a master list of all the relevant rule files to be processed. The rule processing function converts compiled rules into distinct logical units in an IF ... THEN ... ELSE ... block, where the expression inside each IF statement can be evaluated.

Once the pertinent rules are selected, the rule server 402 recursively traverses each patient data tree structure using left and right side token paths, and returns TRUE or FALSE based on the rules of operator evaluation. The rule server 402 determines where to proceed to process a next rule based on the result of each evaluation. Such rule-based systems are well- known in general. For example, a rule-based program called ILIAD has been under development at the University of Utah School of Medicine. ILIAD uses Bayesian reasoning to calculate the posterior probabilities of various diagnoses under consideration, given the findings present in a case. The Harvard Medical School is developing a decision support system called DXplain which acts on a set of clinical findings using a modified form of Bayesian logic to produce a ranked list of diagnoses which might be associated with the clinical manifestations. The Dallas VA Medical Center is developing an expert system designed to handle routine care in an epilepsy follow-up clinic. The system guides nurses in gathering patient histories and then generates preliminary progress notes along with a personalized patient information sheet. The 3M Corporation has developed a system called HELP which uses decision support rules to provide alerts/reminders, data interpretation, patient diagnosis, patient management suggestions and clinical protocols.

The preferred embodiment structures a huge medical knowledge base into sections which emulate a set of expert care givers in particular specialties, resulting in recommenda- tions to a user in acceptably short response times. For example, when a patient complains of chest pain and shows a history of cardiac problems, the system need access and apply only the pertinent rules from the knowledge base 408. Preferably, the knowledge base can be updated with the system in use, and knowledge bases may be exchanged between systems. A user preferably interacts with the preferred embodiment of the invention through a graphical user interface by entering symptoms and vital signs, and accesses the knowledge base and inference engine through a network to retrieve recommended interventions and outcomes. The preferred embodiment implements an electronic record charting capability which is accessible and updated during the individual contact. Data is preferably recorded using point and click mechanisms (e.g., by means of a mouse) or touch screen technology, and data is presented in graphical menus that are site-configurable. The software facilitates real-time decision support by utilizing the expert knowledge base 408 to identify individual problems, desired outcomes, and recommended interventions. The user is automatically alerted when critical conditions exist. Data Structures

FIG. 5 A is a diagram of hierarchical structure levels of a preferred tree data structure for storing patient data. A root node 500 includes a unique identifier for each patient. One or more container nodes 502 are attached to the root node 500. Container nodes 502 categorize patient data into logical groups. In the preferred embodiment, the number of container nodes 502 for an individual patient is variable and is defined as a user enters data into a patient's record. Each container node 502 may have n lower levels of descriptor nodes 504 for further categorization and characterization. The number of descriptor nodes 504 in each level of the tree and the depth of the tree is variable, based upon the data that is entered. In the preferred embodiment, the nodes of each patient tree data structure are stored as a vector of entries under the next higher level structural element. Each database entry at every node preferably includes a time stamp, a user name, and a data field. The order of the entries in the database depends upon the chronological order in which they were entered. In the preferred embodiment, if a particular node is no longer part of an individual's history, that node is not deleted from the tree but is simply marked as deleted in order to maintain an accurate historical record in the database.

FIG. 5B is a representative patient data tree structure root node 506 showing only major container nodes 508. In the preferred embodiment, the root node 506 contains a patient unique identifier (PatientID). The immediate "children" nodes of the PatientID root node 506 are container nodes 506. The illustrated container nodes 508 categorize patient data into logical groups which represent general patient information, rapid scan information (described below), assessments, a care plan of desired outcomes and interventions, and test information.

FIG. 6 is a diagram of specific tree data structure for storing patient data. In particular, FIG. 6 shows the logical elements of a "Vitals" tree data structure 600. The following rule (used as an example above) can be explained using this tree data structure:

IF ("Vitals"."Last"."List"."Left Systolic Blood Pressure"."Any"."*"."Last" >= 140) { "Problems" . "Hypertension" . "Likelihood" . "Last" := "Confirmed" ; }

1. "Vitals" is the container shown at the top of the tree. 2. "Last". "List" is a qualifier/token pair that points to the "List" which was created by the most recent "Last" vitals data entered into the intervention.

3. "Left Systolic Blood Pressure" is one of the four nodes attached to each "List".

4. "Any"."*" is a qualifier/token pair that references "Left Systolic Blood Pressure" collected in "Any" of three positions (sitting, standing, or lying down).

5. "Last" points to the most recent number attached as a node to any position.

6. >= is the greater than or equal to operator.

7. 140 is the comparison value.

8. "Problems". "Hypertension" establishes hypertension as a patient problem if the foregoing evaluation is true and, if hypertension is not currently recognized as a patient problem, a "Hypertension" node is attached to the "Problem" container of the patient tree.

9. "Likelihood". "Last" := "Confirmed" replaces the most recent "Likelihood" of the problem with likelihood "Confirmed". Hence, the example rule is checking the most recent left systolic blood pressure entered into the system for this patient, independent of the patient's physical position at the time, and confirming a problem of hypertension if the data is greater than 140.

Further description of the preferred data structures is set forth in co-pending U.S. Patent Application Serial No. , entitled "Expert System Data Storage Format and Conversion System", filed January 12, 2000, assigned to the assignee of the present invention and incorporated by reference.

Patient Data Entry

FIG. 7 A is a general flowchart of the process for entering individual patient informa- tion and opening an individual chart in accordance with a preferred embodiment of the invention. FIG. 7B is a depiction of a graphical user interface that may be used to implement part of the functions set forth in FIG. 7A.

Upon entering the system (STEP 700), a health care user has the option to select an existing individual (STEP 702) or add a new individual (STEP 704). If a new individual is added, the user is prompted to enter demographic data (STEP 706). The user is then prompted to enter the patient's chief complaint or follow up information (STEP 708). In the preferred embodiment, menus for "Chief Complaints" (e.g., "bleeding", "nausea, vomiting", "fever", etc.) or "Follow Up For" (e.g., "angina", "hypertension", etc.) are presented, with entries selectable by simply pointing and clicking. Once these actions are complete, the user proceeds to open an individual chart by clicking, for example, an "open current chart" button (STEP 710). At this point, the system retrieves the individual patient information from the patient database 406 and the chart is opened. In the preferred embodiment, the user is then presented with a new graphical interface. FIG. 7C is a depiction of a graphical user interface that may be used to display a patient's chart information in accordance with one embodiment of the invention. In particular, information about the patient may be shown in windows 730 that allow for the charting of vital sign measurements, medical history, Patient Problems, Physical Exam, and Labs/Diagnostic Tests. This embodiment also provides access to any of several modules, such as Patient 750, Rapid Scan 800, Assessment 900, Care Plan 752, Diagnostic Tests 1000, and Help 754 (STEP 712). A description of the function of the preferred embodiment of each of these modules is set forth below.

The Patient module 750 preferably provides mechanisms to open another individual chart, review admission data, print the electronic record, close the chart, discharge the individual, or exit the application.

The Care Plan module 752 provides a direct path to the Determine Problems 812, Determine Interventions 814, and Determine Outcomes 816 submodules described below, preferably using a pull down menu. The user simply clicks one of the three items and the appropriate processing described below is invoked, utilizing data that has been provided to the system.

The Help module 754 provides a pull down menu which offers access to files describing how to operate the system and displays the version number of the executable software. Numerous help files may also be provided through point and click mechanisms in each of the modules which are described below. Rapid Scan

In the preferred embodiment, the Rapid Scan module 800 was developed to enter patient assessment data normally associated with urgent cardiac care. Additional focused assessment modules may be developed and associated with additional disciplines. The Rapid Scan module 800 presents an additional graphical interface which provides the mechanisms illustrated in FIG. 8A. More particularly, FIG. 8A is a general flowchart and data diagram of the process for entering rapid scan data, determining problems, determining recommended interventions, and determining outcomes. Selection of Start Rapid Scan function 802 leads the user through a series of submodules. Each submodule presents a menu which allows the user to select data entry items for a patient. Data entry can be through a keyboard or by free text entries. A button keypad may also be provided to enter data through point and click operations using embedded numeric keys.

The Start Rapid Scan function 802 starts the user with a Vital Signs submodule 804. The Vital Signs submodule 804 preferably presents a menu which allows the user to select Enter Vital Signs in order to enter such data as blood pressure, pulse rate, respiratory rate, temperature, height, weight, etc. The user is also presented with options to select Edit Last, which displays the last set of vital signs recorded for the patient; Cancel to return to the Rapid Scan window; Notes, to include any appropriate notations; Date and Time to override the current date and time, which are automatically displayed; or Record to store the data and proceed to the Physical Exam submodule 806. FIG. 8B is a depiction of a graphical user interface that may be used to implement part of the functions of the Vital Signs submodule 804.

The Physical Exam submodule 806 presents another set of menus which allow the user to select menu entries for physical Observations, Abnormal Signs, Mentation, and Other parameters of interest. The user is also presented with options to Cancel and return to the

Rapid Scan window; proceed to the Pain Profile submodule 808 or Risk Factors submodule 810; or Close the exam, which causes the data to be recorded and executes a Determine Problems submodule 812. FIG. 8C is a depiction of a graphical user interface that may be used to implement part of the functions of the Physical Exam submodule 806. The Pain Profile submodule 808 presents another set of menus which allow the user to select menu entries for a pain profile, such as activities that cause Precipitation of pain; Location of the pain; actions that cause Relief of the pain; the Character of the pain; and a Time Line, which includes parameters such as pain duration, onset, intensity, and pattern. The user is also presented with options to Cancel or Close, as in the Physical Exam sub- module, or proceed to the Risk Factors submodule 810. FIG. 8D is a depiction of a graphical user interface that may be used to implement the Pain Profile submodule 808 for chest pain. Other interfaces may be used to implement part of the functions of the Pain Profile sub- module 808 for other types of pain, as determined by applying the patient's complaint data to the assessment knowledge base 104 (see FIG. 1).

The Risk Factors submodule 810 presents another set of menus which allow the user to select menu entries regarding historical and physical Factors affecting risk (e.g., smoking, family history of similar disease, etc.), and a set of Recent Problems which may exacerbate additional problems. The user again has the option to Cancel or Close as is the previous modules. FIG. 8E is a depiction of a graphical user interface that may be used to implement the Risk Factors submodule 810 for cardiac risk. Other interfaces may be used to implement part of the functions of the Risk Factors submodule 810 for other types of risk, as determined by applying the patient's complaint data to the assessment knowledge base 104 (see FIG. 1). Once sufficient available data is collected, the Determine Problems 812, Determine Interventions 814, and Determine Outcomes 816 submodules are executed in the manner described below.

Assessment

The Assessment Module 900 depicted in FIG. 7A presents an additional graphical user interface which provides the mechanisms illustrated in FIG. 9A. More particularly, FIG. 9 A is a general flowchart of the process for entering assessment data, determining problems, determining recommended interventions, and determining outcomes. The user may enter data through point and click mechanisms on pull down menus and by free text entries. A Record button may be provided to save each entry, or saving may be automatic. All lists preferably are completely configurable to the requirements of individual facilities. An Allergies submodule 902 permits entry of a history record of allergies, including specific substances or events and reactions, and categorizes each allergy, for example, as "environmental", "food", or "medication" (this may be done by means of a simple lookup table). FIG. 9B is a depiction of a graphical user interface that may be used to implement part of the functions of the Allergies submodule 902.

A Drug Profile submodule 904 permits entry of a drug history record of drug use. The user may enter medication, indication, dose, route, frequency, history of usage, ordering and administering clinicians as applicable, whether or not the drug was taken as prescribed, and any additional notes desired. FIG. 9C is a depiction of a graphical user interface that may be used to implement part of the functions of the Drug Profile submodule 904.

A History submodule 906 permits entry of a complete patient medical history (PMH). In the preferred embodiment, a comprehensive history can be recorded which may include entries related to cardiovascular, pulmonary, neurological, urinary, peripheral vascular, gastrointestinal, musculoskeletal, reproductive, surgical, immunological, accidents, family history and additional maladies. In the preferred embodiment, the comprehensive list encompasses all areas of medicine and is completely configurable to the requirements of individual facilities. FIG. 9D is a depiction of a graphical user interface that may be used to implement part of the functions of the History submodule 906. Multiple windows may be required to record a complete PMH. A Psych/Social module 908 permits entry of a complete psychological and sociological profile history. The profile may include information on daily habits, diet, lifestyle, an environmental survey, immunization history, exercise habits, interpersonal relationships, and additional factors. FIG. 9E is a depiction of a graphical user interface that may be used to implement part of the functions of the Psych/Social module 908. A Vital Signs submodule 910 provides an additional path to the Vital Signs sub- module 804 described above as a portion of the Rapid Scan module 800. Similarly, a Physical Exam submodule 912 provides an additional path to the Physical Exam submodule 806 described above as a portion of the Rapid Scan module 800. A Review of Systems 914 submodule provides a comprehensive list of items for which diagnostic data and observations may be entered. The list may include head, ears, eyes, skin, nose, throat, neck, respiratory, reproductive, psychological, and additional items.

Once a complete assessment or any submodule within the assessment is complete, the user closes the exam, which causes the data to be recorded. Thereafter, the Determine

Problems 812, Determine Interventions 814, and Determine Outcomes 816 submodules are executed in the manner described below.

Determining Problems, Interventions, and Outcomes The Determine Problems submodule 812 accesses the knowledge base 408 and engages an inference engine in the rule server 402 to display a set of possible problems based on the entered data for the patient. Each problem displayed has an associated likelihood that may include an action recommendation and status. The likelihood may categorize each problem, for example, as "confirmed", "probable", and "possible". Action recommendations may be, for example, either "immediate" or "stat". Status entries preferably are included to provide the user the ability to record agreement or disagreement with the inference engine recommendations. Possible status entries may include "confirmed", "potential", "controlled", "ruled out" and "resolved". In the preferred embodiment, status entries invoke a problem confirmation display that requires the user to enter the name of the confirming diagnostician. Also, the problem list may be filtered to display only a user-defined set of likelihood and status entries. Entry buttons are included which allow the user to append notes regarding any recommendation or status entries. The graphical display also includes a "rationale" button that provides the opportunity to scrutinize the interpretation provided by the inference engine. A date and time field preferably is included to allow the user to record entries other than for the current date and time. FIG. 9F is a depiction of a graphical user interface that may be used to display the results of the Determine Problems submodule 812.

The Determine Interventions submodule 814 displays all interventions recommended by the inference engine and provides a set of controls (e.g., buttons) which allow the user to exercise point and click mechanics to include the status and time tag associated with each intervention. The intervention list may be filtered to display only those actions recommended for a particular problem. A filtering mechanism is also included to display recommended interventions according to type, which may include diagnostics, treatment, education, and follow up. FIG. 9G is a depiction of a graphical user interface that may be used to display the results of the Determine Interventions submodule 814. The Determine Outcomes submodule 816 displays all desired outcomes and also provides a set of controls (e.g., buttons) which allows the user to exercise point and click mechanics to include the status and time tag associated with each outcome. Status information may include entries such as "deteriorated", "improved", "unchanged", or "not applicable". In the preferred embodiment, entry buttons are included which allow the user to append notes regarding any outcome. Entry buttons are also included to allow the user to include additional desired outcomes to the list generated by the inference engine. FIG. 9H is a depiction of a graphical user interface that may be used to display the results of the Determine Outcomes submodule 816.

Accordingly, FIGS. 9B-9H depict graphical user interfaces that together define a patient medical history chart. Data preferably is entered for the patient, whose name is shown at the top left of each chart, by simple point and click mechanisms. Each entry selected is highlighted during the point and click process. Note in FIG. 9D that window slider bars 920 are defined down the right edges of the windows 922 for Cardiovascular, Renal/Urinary, and Gastrointestinal information. These windows 922 include more entries than are visible in the allotted space. The additional entries are revealed by simply clicking and dragging the window slider bar or clicking the up/down arrows. Also note the tabs 924 across the top of the chart. Additional windows are opened by simply clicking the appropriate tab 924 for Allergies, Drug Profile, three additional Patient Medical History charts, or two different Psych/Social charts. Windows and buttons may be included in various displays (see, e.g., FIG. 9F) to add clinical notes, view the system rationale for a selected problem, or to add additional problems by creating supplementary terminal nodes at the user's discretion.

Diagnostic Testing

The Diagnostic Tests module 1000 depicted in FIG. 7A presents an additional graphical interface which provides the selections illustrated in Figure 10A. More particularly, FIG. 10A is a flowchart of the process for evaluating assessment data to determine individual problems, determine recommended interventions, and determine outcomes. In the preferred embodiment, point and click buttons are provided to delete, correct and record data. A button keypad may also be provided to enter data through point and click operations using embed- ded numeric keys. All lists preferably are completely configurable to the requirements of individual facilities.

A Labs submodule 1002 preferably provides pull down menus for lab orders and lab names, with entry windows for test dates and test results. In the preferred embodiment, test results are entered in a window with the range of normal values displayed alongside the user entered results. Subsequent test names and normal range of values are automatically displayed as each set of test data is recorded. FIG. 10B is a depiction of a graphical user interface that may be used to implement part of the functions of the Labs submodule 1002.

An ECG submodule 1004 preferably provides a comprehensive list of electrocardiograph test related items for selection. The list may include sinus rhythm rate, ventricular rhythm/rate, ECG interpretation, atrial rhythm/rate, findings, junctional rhythm/rate, bundle branch block, and an additional window for miscellaneous entries. FIG. 10C is a depiction of a graphical user interface that may be used to implement part of the functions of the ECG submodule 1004. The ECG submodule 1004 applies to data collected once or at periodic intervals. An ECG monitor submodule 1006 preferably provides a comprehensive list of items for selection. The list may include sinus rhythm/rate, blocks, ectopy, atrial rhythm/rate, ventricular rhythm/rate, junctional rhythm/rate, other rhythm, and ECG monitor interpretation. FIG. 10D is a depiction of a graphical user interface that may be used to implement part of the functions of the ECG monitor submodule 1006. The ECG monitor submodule 1006 applies to a continuous monitoring device.

Selections for "Echo" 1008, "X-ray" 1010, and "ETT" 1012 preferably provide access to a Diagnostic (Dx) Tests submodule which presents a comprehensive list of diagnostic tools for selection. The list may include echochardiographic or sonographic interpretation, X-ray interpretation, an ETT (Exercise Treadmill Test) pass/fail entry, and a window to enter test dates. FIG. 10E is a depiction of a graphical user interface that may be used to implement part of the functions of the Dx Tests submodule.

As will be appreciated by the comprehensiveness of the medical data captured by the preferred embodiment of the invention, and the ease of use of the software implementing the invention, the invention provides a primary care giver virtually instant access to every aspect of a patient's medical history, including diagnosis, prescribed interventions, and desired outcomes. Thus, the use of an expert system provides a means for real-time point of care medical decision support. The invention also includes an expert system for data entry and processing that provides an electronic record of an individual interview. Use of the decision support system and electronic medical record to provide real-time medical decision support driven by evidence-based practice guidelines and augmented by expert opinion should result in more uniform and appropriate application of medical care which results in improved patient outcomes at reduced cost.

Embodiments of the invention include methods and corresponding computer programs that permit a computer system to collect and process patient data from a plurality of sites and perform statistical, cross-patient analyses. For example, multiple patient data can be processed to consolidate information indicative of any one of:

• a trend analysis of standard quality of care metrics, for quality of care visibility;

• evidence of a disease or chemical threat epidemiology in a population, thus providing real-time observability of such events;

• correlations between probable diagnoses, desired outcomes, and patient recovery progress, in order to change or fine-tune the knowledge base rules and standards of care for each health condition; or

• time spent by each provider with each patient, for cost visibility.

Further embodiments of the invention include methods and corresponding computer programs that provide a training or educational mode wherein the input data comprises constructed patient scenarios rather than data about an actual patient being treated. Such a system includes computer based training that provides the following functions: • requiring a student operator to determine a correct answer at each step of an analysis process before the student operator is allowed to proceed to a next step in the analysis process;

• providing rationale and help screens for interactive learning by the student operator;

• recording student operator attempts and success rates at providing such correct answers; and

• providing a test phase during which a testing student operator's knowledge is tested (e.g., presentation of multiple choice or true- false questions).

A number of embodiments of the invention have been described. Nevertheless, it will be understood that various modifications may be made without departing from the spirit and scope of the invention. Accordingly, other embodiments are within the scope of the following claims.

Claims

WHAT IS CLAIMED IS:

1. An expert system for real-time decision support, comprising: (a) an assessment system for receiving input data about a patient's health status; (b) a problem identification system for analyzing the input data against a problem identification healthcare knowledge base and associating a probable diagnosis based on the analysis of the input data; (c) a desired outcome generation system for analyzing the probable diagnosis against a desired outcome healthcare knowledge base and associating a desired outcome based on the analysis of the probable diagnosis; (d) an intervention generation system for analyzing the probable diagnosis and desired outcome against an intervention healthcare knowledge base and associating an intervention based on the analysis of the probable diagnosis and desired outcome; and (e) a recording system for analyzing patient recovery progress relative to desired outcomes.

2. The expert system of claim 1, wherein the intervention includes defining for the patient at least one of diagnostic tests, specific treatment, education, or follow up.

3. The expert system of claim 1, wherein each knowledge base incorporates rule-based medical practice guidelines for enforcing compliance with established medical practice guidelines.

4. The expert system of claim 1, wherein the problem identification system determines whether immediate treatments and diagnostic tests for the patient are necessary, and prompts a medical care giver to provide such necessary treatments and diagnostic tests.

5. The expert system of claim 1, further including a record creation system for creating an electronic record for each patient concurrent with a patient encounter, the electronic record including the input data, probable diagnosis, desired outcome, and intervention.

6. The expert system of claim 1, wherein each of the assessment system, problem identifi- cation system, desired outcome generation system, intervention generation system, and actual outcome recording system include a computer-implemented graphical user interface for ease of data entry and display, and substantially all medical categorizations are selectable from pre-defined lists.

7. A method for implementing real-time medical decision support, comprising the steps of: (a) receiving input data about a patient's health status; (b) analyzing the input data in a computer against a problem identification healthcare knowledge base and associating a probable diagnosis based on the analysis of the input data; (c) analyzing the probable diagnosis in a computer against a desired outcome health- care knowledge base and associating a desired outcome based on the analysis of the probable diagnosis; (d) analyzing the probable diagnosis and desired outcome against an intervention healthcare knowledge base and associating an intervention based on the analysis of the probable diagnosis and desired outcome; and (e) analyzing patient recovery progress relative to desired outcomes.

8. The method of claim 7, wherein the step of associating an intervention includes the step of defining for the patient at least one of diagnostic tests, specific treatment, education, or follow up.

9. The method of claim 7, wherein each knowledge base incorporates rule-based medical practice guidelines for enforcing compliance with established medical practice guide- lines.

10. The method of claim 7, wherein the step of associating a probable diagnosis includes the step of determining whether immediate treatments and diagnostic tests for the patient are necessary, and prompting a medical care giver to provide such necessary treatments and diagnostic tests.

11. The method of claim 7, further including the step of creating an electronic record for each patient concurrent with a patient encounter, the electronic record including the input data, probable diagnosis, desired outcome, intervention, and actual outcome.

12. The method of claim 7, further including the step of providing a computer-implemented graphical user interface for ease of data entry and display, and accepting entry of substantially all medical categorizations as selections from pre-defined lists.

13. The method of claim 7, further including the steps of: (a) receiving input data about a patient's health status; (b) analyzing the input data in a computer against an outcome evaluation healthcare knowledge base and associating an outcome evaluation based on the analysis of the input data; (c) comparing the outcome evaluation to the desired outcome to determine a patient's intervention progress.

14. A computer program, stored on a computer-readable medium, for implementing real-time medical decision support, comprising instructions for causing a computer to: (a) receive input data about a patient's health status; (b) analyze the input data in a computer against a problem identification healthcare knowledge base and associate a probable diagnosis based on the analysis of the input data; (c) analyze the probable diagnosis in a computer against a desired outcome healthcare knowledge base and associate a desired outcome based on the analysis of the probable diagnosis; and (d) analyze the probable diagnosis and desired outcome against an intervention health- care knowledge base and associate an intervention based on the analysis of the probable diagnosis and desired outcome; and (e) analyze patient recovery progress relative to desired outcomes.

15. The computer program of claim 14, wherein the instructions for causing the computer to associate an intervention includes instructions for causing the computer to define for the patient at least one of diagnostic tests, specific treatment, education, or follow up.

16. The computer program of claim 14, wherein each knowledge base incorporates rule- based medical practice guidelines for enforcing compliance with established medical practice guidelines.

17. The computer program of claim 14, wherein the instructions for causing the computer to associate a probable diagnosis includes instructions for causing the computer to deter- mine whether immediate treatments and diagnostic tests for the patient are necessary, and to prompt a medical care giver to provide such necessary treatments and diagnostic tests.

18. The computer program of claim 14, further including instructions for causing the computer to create an electronic record for each patient concurrent with a patient encounter, the electronic record including the input data, probable diagnosis, desired outcome, intervention, and actual outcome.

19. The computer program of claim 14, further including a graphical user interface for ease of data entry and display.

20. The computer program of claim 19, further including instructions for causing the computer to accept entry of substantially all medical categorizations as selections from pre-defined lists.

21. The computer program of claim 14, further including instructions for causing the computer to: (a) receive input data about a patient's health status; (b) analyze the input data in a computer against an outcome evaluation healthcare knowledge base and associate an outcome evaluation based on the analysis of the input data; (c) compare the outcome evaluation to the desired outcome to determine a patient's intervention progress.

22. The computer program of claim 14, further including instructions for causing the computer to collect and process patient data from a plurality of sites and consolidate from such patient data information indicative of any one of: (a) a trend analysis of quality of care metrics, for quality of care visibility; (b) evidence of a disease or chemical threat epidemiology in a population; (c) correlations between probable diagnoses, desired outcomes, and patient recovery progress; or (d) time spent by each provider with each patient, for cost visibility.

23. The computer program of claim 14, including further instructions for causing the computer to provide a training mode wherein the input data comprises constructed patient scenarios rather than data about an actual patient being treated, and the instruc- tions: (a) require a student operator to determine a correct answer at each step of an analysis process before the student operator is allowed to proceed to a next step in the analysis process; (b) provide rationale and help screens for interactive learning by the student operator; (c) record student operator attempts and success rates at providing such correct an- swers; and (d) provide a test phase during which a testing student operator's knowledge is tested.