MEDKAT: multiple expert DELPHI-based Knowledge Acquisition Tool

~ - y ~bull ~ middot ~ I )

r

(

(

PROCEEDINGS

SECOND ANNUALaem NORTHEAST REGIONAL CONFERENCE

INTEGRATING THE INFORMATION WORKPLACE

THE KEY TO PRODUCTIVITY

GENERAL CHAIRMAN BRYAN KOCHER

PROGRAM CHAIRMAN PAUL KORROL

acmAbullbullocUitlonfCompul ingM~chinary

~~ ~J~~ bull J_-bullbull _ ~ gt0t4 bull o(_( ~-t~ 04tgtJ~o(tt_ootol~iCIIamp koc~~~lOCMoo( -t-~ ~ Iut~~ -lJvwW~ oJmiddotoI - - -lt ~ ~~_-~ ~ bull - ~ = bull bull bull bull ~ bull bull bull bull bull bull bull bull ~ bull bull bullbull bull bull-r ~ bull bull bull

~ ~ - ~~ ~ ~ middot 1middot ~ - ~ - bull s - - -

- ~ ~ ~ - t

- -

The SECOND ANNUALacm NORTHEAST REGIONAL CONFERENCE

INTEGRATING THE INFORMATION WORKPLACETHE KEY TO PRODUCTIVITY

OCTOBER 28-30 middot1985

(Sheraton-Tara Hotel

Framingham Massachusetts

~

v JAGANNATHANmiddotArtificial Intelligence CenterAdvanced Technology Application DivisionBoeing Computer ServicesBellevue Washington

INTRODUCTION

AS ELMJGHRABYEngineering Mathematics andComputer Science DepartmentUniversity of LouisvilleLouisville Kentucky

( )

Most current day expert systems involve capturing the knowledge of the problemdomain from one expert However capturing differing viewpoints to bear on a givenproblem has its advantages Recognition of this fact has lead to research in usingmultiple domain experts (Mittal and Dym 1985) Getting a group of experts toprovide expertise on a given problem is a tricky issue The problem is compoundedwhen the experts disagree on some aspect of the problem (Jagannathan et al 1982)When obtaining knowledge from a single expert generally addressed issues are (1)determining the relevancy of the application (2) determining the degree of expertiseavailable (3) identifying user interface questions and (4) selecting representationand reasoning schemes to be adopted In addition to the above the acquisition ofknowledge from multiple experts requires that group dynamics be studied

In the past knowledge acquisition was done strictly by interviewing the expertsSeveral tools are available now to facilitate this process of obtaining the knowledgefrom an expert (Hayes-Roth et al 1983 Boose 1984 Caviedes 1984) Howeverhitherto the issue of building a tool that would help in the process of obtainingknowledge from multiple experts has not been addressed MEDKAT MultipleExpert Delphi-Based Knowledge Acquisition Tool has been developed to assist withthis process (

The knowledge acquisition scheme discussed in this paper is adapted from group tlldecision-making techniques Computer-assisted knowledge acquisition frorrimultiple experts (typically from a small group) lends itself to the use o~elphitechnique This technique evolved from the Rand Corporation studies on groupdecision making It has been applied successfully in a wide variety of problemssuch as technology forecasting and building multiattribute decision models (Amara1975 Dalkey and Helmes 1963 Elmaghraby 1978 Enzer 1970 and Linstone andTuroff 1975)

The main features of the Delphi methodology are

1 Anonymity sThe response of each expert is not disclosed to other expertwithin the group

2 Information feedback

Group response is derived from all experts after each round of informationgathering Then it is made available to the individual ex perts for reshyevaluation of their earlier decisions

3 Controlled iterations

The degree of consensus reached at the end of each round determines thepossible need for further iterations

This work done while author-was associated with the University of Louisville

30

t

( ) The knowledge acquisition phase is based on a representation model that combinesthe domain knowledge in the fonn of a decision tree with associated rule bases(Caviedes 1984) A decision problem is used here to explain the strategy used

The logical representation used in capturing the knowledge of each expert is doneusing a hierarchy of frames The representation scheme chosen agrees in principleto organizing knowledge as a hierarchy of specialists proposed by Bylander Mittaland Chandrasekaran in their Conceptual Structures Representation Language(1983) A portion of the decision-process hierarchy for the selection of simulationtechniques is presented in the following figure (Elmaghraby and Jagannathan 1985J agannathan and Elmaghraby 1984)

l~c E jO P ~ ~

~ j~ O IO i

--

( OM Pl EVl f Y

Fcross

HUMAN middotRA l ED

~JE middotWE ICHT

l~ EmiddotWE IGH I

--

-

FACTORS

)NMAN-RELATED1

PRODUCT EXPENSIVE

ACCURACY

SUBCATEGORIES~~===============_--_ DIGITAL ~

-~~D

ANALOG-_

SIMULATIONTECHNIQUES

FAGORSSUBCATEGORIES

Sample Knowledge Frame

The frame representation as demonstrated in the figure allows recursive definitionssuch as defining subcategory frames to have the same structure as the higher levelproblem frame The problem specified at the top-level frame is assumed to have adecision value for each subcategory The factors frame for a subcategory is therepresentation of its decision rule

For example the decision to use simulation techniques in the figure is based onrules based on the factors human rated product expensive and accuracyThe true and false weights represent the importance of each factor and are treatedappropriately at the inference level (Jagannathan et al 1982 Sandel l 1984) Thebasic idea is that there is a hierarchy of decision nodes modeled by the subcategoryslot of frame at any given level Orthogonal to this there are rules determining thevalidity of each node based on factors germane to this node

Iv

middot 31

~

~ ~-gt-~---~~~ ~~~~~ ~~~~~ ~ - ~

-

- bull -

_ ~ -f bull bullbull bull - - 0 bull bull ~ ~ ~ 0-

~ - - ~ 1 bullbull ~ bull

~NOWEDGEiNTEGmiddotRATION middot -

-

bull _ ~ 0

~ 0 bull

The objective is to ~btain a consensus model of the problem do~ain integrating the (knowledge acquired from each expert The knowledge integration scheme is based on the Delphi technique The main steps in the algorithm for building the systemare outlined below

Phase 1Obtain from coordinator1 Problem domain (eg simulation techniques)2 List ofexperts in the domain to be consulted

Phase 2Obtain subcategories (to build model) from each expert

Apply integration rules to combine individual responses Example rules to aidthis process include1 Synonym pruning2 Union operation to obtain compre hensive list of categories

Provide as feedback to individual experts the combined model at this levelAllow each user to changemodify hisher initial responses

Repeat process until consensus is reached

Phase 3Obtain from each expert1 The set of factors pertinent to the current category2 For each factor the weight of the factor as regards the category when the

factor is known to be true and when it is known to be false

Obtain integrated model from the responses of all experts by applying a set ofrules Sample rules include1 For the factors themselves use rules as detailed in phase 22 For the weights of factors keep track of the upper and lower quartiles of

responses and use solution integration in the form of weighted averaging


(

Phase 4Obtain from each expert for each factor at this level the rulers) to evaluate thatfactor

Give the merged set of rules to each expert as feedback and ask each user to rankthe rules in the set

Prune the rules and repeat the process for consensus

Phase 5Apply a depth-first (recursive) or breadth-first strategy to build incrementallythe complete knowledge base

32

~ - - ~

-

~

~ ~ -

-

The current implementation of MEDKAT incorporates a ll of the above phasesexcept phase 4 It does not have an interface to a dictionary to do synonym pruningThe individual expert however can recognize a synonym and react by removingitorrenaming the variables involved

The entire system is built using IQUSP on an ffiM PC and GENIE (Sandell 1984)GENIE a general-purpose inference engine developed at Vanderbilt University is aframe-based system and allows the representation presented earlier Eachexpertsresponse is stored independently thereby providing a history of responses that leadto the final knowledge base This history can be examined in case of discrepanciesbetween experts

The sample knowledge frame presented earlier is a portion of the actual framegenerated using MEDKAT The sample case selected represents highest-level rulesin a currently active application of expert systems to the problem of simulationlanguage selection and categorization (Elmaghraby and Jagannathan 1985)

CONCLUSION

(

A knowledge acquisition strategy from a group of experts has been presented Themethodology as proposed has further implications in the application of expertsystems to cooperative problem solving (Jagannathanand Elmaghraby 1985) Theresearch in adapting the Delphi technique has raised several important issues

One issue is the determination of what applications allow this form of knowledgeacquisition The methodology seems well suited for classificatory problems Thequestion is Can the technique be adapted to obtain knowledge for planning typeproblems

Another issue not addressed here is the user interface requirements Currentlyeach expert is required to provide information in a strictly orderly fashion For themethodology to be practical it should allow for concurrent acquisition which wouldin turn allow the experts to input information at varying degrees of detail and paceBut this raises a issue that the orderly input (providing information for one node ata time) circumvents The question is how to provide enough intelligence to theDelphi Coordinator module to take care ofsubsumption problems - that is when oneexperts subcategory appears as another experts category at a different level in thetree

In conclusion MEDKAT appears to present a promising approach to obtainingknowledge from multiple experts Particularly since the approach is based on aproven methodology already tested and used in group decision making

33

~ ~

rmiddot1 0 J 0

- ~ - - - -~~~~~~~ middot~~~-~~77~l~~~~middot~middot middot~middotT~~ middot middot~jr~~~~~ ~~~~fs~~_~=---r~~

-i 0 bull bull I bull o bull bull bullbull bull ~ o bull bull bull bullbull bull bull bull bull bull -

_ 0 bull bull ~ bull bull bull bull - 0

~ ~ ~ ~ gt~ o~ ~ - _ - - gt ~ _ ~~ ~ ~~

bull ~ bull bullbull bull bull bull bull bull bull l bullbullbullbull bull 0 I bull bull bull bull bull bull bull

o bull 0middotbull bull bull - o r -

REFERENCES

Amara R Some methods of future research Institute for the Future WP-23December 1975

Boose J H Personal Construct Theory and the Transfer of Human ExpertiseProceedings ofAAAI-84 August 1984

Bylander T Mittal S and Chandrasekaran B CSRL A Language for ExpertSystems for Diagnosis Proceedings ofIJCAI1983

Caviedes J E C MEDKAS A medical knowledge engineering assistant PhDdissertation Vanderbilt University December 1984

Dalkey N and Helmes 0 An Experimental Application of the Delphi Method tothe Use ofExperts Management Science 9 No3 April 1963

Enzer S Delphi and Cross-Impact Techniques Kadansha Ltd Tokyo Japan 1970~

Elmaghraby A S and Basic Needs Index Delphi Application Proceedings of theWisconsin-Madison 1978

(

~

Elmaghraby A S Jagannathan V An Expert System for Simulationists inA rtificial Intelligence Graphics and Simulation Graham Birtwistle (ed) SCS 1985

Hayes-Roth F Waterman D A and Lenat D B (editors) Building ExpertSystems Addison-Wesley 1983 (

Jagannathan V Bourne J R Jansen B H Ward J W Artificial IntelligenceMethods in Quantitative Electroencephalogram Analysis Computer Programs inBiomedicine 15 1982

J agannatha n V ElmaghrabyA S and Alexander S Delphi-based distributedexpert decision-making working paper 1985

Jagannathan V Elmaghraby and A S Computer-aided learning tool forsimulation Proceedings of the Third Annual Workshopfor Interactioe ComputingOctober 1984

Keeney R and Raifa H Decisions with Multiple Objectives Preferences and ValueTradeoffs John Wiley amp Sons NY 1976

Linstone and Turoff The Delphi Method Techniques and Applications AddisonshyWesley 1975

Mittal S and Dym C L Knowledge acquisition from multiple experts The AIMagaz ine Summer 1985 pp 32-36

Sandell H S H laquoA knowledge engineering tool for creating frame - and rule-basedexpert systems PhD dissertation Vanderbi It Uni versi ty August 1984

34

~~ ~J~~ bull J_-bullbull _ ~ gt0t4 bull o(_( ~-t~ 04tgtJ~o(tt_ootol~iCIIamp koc~~~lOCMoo( -t-~ ~ Iut~~ -lJvwW~ oJmiddotoI - - -lt ~ ~~_-~ ~ bull - ~ = bull bull bull bull ~ bull bull bull bull bull bull bull bull ~ bull bull bullbull bull bull-r ~ bull bull bull

~ ~ - ~~ ~ ~ middot 1middot ~ - ~ - bull s - - -

- ~ ~ ~ - t

- -

The SECOND ANNUALacm NORTHEAST REGIONAL CONFERENCE

INTEGRATING THE INFORMATION WORKPLACETHE KEY TO PRODUCTIVITY

OCTOBER 28-30 middot1985

(Sheraton-Tara Hotel

Framingham Massachusetts

~


INTRODUCTION


( )











30

t



l~c E jO P ~ ~

~ j~ O IO i

--

( OM Pl EVl f Y

Fcross

HUMAN middotRA l ED

~JE middotWE ICHT

l~ EmiddotWE IGH I

--

-

FACTORS

)NMAN-RELATED1

PRODUCT EXPENSIVE

ACCURACY


-~~D

ANALOG-_


FAGORSSUBCATEGORIES




Iv

middot 31

~

~ ~-gt-~---~~~ ~~~~~ ~~~~~ ~ - ~

-

- bull -




-

bull _ ~ 0

~ 0 bull












(





32

~ - - ~

-

~

~ ~ -

-




CONCLUSION

(





33

~ ~

rmiddot1 0 J 0




~ ~ ~ ~ gt~ o~ ~ - _ - - gt ~ _ ~~ ~ ~~



REFERENCES








(

~










34

~


INTRODUCTION


( )











30

t



l~c E jO P ~ ~

~ j~ O IO i

--

( OM Pl EVl f Y

Fcross

HUMAN middotRA l ED

~JE middotWE ICHT

l~ EmiddotWE IGH I

--

-

FACTORS

)NMAN-RELATED1

PRODUCT EXPENSIVE

ACCURACY


-~~D

ANALOG-_


FAGORSSUBCATEGORIES




Iv

middot 31

~

~ ~-gt-~---~~~ ~~~~~ ~~~~~ ~ - ~

-

- bull -




-

bull _ ~ 0

~ 0 bull












(





32

~ - - ~

-

~

~ ~ -

-




CONCLUSION

(





33

~ ~

rmiddot1 0 J 0




~ ~ ~ ~ gt~ o~ ~ - _ - - gt ~ _ ~~ ~ ~~



REFERENCES








(

~










34

t



l~c E jO P ~ ~

~ j~ O IO i

--

( OM Pl EVl f Y

Fcross

HUMAN middotRA l ED

~JE middotWE ICHT

l~ EmiddotWE IGH I

--

-

FACTORS

)NMAN-RELATED1

PRODUCT EXPENSIVE

ACCURACY


-~~D

ANALOG-_


FAGORSSUBCATEGORIES




Iv

middot 31

~

~ ~-gt-~---~~~ ~~~~~ ~~~~~ ~ - ~

-

- bull -




-

bull _ ~ 0

~ 0 bull












(





32

~ - - ~

-

~

~ ~ -

-




CONCLUSION

(





33

~ ~

rmiddot1 0 J 0




~ ~ ~ ~ gt~ o~ ~ - _ - - gt ~ _ ~~ ~ ~~



REFERENCES








(

~










34

~

~ ~-gt-~---~~~ ~~~~~ ~~~~~ ~ - ~

-

- bull -




-

bull _ ~ 0

~ 0 bull












(





32

~ - - ~

-

~

~ ~ -

-




CONCLUSION

(





33

~ ~

rmiddot1 0 J 0




~ ~ ~ ~ gt~ o~ ~ - _ - - gt ~ _ ~~ ~ ~~



REFERENCES








(

~










34

~ - - ~

-

~

~ ~ -

-




CONCLUSION

(





33

~ ~

rmiddot1 0 J 0




~ ~ ~ ~ gt~ o~ ~ - _ - - gt ~ _ ~~ ~ ~~



REFERENCES








(

~










34

~ ~

rmiddot1 0 J 0




~ ~ ~ ~ gt~ o~ ~ - _ - - gt ~ _ ~~ ~ ~~



REFERENCES








(

~










34

Date post:	16-Nov-2023
Category:	Documents
Upload:	louisville
View:	1 times
Download:	0 times

MEDKAT: multiple expert DELPHI-based Knowledge Acquisition Tool

Documents