Grounding Knowledge Acquisition with Ontology Explanation: A Case Study

Abstract

Knowledge validation is still a challenge when constructing knowledge-based systems. It is one of the major reasons for user rejection and disagreement between project participants. Systematic and periodic reviews of the domain ontology, with a formal agreement of the whole development team (including the experts) are a recommended good practice. Nevertheless, these reviews do not guarantee system success. This paper presents a case study of the construction process of a knowledge-based system. The process involved a group of experts with varied work experience. A great deal of negotiation happened during knowledge acquisition meetings, which took place during a 6-month period. After each meeting, changes in the ontology were verified through a web-based questionnaire, from which either consensual agreement was reached (and changes implemented) or the need for a new meeting was ascertained. An explanatory review at the beginning of each meeting further solidified the understanding of all participants. This cyclic process led to a final version of the ontology, ratified by all participants. This model supports diagnosis and prediction of failures in mechanical drilling rigs in oil exploration sites. Unexpectedly, during system trials, experts disagreed with results, which raised questions about the validity of the domain ontology. The system’s explanation module provided a cornerstone for a reflective process that helped identify inconsistencies and corrections needed. These reflections led to adjustments to the ontology, and a reflection about previous decisions and element definitions. Explanations, derived from the ontology and instantiated using real scenarios, shed light on knowledge gaps and semantic inconsistencies of the domain model. In this paper we have three main goals: (1) to present our ontology construction process; (2) to highlight a particular situation where results were inadequate; and (3) to show how the explanation system helped experts and knowledge engineers identify gaps. We also present lessons learned from the whole process, that may apply in other situations.

Keywords: Explanation, knowledge acquisition, ontology validity, ontology engineering