As techniques for conceptual modelling, enterprise modelling, and business process modelling have proliferated over the years (e.g. Olle et al., 1991), researchers and practitioners alike have attempted to determine objective bases on which to compare, evaluate, and determine when to use these different techniques (e.g. Karam and Casselman, 1993; Gorla et al., 1995) . However, throughout the 1980s, 1990s, and into the new millennium, it has become increasingly apparent to many researchers that without a theoretical foundation on which to base the specification for these various modelling techniques, incomplete evaluative frameworks of factors, features, and facets will continue to proliferate. Furthermore, without a theoretical foundation, one framework of factors, features, or facets is just as justifiable for use as another (e.g. Bansler and Bodker, 1993).

Wand and Weber (1989; 1990; 1993; 1995) have investigated the branch of philosophy known as ontology as a foundation for understanding the process of developing an information system. Ontology is a well-established theoretical domain within philosophy dealing with identifying and understanding elements of the real world. However, interest in, and the applicability of, ontologies today extends to areas far beyond philosophy. As Gruninger and Lee (2002, p. 13) point out, ‘…a Web search engine will return over 64 000 pages given ‘ontology’ as a keyword … the first few pages are phrases such as “enabling virtual business”, “gene ontology consortium”, and “enterprise ontology”.’ The usefulness of ontology as a theoretical foundation for knowledge representation and natural language processing is a fervently debated topic at the present time in the artificial intelligence research community (Guarino and Welty, 2002). The use of ontologies as a basis for the analysis of techniques that purport to assist analysts to develop models that emulate portions of the real world has been growing steadily more popular. The Bunge-Wand-Weber (BWW) ontological models (Weber, 1997), for example, have been applied extensively in the context of the analysis of various modelling techniques. Wand and Weber (1989; 1990; 1993; 1995) and Weber (1997) have applied the BWW representation model to the ‘classical’ descriptions of entity-relationship (ER) modelling and logical data flow diagramming (LDFD). Weber and Zhang (1996) also examined the Nijssen Information Analysis Method (NIAM) using the ontology. Green (1997) extended the work of Weber and Zhang (1996) and Wand and Weber (1993; 1995) by analysing various modelling techniques as they have been extended and implemented in upper CASE tools. Furthermore, Parsons and Wand (1997) proposed a formal model of objects and they use the ontological models to identify representation-oriented characteristics of objects. Along similar lines, Opdahl and Henderson-Sellers (2001) have used the BWW representation model to examine the individual modelling constructs within the OPEN Modelling Language (OML) version 1.1, which is based on ‘conventional’ object-oriented constructs. Green and Rosemann (2000) have extended the analytical work into the area of integrated process modelling based on the techniques presented in Scheer (2000). Most recently, Green et al. (2003) have extended the use of this evaluative base into the area of enterprise systems interoperability using business process modelling languages like ebXML, BPML, BPEL4WS, and WSCI. Clearly, ontology is a fruitful theoretical basis on which to perform such analyses. However, while ontological analyses are frequently utilised, particularly in the area of conceptual modelling technique analysis, the actual process of performing the analysis remains problematic. The current process of ontological analysis is open to the individual interpretations of the researchers who undertake the analysis. Consequently, such analyses are criticised as being subjective, ad hoc, and lacking in relevance. There is a need, therefore, for the systematic identification of shortcomings of the current ontological analysis process. The identification of such weaknesses, and their subsequent mitigation, will lead to a more rigorous, objective, and replicable analytical process.

Accordingly, this paper has several objectives. First, we aim to identify comprehensively the shortcomings in the current practice of ontological analysis. The identification of such shortcomings will provide a basis upon which the practice of ontological analysis can be improved. Second, we want to develop several propositions and methodology extensions that enhance the ontological analysis process by making it more objective and structured.

There are several contributions this paper aims to make. They are based on previous experiences with ontological analyses as well as observations derived from published analyses. First, the work presents a detailed analysis of the actual process of performing an ontological evaluation. We identify eight shortcomings of the current ontological analysis process, viz. lack of understandability, lack of comparability, lack of completeness, lack of guidance, lack of objectivity, lack of adequate result representation, lack of result classification, and lack of relevance. Each of the identified shortcomings is then classified as belonging to one of three phases of analysis, viz., input, process, and output. Second, the paper presents recommendations on how each of the shortcomings in the three phases can be overcome. The recommendations, inter alia, include an extended methodology for improving the objectivity of the analysis as well as a weighting model that aims to improve the classification of the results of any ontological analysis.

The remainder of the paper is structured as follows. The next section identifies eight current shortcomings of ontological analyses that are classified with respect to the three phases of analysis. The third section provides recommendations concerning how to overcome the identified shortcomings in each of the three phases. The final section provides a brief summary of the work and outlines possible future research in this area.