The formal specification of ontologies, together with the differences in the languages used to specify the ontologies and the grammars under analysis, have been classified as issues pertaining to the lack of understandability and comparability.

In order to overcome these shortcomings, it is proposed to convert the ontology as well as the selected modelling grammar to meta models using the same language (e.g. ER Models or UML Class Diagrams). This facilitates a pattern-matching approach towards the ontological analyses of completeness and clarity of a grammar. As a first step we converted, for example, the Bunge-Wand-Weber ontology into an ER-based meta model. This meta model includes 50 entity types and 92 relationship types. It has clusters such as system, property or class/kind. Such a meta model explains, in a language familiar to the information systems (IS) community, the core constructs of the ontology. It also highlights the underlying focus of the ontology. In the case of the BWW model, for example, it is obvious from a visual inspection of the meta model that the ontology is centred around the existence of a thing, which is the central entity type in the meta model.

The obtained meta model can now be used for a variety of ontological analyses. Moreover, it allows a critical review of the BWW model by a wider community. The approach, however, is not without its limitations. Commonly used modelling techniques such as ER or UML are often widely accepted but they have not been designed for the purposes of meta modelling. Thus, they occasionally lack the required expressiveness. Figure 13.1, “The BWW meta model.” provides an impression of the size and complexity of the meta model for the BWW ontology.

While an ER-based meta model helps to overcome issues related to the understandability of an ontology, a corresponding meta model of the analysed grammar is required to deal with the lack of comparability issue. Many popular modelling techniques (e.g. ARIS or UML, and also interoperability standards such as ebXML) are already specified in meta models using ER-notations or UML Class Diagrams. If the meta models for the ontology and the modelling technique are specified in the same language, the ontological analysis turns into a comparison of two conceptual models. As part of the analysis, it will be required to identify corresponding entity types and relationship types in both models. It also becomes immediately obvious if the paradigm of the analysed grammar differs from the ontology. In the case of ARIS or many Web Services standards, for example, the meta models are centred around functions or activities instead of being centred around things.

The issues related to the process of conducting an ontological analysis have been described as lack of completeness, lack of guidance and lack of objectivity.

Based on the assumption that corresponding meta models for the ontology and the analysed grammar are available, it is possible to clearly specify the scope of an analysis using those meta models. A selection of clusters, entity types and relationship types would define all elements that are perceived of relevance for the analysis. An analysis of an ER-based notation, for example, could be focused on the BWW clusters thing, system and property and could exclude the more behavioural-oriented clusters event and state. Such boundaries of an analysis could be easily visualised in the meta model and would provide a clear description of the comprehensiveness of the analysis.

The existence of two corresponding meta models and a clear definition of the scope of the analysis are necessary but not sufficient criteria for a well-guided process. Further guidelines are required regarding the starting point of such a process and the actual sequence of activities. Based on our experiences, we recommend starting with the representation mapping; that is, selecting the meta model of the ontology and subsequently identifying corresponding elements in the modelling grammar. The first construct to be analysed should be the most central entity type. For example, in the case of the BWW model, the entity type thing is the appropriate starting point. Our previous work provides a strong argument that this analysis should follow a cluster-by-cluster approach. Starting with the core constructs in a cluster allows a more structured and focused analysis of the completeness of a modelling grammar. The analysis of the entity types is followed by the relationships and the cardinalities. Constructs in the meta model that have only been introduced for reasons of correctness of the meta model, but that do not reflect ontological constructs, are excluded from the analysis. The representation mapping is followed by an analysis of the clarity of the target grammar, i.e. the interpretation mapping. In this case the meta model of the grammar under analysis is the starting point. The general procedure is similar. A primary advantage of a cluster-based analysis is that the structure of the two meta models provides valuable input for the ontological analysis. An example is the analysis of generalisation-specialisation relationships in the meta model of the grammar. We propose to ontologically classify the super-type first and then to inherit this ontological classification to all sub-types. This streamlines the process of the analysis and increases consistency.

The lack of objectivity issue, on the other hand, frequently stems from the analysis being performed by a single researcher. This situation results in an analysis that is almost certainly biased by the researcher’s background as well as their interpretation of the specification of the grammar. In order to improve the validity of the analysis, a research method can be adopted that involves individual analyses of a particular grammar by at least two members of a research team, followed by discussion and hopefully consensus as to the final analysis by the entire team of researchers. The method consists of three steps:

Just such a method was employed in a project that sought to apply the BWW representation model analysis to a number of the leading potential Web Services standards: ebXML, BPML, BPEL4WS and WSCI. The project team was composed of four researchers and the standards were analysed in the order: ebXML à BPML à BPEL4WS à WSCI. Two researchers were involved in Steps 1 and 2 of the method (the individual analysis of a standard followed by a meeting of the two researchers in order to obtain an agreed mapping). This was followed by a meeting of the entire team in order to discuss the mapping and arrive at the final analysis. The process was performed for each of the four standards. Table 13.1, “Summary of Step 2 mapping agreement between both researchers” shows the recorded agreement statistics at the second step of the applied method while Table 13.2, “Summary of Step 3 mapping agreement” shows the recorded agreement statistics at the third step of the method.

The adoption of such a method can be seen to have greatly improved the objectiveness of the carried-out analyses.

The three main shortcomings related to the outcome of an ontological analysis have been characterised as the lack of adequate result representation, lack of result classification and the lack of relevance.

The meta models, which have been used as input for the ontological analyses, are an appropriate medium to visualise the outcomes of the entire analysis process. In our work on the analysis of ARIS, we derived a meta model of the BWW model that highlighted all constructs of the ontology that did not have a corresponding construct in the grammar under analysis. That is, we visualised incompleteness in the model using simple colour coding. In a similar way, we derived three ARIS meta models that highlighted excess, overload and redundancy in ARIS. Such models form a very intuitive way of representing the identified ontological shortcomings. The underlying clustering of the models also helps to quickly comprehend the main areas in which there are shortcomings.

At the present time, the process of an ontological analysis results in the identification of ontological incompleteness and ontological clarity through the identification of missing, overloaded or redundant grammatical constructs. While the end result identifies such problems, it fails to account for their relative importance. For example, thing is one of the fundamental constructs of the BWW model. Therefore, a lack of mapping to a modelling grammar for this construct should be considered a more important shortcoming than the lack of mapping for, say, the well-defined event construct. There is a need for the development of a scoring model that enables the calculation of the ‘goodness’ of a grammar with respect to the ontology. In such a scoring model, each of the ontological constructs has a value assigned to it that reflects the relative importance of the construct in the ontology. Core constructs would therefore have high weightings whereas less important constructs would attract lower weightings. Following an ontological analysis of a particular grammar, the weighting of all missing constructs would be calculated to arrive at one value that generally reflects the outcome of the analysis.

An example for such a classification could have the following structure. All core constructs of an ontology (and the modelling grammar) would get the value one. All other constructs represented as an entity type in the meta model of the ontology would receive the value 0.7, and all other constructs get the value 0.3. Such a weighting would then be applied to the outcomes of the ontological analysis. The scores would be aggregated across the ontology and modelling grammar. They could also be calculated separately for completeness, excess, overload and redundancy. Furthermore, they could be aggregated per cluster, which allows a more differentiated view of the particular strengths or weaknesses of a modelling grammar. Though the consolidated score of such an evaluation should not be overrated, it provides better insights into the characteristics of the ontological deficiencies and provides a first rating of the significance and importance of the identified shortcomings.

Apart from the lack of result classification that is addressed by the scoring model, another problem with the outcome of the analyses has been the perceived lack of relevance of the results. Since most modelling grammars focus on modelling a subset of the phenomena that occur in the real world, it would follow that not all constructs of an ontology are necessary in order to analyse such a grammar. If the full ontology is used in the analysis, the result may identify potential problems that would not, in reality, occur, because the modelling grammar is not used to model any phenomena described by the missing constructs. Further, there may also be a need for specialisation of some of the ontological constructs in order to enhance analysis of a grammar pertaining to a particular domain.

Indeed, the outcomes of the ontological analyses of different modelling grammars to date appear to support the need for a focused ontology, which consists of different subsets of the ontological constructs for different domains. The analyses of the examined grammars consistently show that the constructs conceivable state space, conceivable event space and lawful event space, for example,have no representative constructs in the grammars. Such missing constructs, if identified as unnecessary for the particular domain, can be ignored, leading to a simpler analysis that does not consider phenomena that are deemed to be outside of the scope of the target grammar.