15.1 Conformance to ODP standards
Conformance relates an implementation to a standard. Any proposition that is true in the specification must be true in its implementation.
A conformance statement is a statement that identifies conformance points of a specification and the behaviour which must be satisfied at these points. Conformance statements will only occur in standards which are intended to constrain some feature of a real implementation, so that there exists, in principle, the possibility of testing.
The RM-ODP identifies certain reference points in the architecture as potentially declarable as conformance points in specifications. That is, as points at which conformance may be tested and which will, therefore, need to be accessible for test. However, the requirement that a particular reference point be considered a conformance point must be stated explicitly in the conformance statement of the specification concerned.
Requirements for the necessary consistency of one member of the family of ODP standards with another (such as the RM-ODP) are established during the standardization process. Adherence to these requirements is called compliance.
If a specification is compliant, directly or indirectly, with some other standards then the propositions which are true in those standards are also true in a conformant implementation of the specification.
15.2 Testing and reference points
The truth of a statement in an implementation can only be determined by testing and is based on a mapping from terms in the specification to observable aspects of the implementation.
At any specific level of abstraction, a test is a series of observable stimuli and events, performed at prescribed points known as reference points, and only at these points. These reference points are accessible interfaces. A system component for which conformance is claimed is seen as a black box, testable only at its external linkages. Thus, for example, conformance to OSI protocol specifications is not dependent on any internal structure of the system under test.
15.3 Classes of reference points
A conformance point is a reference point where a test can be made to see if a system meets a set of conformance criteria. A conformance statement must identify where the conformance points are, and what criteria are satisfied at these points. Four classes of reference points at which conformance tests can be applied are defined.
15.3.1 Programmatic reference point: A reference point at which a programmatic interface can be established to allow access to a function. A programmatic conformance requirement is stated in terms of a behavioural compatibility with the intent that one object be replaced by another. A programmatic interface is an interface which is realized through a programming language binding.
NOTE -
For example, a programmatic reference point may be established in a database standard to support a language binding at some level of abstraction.
15.3.2 Perceptual reference point: A reference point at which there is some interaction between the system and the physical world.
NOTES
1 - A perceptual reference point may be e.g. a human-computer interface or a robotic interface (specified in terms of the interactions of the robot with its physical environment).
2 - A human-computer interface perceptual conformance requirement is stated in terms of the form of information presented to a human being and the interaction metaphor and dialogues the human may be engaged in.
3 - A perceptual reference point may, for example, be established in a graphics standard.
15.3.3 Interworking reference point: A reference point at which an interface can be established to allow communication between two or more systems. An interworking conformance requirement is stated in terms of the exchange of information between two or more systems. Interworking conformance involves interconnection of reference points.
NOTE - For example, OSI standards are based on the interconnection of interworking reference points (the physical medium).
15.3.4 Interchange reference point: A reference point at which an external physical storage medium can be introduced into the system. An interchange conformance requirement is stated in terms of the behaviour (access methods and formats) of some physical medium so that information can be recorded on one system and then physically transferred, directly or indirectly, to be used on another system.
NOTE - For example, some information interchange standards are based on interchange reference points.
The testing of conformance may take place at a single reference point, or it may involve some degree of consistency over use in a series of configurations involving several reference points. This may involve the testing of conformance to
a) the requirement for a component to be able to operate after some preparatory process to adapt it to the local environment;
b) the requirement for a component to operate according to its specification at a particular reference point from initialization onwards;
c) the requirement for a component to continue to work when moved into a similar environment during operation.
The properties being tested above give rise to attributes of the objects or interfaces involved, as follows.
15.4.1 Portability: The property that the reference points of an object allow it to be adapted to a variety of configurations.
NOTE - If the reference point is a programmatic reference point, the result can be source-code or execution portability. If it is an interworking reference point, the result is equipment portability.
15.4.2 Migratability: The ability to change the configuration, substituting one reference point of an object for another while the object is being used.
15.5 The conformance testing process
Conformance is a concept which can be applied at any level of abstraction. For example, a very detailed perceptual conformance is expected to a standard defining character fonts, but a much more abstract perceptual conformance applies to screen layout rules.
The more abstract a specification is, the more difficult it is to test. An increasing amount of implementation-specific interpretation is needed to establish that the more abstract propositions about the implementation are in fact true. It is not clear that direct testing of very abstract specifications is possible at reasonable cost using currently available or foreseeable techniques.
The testing process makes reference to a specification. To be complete, the specification must contain:
a) the behaviour of the object being standardized and the way this behaviour must be achieved.
b) a list of the primitive terms used in the specification when making the statements of behaviour.
c) a conformance statement indicating the conformance points, what implementations must do at them and what information implementors must supply (corresponding to the OSI notions of PICS and PIXIT).
There are two roles in the testing process: the implementor and the tester. The first role is that of the implementor, who constructs an implementation on the basis of the specification. The implementor must provide a statement of a mapping from all the terms used in the specification to things or happenings in the real world. Thus the interfaces corresponding to the conformance points must be indicated and the representation of signals given. If the specification is abstract, the mapping of its basic terms to the real world may itself be complex. For example, in a computational viewpoint specification (see Recommendation X.903 | International Standard 10746-3), the primitive terms might be a set of interactions between objects. The implementor wishing to conform to the computational viewpoint specification would have to indicate how the interactions were provided, either by reference to an engineering specification or by providing a detailed description of an unstandardized mechanism (although this course limits the field of application of the implementation to systems in which there is an agreement to use the unstandardized mechanism).
The second role is that of the tester, who observes the system under test. Testing involves some shared behaviour between the testing process and the system under test. If this behaviour is given a causal labelling, there is a spectrum of testing types from
a) passive testing, in which all behaviour is originated by the system under test and recorded by the tester;
b) active testing, in which behaviour is originated and recorded by the tester.
Normally, the specification of the system under test is in the form of an interface, as is the specification of the tester and test procedures. When testing takes place, these interfaces are bound.
The tester must interpret its observations using the mapping provided by the implementor to yield propositions about the implementation which can then be checked to show that they are also true in the specification.
The testing process succeeds if all the checks against the specification succeed. However, it can fail because
a) the specification is logically inconsistent or incomplete, so that the propositions about the implementation cannot be checked (this should not occur);
b) the mapping given by the implementor is logically incomplete, so that it is inconsistent or observations cannot be related to terms in the specification; testing is impossible.
c) the observed behaviour cannot be interpreted according to the mapping given by the implementor. The behaviour of the system is not meaningful in terms of the specification, and so the test fails.
d) the behaviour is interpreted to give terms expressed in the specification, but these occur in such a way that they yield propositions which are not true in the specification, and so the test fails.
15.7 Relation between reference points
The flow of information between modelled system components may pass through more than one reference point. For example, a distributed system may involve interactions of two components A and B, but communication between them may pass in turn through a programmatic interface, a point of interconnection and a further programmatic interface.
A refinement of the same system may show interconnected components that have more than one component on the communication path between them.
In either case, conformance testing may involve
a) testing the information flow at each of these reference points;
b) testing the consistency between events at pairs of reference points.
In general, testing for correct behaviour of a configuration of objects will require testing that statements about communication interfaces are true, but it will also require observation of other interfaces of these objects, so that the statements about the composition can also be checked.
The general notion of conformance takes into account the relation between several conformance points. Since the specification related to a given conformance point may be expressed at various levels of abstraction, testing at a given conformance point will always involve interpretation at the appropriate level of abstraction. Thus, the testing of the global behaviour requires coordinated testing at all the conformance points involved and the use of the appropriate interpretation at each point.
In particular, conformance of a template to a given programmatic interface can only be asserted when considering the language binding for the language in which the template has been written, and compliance of the written template to the language binding specification, which must itself be conformant with the abstract interface specification.
NOTE - Associated with each index entry are the page number and the clause or subclause where the index entry is defined.
Compliance p.16, clause 15.1