Conformance Checking: Comparing Actual Processes with Process Models

What conformance checking means

Conformance checking answers a different question from automatic process discovery. Discovery generates a model of observed behavior from event data. Conformance checking, by contrast, presupposes a comparison model and examines how recorded processes relate to it. The Process Mining Manifesto identifies this comparison of expected and actual behavior as one of the three basic types of process mining, alongside discovery and enhancement.^[1]

“Conformance” must not be hastily equated with legal or regulatory compliance. The method initially establishes a technical relationship between event data and a model. Whether a detected deviation is legally relevant, operationally permissible, desirable, or merely the result of an incomplete model must then be assessed against the specific purpose.

Conformance checking can examine individual cases, identify deviation patterns across many executions, and calculate measures of agreement. Its particular value lies in tracing differences back to specific activities and process locations rather than merely returning an overall score.

A release event precedes the expected review

An illustrative comparison model requires a documented review before the release of batch C-24. In the selected log, the release appears first; the associated review event follows afterward. The comparison locates this difference between the event sequence and the model.

Three types of explanation remain open. First, execution may genuinely have deviated from the requirement. Second, capture may misrepresent the process, for example through an inappropriate time reference or a missing import. Third, the comparison basis may be unsuitable: a different model version, an overly coarse event mapping, or a requirement that does not apply to this case.

The diagnosis must therefore lead back to the specific release event, its review reference, and the relevant model version. Correcting the mapping and correcting the operational process would be different measures. The technical finding does not yet determine which is required.

The event log and process model provide different perspectives

The comparison requires two separate descriptions. The event log documents what was actually observed. The process model describes which behavior is considered possible, intended, or expected.

A classic event log contains ordered traces. Each trace represents a case, such as an order, service operation, or production batch. Its process events denote executed activities and often carry additional information such as times, roles, resources, or results. Their meaning must be unambiguous so that an observation can be mapped to the correct model activity.

The comparison model may use different notations. Petri nets, BPMN models, process trees, or declarative rules are common. What matters is that the model describes behavior that can be evaluated. A process diagram without unambiguous semantics is insufficient for an algorithmic check.

The two sides must not be confused. An event log is not a complete representation of every possible process, but a collection of observed executions over a particular period. A model is not a neutral truth, but a description created for a purpose. It may normatively prescribe how work should be done or descriptively summarize behavior considered typical.

Events and model must correspond before comparison

A method can compare only what has been mapped in domain terms. The activity “Review completed” in the event log must correspond to an appropriate element of the process model. Different labels, differing granularity, or missing events make this mapping harder.

A model may, for example, contain a single step “Test material,” while the event log records sampling, measurement, four-eyes check, and release separately. Conversely, the model may distinguish several substantive checks even though the source system supplies only a generic “Check completed” event.

Mapping is therefore required before conformance checking. It specifies which event classes correspond to which model activities, which events may be ignored for comparison, and which lifecycle transitions matter. Start and end conditions must also be unambiguous.

Standards structure data, but do not automatically resolve its domain mapping. XES and OCEL describe different event-log models. Whether an event type and a model activity mean the same thing remains a domain agreement.

Incorrect mapping produces apparent deviations even when log and model merely express the same circumstances differently. Overly coarse mapping can likewise conceal real differences. The comparison basis must therefore be documented, versioned, and reviewed for the specific analysis purpose.

Token replay replays the observed sequence in the model

Token replay is an early and intuitive conformance-checking method, applied particularly to Petri nets. The observed trace is replayed in the model event by event. For each activity, the method checks whether the corresponding transition is enabled in the current model state.

If an observed activity can be executed only by supplying a token missing from the model, that deficit is recorded. After the full replay, the reached marking is compared with the intended final marking. Tokens expected there are not an error; missing and excess remaining tokens are counted. Produced, consumed, missing, and remaining tokens can be combined into a fitness measure.

Rozinat and van der Aalst developed this approach to compare real process executions with models and make deviations visible at model level.^[2] The method is understandable and comparatively efficient, but depends on the model notation and the local treatment of a deviation.

When an error occurs during replay, the method must be able to continue the sequence artificially. A local repair can make later differences depend on the intermediate correction chosen. Token replay therefore does not always provide the globally best explanation of the deviation between trace and model.

Alignments seek the best-fitting model execution

Alignment-based methods consider the event sequence and model behavior together. The aim is to align the observed trace with an allowed execution of the model. Each step distinguishes whether log and model agree or whether a move on only one side is required.

A synchronous move means that the observed event matches the next enabled model activity. A log-only move indicates an observed event for which no corresponding model activity is executed at that point. A model-only move indicates a model step executed without a corresponding log event. For a visible activity, this may indicate a missing observation. Silent model steps, by contrast, intentionally have no associated log event and are not a deviation merely for that reason.

Costs can be assigned to the different moves. The method then seeks an allowed alignment with the lowest possible total cost. Van der Aalst, Adriansyah, and van Dongen show how this relationship between events and model elements can support conformance and performance analysis.^[3]

An optimal alignment is not automatically the only correct domain explanation. Different alignments may have the same cost. The chosen costs also express which deviations are considered more or less serious. Weighting every skipped activity equally may greatly simplify its domain significance.

Tracing a finding to a reviewable location

For C-24, the identified location is more precise than saying “poor conformance”: which release event was mapped to which model element, and which expected step is missing at that point? Replay or alignment provides a technical reference for this.

The subsequent investigation keeps finding and explanation separate. It documents the mapping used, the original events affected, and the assumptions still to be checked. If several equally low-cost alignments are found, that ambiguity also matters. A computationally optimal alignment does not by itself determine the actual cause of the error.

Two metrics complement the local diagnosis

Fitness and precision assess different properties of the comparison: fitness considers how well the model can reproduce recorded sequences. Precision considers how strongly additionally allowed behavior not observed in the selected log is restricted. Values can be meaningfully compared only with their respective calculation methods taken into account.

An overall score does not replace the localized finding in the release case. It depends on factors including the method, parameters, and data selection; research distinguishes numerous measures with different properties.[4]

The model's purpose changes the interpretation

With a normative model, a deviation describes a difference from intended behavior. This may trigger a need for review. Whether a violation actually occurred depends on whether the model correctly and completely represents the relevant requirement and applies to the specific case.

With a descriptive model, a deviation initially means only that the case does not match the modeled normal or reference behavior. The unusual sequence may be erroneous, rare, innovative, or due to a legitimate exception.

Discovered models can also be used for conformance checking, for example to compare different periods or locations. It must then remain clear that the reference process was derived from historical data. Frequent behavior is not automatically correct; rare behavior is not automatically wrong.

Before each analysis, the model's meaning should therefore be specified in writing: an approved way of working, technical control flow, a historical reference state, or an analytical hypothesis. Without this purpose, the meaning of a deviation score remains unresolved.

Data quality limits what the result can establish

Conformance checking assesses only observable behavior. Unlogged work steps remain invisible. Duplicate events, incorrect timestamps, changing case identifiers, or unclear activity names can alter the reconstructed sequence.

Case construction also matters. If a complex process is viewed solely through an order number, shared materials, equipment, or tests may belong to several cases. Forcing a single perspective can then produce sequences that did not exist in real work.

Before analysis, at least completeness, unambiguity, time order, activity semantics, and object references should therefore be checked. Filters and exclusions must be reproducible. A cleaned log must not be tailored to the model so strongly that relevant deviations disappear before comparison.

The technical export is also part of the result's meaning. If a source system exports only successful completions, aborted or rejected steps cannot be investigated. High fitness would then describe the data selection more than the actual process.

A localized difference becomes a specific review question

In the release case, the next step is not a blanket assessment of the entire batch. First, it must be established why the associated review event appears too late in the comparison. The finding provides a specific location in the log and model for this purpose.

This gives conformance checking a clear benefit: it narrows down what must be investigated. The documented explanation then determines whether the data, comparison basis, or execution needs correction.

Conformance checking does not automatically assess data accuracy. An event that fits formally may be incorrect in substance.