Fitness and Precision: How Well Event Data and Process Models Match

Two perspectives on the same log–model pair

A model that allows many paths can explain every observation while barely distinguishing between them. A restrictive model, conversely, may exclude real sequences. The metrics must therefore be considered together and alongside their calculation method.

M1 and M2 explain the same observations with different restrictions

An illustrative log contains the sequences A → B → D and A → C → D. M1 allows exactly these two paths. M2 additionally allows paths through E and F, as well as repetitions. Both can fully reproduce the two recorded sequences.

The difference lies in the additional behavior allowed. A precision method that evaluates possible but unobserved continuations at observed states may rate M2 less favorably. This sketch does not imply a specific numerical value or a ranking for arbitrary precision measures; the formal model, measure, and calculation must first be defined.

If the path through C is removed from M1, it can no longer fully reproduce the second observed sequence. Greater restriction alone is therefore not a sufficient quality goal. It also remains a domain question whether E and F are legitimate rare paths or unintended freedoms in the model.

Fitness assesses reproduction of observed sequences

Fitness looks from the event log toward the model. An execution has high fitness if its event sequence can be replayed in the model with no or only minor deviations. If the analysis method must skip many observed events or add unrecorded steps in the model, fitness decreases.

The exact calculation depends on the method. “Fitness 0.92” without the method, model type, costs, and aggregation is therefore not a complete statement. Values from different tools or configurations must not automatically be treated as equivalent.

Even perfect fitness does not prove that a model is good. A very permissive model may allow all traces and thereby achieve fitness of one. Only considering precision and other quality dimensions alongside it shows whether the model also restricts behavior meaningfully.

Precision restricts additionally allowed behavior

Precision looks in the opposite direction: from the model toward the observations. A common approach examines states reached through recorded executions, asking which next steps the model allows and which of them were actually observed in the log. If the model allows many unobserved alternatives, precision decreases.

Muñoz-Gama and Carmona developed an approach to precision that counts situations in which the model deviates from the log, traversing the model along the log rather than examining its entire behavior.^[3] Such deviations are called “escaping edges”: activities that the model allows as the next step after an observed prefix but that were never observed after that prefix in the log.^[6]

Low precision is often understood as an indication of underfitting. The model is then too general to represent characteristic restrictions of the real process. Nevertheless, an unobserved alternative is not automatically wrong. It may represent a rare but intended emergency or special-case process.

The event log and process model form the comparison basis

Both metrics require at least an event log, a formally evaluable process model, and a traceable mapping between event classes and model activities. The log contains observed executions within a particular period. The model describes allowed or possible execution sequences.

These two sides are not equally complete. A log contains only behavior actually recorded and included in the analysis. It may omit rare, seasonal, or not-yet-encountered special cases. A model, by contrast, may describe operationally intended alternatives that did not occur during the period considered.

Fitness and precision therefore depend on the selected data subset. Period, case construction, event granularity, filters, model version, and cost parameters form part of a measurement's meaning. Without them, a single value is difficult to compare reliably.

Token-based values depend on replay

Token-based fitness relates missing and excess tokens during Petri-net replay to produced and consumed tokens.[1] The intended tokens of the final marking are not residual errors.

In token-based conformance checking, comparing two values requires knowing how the method adds missing tokens and handles silent or ambiguous model steps. These decisions can change the subsequent replay and therefore the result.

Costs and normalization shape alignment-based fitness

Alignment-based fitness evaluates deviations between the log and an allowed model execution using a cost function.[2] The weighting may, for example, treat a missing visible review step differently from an additional event. Silent model steps intentionally have no associated log event.

The cost function, treatment of such steps, and normalization therefore belong with the result. Two tools may produce different values for the same log–model pair if these choices differ. A comparison should first reconcile the calculation basis before claiming a quality difference.

A higher overall score may result from selection

Suppose the sequence through C is filtered out of the illustrative log. The restricted M1 without C would now fit the remaining data better. Actual execution has not improved; only the comparison basis has narrowed.

Aggregation also affects the impression. An average across all cases weights frequent variants differently from an analysis that considers each distinct variant once. A high overall score may conceal a rare, operationally important deviating path.

A results report should therefore disclose selection, frequencies, and aggregation. Changes from an earlier analysis can be interpreted as a trend only when changes to the underlying basis are known.

Fitness and precision do not form a simple target score

Fitness and precision can be combined into a harmonic mean. This overall score makes ranking easier but conceals why a model performs better or worse. Two models may have the same combined score even though one excludes real cases and the other allows much additional behavior.

Alongside fitness and precision, the Process Mining Manifesto identifies generalization and simplicity as quality dimensions of a process model.^[4] Generalization concerns the ability to represent plausible, not-yet-observed behavior appropriately. Simplicity favors understandable models without unnecessary complexity.

These goals can compete. A model that exactly reproduces every individual trace may maximize fitness and precision on the available log yet generalize poorly and be difficult to understand. Selection should therefore be guided by the analysis purpose, the expected range of processes, and the ability to assess the model in domain terms.

Data quality affects both metrics

Missing events apparently reduce fitness when the model expects a step that was not technically recorded. Duplicate or incorrectly ordered events may produce additional log moves. Inconsistent activity names split domain-equivalent steps into different classes.

Precision also responds to the data subset. If a log contains only the most frequent product types or only successfully completed cases, legitimate model alternatives appear as unobserved behavior. A short period may systematically omit rare release, escalation, or safety paths.

The analysis must therefore disclose which cases are complete, which filters apply, and which process version is being investigated. A low value initially calls for investigation: it may indicate a model error, a real special-case path, incomplete data, or incorrect mapping.

Discovered models need a separate assessment

Process discovery generates process models from observed event sequences. Fitness and precision then help assess whether a discovered model represents the data appropriately without allowing excessive behavior.

Evaluation on the same data used to generate the model may be too optimistic, however. For robust conclusions, it may be useful to split data into training and test sets by time or case. This reveals whether the model merely reproduces existing individual cases or also describes further comparable executions appropriately.

Process variants also remain relevant. An aggregated fitness value may conceal that the most frequent variant fits perfectly while an operationally important special-case path regularly deviates. Metrics should therefore be available broken down by variant, product group, location, or model version.

Object-centric processes need appropriate definitions

Classic fitness and precision methods often assume case-based traces. Real production processes, however, simultaneously connect batches, materials, samples, tasks, equipment, and orders. A single case identifier may artificially simplify these relationships.

For object-centric process mining, dedicated notions of fitness and precision have therefore been developed for object-centric Petri nets and event logs. Adams and van der Aalst account for events being connected to multiple objects and object types.^[5]

Values from classic and object-centric analyses are not directly comparable. First, the process perspective that appropriately represents the operational question must be established.

The next chart complements the qualitative M1/M2 example with published results for object-centric models. A, B and C refer to the study’s models, not to M1 and M2.

Read B’s high precision together with its lower fitness and excluded events. It is not evidence that the model covers the log comprehensively.

What fitness and precision do not prove

• Unobserved does not mean impermissible. Rare or future alternatives may be operationally necessary.
• A deviation does not explain a cause. Model, mapping, data, and execution must be reviewed separately.
• The metrics do not replace a compliance assessment. Legal and substantive permissibility require accountable judgment.

The number needs its comparison context

M1 and M2 show why good reproduction of the log does not yet mean precise restriction of model behavior. The filtered log also shows how a score can improve without any change to the work.

A reliable interpretation therefore always identifies the log, model, and measure. Only with data selection, parameters, and aggregation can one assess what a difference between two results actually means.