Workflow Engine: Controlled Execution of Process Models

What a Workflow Engine Is

The engine is therefore not the workflow itself. Workflow models describe activities, sequences, branches, and completion conditions. The workflow engine implements such a definition at runtime and manages the state of each specific operation.

For a production batch, many instances of the same approved model can be active simultaneously. Each instance has its own data, deadlines, tasks, and states. One batch is waiting for a test result, another needs rework, and a third has already been released. The engine keeps these executions separate and continues them under the same versioned definition.

One Definition Produces Many Process Instances

A process definition is a template. Only a domain trigger creates an instance from it: a new batch, a deviation, a material test, or a release request. The instance receives a unique identity and is connected to the affected objects.

The workflow engine stores which steps are complete, which are running in parallel, which data are available, and what the process is waiting for. This state must not be lost during a restart. Long-running processes can last days or months and survive technical interruptions.

Instance data and domain object data should remain distinguishable. “Review task open” describes execution state. The measurement result, the batch tested, and the responsible person are domain data. The engine coordinates their processing but is not automatically the system of record for every data type.

Resuming the Correct Waiting Instances

An instance can wait hours for a finding. Its waiting state, expected message type, and correlation attributes must therefore be stored. An open network connection is no substitute for this persistent state.

The batch ID alone is not always enough: several investigation orders can exist for the same batch. Correlation must distinguish the specific order and expected response. WS-BPEL uses correlation sets made up of message properties for this purpose.^[3]

After successful correlation, a separate check must establish whether the response is still expected in the current state. A late result may belong to an order that has already been canceled. A second delivery of the same message must not produce a second business effect. State checks, persistence, and handling of repetitions must fit together.

A Response Arrives Twice After a Restart

In the following illustrative runtime example, instance P-17 is waiting for laboratory order L-08. The model version applied, waiting state, and order identifier are stored. A server restart ends the executing program process, but the operational instance must subsequently be able to continue from its stored state.

Response R-52 identifies L-08. The runtime assigns it to P-17, checks that receipt is still pending, and stores the processing together with the resulting transition. Only the process definition determines whether the result triggers a further assessment or the next work step.

If R-52 is delivered again, it must be recognized as already processed. This differs from a corrected finding with a new identity. The latter can require substantive reassessment and must not simply be discarded as a duplicate.

The example describes a design objective. Whether a specific system achieves it must be tested through restart, duplicate-delivery, and error cases. In particular, a crash between an external transaction and internal confirmation requires coordinated handling in both components involved. When selecting compliance software, restart and duplicate-delivery cases should form part of the product demo so that the observed effect can be compared with operational requirements.

Errors, Retries, and Compensation Need Their Own Rules

Technical calls can fail or receive repeated acknowledgments. An engine must distinguish whether a step can safely be retried, whether manual intervention is needed, or whether an effect that has already occurred must be compensated.

Idempotent operations do not accidentally produce a second business effect when repeated. A redelivered message must not, for example, create two material transactions. The called service therefore needs a unique operation identifier and controlled handling of commands already processed.

Compensation is not technically turning back time. If material has been physically consumed or a decision communicated, the earlier state often cannot simply be restored. The process then needs an explicit counteracting or corrective operation.

Running Instances and New Model Versions Must Fit Together

When a process definition changes, the question arises of what happens to instances already running. A new version may apply only to operations started in the future. Alternatively, selected instances can be migrated if the source and target states can be mapped unambiguously to one another.

An uncontrolled switch can cause expected steps to disappear, create new obligations without a suitable starting state, or reactivate activities already performed. Every migration therefore needs rules, checks, and a documented decision.

For later analysis, it must remain clear under which definition an instance began and which version was actually applied to it. Only then can different execution paths and results be interpreted correctly.

When implementing workflow software, the operational changeover must be organized alongside technical migration: who takes ownership of open work, and how are the old and new systems reconciled?

Executable Models Need Unambiguous Semantics

An intuitive process diagram is insufficient. The engine must be able to determine unambiguously when a step is activated, how branches are evaluated, and when parallel paths merge again. Events, timers, errors, and end states also need defined meanings.

Within process modeling, BPMN 2.0.2 defines the conformance subclasses Descriptive, Analytic, and Common Executable. They describe different scopes of supported model elements and attributes. The specification also defines execution semantics, including those for activities, events, gateways, and sequence flows.^[2]

Not every formally valid model is operationally easy to execute. Highly complex branches, unclear termination conditions, or hidden side effects make review and maintenance more difficult. The definition should therefore remain as precise as necessary and as understandable as possible.

Orchestration Connects Human and Technical Work

A workflow engine can coordinate different types of work. Human tasks are offered to a role or person. Service tasks call applications. Message handling waits for an external event. Timers establish deadlines or trigger escalations. Rules select a permissible path.

Weske distinguishes system workflows from human-interaction workflows at the architecture level. For system workflows, the engine invokes the applications defined by the model. For human-interaction workflows, the same runtime combines automatically invoked applications with human interactions; organizational information such as availability and capability can inform how work is offered.^[6]

In orchestration, a central process instance determines which interaction is expected next. WS-BPEL, for example, describes executable business processes that connect web-service interactions into stateful process logic.^[3] This distinguishes orchestration from a mere sequence of independent API calls.

Technical automation and human responsibility remain separable. The engine can prepare data, check responsibilities, and offer the next step. This does not automatically turn a substantive review or approval into a machine decision.

Connecting Task Status to Process Continuation

The engine must recognize whether an expected human task has delivered a usable result, failed, or been withdrawn. Claiming and delegation are part of the task lifecycle described, for example, by WS-HumanTask.^[4] For the runtime, what matters is which of these states permits a transition.

Digital work instructions can support performance of the task. Confirmation of their final step continues the process instance only if the agreed result type and associated prerequisites are satisfied. Closing a browser window does not provide such a result.

Decision Logic and Process Control Are Not the Same

Gateways and conditions select between modeled paths. A simple condition might be: if the measurement lies within the approved range, documentation review follows; otherwise, a substantive assessment is initiated.

Complex decision rules should not be hidden without control in the process diagram. They can be maintained as versioned decision tables or separate rule models. The engine invokes the decision, stores inputs and result, and then continues along the selected path.

Technical evaluation of a condition does not replace a substantive decision when that decision is assigned to a human. With Human in the Loop, the engine must therefore take the result of the required review or approval into account before continuing the dependent step.

Execution State and Process History Serve Different Purposes

The engine needs a current state to determine the next step. That state alone is insufficient for traceability. History is equally important: when an instance started, which task was offered to whom, which result arrived, and why a path was selected.

Event Sourcing is one possible architectural principle in which states are derived from stored events. A workflow engine does not, however, necessarily have to be implemented entirely through event sourcing. What matters is that substantively relevant operations are preserved traceably and that technical runtime data are not confused with complete audit evidence.

Engine logs can contain technical details while omitting substantive rationales. Conversely, not every internal retry of a technical call belongs in the substantive process history. The two levels need a clear mapping.

The Runtime Has a Different Job from the Editor

The distinction between modeling and runtime is more than an interface distinction. Weske separates build time from runtime in workflow management systems: at build time, the workflow model is enriched with the technical information needed for execution. At runtime, concrete workflow instances are created from that model and their enactment is controlled by the engine.^[6]

A modeling interface therefore edits the process definition. The runtime, by contrast, manages instances that have already started. Task clients and connected domain applications interact with this runtime but serve other functions.

This separation is also visible in the Workflow Reference Model reproduced in the OMG Workflow Management Facility specification: process definition, execution, invoked applications, task clients, and administration are distinguished.^[5] The historical WfMC version is additionally listed in the bibliography.

A product can offer these components together. Separating them conceptually helps assess changes in a targeted manner. This does not by itself prevent an administrator from changing domain data; that requires permissions that are actually enforced and traceable interventions.

Distinguishing a Workflow Engine, State Machine, and Task Control

A state machine describes permissible states and transitions of a single object. A workflow engine typically coordinates multiple activities, participants, and objects over a longer process. It can use state machines internally but is not identical to them.

Task Orchestration focuses more narrowly on creating and continuing tasks depending on results. An Approval Workflow describes the specific flow of review, approval, rejection, and rework. Both can be executed by a workflow engine but do not cover its entire runtime responsibility.

A queue is not a workflow engine either. It transports or buffers messages. Only a process definition, instance state, transition logic, correlation, and controlled continuation together provide orchestration of a complete process.

What a Workflow Engine Does Not Provide on Its Own

• It does not guarantee data quality. Inputs, equipment interfaces, and assignments need separate controls.
• It does not automatically create compliance. Requirements must be correctly translated into processes, roles, and controls.
• It is not a complete domain application. Materials, documents, and master data may reside in other authoritative components.
• It does not prevent uncontrolled complexity. Too many special paths and hidden rules make operation and review harder.

The Decisive Test Lies in the Interruption

A demonstration of the normal path does not yet show whether an engine reliably continues a long-running operation. Interruptions are more informative: a restart while waiting, a late response, a repeated delivery, or a version change while steps remain open.

These cases reveal whether the stored state, message correlation, and transition logic tell the same story. Only then can it be explained why a specific instance continues at precisely this point.