Append-only Log: Why Records Are Added Rather Than Overwritten

The writing rule of an append-only log

The principle applies whether a log records measurements, access, or operational activities. The ordered sequence and the treatment of entries already included determine its structure.

A later correction appears as an additional entry. Technical protection against interventions outside the intended writing model must be considered separately.

Two entries, with the original transaction preserved

A fictional consumption log contains entry B-104 at position 104: 12.5 kilograms consumed, linked to a specific batch and task. At position 105, K-105 follows, referencing B-104 and correcting the amount to 11.5 kilograms. B-104 remains readable with its original content.

The defining feature of append-only behavior is that later information is added while the previous sequence is preserved. Whether K-105 has already been reviewed and how consumption affects inventory are questions for additional operational rules. A read view must not infer these details solely from the higher position number.

Data provenance additionally describes provenance relationships: for example, who recorded the consumption and which supporting record led to the correction. The log position, by contrast, identifies where the entry was added to the stored sequence.

Order is more than a timestamp

An append-only log needs a defined order. Within a single process, a consecutive sequence number may suffice. In distributed systems, several components may write almost simultaneously, clocks may differ, or entries may arrive late. In that setting, “sort by timestamp” is not automatically equivalent to the actual operational sequence.

The architecture should therefore distinguish between the time something happened, the time it was recorded, and the position at which it was committed to the log. Causal relationships can additionally express that an entry is based on or was triggered by another entry.

A global order is not always necessary. For some questions, a reliable order per batch, task, or data stream is sufficient. What matters is that the chosen order fits the operational purpose and that behavior is defined for concurrent entries.

What a reliable log entry should contain

A log is only as informative as its entries. A sequence of unstructured text messages is difficult to attribute, filter, or assess in its domain context later. A reliable entry therefore needs a stable identity and an unambiguously described context.

Typical components include:

• Entry identifier: a unique ID through which other entries and analyses can reference the operation;
• Event type: a domain-specific label such as “measurement recorded,” “material transaction posted,” “review completed,” or “correction created”;
• Time information: at least the time of recording and, if different, the time of the operational event;
• Actor: the identified person, role, application, or machine that triggered the entry;
• Related objects: such as a batch, product, task, device, material, or document;
• Payload: the recorded value, state change, or operational content;
• Versions: the SOP, model, schema, or rule versions used;
• Relationships: references to preceding, corrected, or triggered entries.

The fields do not need identical names in every system. What matters is their defined meaning. A timestamp without a known time zone or a username without a stable identity may be formally present yet still fail to support reliable attribution.

Logical history and physical storage must be distinguished

The term becomes misleading when “only add” is read as a prohibition on all physical changes. Databases reorganize pages, rebuild indexes, create backups, or move data to other storage classes. A controlled migration may also transfer entries into a new technical structure.

Such operations do not necessarily conflict with the logical append-only principle. They must, however, preserve the entries visible to operational users, their order, identities, and relationships. Where compaction or archiving is permitted, it requires defined rules. A summary may speed up operational queries, for example, but should not silently replace the only copy of the original history if that history must still be retained.

Deletion requirements must also be addressed explicitly. Data protection, retention periods, or justified cleanup may conflict with an unlimited log. The solution is not a blanket claim that data is immutable, but a coordinated strategy for retention, access restrictions, archiving, and deletion.

The same sequence must be readable after restoration

A successful backup run alone does not show that the log can be used again. Restoration requires checking that entries, positions, identities, and necessary referenced versions fit together. If the referenced original B-104 is missing, for example, a preserved correction K-105 no longer fully explains the operation.

The operating concept must also define which confirmed state was backed up and how entries created afterward are handled. A restored older state must not silently appear to be the complete current history. Indexes and archive storage can support access; evidence that the sequence has been preserved remains a separate matter.

Append-only alone does not protect against tampering

An application interface may allow only new entries to be added, while privileged database access could still alter older records. The logical writing rule is therefore only one layer of control. Technical and organizational measures must protect it.

These include restrictive write and administration permissions, separate responsibilities, logged maintenance, secured time sources, regular backups, restoration tests, and monitoring for unexpected changes. Requirements depend on criticality, the threat model, and the context of use.

Cryptographic methods can provide additional verifiability. Hash chaining links consecutive entries through their hash values; RFC 9162 describes a publicly verifiable append-only log for Certificate Transparency whose Merkle tree supports consistency proofs between an earlier and a later state.^[1] This example also illustrates the distinction: verifiable append-only behavior results from a specific cryptographic structure and verification procedure. It is not an automatic property of every table that permits only INSERT operations.

NIST describes blockchains as tamper-evident and tamper-resistant distributed ledgers, while also noting different permission and trust models.^[2] An append-only log requires neither a blockchain nor tokens. Whether hash chaining, external checkpoints, or digital signatures are needed is a separate architectural decision.

Corrections extend the history

In a conventional record, an incorrect value is often replaced directly. Without additional logging, the previous value can then no longer be identified. In the append-only model, the correction is treated as a separate operation.

A correction event should unambiguously reference the original entry, specify the corrected content, and retain the reason required by the operational context. Depending on the process, it may also need the person performing the action, a reviewing role, and an approval. The application can then generate a current view in which the corrected value applies. The underlying history still contains both entries.

Errors are not preserved so that they can continue to be used. They are retained so that the actual sequence remains traceable. The current presentation must clearly distinguish the valid value from one that merely documents an earlier state. Otherwise, a complete history may produce an unclear operational view.

For controlled changes, an audit trail provides the more specific control perspective: it describes which relevant inputs and changes must be traceable with identity, time, and, where applicable, a reason. PIC/S and the FDA emphasize preserving original information and making relevant changes reconstructable.^[3][4] An append-only log, by contrast, is the more general storage principle that can preserve such changes as additional entries.

A current view can complement the log

An application can derive a view of valid consumption from B-104 and K-105 once the meaning and review status of both entries are established. An access log may not need this kind of state calculation at all. Projections are therefore a possible use of a log, not part of its definition.

Event sourcing goes further: domain events form the authoritative basis of application state. The rules for rebuilding that state are part of this design.[5]

A complete history can still contain incorrect data

Preserving earlier entries makes what was recorded visible. It does not prove that the content was factually correct. An incorrectly calibrated device, a mixed-up material identifier, or an inaccurate manual input may all be stored correctly in an append-only log.

Data integrity therefore requires more than a particular storage behavior. Completeness, consistency, and accuracy are established throughout the data lifecycle: through appropriate capture, attribution, review, protection, retention, and controlled correction. The append-only principle supports traceability but does not replace these controls.

Completeness is equally important. If relevant operations bypass the log, the stored sequence can remain internally consistent while still failing to represent the full operational reality. Interface errors, offline work, and subsequent bulk postings must therefore be identifiable and handled in a controlled manner.

What an append-only log is not automatically

• Not a complete audit trail: Without an actor, time, reason, and operational context, a change may remain inadequately explained.
• Not event sourcing: A log can accumulate information without being the authoritative source of all application states.
• Not immutability: Administrative access or technical errors can affect older data if no additional controls exist.
• Not a blockchain: Distributed consensus mechanisms are not required for the basic storage principle.
• Not automatic data accuracy: An incorrect value can be stored completely and in order.
• Not unlimited retention: Retention periods, data protection, and archiving rules must be defined separately.

The earlier sequence must remain identifiable in the new state

The transaction example comes down to a simple control question: after K-105 is added, does B-104 still exist with the same identity and content? For larger logs, this question must remain answerable for the relevant earlier state and its order.

This makes append-only behavior concretely verifiable. The writing rule, its protection, and its preservation during operation work together. The substantive assessment of a correction and the calculation of a current view build on that foundation, but require their own rules.