Technical Article

Deterministic PDF ID in Delphi for Reproducible Builds

Generate the same invoice twice from the same template and data, then diff the two PDFs. Two places differ: the hex strings in the trailer /ID array and the ModDate string in the document information dictionary, both taken from the clock. In losLab PDF Library, SetDeterministicDocumentID(1) plus a fixed ModDate makes the files byte identical, for a document without per-run XMP UUIDs or attachments

The obvious first attempt is to pin the dates and rerun. CreationDate and ModDate now match, and the hashes still differ, now only in two 32-digit hex strings near the very end of the file, inside /ID [<…> <…>]. That identifier is computed separately from the metadata, and until it stops depending on the time of the save, a build cache keyed on the file hash never hits and a content addressable store keeps one blob per run instead of one per document

What the trailer ID array is for

The trailer /ID identifies a file; it does not checksum its content. ISO 32000-1 §14.4 defines it as an array of two byte strings. The first is the permanent identifier, assigned when the document is created and meant to survive every later edit. The second is the changing identifier, which a writer refreshes whenever it modifies the file. A document management system compares the pair to decide whether two files are revisions of one document or two unrelated documents. Table 15 in §7.5.5 makes the entry required when the trailer carries /Encrypt and optional otherwise, and §14.4 notes that leaving it out can get in the way of workflows that rely on file identity

The specification never says how to compute the value. It suggests an MD5 digest over things like the current time, the file path, the file size and the values in the information dictionary, and the clock is the ingredient that makes the result unique. Uniqueness is exactly what you want from an identity marker and exactly what wrecks reproducibility. By default losLab PDF Library follows that suggestion in its simplest form, an MD5 of the current timestamp, so two documents created a second apart get different permanent identifiers even when every other byte matches

Diagram comparing identical Delphi build runs whose default wall clock based PDF trailer IDs differ against byte identical deterministic output
The default trailer id folds the wall clock into the file identity while deterministic mode derives it from a stable seed

Turning on a reproducible identifier

Deterministic mode is a per document switch and it is off by default, so existing output does not change until you ask for it. SetDeterministicDocumentID takes 0 or 1 and returns 1 when the value was accepted, 0 for anything else; GetDeterministicDocumentID reports the current setting. SetDocumentIDSeed supplies an explicit seed string, and with a seed in place the identifier is an MD5 of that string, written into both elements of the array. GetDocumentFileID reads /ID[0] back after the save, which is handy for logging or for a test assertion

var
  Lib: TPDFlib;
  FileID: WideString;
begin
  Lib := TPDFlib.Create;
  try
    Lib.SetDeterministicDocumentID(1);        // right after Create
    Lib.SetDocumentIDSeed('invoice-4471-rev3');
    Lib.SetInformation(7, 'D:20260101000000Z');   // CreationDate
    Lib.SetInformation(8, 'D:20260101000000Z');   // ModDate
    Lib.SetOrigin(1);
    Lib.DrawText(100, 700, 'Invoice 4471');
    Lib.SaveToFile('invoice.pdf');
    FileID := Lib.GetDocumentFileID;          // same value on every run
  finally
    Lib.Free;
  end;
end;

The identifier is computed at save time. Flipping the switch late in a document build still works, and a changed seed reaches the file on the next full save: save with seed A, save again with seed B, and the two outputs carry different identifiers, while going back to seed A gives you the original value again

Turn the switch on immediately after creating the object, though. Fonts, images, graphics states and other page resources get names in the page's resource dictionary, and while deterministic mode is on those names come from a counter. Anything added before the switch keeps a randomly generated name, which leaves the identifier stable but changes a few bytes in the content streams on every run

Why does the identifier still change when no seed is set?

Because the information dictionary is part of the hash. Leave the seed empty, or pass an empty string to SetDocumentIDSeed to clear one, and losLab PDF Library derives the identifier from the file itself: it runs the serialised body through SHA-256 while writing it and uses the first 16 bytes of the digest for both elements of /ID. Any change to page content changes the identifier, which is the whole point. The catch is that CreationDate and ModDate live in that body too, and the save routines stamp ModDate with the current time unless the caller has set it

PDF Library for Delphi decision flow showing SetDocumentIDSeed taking priority over the derived document-state seed used for deterministic PDF ids
An explicit seed overrides everything else while the derived path hashes info dictionary entries including CreationDate and ModDate
Lib.SetDeterministicDocumentID(1);
Lib.SetDocumentIDSeed('');   // derive /ID from the saved bytes
// The body hash covers the Info dictionary, so the dates must be
// build inputs. Key 8 also stops the save from stamping ModDate.
Lib.SetInformation(2, 'Quarterly Report');        // Title
Lib.SetInformation(5, 'reporting-service 4.2');   // Creator
Lib.SetInformation(7, 'D:20260101000000Z');       // CreationDate
Lib.SetInformation(8, 'D:20260101000000Z');       // ModDate
Lib.SaveToFile('report.pdf');

Key 8 does double duty here, and it is the line people most often leave out. Setting ModDate through SetInformation marks the date as caller supplied, and from then on SaveToFile and SaveToString leave it alone. Without it every save writes a fresh timestamp, the hashed body changes, and the content-derived identifier changes with it. Take the dates from the source record or from a fixed epoch. Now has no business in a reproducible build

A few things are kept out of the hash on purpose. Randomly generated resource names and the UUID in an XMP xmpMM:InstanceID entry are normalised before hashing, so two runs that differ only in those tokens get the same /ID. The file bytes still differ in those places, so a PDF/A style document with a fresh instance UUID per run gives you a reproducible identifier but not a byte identical file. Embedded attachments are another source of drift: AddAssociatedFileFromString stamps the attachment's own /ModDate with the current time, and that date is hashed like any other body byte

Explicit seed or content hash

For build artefacts we would leave the seed empty. A content-derived identifier changes exactly when the bytes change, so a cache keyed on it can never serve a stale file, and you get no second key to keep in sync with the data. An explicit seed fits a different job: a downstream archive that treats /ID[0] as document identity and expects the same identifier across regenerations of the same invoice, even after the template changes. That matches the intent of §14.4 better, at the price that two visibly different files can share an identifier

One consequence of the content path deserves a deliberate decision. When you load an existing PDF and save it in full with deterministic mode on, the incoming /ID is discarded and the new one depends only on what gets written. Two source files that differ only in their trailer identifiers come out with the same /ID. That is what a normalising build step wants. If you are editing a customer's document and its identity has to survive, leave deterministic mode off for that job or save it incrementally

Why does rewriting the ID break an encrypted PDF?

Because in an encrypted document /ID[0] is key material. ISO 32000-1 §7.6.3.3 Algorithm 2 feeds the first element of the file identifier into the encryption key computation for the standard security handler at revisions 2 through 4, together with the padded password, the /O value and the permission bits. That key then produces the /U string a reader checks on open. The key is derived when you call Encrypt or load an encrypted file, before any save, so replacing the identifier during the save would produce a well formed file whose password check fails for everyone, including you

losLab PDF Library therefore refreshes the deterministic identifier only on unencrypted full saves. An encrypted document keeps whatever /ID it already had, and the switch has no effect on that path. Reopening such a file with the user password works as usual. Do not put encrypted output into a byte comparison, either: AES encryption under §7.6.2 starts every encrypted string and stream with a random 16-byte initialisation vector, so two encryptions of the same document never match byte for byte. If you need a regression check there, decrypt both and compare what you get back. Revision handling and permission bits are covered in more depth in the walkthrough of PDF encryption and permission auditing

PDF Library for Delphi: Guard rail map of the deterministic PDF id covering full saves encrypted documents incremental updates and the encryption restore path
Only unencrypted full saves may refresh the id since encrypted files treat /ID[0] as key material and append revisions must stay linked

Incremental saves keep the original identifier

An incremental update leaves every earlier byte of the file in place and appends a new revision after it. The permanence of /ID[0] is what tells a consumer that the appended revision belongs to the same document as the ones before it. Rewriting it would contradict the revisions already in the file and would undercut signature workflows, since a signature covers a byte range of one specific revision. Append mode never touches the identifier in deterministic mode, which keeps the behaviour described in the article on PDF incremental updates and append to stream intact

Keeping output reproducible over time

Every later change to the generator is a chance to put a clock value back into the file: a watermark that prints the generation time, a code path that attaches a file, a different save option. Save options matter as much as content: packing objects into compressed object streams produces a completely different file layout from the classic cross reference table, as explained in the notes on object streams and cross reference streams, so keep those settings fixed per pipeline. The cheapest defence is a test that builds the same document twice in memory and compares the bytes

function BuildQuote(const Seed: WideString): AnsiString;
var
  Lib: TPDFlib;
begin
  Lib := TPDFlib.Create;
  try
    Lib.SetDeterministicDocumentID(1);
    Lib.SetDocumentIDSeed(Seed);
    Lib.SetInformation(7, 'D:20260101000000Z');
    Lib.SetInformation(8, 'D:20260101000000Z');
    Lib.SetOrigin(1);
    Lib.DrawText(100, 700, 'Quote 8812');
    Result := Lib.SaveToString;
  finally
    Lib.Free;
  end;
end;

// Two independent builds must give one byte sequence
if BuildQuote('quote-8812') = BuildQuote('quote-8812') then
  WriteLn('reproducible')
else
  WriteLn('nondeterminism leaked into the output');

The same harness can compare a serial and a parallel RecompressStreams run on the same source: load it into two TPDFlib instances, enable deterministic IDs and pin ModDate on both, set SetOptimizationWorkers(1) on one and a higher worker count on the other, and diff the two SaveToString results. With the clock taken out of both files, whatever difference remains comes from the recompression itself

Reproducible trailer identifiers, metadata control and incremental saving are all part of losLab PDF Library for Delphi and C++Builder, whose reference also lists the encryption options mentioned here