Reproducible PDF Output in Delphi: Byte-Identical Saves

Two PDFs written by the same invoice code one second apart are almost never the same file. A binary compare first trips inside the Info dictionary, where /CreationDate has moved by a second, then in the XMP packet, then at the trailer /ID. Turn on AES-256 and the comparison collapses on nearly every stream, because the file encryption key itself was freshly generated. Masking the dates before the compare repairs the first two differences and none of the others. A golden-file test only becomes trustworthy when every value the writer takes from somewhere other than the document is fixed at the point where it is read

Four values that never came from the document

HotPDF Delphi Component, like any conforming PDF writer, fills four kinds of value from outside the page content: the clock, the document identifier, the cryptographic random source and the in-memory order of dictionary entries. ISO 32000-1 has a reason for each of them

• The clock. The Info dictionary carries /CreationDate and /ModDate (ISO 32000-1 §14.3.3, Table 317) as D:YYYYMMDDHHmmSS strings with a time-zone suffix (§7.9.4), and the XMP packet repeats the same instant as xmp:CreateDate and xmp:ModifyDate. All of them are rendered from one creation timestamp that the component takes from Now when it is constructed
• The identifier. The trailer /ID array (§14.4) holds a permanent identifier and a modification identifier. By default the first is an MD5 of the file name plus the current time down to the millisecond, and the second hashes that value again together with the system tick count
• The random bytes. An AES-256 document needs a 32-byte file encryption key, an 8-byte validation salt and an 8-byte key salt for /U, another pair for /O, four filler bytes inside /Perms, and a 16-byte initialization vector in front of every encrypted string and stream. ISO 32000-2 §7.6.4.4.7 wants the salts random. RC4 and AES-128 derive their key from the passwords and the first /ID element (ISO 32000-1 §7.6.3.3), so a fresh identifier on its own is enough to re-key the whole file
• The order. A PDF dictionary is an unordered mapping (§7.3.7). A writer that walks its in-memory list emits keys in the order code happened to add them, so two code paths that build the same resource dictionary in a different sequence produce a legal but textually different file

Everything else already follows from the input: content streams, compressed image data and cross-reference offsets repeat for the same drawing calls. Even the one field that looks random is not: the six-letter prefix on an embedded font subset (ISO 32000-1 §9.6.4, the ABCDEF+ part of ABCDEF+Arial) is computed from the set of glyphs the subset keeps, so the same text yields the same prefix on every run. The noise sits in the metadata and the security layer, which is why a single property can remove it

What ReproducibleOutput pins

Setting ReproducibleOutput := True before BeginDoc replaces each of the four sources with a fixed value, in the same code paths that would otherwise reach for the clock or the random generator. There is no cleanup pass afterward. The property defaults to False, and nothing in the library turns it on for you

var
  Pdf: THotPDF;
begin
  Pdf := THotPDF.Create(nil);
  try
    Pdf.AutoLaunch := False;
    Pdf.FileName := 'golden-invoice.pdf';
    Pdf.ReproducibleOutput := True;     // must be set before BeginDoc
    Pdf.BeginDoc;
    Pdf.CurrentPage.SetFont('Arial', [], 12);
    Pdf.CurrentPage.TextOut(40, 40, 0, 'Invoice 2026-0042');
    Pdf.EndDoc;
  finally
    Pdf.Free;
  end;
end;

Under the flag, BeginDoc sets the creation timestamp to 1 January 2026, 00:00 local time, and because both Info dates and the XMP dates are rendered from that one value, a single assignment covers all of them. The same branch seeds the internal document identifier from a constant string instead of the file name and the clock. When the file is written, the trailer /ID is computed as an MD5 over a canonical export of the whole object graph in which the digits of every D: date string are zeroed first, so a timestamp cannot sneak back in through the hash. Both elements of /ID receive that value. A loaded document that you encrypt without ever calling BeginDoc (set ActivateProtection after LoadFromFile) gets the same content-derived identifier

The random bytes are replaced with constants. The AES-256 file key, all four salts and the /Perms filler are filled with $5A, and every CBC initialization vector, for AES-128 strings and streams as well as AES-256, becomes the pattern 14 * (1 + I) for byte I. That pattern is non-zero on purpose. An all-zero IV is the textbook weak choice for CBC, and a regression suite has no business normalising it. With key, salts and vectors fixed, /U, /UE, /O, /OE, /Perms and every encrypted stream come out identical on the second run

Finally, SaveToStream switches DeterministicDictionaryOrder on whenever the reproducible flag is set, and it does so before it picks between the linearised, object-stream and classic layouts, so output with LinearizeOutput or UseObjectStreams is sorted too. Keys are ordered by their raw bytes, a key that is a prefix of another sorts first, and the original position breaks ties. That is the ordering the diagnostic writer uses, described in the article on hand-editing a PDF and repairing it afterward; the reproducible flag borrows the ordering and none of that writer's plain-text layout. Turning on DeterministicDictionaryOrder by itself is a half measure. The keys line up, but the Info dates and both trailer /ID strings still carry the clock, so two saves keep differing in the same four places

A test that cannot pass by luck

The date is fixed in BeginDoc because the constructor runs before the caller has had any chance to set the property. The normal sequence is Create, then ReproducibleOutput := True, then BeginDoc, and only the last of those sees the flag with its final value. That ordering has a direct consequence for test design. Two saves that land inside the same wall-clock second write the same D: string by accident, so a fast test passes whether or not the date is pinned, and then fails months later on a slower build agent

The fix is cheap: wait 1100 ms between the two saves so the timestamp is guaranteed to cross a second boundary, run the case for plain, AES-128 and AES-256 output with real passwords on the encrypted variants, and report the first differing byte offset rather than a bare true or false. An offset points at one object. A boolean points at the whole file

uses
  System.SysUtils, System.IOUtils, HPDFDoc;

function SaveOnce(const Target: string; Encrypt: Boolean;
  KeyType: THPDFKeyType): TBytes;
var
  Pdf: THotPDF;
begin
  Pdf := THotPDF.Create(nil);
  try
    Pdf.AutoLaunch := False;
    Pdf.FileName := Target;
    Pdf.ReproducibleOutput := True;
    Pdf.VerifySavedOutput := True;   // strict reload right after EndDoc
    if Encrypt then
    begin
      Pdf.OwnerPassword := 'owner';
      Pdf.UserPassword := 'user';
      Pdf.CryptKeyLength := KeyType; // aes128 or aes256
      Pdf.ActivateProtection := True;
    end;
    Pdf.BeginDoc;
    Pdf.CurrentPage.SetFont('Arial', [], 12);
    Pdf.CurrentPage.TextOut(40, 40, 0, 'reproducible save');
    Pdf.EndDoc;
  finally
    Pdf.Free;
  end;
  Result := TFile.ReadAllBytes(Target);
end;

// Your own helper: -1 when equal, otherwise the first differing offset
function FirstDifference(const A, B: TBytes): Integer;
var
  I, N: Integer;
begin
  N := Length(A);
  if Length(B) < N then
    N := Length(B);
  for I := 0 to N - 1 do
    if A[I] <> B[I] then
      Exit(I);
  if Length(A) <> Length(B) then
    Result := N
  else
    Result := -1;
end;

procedure CheckReproducible(Encrypt: Boolean; KeyType: THPDFKeyType);
var
  A, B: TBytes;
  Offset: Integer;
begin
  A := SaveOnce('run-a.pdf', Encrypt, KeyType);
  Sleep(1100);                     // force a different timestamp second
  B := SaveOnce('run-b.pdf', Encrypt, KeyType);
  Offset := FirstDifference(A, B);
  if Offset >= 0 then
    raise Exception.CreateFmt('Saves differ at byte offset %d', [Offset]);
end;

Call CheckReproducible(False, aes128), CheckReproducible(True, aes128) and CheckReproducible(True, aes256); the key type is ignored on the plain run. VerifySavedOutput earns its place in this test. With it on, EndDoc reloads the file it has just written in a fresh THotPDF instance with cross-reference recovery disabled, using the user password when the document is protected, and raises if that load fails. A change that makes the output stable and unreadable at once would sail through on a green diff; the reload stops it

Is a reproducible encrypted PDF still secure?

No. A document encrypted under ReproducibleOutput has no real protection, and the flag belongs off for anything that leaves the test directory. The AES-256 file key is thirty-two bytes of $5A. The password still gates the /UE and /OE wrappers, but what they wrap is a published constant, so anyone who knows it can decrypt every content stream without a password. Fixed salts also remove the per-file uniqueness that ISO 32000-2 §7.6.4.4.7 relies on to keep the same password from producing the same /U string in two files. The AES-256 setup article describes what the encryption properties deliver with the random source intact; under the reproducible flag none of that applies. Gate the property behind a test-only configuration value, not a setting a production code path can reach

AES-GCM is handled differently. With CryptKeyLength := aesgcm (PDF 2.0 only), the component draws the 8-byte nonce prefix from the system random source whatever ReproducibleOutput says, and then counts upward for each encrypted string or stream. That is the right behaviour. GCM fails badly when a nonce repeats under the same key: two ciphertexts leak the XOR of their plaintexts, and an attacker can recover enough to forge authentication tags. A pinned key plus a pinned nonce would make that repeat certain across every file the suite produces. Encrypted golden files therefore use aes128 or aes256, which run in CBC mode

The identifier is a softer trade-off. ISO 32000-1 §14.4 intends the second /ID element to change on every modification, and a reproducible save writes the same value into both slots. Because that value hashes the object graph, different content still gets different identifiers. A reader that keys a cache on /ID, an annotation store or a form-data sidecar for instance, will still treat two reproducible saves of the same content as one file. In a test suite that is exactly what you want

Inputs that stay outside the writer

ReproducibleOutput covers the entropy the writer introduces on its own. A few more inputs come from the build machine or from operations that record a new moment by definition

• The time-zone suffix. Every PDF date string ends in the local UTC offset, so the same fixed date reads D:20260101000000+08'00' on one build agent and D:20260101000000-05'00' on another. Golden files hold across runs on one machine, or across machines that share a zone. If the files travel between offices, pin the agents' time zone
• Incremental updates. SaveIncrementalUpdate keeps the source's permanent identifier and builds a new modification identifier from the previous one, the target path, the tick count and the current time. An appended section is a new modification by definition. Compare full rewrites, not appended deltas
• The passthrough shortcut. SaveLoadedDocument normally copies an unmodified, unencrypted source byte for byte instead of serialising it again. Under the flag the file is always rewritten so the ordering and identifier rules apply, which makes the save slower and never a copy of the input. Diff it against a previous reproducible save, never against the original
• Timestamps you add yourself. AddDocumentAttachment stamps the embedded file's /Params /CreationDate and /ModDate (ISO 32000-1 §7.11.4) with the moment of the call, and a signature records its signing time in /M. Those dates describe events the caller asked for, so the flag leaves them alone. Keep attachments and signatures out of byte-level cases, or mask those strings in the compare
• Installed fonts. A TrueType subset is cut from the font file installed on the machine that runs the test. Two agents with different releases of Arial embed different glyph programs for the same text, and the subset prefix can change with them. Ship the font files with the build image

Loaded documents follow the same rules. Set the property before loading, make your edits, and save through SaveLoadedDocument; the output is a normalised rewrite you can commit as the reference for the next run

var
  Pdf: THotPDF;
begin
  Pdf := THotPDF.Create(nil);
  try
    Pdf.AutoLaunch := False;
    Pdf.ReproducibleOutput := True;   // set before loading
    if Pdf.LoadFromFile('incoming.pdf') <= 0 then
      raise Exception.Create('incoming.pdf did not load');
    // ... edits under test ...
    Pdf.SaveLoadedDocument('normalized.pdf');  // full rewrite, sorted keys
  finally
    Pdf.Free;
  end;
end;

Byte-identical saves also corner a flickering conformance check. Once the writer is stable, a check that passes on one run and fails on the next is reacting to the object graph the test builds, such as one direct dictionary handed to two parents, so inspect the test input before the validator. A structural diff from the catalog down shows which object moved between the two runs

The ReproducibleOutput, DeterministicDictionaryOrder, VerifySavedOutput and encryption properties used above are part of the HotPDF Delphi Component, which ships for both Delphi and C++Builder