Reproducible PDF Output in Delphi: Byte-Identical Saves

HotPDF Delphi Component produces byte-identical PDF output across saves when the ReproducibleOutput property is True: it pins the Info /CreationDate and /ModDate to a fixed date, replaces the wall-clock document identifier with a seeded or content-derived hash, substitutes constants for every random byte the AES encryption paths would otherwise draw, and sorts every dictionary it serializes. The flag exists for regression suites and build-artifact comparison, not for production documents, and the reasons for that boundary are the interesting part. The scenario that drives the feature is a golden-file test. You render an invoice, commit the PDF, and assert that tomorrow's build produces the same bytes. It never does. The file opens fine in every viewer, the text is identical, the page tree is identical, and the diff still lights up in four or five places. Anyone who has tried to put a PDF generator under a byte-level regression test has hit this wall, and the fix is not "strip the timestamps" but a precise accounting of every place the writer consults something other than the document itself

Why do two saves of the same PDF differ?

Two saves of the same document differ because a PDF writer, HotPDF included, consults four sources of entropy that have nothing to do with page content: the wall clock, the document identifier, the cryptographic random number generator, and the memory order of dictionary entries. Each one is legitimate on its own. ISO 32000-1 wants them there. They simply make the file a function of when and where it was written rather than of what it contains

• The clock. The Info dictionary carries /CreationDate and /ModDate (ISO 32000-1 §14.3.3, Table 317) as D:YYYYMMDDHHmmSS strings with a time-zone suffix (§7.9.4), and the XMP packet repeats the same instant as xmp:CreateDate and xmp:ModifyDate. HotPDF stamps both from FCreationDate, which the constructor initializes to Now, so the two saves differ in the second they were written
• The identifier. The trailer /ID array (ISO 32000-1 §14.4) holds a permanent identifier and a modification identifier. HotPDF's default recipe hashes the file name together with the current time down to the millisecond for the first element, and hashes that plus GetTickCount for the second. Two identifiers, two fresh values on every run
• The random bytes. Standard security depends on the identifier and on genuine randomness. For AES-256 the file encryption key, the validation and key salts, and every CBC initialization vector are drawn from the system random source (ISO 32000-2 §7.6.4.4.7 requires random salts). Because /U, /UE, /O and /OE are all computed from those bytes, an encrypted document changes in its entirety even when the plaintext does not. The older algorithms fold the first /ID element into the key (ISO 32000-1 §7.6.3.3, §7.6.3.4), so a fresh identifier alone is enough to re-key the file
• The order. A PDF dictionary is an unordered mapping, and a writer that walks its in-memory list emits keys in insertion order. Any code path that builds a resource dictionary in a different sequence, or a loaded document that was parsed from a different layout, produces a legal but textually different file

What does ReproducibleOutput pin down?

Setting ReproducibleOutput := True before BeginDoc or before SaveLoadedDocument replaces each of the four sources with a fixed value, and it does so in the same code paths that would otherwise reach for the clock or the random generator, so no separate cleanup pass is needed. Notice what is missing from the list above: the content. Fonts, page streams, image data and the cross-reference table are already deterministic for the same input; the noise lives entirely in the metadata and the security layer, which is why one targeted property can remove it. The property defaults to False and nothing in the library turns it on for you

var
  Pdf: THotPDF;
begin
  Pdf := THotPDF.Create(nil);
  try
    Pdf.AutoLaunch := False;
    Pdf.FileName := 'golden-invoice.pdf';
    Pdf.ReproducibleOutput := True;     // before BeginDoc
    Pdf.BeginDoc;
    Pdf.CurrentPage.SetFont('Arial', [], 12);
    Pdf.CurrentPage.TextOut(40, 40, 0, 'Invoice 2026-0042');
    Pdf.EndDoc;
  finally
    Pdf.Free;
  end;
end;

Inside BeginDoc the reproducible branch assigns FCreationDate := EncodeDate(2026, 1, 1) and seeds the document identifier with MD5CalcString('HotPDF-reproducible-seed') instead of the file-name-plus-clock digest. That single assignment covers both Info dates and both XMP dates, because all four are rendered from the same field. When the file is finally written, BuildDocumentIdentifiers asks ComputeCanonicalDocumentIdentifier for the trailer identifier: it exports the whole object graph in canonical order, zeroes the digits of any D: date string it finds so the timestamps cannot leak back in through the hash, and takes the MD5 of the result. Both elements of /ID receive that value. The same content-derived identifier is used when a loaded document is encrypted without ever passing through BeginDoc, which is the case for ActivateProtection on a file you opened with LoadFromFile

The random bytes are the least obvious substitution. The AES-256 key routine wraps its random source in a local helper that, under the flag, calls FillChar(P^, Count, $5A) for the 32-byte file encryption key and for each 8-byte salt, and the AES-128 and AES-256 string and stream encryptors switch from AESGenerateRandomIV to AESGenerateStaticIV, which fills the initialization vector with 14 * (1 + I) for slot I. With the key, the salts and the vectors all fixed, /U, /UE, /O, /OE and every encrypted stream come out identical on the second run. Finally, SaveToStream switches DeterministicDictionaryOrder on whenever the reproducible flag is set, and the serializer then insertion-sorts each dictionary by the raw bytes of its key names, shorter prefix first, with the original index as a tie-breaker. That is the same ordering the diagnostic writer uses, described in the article on hand-editing a PDF and repairing it afterward; the reproducible flag borrows only the ordering, not the rest of that writer's plain-text layout

Why did the fixed date still leak the wall clock?

The v2.752.2 fix exists because the fixed creation date was originally decided in the constructor, and the constructor cannot know a property the caller has not set yet. The normal call sequence is Create, then ReproducibleOutput := True, then BeginDoc. At construction time FReproducibleOutput is still False, so FCreationDate received Now and kept it. The identifier and the random bytes were correctly pinned, so the two files agreed almost everywhere and disagreed in exactly two date strings and two XMP fields. Moving the assignment into the reproducible branch of BeginDoc, next to the seeded identifier, put the decision at the point where the property has its final value

The regression test that missed this is worth more than the fix. Two saves that both run inside the same wall-clock second write the same D: string by accident, and the byte comparison passes for a bug that fails on any slower machine. The corrected test sleeps 1100 ms between the two saves so the PDF timestamp is guaranteed to cross a second boundary, runs the case for plain, AES-128 and AES-256 output with real passwords on the two encrypted variants, and compares the two buffers with CompareMem, reporting the first differing offset on failure so the diff points at a specific object instead of a whole file. A byte comparison proves determinism and nothing else, so keep a separate assertion that reloads the encrypted output with the user password and reads a page count; a change that makes the file stable and unreadable at the same time must not slip through on the strength of a green diff

function SaveOnce(const Target: string): TBytes;
var
  Pdf: THotPDF;
  Stream: TFileStream;
begin
  Pdf := THotPDF.Create(nil);
  try
    Pdf.AutoLaunch := False;
    Pdf.FileName := Target;
    Pdf.ReproducibleOutput := True;
    Pdf.OwnerPassword := 'owner';
    Pdf.UserPassword := 'user';
    Pdf.CryptKeyLength := aes256;
    Pdf.ActivateProtection := True;
    Pdf.BeginDoc;
    Pdf.CurrentPage.SetFont('Arial', [], 12);
    Pdf.CurrentPage.TextOut(40, 40, 0, 'reproducible save');
    Pdf.EndDoc;
  finally
    Pdf.Free;
  end;
  Stream := TFileStream.Create(Target, fmOpenRead or fmShareDenyWrite);
  try
    SetLength(Result, Stream.Size);
    if Stream.Size > 0 then
      Stream.ReadBuffer(Result[0], Stream.Size);
  finally
    Stream.Free;
  end;
end;

// in the test body
A := SaveOnce(PathA);
TThread.Sleep(1100);          // force a different PDF timestamp second
B := SaveOnce(PathB);
Assert.AreEqual<Integer>(Length(A), Length(B));
Assert.IsTrue(CompareMem(@A[0], @B[0], Length(A)),
  'two saves under ReproducibleOutput must be byte-identical');

Is a reproducible encrypted PDF still secure?

No. A document encrypted under ReproducibleOutput is not protected in any meaningful sense, and the flag must be off for anything that leaves the test directory. The AES-256 file encryption key is thirty-two bytes of $5A, the salts are eight bytes of $5A, and the initialization vectors follow a published arithmetic pattern. The password still gates the /UE and /OE wrappers, but the wrapped key is a constant, so anyone who knows the constant can decrypt every content stream without a password at all. The salts being fixed also removes the per-document uniqueness that ISO 32000-2 §7.6.4.4.7 relies on to keep identical passwords from yielding identical /U strings across files. Read the AES-256 setup article for what the encryption properties promise when the random source is intact; under the reproducible flag those promises are suspended

The identifier trade-off is subtler. ISO 32000-1 §14.4 intends the second /ID element to change on every modification so that tools can tell an updated file from its ancestor, and a reproducible save writes the same value into both slots. Because that value is a hash of the canonical object graph, two documents with different content still get different identifiers, which is better than a constant. But the seed that BeginDoc uses for key derivation is the same string for every document on every machine, and a reader that keys on /ID to distinguish files, an annotation cache or a form-data sidecar for instance, will conflate every reproducible file that happens to hash the same

What does the flag not cover?

ReproducibleOutput removes the entropy the writer introduces on its own; it cannot remove entropy that enters through the environment or through code paths it does not control, and three of those are easy to trip over

• The time-zone suffix. _DateTimeToPdfDate appends the local UTC offset, so D:20260101000000+08'00' on one build agent and D:20260101000000-05'00' on another are different bytes for the same fixed date. Reproducibility holds across runs on one machine, or across machines that share a time zone; pin the agent's zone if your golden files travel
• Incremental updates. SaveIncrementalUpdate computes its modification identifier from the target path, GetTickCount and the current time with no reproducible branch, because an incremental section is by definition a new modification. Compare full rewrites, not appended deltas
• The passthrough shortcut. SaveLoadedDocument normally copies an unmodified, unencrypted source file byte for byte instead of re-serializing it. The reproducible flag disables that shortcut and forces a full rewrite so the ordering and identifier rules apply, which means the reproducible save of a loaded file is slower than the default and is never a copy of the input. Diff it against a previous reproducible save, never against the original

One more lesson from the same release, about what a passing check does and does not prove. A PDF/X-6 test fixture called CharProcs.DeleteValue('A'), which freed a directly held glyph stream, then re-inserted the same pointer, and separately handed one direct ExtGState object to both a resource dictionary and a pattern. The conformance validator passed intermittently on that use-after-free and double ownership because it was reading whatever the freed memory happened to hold. When a structural check flickers, look at the ownership of the test input before you look at the validator. Reproducible output makes that discipline cheaper: once two saves are byte-identical, the only remaining source of a flicker is the object graph itself, and a structural diff from the catalog down will find it

The ReproducibleOutput, DeterministicDictionaryOrder and encryption properties described here ship in the standard HotPDF Delphi Component for Delphi and C++Builder, and the same flag drives the library's own regression corpus, so the behavior you get in a test suite is the behavior the component is tested with