When an AES-256 revision 6 PDF rejects a password you know is correct, the usual culprit is the round counter in the R6 hardened hash (ISO 32000-2 Algorithm 2.B). The PDFium Component for Delphi had exactly this bug before v2.57.2: TPdf.SaveAsEncrypted counted rounds from the wrong base, and roughly 30% of the files it produced were refused by FPDF_LoadDocument with FPDF_ERR_PASSWORD
The nasty part is not the arithmetic. The nasty part is that the defect was invisible to every test that encrypted a file and then checked it with the same code, and that a single successful round trip proved nothing. This post walks through what Algorithm 2.B actually asks for, where a one-round slip hides, why it only fails some of the time, and how to build a regression test that catches a bug like this before a customer does
What does the R6 hash termination rule actually say?
Algorithm 2.B stops only after at least 64 rounds, and then only when the last byte of the most recent AES output E is less than or equal to the round number minus 32, where the round number is the 1-based count of rounds already executed. After the Nth round you compare against N. Not N − 1, not the index of the next round. That one definition is the entire bug surface
The rest of the algorithm is mechanical. Start with K as the SHA-256 of password, salt and (for the owner key) the 48-byte /U value. Each round builds K1 as password ‖ K ‖ UK repeated 64 times, encrypts it with AES-128-CBC without padding using the first 16 bytes of K as key and the next 16 as IV, takes the first 16 bytes of E as a big-endian 128-bit integer mod 3, and picks SHA-256, SHA-384 or SHA-512 accordingly to produce the next K. The final hash is the first 32 bytes of K. ISO 32000-2 §7.6.4.3 defines it; the data-dependent loop length is deliberate, because it makes the work per password guess unpredictable for a GPU cracker
Here is the correct shape of the loop, written as an illustration rather than a drop-in implementation. BuildK1, AesCbcNoPad128, FirstBlockMod3 and the hash helpers stand in for whatever primitives your codebase already has
// Illustration: Algorithm 2.B loop skeleton (ISO 32000-2, revision 6)
K := Sha256(Password + Salt + UK);
RoundNumber := 0; // rounds executed so far
repeat
Inc(RoundNumber); // this is round 1, 2, 3 ...
K1 := BuildK1(Password, K, UK); // (Password || K || UK) repeated 64 times
E := AesCbcNoPad128(Copy(K, 0, 16), Copy(K, 16, 16), K1);
case FirstBlockMod3(E) of // first 16 bytes of E, big-endian, mod 3
0: K := Sha256(E);
1: K := Sha384(E);
2: K := Sha512(E);
end;
until (RoundNumber >= 64) and (E[High(E)] <= RoundNumber - 32);
Result := Copy(K, 0, 32);Read the termination line with the counter in mind. The increment sits at the top of the body, so by the time the until test runs, RoundNumber holds exactly the number of completed rounds, which is what the standard means by the round number
Where the off-by-one hides in a Delphi port
The PDFium Component bug was a counter that started at −1 with the increment at the top of the loop, so the value tested in the until clause was always the executed round count minus one. Two things go wrong at once: the 64-round minimum becomes 65, and the byte threshold is one notch tighter than the standard says. Both shift the exit point by exactly one round in the cases where it matters
The same slip has several equally plausible disguises. Start at 0 and increment at the bottom of the body, after the test has already been written against the pre-increment value. Use a for loop index that is 0-based and reuse it in the comparison. Translate a C reference whose loop variable is checked before ++. Each of these reads naturally in code review, and each produces a hash that looks perfectly random, because it is: it is just a different, equally valid-looking SHA output that no other reader computes. The fixed CalculateR6Hash in FPdfEncrypt.pas now carries a comment that pins the 1-based semantics, precisely because the wrong version does not look wrong
Why does an AES-256 R6 PDF fail only some of the time?
The failure is probabilistic because the extra round only changes the result when the terminating byte lands in the one-round window between the correct threshold and the shifted one. In every other case both versions exit on the same round and produce the same K, so the file opens fine. In the PDFium Component measurement, 18 of 60 encryptions (about 30%) produced /U, /O, /UE and /OE values that disagreed with a standard implementation; the other 70% were byte-for-byte correct
Fresh random salts are drawn for every save, so the same document and the same password succeed on one run and fail on the next. From a support desk that looks like flaky storage, a corrupt upload, or a user mistyping a password. Nobody thinks to blame a loop counter when seven out of ten files open. If your product sees intermittent "wrong password" reports on PDFs your own code encrypted, and only on revision 6, this is the first place to look
Revision 5 is immune, since it uses a single SHA-256 pass with no iteration. The ISO/TS 32003 AESV4 path is not: TPdfEncryptionRevision value erR7 reuses Algorithm 2.B for its password hashes, so the same fix covers the GCM files described in AES-256-GCM encryption and PDF MAC integrity with ISO/TS 32003
The self-consistency trap: why round-trip tests pass
A round trip through your own encrypt and your own verify always succeeds, because both sides run the same wrong loop and agree on the same wrong hash. The defect only exists relative to somebody else's implementation, so the judge must be an independent one: the PDFium engine in pdfium.dll, Acrobat, qpdf, or any reader you did not write
This is the uncomfortable general lesson for anyone writing crypto-adjacent document code. Unit tests that compute a hash and compare it to the output of the function under test prove determinism, not correctness. Known-answer vectors from the standard or a reference tool are better, and a cross-implementation round trip is the closest thing to ground truth you get. The PDFium Component is in a lucky position here: encryption is done in Pascal by EncryptDocumentBytes, while decryption on load goes through PDFium's own C++ security handler, so reopening the saved file with TPdf is a genuine independent check rather than a mirror
uses
SysUtils, PDFium, FPdfEncrypt;
procedure SaveR6(const Source, Target: string);
var
Pdf: TPdf;
Opts: TPdfEncryptOptions;
begin
Pdf := TPdf.Create(nil);
try
Pdf.FileName := Source;
Pdf.Active := True;
Opts := TPdfEncryptOptions.Default;
Opts.UserPassword := 'userpw';
Opts.OwnerPassword := 'ownerpw';
Opts.Revision := erR6; // AESV3, Algorithm 2.B password hash
if not Pdf.SaveAsEncrypted(Target, Opts) then
raise Exception.Create('SaveAsEncrypted failed');
finally
Pdf.Free;
end;
end;TPdfEncryptOptions.Default starts at revision 5 with all permissions allowed and metadata encrypted, so set Revision explicitly when you want R6. TPdf.SaveAsEncryptedToStream takes the same options record when the output goes to a TStream instead of a file. Note that the passwords are AnsiString in the options record, which matters if you are passing non-ASCII credentials; normalise and encode them before they reach the record
How do you regression-test a probabilistic encryption bug?
Repeat the cross-implementation round trip enough times that a 30% failure rate cannot slip through. A single run misses the PDFium Component bug about 70% of the time. Twenty runs, each with fresh salts, drive the miss rate to about 0.08% (0.7 to the 20th power), which is why the shipped regression test loops 20 times instead of once
procedure TEncryptTests.R6_EncryptedDocumentReopensWithUserPassword;
var
Pdf: TPdf;
I: Integer;
Target: string;
begin
Target := TempOutputPath('encrypt_r6_roundtrip.pdf');
for I := 1 to 20 do
begin
SaveR6(SamplePdf, Target); // fresh salts on every save
Pdf := TPdf.Create(nil);
try
Pdf.Password := 'userpw';
Pdf.FileName := Target;
Pdf.Active := True; // PDFium verifies /U independently
AssertTrue('Run ' + IntToStr(I) + ' must reopen', Pdf.Active);
finally
Pdf.Free;
end;
end;
end;Pair it with the negative case, a wrong password that must fail, so that a broken loader cannot make the positive test pass vacuously. And if you build with Lazarus, rebuild the library package before running the test project after touching Lib\*.pas; the test build links the precompiled .ppu units of the package, and running against stale object code gives you a reassuring green on a bug you have not actually fixed. Seeding is a related trap worth reviewing in the same pass: FPC Random producing constant encryption keys is another case where the output looks random and is quietly wrong
FAQ
Does the R6 round-counter bug affect files encrypted with revision 5?
No. Revision 5 hashes the password with a single SHA-256 pass and has no iteration count, so there is no round counter to get wrong. Only revision 6 and the ISO/TS 32003 AESV4 revision, which both use Algorithm 2.B, are affected
Can a file written by the buggy version still be opened?
About 70% of them open normally because their hash happened to match. The rest carry /U and /O values no standard reader reproduces, so the password check fails everywhere; the practical fix is to re-encrypt from the plaintext source with v2.57.2 or later
Why test against PDFium instead of my own decryption code?
Your own decryptor shares the same Algorithm 2.B implementation as your encryptor, so a wrong round count cancels out and every round trip passes. An independent implementation such as PDFium's security handler is the only check that can disagree with you
Encryption, PDF MAC validation and the rest of the save pipeline ship in the PDFium Component for Delphi and C++Builder, with source and regression tests for both Delphi and Lazarus