An AES-256 PDF encrypted with a non-ASCII password opens in the program that wrote it and nowhere else. The cause is almost always a missing preparation step: ISO 32000-2 §7.6.4.3.3 requires the password to be processed with the SASLprep profile of stringprep before it is UTF-8 encoded and hashed. PDFlibPas, the PDF library for Delphi and C++Builder, performs that preparation inside Encrypt, EncryptFile and DecryptFile
This is not the wrong-password story and not the permission-bit story. If your users are typing a password you never issued, the retry machinery in the article on retrying encrypted PDF passwords is what you want, and if you are trying to work out what an existing file actually enforces, the encryption and permissions audit covers that ground. This one is narrower and stranger: the password is correct, the user typed it correctly, and the file still refuses to open somewhere else
Why does a non-ASCII password open in one reader but not another?
Because the two programs hash different byte sequences from the same keystrokes. The revision 6 key derivation in ISO 32000-2 §7.6.4.3.3 takes the password as UTF-8 bytes, truncates to 127 bytes, appends a salt, and runs the hardened hash; the result is checked against the /U and /O entries in the encryption dictionary. Nothing in that chain is fuzzy. One differing byte anywhere in the input produces a completely different digest, the validation fails, and the reader has exactly one thing it can say: wrong password
The bytes diverge because Unicode offers several ways to type what looks like the same password. A Chinese password may arrive as precomposed characters from one input method and as compatibility forms from another. A German or French password copied out of a word processor can carry a NO-BREAK SPACE (U+00A0) where the user believes there is an ordinary space, or a SOFT HYPHEN (U+00AD) that renders as nothing at all. SASLprep exists to collapse all of those into one canonical form before anybody hashes anything, so that every conforming implementation derives the same key from the same intent
What does SASLprep actually change about a password?
RFC 4013 defines SASLprep as a profile of the stringprep framework in RFC 3454, and it is four ordered steps rather than one transformation. Mapping comes first: RFC 3454 table C.1.2 (non-ASCII spaces) is mapped to U+0020, and table B.1 (characters commonly mapped to nothing) is deleted outright. Normalisation to Unicode NFKC follows, which is the step that folds compatibility characters and combining sequences. Then the prohibited-output check rejects anything in tables C.2.1 through C.9. Finally the bidirectional rule from RFC 3454 section 6 is applied to the normalised string
PDFlibPas implements the whole profile in the PDFlibSASLprep unit, which exposes a single entry point. PLSASLprepPassword takes the raw password, writes the prepared form into a var parameter, and returns False when the password must be refused. The function is deliberately total on the happy path: an ASCII-only password comes back byte-identical, so nothing about existing deployments changes
uses
PDFlibSASLprep;
var
Prepared: WideString;
begin
// RFC 4013: mapping, then NFKC, then prohibited output, then the bidi rule
PLSASLprepPassword('I' + WideChar($00AD) + 'X', Prepared); // -> 'IX' B.1 deletes SOFT HYPHEN
PLSASLprepPassword('a' + WideChar($00A0) + 'b', Prepared); // -> 'a b' C.1.2 maps NBSP to U+0020
PLSASLprepPassword(WideString(WideChar($00AA)), Prepared); // -> 'a' NFKC folds ORDINAL INDICATOR
PLSASLprepPassword(WideString(WideChar($2168)), Prepared); // -> 'IX' NFKC folds ROMAN NUMERAL NINE
PLSASLprepPassword('user', Prepared); // -> 'user' ASCII is never touched
end;The U+200B ambiguity that the tables do not resolve
One code point lands in two RFC 3454 tables at once, and the two tables disagree. ZERO WIDTH SPACE (U+200B) falls inside the C.1.2 range U+2000 to U+200B, where the rule says map it to U+0020, and it also falls inside the B.1 range U+200B to U+200D, where the rule says delete it. Read the mapping step in either order and you get different bytes out of the same password: a+U+200B+b prepares to a b under C.1.2 and to ab under B.1. RFC 4013 names both tables and does not say which wins, so this is a genuine ambiguity in the specification rather than a reading error. PDFlibPas tests C.1.2 membership first and therefore maps U+200B to a space, which is the behaviour other widely deployed stringprep implementations settled on; matching them is the only thing that matters here, because the goal is byte agreement with whatever reader the customer happens to use
Reading old files: prepared first, raw second
The fix creates its own compatibility problem. Every AES-256 file written before the change hashed the raw UTF-8 password, so making the reader strictly conformant would lock customers out of their own archives. PDFlibPas resolves this on the read side by trying two candidates in order. TPDFDocument.SetPassword builds a candidate list that starts with the prepared form and falls back to the raw form, and it only adds the prepared entry when the document is actually AES-256 and the two forms differ. For an ASCII password the forms are identical, the list holds one entry, and the cost of the whole mechanism is a single string comparison. DecryptFile does the same thing along its direct AES-256 rewrite path, calling PLDirectDecryptFileAES256 with the prepared password first
var
Lib: TPDFlib;
Bytes: AnsiString;
begin
Lib := TPDFlib.Create;
try
Lib.SetOrigin(1);
Lib.DrawText(100, 100, 'saslprep roundtrip');
// Strength 3 and 4 are the two AES-256 values; both are prepared before hashing
Lib.Encrypt('ow' + WideChar($00AD) + 'ner', 'pa' + WideChar($00AD) + 'ss', 4,
Lib.EncodePermissions(1, 0, 0, 0, 0, 0, 0, 1));
Bytes := Lib.SaveToString;
finally
Lib.Free;
end;
Lib := TPDFlib.Create;
try
// 'pass' is what SASLprep produced and what any conforming reader computes,
// so the plain ASCII form opens a file created with the soft-hyphen form
if Lib.LoadFromString(Bytes, 'pass') = 1 then
Caption := IntToStr(Lib.PageCount);
finally
Lib.Free;
end;
end;The fallback carries one guard worth copying. The second attempt in DecryptFile runs only when the prepared and raw forms differ and the first attempt reported no hard error code. A structural failure means the input is damaged or is not the encryption revision you assumed, and retrying a broken file with another password just burns a second full parse over hostile input; the reasoning behind that reflex is laid out in the note on parsing untrusted PDFs safely. Note also that there is no fallback on the write side, and that asymmetry is intentional. Reading tolerates history, writing does not: every new AES-256 file gets the conformant bytes
Which passwords get rejected outright, and what is error 604?
SASLprep can refuse a password entirely, and when it does, encryption must fail loudly rather than silently substitute something. Encrypt and EncryptFile prepare both the owner and user passwords whenever Strength is 3 or 4, return 0 on refusal, and set LastErrorCode to PDFLIB_ERROR_PASSWORD_SASLPREP, which is 604. Two families of input trigger it. The prohibited-output tables reject control characters (C.2.1 and C.2.2), private-use code points (C.3), non-characters (C.4), lone surrogates (C.5), U+FFFD (C.6), ideographic description characters (C.7), and the display-control and tagging ranges (C.8 and C.9). Separately, the RFC 3454 section 6 bidi rule rejects any string that contains a RandALCat character from table D.1 unless the string both begins and ends with one and contains no left-to-right letters at all
var
Lib: TPDFlib;
begin
Lib := TPDFlib.Create;
try
// U+0007 is a C.2.1 control character, so preparation refuses the password
if Lib.Encrypt('owner', 'bad' + WideChar($0007), 4,
Lib.EncodePermissions(1, 0, 0, 0, 0, 0, 0, 1)) = 0 then
begin
if Lib.LastErrorCode = PDFLIB_ERROR_PASSWORD_SASLPREP then // 604
ShowMessage('The password contains characters that PDF encryption does not permit.');
end;
finally
Lib.Free;
end;
end;That bidi rule is the one that will surprise your support desk. An Arabic or Hebrew password ending in a Western digit, or one with a stray Latin letter in the middle, is refused by the specification even though it looks perfectly reasonable in the entry field. Surface 604 as a message about the password characters, not as a generic encryption failure, or somebody will spend an afternoon looking for a bug in your key derivation
Honest limits: NFKC, an approximate LCat, and one Delphi trap
Two parts of the implementation are approximations, and both deserve to be stated plainly rather than buried. NFKC normalisation is performed by the Windows NormalizeString API, loaded dynamically from Normaliz.dll. When that library is unavailable the mapped string is used unnormalised, which means the mapping and prohibition steps still run but compatibility folding does not. In practice the DLL has shipped with every Windows release since Vista, so the degraded path is a pre-Vista and non-Windows concern rather than a live one, but a password that relies on NFKC folding would produce different bytes there and that is a real, if remote, divergence. The bidi check is the second approximation: detecting LCat characters uses the common letter ranges instead of the full RFC 3454 table D.2, and the direction of that error is what makes it acceptable. A missed LCat character can only cause the bidi rule to pass where the specification would have rejected, never the reverse, and it never touches the mapping or normalisation steps, so the prepared byte sequence of an accepted password is unchanged. The residual risk is therefore a policy divergence rather than a byte divergence: an exotic-script password that a stricter implementation would refuse to accept at all. Every password both sides accept hashes identically, which is the property interoperability actually depends on
Finally, a Delphi syntax trap that costs an hour if you have not hit it before. When a function returns a procedural type, assigning it without parentheses does not call it. The compiler reads Proc := GetNormalizeProc; as taking the address of GetNormalizeProc itself, then reports E2009 with the unhelpful complaint that calling conventions differ, because the accessor uses the default convention while the imported API type is stdcall. The empty parentheses are mandatory
type
TNormalizeString = function(NormForm: Integer; SrcString: PWideChar; SrcLength: Integer;
DstString: PWideChar; DstLength: Integer): Integer; stdcall;
function GetNormalizeProc: TNormalizeString; // loads Normaliz.dll on first use
...
var
Proc: TNormalizeString;
begin
// Proc := GetNormalizeProc; // E2009: reads as @GetNormalizeProc, conventions differ
Proc := GetNormalizeProc(); // correct: calls the accessor and assigns its result
if not Assigned(Proc) then
Exit; // no NFKC available, mapped string is used as-is
end;Password preparation is one of those details that never appears in a feature list and decides whether an encrypted document survives contact with a customer in another locale. The Encrypt, EncryptFile, DecryptFile and SetPassword entry points described here are part of the losLab PDF Developer Library Pascal Edition for Delphi and C++Builder, whose product page carries the full encryption reference and the complete error-code table