PDFlibPas implements SHA-3 and SHAKE256 on a single Keccak-f[1600] permutation in Object Pascal. The two families differ in exactly two places: the pad10*1 domain byte, $06 for SHA-3 and $1F for SHAKE per FIPS 202 §6.2 and §6.3, and the rate rule. SHAKE256 pins its rate at 136 bytes regardless of output length
That second difference is the one people get wrong, because it does not look like a difference at all. SHA3-256 also runs at rate 136, so a SHAKE256 built by copying the SHA-3 code path passes every 32-byte test you throw at it and then produces garbage the first time somebody asks for 64. What follows is how the two families ended up sharing one context record in PDFlibDigest, where the one-shot entry point stops, and why the streaming XOF had to exist before Ed448 or ML-DSA could
Why do SHA-3 and SHAKE256 share one Keccak context?
Because the only per-family state is a single byte, and PDFlibPas stores it in TSHA3Context as Suffix. Once the domain separator lives in the context instead of being hard-coded in the finalization routine, SHA3AbsorbBlock, SHA3Update and SHA3Final serve both families unchanged, and there is exactly one copy of the 24-round permutation in the library. The initializers are where the two paths diverge, and they diverge completely
Type
TSHA3Context= Record
State: Array [0.. 24] Of UInt64;
Buffer: Array [0.. 135] Of Byte;
BufferLength: Integer;
Rate: Integer;
OutputLength: Integer;
// Domain separator: $06 for SHA3, $1F for SHAKE (FIPS 202 §6.2/6.3)
Suffix: Byte;
End;
Procedure SHA3Init(Var Context: TSHA3Context; OutputBits: Integer);
Begin
FillChar(Context, SizeOf(Context), 0);
Context.OutputLength:= OutputBits div 8;
Context.Rate:= 200- 2* Context.OutputLength; // rate follows the digest size
Context.Suffix:= $06;
End;
// SHAKE256: rate is fixed at 136 bytes whatever the caller asks for,
// because SHAKE is an extendable-output function
Procedure SHAKE256Init(Var Context: TSHA3Context; OutputBytes: Integer);
Begin
FillChar(Context, SizeOf(Context), 0);
Context.OutputLength:= OutputBytes;
Context.Rate:= 136;
Context.Suffix:= $1F;
End;The formula in SHA3Init gives 136, 104 and 72 bytes of rate for SHA3-256, SHA3-384 and SHA3-512, so rate and security level are welded together for the fixed-length family. SHAKE256Init deliberately breaks that link: OutputLength becomes a request for bytes, not a declaration of security strength, and the sponge capacity stays at 512 bits no matter how much output you draw. That decoupling is the definition of an extendable-output function, and writing it as two initializers over one context makes the distinction visible in the source rather than buried in a comment
The 136-byte ceiling on the one-shot entry point
The convenience functions in PDFlibPas refuse anything outside 1..136 bytes, and that limit is honest rather than arbitrary. SHAKE256StreamRange and SHAKE256ByteArray both route through SHA3Final, which pads the last block, permutes once, and reads output straight out of the state. Only the first Rate bytes of that state are legal output, so one squeeze yields at most 136 bytes; past that you need a fresh permutation per block. Both real consumers fit inside the window anyway: the PDF signature profile digests 32 bytes, and Ed448 needs 114 per RFC 8032 §5.2. SHAKE256StreamRange takes the same FirstLength, SecondStart, SecondLength triple as SHA256StreamRange and SHA3_256StreamRange, so validation code walks a signature /ByteRange the same way for every algorithm and never special-cases the XOF
What goes wrong in pad10*1 when the position lands on rate-1?
Nothing, provided you XOR both pad bytes into the state instead of assigning them. FIPS 202 §B.2 puts the domain suffix at the current absorb position and $80 at the last byte of the block. When the message happens to leave the position at Rate- 1, those two writes target the same byte and must combine into $9F. Store rather than XOR and one of them wins, the padding is silently wrong for exactly one message length in 136, and every regression vector you own passes because none of them lands there. PLShakeXOFFinalize in PDFlibPas XORs both, which makes the overlapping case fall out of the general one with no branch
// Pads the final partial block and enters the squeeze phase
Procedure PLShakeXOFFinalize(Var Ctx: TPLShakeXOF);
Begin
If Not Ctx.Absorbing Then
Exit;
Ctx.Absorbing:= False;
// suffix at SqueezePos, $80 at Rate-1; when SqueezePos = Rate-1 the two
// XORs land on the same byte and correctly combine to $9F
Ctx.State[Ctx.SqueezePos div 8]:= Ctx.State[Ctx.SqueezePos div 8]xor
(UInt64(Ctx.Suffix)shl (8* (Ctx.SqueezePos and 7)));
Ctx.State[(Ctx.Rate- 1)div 8]:= Ctx.State[(Ctx.Rate- 1)div 8]xor
(UInt64($80)shl (8* ((Ctx.Rate- 1)and 7)));
SHA3Permute(Ctx.State);
Ctx.SqueezePos:= 0;
End;TPLShakeXOF: absorbing straight into the sponge
PDFlibPas exposes unbounded output through a separate record, TPLShakeXOF, holding only State as 25 UInt64 words, Rate, Suffix, an Absorbing flag and SqueezePos. There is no message buffer: PLShakeXOFAbsorb XORs incoming bytes directly into the state and permutes whenever it fills a block, with SqueezePos serving as the offset in both phases. PLShakeXOFInit picks the variant with one boolean, True for SHAKE128 at rate 168 and False for SHAKE256 at rate 136, and PLShakeXOFSqueeze keeps permuting until the caller has the byte count it asked for. Cross-block output was checked at 200 bytes, past the first block boundary in both variants
Var
Ctx: TPLShakeXOF;
Digest: Array [0.. 113] Of Byte; // Ed448 wants 114 bytes, RFC 8032 §5.2
Begin
PLShakeXOFInit(Ctx, False); // False = SHAKE256, rate 136
PLShakeXOFAbsorb(Ctx, @Dom4[1], Length(Dom4));
PLShakeXOFAbsorb(Ctx, @Prefix[1], Length(Prefix));
PLShakeXOFAbsorb(Ctx, @Msg[1], Length(Msg));
PLShakeXOFFinalize(Ctx);
PLShakeXOFSqueeze(Ctx, @Digest[0], SizeOf(Digest));
End;
// Or, when one buffer in and one buffer out is all you need:
// PLShakeXOF(False, Input, Output, 114)Absorbing in several calls matters more than it looks. Ed448 prefixes the message with a dom4 string and a 57-byte value before hashing, and a rejection sampler wants to feed a seed and index bytes without concatenating strings first. Having the streaming record land before either consumer was the point: the pure Pascal Ed448 and Brainpool curve work needs 114 bytes in one shot, and the FIPS 204 ML-DSA port squeezes indefinitely from both variants. Neither could have been written against the one-shot API
Why does PDFlibPas refuse to expose SHAKE256 as a digest selector?
Because there is no legal way to pick it. TPDFlibDigestAlgorithm in PDFlibDigitalSign runs daAuto, daSHA1, daSHA256, daSHA384, daSHA512, daSHA3_256, daSHA3_384, daSHA3_512, and stops. There is no daSHAKE256, and the omission is a decision, not an oversight. RFC 8702 registers SHAKE in CMS, but inside a PDF signature the only place ISO/TS 32002 puts SHAKE256 is as the internal digest of Ed448, where the signature algorithm chooses it and the caller does not. Publishing it as a free-standing DigestAlgorithm value would let anyone pair SHAKE256 with RSA or ECDSA and produce a signature dictionary no validator will accept. The capability ships, the selector does not, and verification-side code calls SHAKE256StreamRange over the /ByteRange layout directly. If you are choosing a digest for signatures a counterparty must validate today, daSHA256 through daSHA3_512 inside a PAdES B-B to B-LTA workflow with timestamping and long-term validation data remains the answer; ISO/TS 32001 is what governs when SHA-3 becomes routine in PDF at all
Testing an XOF: the vector that looks like it passes
Compare the full output length, and use more than one vector. The empty-string SHAKE256 result is the one everybody memorizes, and it is a weak test precisely because it is so widely quoted. The 32-byte SHAKE256 output for abc is worse: its first 16 bytes, 483366601360a8771c6863080cc4114d, coincide with several neighboring vectors, so a truncated comparison reports success while the discriminating half, 8db44530f8f1e1ee4f94ea37e78b5739, goes unchecked. The PDFlibPas regression compares both vectors over their whole length. Add a cross-block case as well, because a squeeze loop that never crosses a rate boundary has not been tested at all
The broader lesson from folding SHAKE into an existing SHA-3 implementation is that the shared code was never the risk. Keccak-f[1600] is the same permutation for both families and it either matches the vectors or it does not. The risk lives in the two bytes and one integer that differ, and in the ceiling you have to state honestly rather than let callers discover. To see how the digest layer fits alongside the signing, encryption and PDF/A tooling in a native Object Pascal codebase, the PDFlibPas Delphi PDF library product page lists the full component set and the supported compiler matrix