PDFium Component builds the CMS signedAttrs block through TDerWriter.SetOfSorted, because X.690 clause 11.6 requires the members of a SET OF to be emitted in ascending order of their own DER encodings. Assemble them in construction order instead and you get signatures your own verifier accepts and every third-party PAdES validator rejects
That last sentence is the whole reason this defect survived a full regression suite. The signing path was covered, the verification path was covered, the round trip was green on every platform, and the output was still not valid DER. A test that signs with your writer and verifies with your reader cannot detect a deviation from the encoding rules, because both halves share the deviation
What X.690 clause 11.6 actually requires
The members of a SET OF value must be sorted, and the sort key is the complete DER encoding of each member, compared as an unsigned octet string, with the shorter encoding ordered first when one is a prefix of the other. This is a canonical restriction rather than a suggestion: DER admits exactly one encoding of a given value, so an unsorted SET OF is not a differently ordered encoding of the same value, it is not a DER encoding at all. Because ASN.1 tags come first in a TLV, the comparison usually resolves inside the first two octets, and since every member of a CMS attribute list starts with 30, the SEQUENCE tag, the tie breaks on the length octet. The required order therefore looks almost like ordering the attributes by size, which is worth knowing when you are staring at a hex dump wondering why the order appears arbitrary
Why is the minimum attribute set already out of order?
Because the natural construction order and the encoded order disagree from the very first signature you produce, with no optional attributes involved. BuildSignedAttrs in FPdfCms.pas appends content-type, then message-digest, then signing-certificate-v2, then cms-algorithm-protection. With SHA-256 digests and an RSA signature algorithm, the four attribute TLVs begin with these bytes
// Construction order produced by BuildSignedAttrs
// content-type 30 18 (24 content octets)
// message-digest 30 2F (47)
// signing-certificate-v2 30 37 (55)
// cms-algorithm-protection 30 2D (45)
//
// DER order required by X.690 11.6
// 30 18 content-type
// 30 2D cms-algorithm-protection
// 30 2F message-digest
// 30 37 signing-certificate-v2Two of the four attributes move. cms-algorithm-protection at 30 2D has to precede signing-certificate-v2 at 30 37, and it also has to precede message-digest at 30 2F. So the smallest conforming PAdES B-B attribute set PDFium Component can emit was already mis-ordered under the old code, before anyone adds signing-time, commitment-type-indication or a signature-policy-identifier. There was never a configuration in which the bug failed to fire
Why do our own tests pass while every external validator fails?
Because the two families of verifier compute the message digest over different bytes. RFC 5652 clause 5.4 says the input to the signature operation is the DER encoding of the SignedAttributes value, with the IMPLICIT [0] tag replaced by the SET tag 31, and the phrase that decides everything is the DER encoding, not the octets you received. The Windows platform verifier reached through CryptVerifyDetachedMessageSignature takes the received octets, swaps the leading tag and hashes them as they arrived, so order never enters into it and a mis-ordered SET OF digests to exactly what the signer computed. OpenSSL and BouncyCastle, which is what the widely deployed European validators are built on, parse the attribute set into structures and re-encode before digesting. Re-encoding applies the sort, the digest no longer matches the one that was signed, and the signature is reported as broken rather than as mis-encoded, which sends you hunting through key material and certificate chains for a defect that lives in a comparison function
Generalize past this one bug, because that is the expensive part. A same-origin round trip is immune to every deviation from an interoperability specification that both halves of your code share; it proves your writer and your reader agree with each other, which is not the property anyone cares about. For anything that leaves the process as a wire format, the only meaningful regression is one that pins bytes against the specification or runs a foreign implementation over your output. The OpenSSL CMS verification backend earns its place here for a second reason beyond portability: it is a genuinely independent reader of your own DER
The fix, and where it already existed
The DER writer had the sorted variant all along. TDerWriter in FPdfAsn1.pas exposes both SetOf, which wraps a pre-concatenated content buffer in a SET tag and preserves whatever order it was handed, and SetOfSorted, which takes the members individually as a TDerByteArrays and merge-sorts them by their encodings before wrapping. The PDF MAC code in FPdfMac.pas was already building its authenticated attribute set with SetOfSorted; only the CMS signer was not. Keeping the members as an array right up to the wrapping call is the structural half of the fix, because once attributes have been concatenated into a single buffer the ordering information is gone and no care downstream can restore it. A writer API that offers only SetOf(Content: TBytes) for SET OF is an API that invites this bug; the array-taking overload is what makes the correct thing expressible
// Before: members keep the order in which AddAttr appended them
Result:= W.SetOf(ConcatMany(AttrList));
// After: X.690 11.6 ordering applied at the point of encoding
Result:= W.SetOfSorted(AttrList);SET OF sorts, SEQUENCE OF does not
This is the distinction most often confused, and getting it backwards is as damaging as ignoring it. A SEQUENCE OF is an ordered collection: its order is part of the value and you must never sort it. A SET OF is unordered as a value, so DER imposes a canonical order to make the encoding unique. Sorting a SEQUENCE OF corrupts data; leaving a SET OF unsorted produces something that is not DER
- SET OF, must be sorted:
SignedAttributesandUnsignedAttributesinSignerInfo, thedigestAlgorithmsset, thecertificatesandcrlssets inSignedData, thesignerInfosset, the value set inside each individualAttribute, and the attribute set inside eachRelativeDistinguishedNameof an X.501 Name - SEQUENCE OF, never sorted: the RDN list that makes up a
Name, thecertsfield ofSigningCertificateV2, extension lists, and the general-name lists inside them
An RDN is a SET OF sitting inside a SEQUENCE OF one level up, so a distinguished name contains both rules within two levels of nesting. That layout also explains why single-member sets are so forgiving: a set with one element is trivially sorted, which is why the attribute value sets built with W.SetOf inside BuildSignedAttrs were correct as written and needed no change. The bug lives only where a set genuinely holds two or more members, and multi-valued RDNs are rare enough that most name encoders get away with the same mistake for years
How do you keep this from coming back?
Assert on bytes, not on verdicts. The cheapest durable check is a self-audit that walks its own output: parse the emitted signedAttrs, compare each member against the previous one with the same unsigned-octet-string comparison the writer uses, and fail the build if any pair is out of order. It runs in microseconds, it needs no external library, and unlike a verification round trip it tests the property the specification actually states
// Walk the members of a SET OF and confirm X.690 11.6 ordering.
// Members is the list handed to SetOfSorted, or the result of
// re-parsing an emitted 31 xx / A0 xx block
function SetOfIsSorted(const Members: TDerByteArrays): Boolean;
var
I: Integer;
begin
for I:= 1 to Length(Members)- 1 do
if CompareDerBytes(Members[I- 1], Members[I])> 0 then
Exit(False);
Result:= True;
end;
// In the signing test, do not stop at "our verifier said valid".
// Pin the bytes the signer hashed
Attrs:= BuildSignedAttrs(W, Digest, CertHash, DigestOid, SigOid, nil, Options);
Assert(Attrs[0]= $31, 'signedAttrs must carry the SET tag when hashed');
Assert(SetOfIsSorted(SplitSetOfMembers(Attrs)), 'signedAttrs not in DER order');Two limits are worth stating plainly. Sorting signedAttrs does not make a signature interoperable on its own, since everything else in the container has to hold up too and the reasons validators turn down otherwise sound PAdES signatures are varied enough to fill their own discussion. And a sorted SET OF is a property of what you write, not of what you read: PDFium Component still has to accept an incoming signature whose signedAttrs arrive unsorted, because such files exist in the wild and RFC 5652 clause 5.4 tells a verifier to re-encode rather than to reject. Strict on output, tolerant on input, as usual. The DER writer, the CMS builder and the signing path described in the PAdES B-B walkthrough all ship as readable Pascal source with the PDFium Delphi component, which is the point of shipping source for a signing library at all: when a validator somewhere in Europe disagrees with your document, you can read the exact bytes your code emitted and the exact clause they violate, instead of filing a ticket and waiting