DER Length Octets and CMS Slicing in Delphi with PDFium

The four bytes A0 82 05 DC open a context-specific [0] element carrying 1500 bytes of content. Read from left to right they are unambiguous: tag A0, an initial length octet 82 announcing two more, then 05 DC. Read from right to left, starting at the first content byte, the first octet you encounter is DC. Its top bit is set and its low seven bits read 92, so a helper that trusts it places the tag 94 bytes before the content. The true distance is 4

Nobody walks backwards deliberately. It happens because most DER readers hand back the content offset and length of each element, precisely what you need to descend into it, whereas copying a sibling element unchanged needs the offset of its tag. Subtracting from the content offset looks like a two-line job, and it stays correct for as long as every length fits in a single octet. The length of a real certificate set never fits in one

What a signature timestamp does to a CMS already signed

Adding a PAdES B-T timestamp forces every enclosing header of the CMS to be re-encoded whilst five sibling elements must come across byte for byte. The signature-time-stamp attribute that ETSI EN 319 122-1 clause 5.3 defines under OID 1.2.840.113549.1.9.16.2.14 lives in the unsignedAttrs of the SignerInfo from RFC 5652 clause 5.3, and the RFC 3161 token within it is computed over the signature value. So the CMS is already built and already signed when the token arrives from the TSA. One extra attribute lengthens the SignerInfo, which lengthens the signerInfos SET, then SignedData, then the [0] EXPLICIT wrapper, then the outer ContentInfo

In PDFium Component that sequence amounts to three exported calls in FPdfCms.pas and FPdfTsa.pas. SignPadesBytes runs them on your behalf when TPadesSignOptions.Level is plB_T or higher and TsaUrl is set, and it raises EPadesCrypto when a B-T level is requested with an empty TsaUrl rather than writing a plain B-B signature. Build the CMS yourself with BuildSignedData and the chain looks like this

// CmsDer holds the ContentInfo returned by BuildSignedData.
// SignPadesBytes performs the same three steps when Options.Level >= plB_T
var
  SigValue: TBytes;
  Ts: TPadesTimestamp;
begin
  if not ExtractSignatureValue(CmsDer, SigValue) then
    raise Exception.Create('CMS carries no signature value');
  // The TSA messageImprint is computed over the signature value octets
  Ts:= RequestTimestamp(TsaUrl, SigValue, Nonce);
  // Re-encodes SignerInfo up to ContentInfo, copies the siblings unchanged
  AddSignatureTimestampToCms(CmsDer, Ts.TokenDer);
end;

AddSignatureTimestampToCms carries five elements across untouched: the contentType OID, the SignedData version INTEGER, the digestAlgorithms SET, the encapContentInfo SEQUENCE and, where present, the certificates [0] set. In the CMS that BuildSignedData emits with SHA-256, four of those are tiny: 11 bytes for the OID, 3 for the version, 17 for the digest algorithm set, 13 for the detached encapContentInfo. The certificate set holds the signer certificate followed by every chain certificate you passed in. A real X.509 certificate runs to several hundred bytes. This is the one slice whose length octets always take the long form

Why is it impossible to walk DER length octets backwards?

The count of length octets is stored in the first of them, and a backwards walk meets the last one first. X.690 clause 8.1.3.4 defines the short form: one octet, bit 8 clear, bits 7 to 1 holding a length from 0 to 127. Clause 8.1.3.5 defines the long form. Its initial octet has bit 8 set, and bits 7 to 1 count the subsequent octets holding the length as an unsigned big-endian integer. Nothing marks a subsequent octet as subsequent. Its bit 8 is a magnitude bit like the other seven. So a helper that tests the top bit of Buf[ContentOffs- 1] is testing data and then reading data as a count

// The tempting helper, given only the content offset. Do not ship this
function NaiveHeaderStart(const Buf: TBytes; ContentOffs: Integer): Integer;
var
  P, LenByte: Integer;
begin
  P:= ContentOffs- 1;            // lands on the LAST length octet
  if P< 0 then
    Exit(ContentOffs);
  LenByte:= Buf[P];
  if (LenByte and $80)= 0 then   // only meaningful for the FIRST one
    Result:= P- 1
  else
    Result:= P- (LenByte and $7F)- 1;
end;

// A0 82 05 DC (1500 bytes): Buf[ContentOffs- 1]= $DC, $DC and $7F= 92
//   Result= ContentOffs- 94, the tag is at ContentOffs- 4
// A0 82 05 10 (1296 bytes): $10 has bit 8 clear, read as short form
//   Result= ContentOffs- 2, which points at 05 inside the length octets

Both failure directions appear with ordinary sizes. A 1500-byte certificate set ends its header in DC, and the helper starts the slice 90 bytes too early. A 1296-byte set ends in 10, the helper mistakes it for a short-form length, and the slice starts two bytes late with no tag at all. Whether a given chain lands on the first or the second case depends on nothing more than the low byte of its total length

The early case can also run clean off the front of the buffer. For a CMS laid out the way BuildSignedData lays it out, with every enclosing length taking two octets, the certificate set's content begins 60 bytes in: 4 for the ContentInfo header, 11 for the OID, 4 apiece for the [0] and SignedData headers, then 3, 17 and 13 for the small siblings and 4 for the certificate set header. Subtract 94 from 60 and the helper returns -34. Its only guard sits on ContentOffs- 1, so the negative result goes straight into the copy

What X.690 clause 10.1 nails down

DER makes the length encoding a pure function of the length. X.690 clause 10.1 restricts DER to the definite form and demands the minimum number of octets, which removes the two freedoms BER permits: the indefinite form, and a long-form length padded with leading zero octets. Under that rule a content length below 128 always takes exactly one length octet. Any other length takes one initial octet plus exactly as many subsequent octets as the value has significant bytes. That gives 81 80 for 128. At 256 the header grows to 82 01 00, and 65536 needs 83 01 00 00. Code already holding the content length can compute the header size without reading a single byte of the buffer

The derivation is only as sound as the claim that the input really is DER. TDerReader.TryReadTlvAt enforces that claim before any slicing happens. It rejects an initial length octet of $80 (the indefinite form), a long-form length whose first subsequent octet is zero, a single subsequent octet below $80 that ought to have been a short form, and more than four subsequent octets. ReadTlv is built on TryReadTlvAt, so every element AddSignatureTimestampToCms walks has already passed those checks. A BER-style padded length never reaches the arithmetic

Inside AddSignatureTimestampToCms each sibling's tag offset is derived from its content offset and content length in this way. Where the derived start falls before the beginning of the buffer, meaning the offsets cannot belong together, the procedure raises an exception bearing the message CMS: TLV header precedes the buffer start and produces no CMS at all. A malformed rebuild that still parses is far harder to trace than an exception at the point where the offsets stopped agreeing

The derivation carries one further assumption: the identifier takes a single octet. X.690 allows multi-octet identifiers for tag numbers of 31 and above, and TryReadTlvAt parses them. CMS SignedData never uses one, its tags being universal SEQUENCE, SET, INTEGER and OID plus the context-specific [0] and [1], so the assumption holds for every element the timestamp rebuild touches

Copying sibling elements within your own DER code

If you write code against TDerReader, skip the derivation altogether. PeekTlv and ReadNextTlv fill a TDerTlv record whose Offset already points at the tag and whose HeaderLength is the measured size of tag plus length octets, and EncodedBytes returns the element from Offset through to NextOffset. That path also copes with multi-octet identifiers, which no quantity of length arithmetic can recover. The four-out-parameter ReadTlv is convenient for descending. It reports only the content side, though, so reach for it when you need content, and for the record form when you need the whole element

// Every child of SignedData as a complete TLV: tag, length octets, content.
// TDerTlv.Offset points at the tag, so no header arithmetic is involved
function SignedDataChildren(const CmsDer: TBytes): TDerByteArrays;
var
  Root, Ci, Explicit0, Sd: TDerReader;
  Tlv: TDerTlv;
begin
  Result:= nil;
  Root.Initialize(CmsDer);
  if not Root.ReadNextTlv(Tlv) or not Root.EnterConstructed(Tlv, Ci) then
    raise Exception.Create('ContentInfo SEQUENCE expected');
  if not Ci.SkipTlv then                            // contentType OID
    raise Exception.Create('contentType OID expected');
  if not Ci.ReadNextTlv(Tlv) or not Ci.EnterConstructed(Tlv, Explicit0) then
    raise Exception.Create('[0] EXPLICIT content expected');
  if not Explicit0.ReadNextTlv(Tlv) or not Explicit0.EnterConstructed(Tlv, Sd) then
    raise Exception.Create('SignedData SEQUENCE expected');
  while not Sd.AtEnd do
  begin
    if not Sd.ReadNextTlv(Tlv) then
      raise Exception.Create('SignedData child malformed');
    SetLength(Result, Length(Result)+ 1);
    Result[High(Result)]:= Sd.EncodedBytes(Tlv);   // e.g. A0 82 05 DC ...
  end;
end;

For the cases where some other API gives you nothing beyond a content offset and a length, the forward derivation is short. DerHeaderStart, shown alongside its boundary check in the next section, is your own code; it mirrors the rule from clause 10.1 and returns -1 for offsets that cannot be consistent, so the caller holds a value it can refuse rather than a plausible-looking start byte

Why do short test certificates conceal the mistake?

A small placeholder certificate keeps the certificate set in the short form, and the backwards walk is correct for precisely that case. A 32-byte set encodes as A0 20 and a 64-byte one as A0 40. Walking back from the content lands on that single length octet, whose top bit is clear because it is the first and only one. Right answer, wrong reason. A timestamp test built on certificates that small can re-parse the result, find the attribute, report B-T and never once push a certificate set of realistic size through the slice

Any code path depending on how a length is encoded needs test input that crosses the encoding boundary. For DER that means content of 128 bytes or more, which forces the long form, and of 256 bytes or more, which forces a second subsequent octet. The check below compares DerHeaderStart against the header the reader measured whilst parsing, at both boundaries and at the 1500-byte case from the opening. An unsorted SET OF in signedAttrs slips past same-origin round trips for the same structural reason: the test only feeds inputs on which wrong code and right code agree

// Your own helper for APIs that report only content offset and length.
// Assumes a one-octet identifier, true for every tag in CMS SignedData
function DerHeaderStart(ContentOffs, ContentLen: Integer): Integer;
var
  LengthOctets, Remaining: Integer;
begin
  LengthOctets:= 1;                  // short form, X.690 8.1.3.4
  if ContentLen>= 128 then
  begin
    Remaining:= ContentLen;          // long form, X.690 8.1.3.5
    while Remaining> 0 do
    begin
      Inc(LengthOctets);             // one per significant byte
      Remaining:= Remaining shr 8;
    end;
  end;
  Result:= ContentOffs- LengthOctets- 1;
  if Result< 0 then
    Result:= -1;                     // inconsistent offsets: let the caller raise
end;

// Boundary check, written as your own test code: the derived tag offset
// must equal the offset TDerReader measured while parsing the header
procedure CheckHeaderDerivation;
const
  Lens: array[0..6] of Integer= (127, 128, 255, 256, 1500, 65535, 65536);
var
  W: TDerWriter;
  R: TDerReader;
  Tlv: TDerTlv;
  Content: TBytes;
  I: Integer;
begin
  W:= TDerWriter.Create;
  try
    for I:= Low(Lens) to High(Lens) do
    begin
      SetLength(Content, Lens[I]);
      R.Initialize(W.Wrap($A0, Content));   // A0 7F / A0 81 80 / A0 82 05 DC ...
      W.Clear;
      if not R.PeekTlv(Tlv) then
        raise Exception.CreateFmt('unreadable TLV for %d bytes', [Lens[I]]);
      if DerHeaderStart(Tlv.ContentOffset, Tlv.ContentLength)<> Tlv.Offset then
        raise Exception.CreateFmt('header start wrong for %d bytes', [Lens[I]]);
    end;
  finally
    W.Free;
  end;
end;

The same boundary is reachable end to end through the exported functions. Hand BuildSignedData chain certificates whose combined size puts the certificate set just under and just over 128 and 256 bytes, run AddSignatureTimestampToCms on each result, then walk the output with SignedDataChildren and compare the certificate set element byte for byte against its counterpart in the CMS before the timestamp was added

What AddSignatureTimestampToCms will accept

AddSignatureTimestampToCms is written for the CMS that BuildSignedData emits: one SignerInfo, an optional certificates [0] set, no crls [1] and no unsignedAttrs as yet. Feed it a signerInfos SET holding more than one SignerInfo and it raises rather than keeping the first signer and discarding the rest. A CMS carrying a crls [1] set stops at the CMS: signerInfos SET expected exception, because the element after the certificates is not the SET the walk is after. Both refusals happen before anything is written back into CmsDer

The new unsignedAttrs holds exactly one attribute, so the SET OF ordering rule of X.690 clause 11.6 is satisfied without a sort. The signed part of the SignerInfo, from its version through to the signature OCTET STRING, is copied as one block, which means a validator re-digesting signedAttrs sees the same bytes before and after the token is added. When a validator still turns the document down, the causes usually lie elsewhere and are covered in this checklist of PAdES rejection reasons. What the token buys you over time, and how B-LT and B-LTA build upon it, appears in the walkthrough of RFC 3161 timestamps and the DSS

The DER reader and writer in FPdfAsn1.pas, the CMS builder and the timestamp step in FPdfCms.pas are all Pascal source shipping with the PDFium Component for Delphi. When a rebuilt SignedData refuses to parse in a validator, you can step through the code that cut each element and check it against the X.690 clause it cites