LlosLab Software Development BlogEngineering notes for PDF, Delphi, and document software developers.

Technical Article

Map PDF Text to Content-Stream Bytes in Delphi

Delphi PDF Library

One wrong character in an invoice number, and the only edit primitive on hand rewrites the whole text run. PDF Library for Delphi closes that gap: GetTextBlockCharContentLocation maps each extracted UTF-16 position back to the content-stream instruction, operand and encoded byte range that produced it, and ReplaceTextBlockCharSourceBytes overwrites only that range. Text extraction normally throws away everything you would need for this. You get Unicode, widths and geometry, and the provenance evaporates, so the character at position 7 of block 3 is just a character. Which stream produced it, which instruction, which operand, which byte inside that operand: gone. Every point-edit strategy built on top of that has to guess, usually by searching the decoded content for a substring and hoping it occurs exactly once. On a real page it does not

Why does rewriting a whole text run wreck the page?

Because the run is not just text. The text-showing operators in ISO 32000-1 §9.4.3 include TJ, whose operand is an array that interleaves strings with numeric adjustments, and those numbers are the typesetting. A line laid out as [(AB) -120 (CD)] TJ carries a 120-thousandths-of-an-em kern between the two strings. Emit a fresh Tj with the concatenated text and the kern is gone, the line reflows by a hair, and on a form the value drifts out of its box. The same objection applies to the font: the operand bytes are codes in whatever encoding Tf selected, not Unicode, and for a composite font they may be two-byte CIDs with no relationship to the character you read out of the extractor. Regenerate the run and you have to be right about the font's encoding, its /ToUnicode map and its glyph coverage. Point editing sidesteps all of that by never leaving the byte domain

What does GetTextBlockCharContentLocation return?

The method resolves one character to a nine-field record, and every field is an address rather than a value. ContentLayer is the 1-based index into the page /Contents array, or 0 when the character came from nested content. StreamObjectNumber and StreamGeneration identify the containing stream. InstructionIndex is the 0-based position in the decoded content program, OperandIndex the text-string operand, and ArrayElementIndex the element inside a TJ array or -1 for a direct string operand. SourceByteOffset and SourceByteLength then name the byte range inside that decoded string

Var
  Lib: TPDFlib;
  ListID, Block, CharPos: Integer;
  ContentLayer, StreamObjectNumber, StreamGeneration: Integer;
  InstructionIndex, OperandIndex, ArrayElementIndex: Integer;
  SourceByteOffset, SourceByteLength, Flags: Integer;
Begin
  Lib:= TPDFlib.Create;
  Try
    Lib.LoadFromFile('invoice.pdf', '');
    Lib.SelectPage(1);
    ListID:= Lib.ExtractPageTextBlocks(3);
    Try
      // Block and CharPos come from your own scan of GetTextBlockText
      If Lib.GetTextBlockCharContentLocation(ListID, Block, CharPos,
        ContentLayer, StreamObjectNumber, StreamGeneration,
        InstructionIndex, OperandIndex, ArrayElementIndex,
        SourceByteOffset, SourceByteLength, Flags)= 1 Then
      Begin
        // ContentLayer = 0 means the glyph lives in a nested Form XObject
        // ArrayElementIndex = -1 means a plain Tj operand, not a TJ array
      End;
    Finally
      Lib.ReleaseTextBlocks(ListID);
    End;
  Finally
    Lib.Free;
  End;
End;

The lookup costs nothing at query time. While the renderer decodes each content layer it registers the logical spans it is walking, so a position query is a binary search over an ordered interval list rather than a linear scan of every content span per character. Nothing is reparsed when you ask; the map was built during the extraction pass you already paid for. If you are already enumerating hits with PDF text search that returns hit coordinates, adding a content location per hit is close to free

Editing bytes, not Unicode

ReplaceTextBlockCharSourceBytes takes an AnsiString of raw replacement bytes in the active PDF font encoding. That is the whole design, and it is deliberate. Nothing transcodes, nothing re-encodes, nothing guesses at the font. The library splices your bytes over the named range of the target string and re-emits the containing content layer. Adjacent strings in the same TJ array and the numeric kerns between them are left byte-identical. Take the layout above: locating the B in [(AB) -120 (CD)] TJ yields ArrayElementIndex 0, SourceByteOffset 1, SourceByteLength 1. Replace it with Z and the emitted content contains (AZ), still followed by -120 and (CD), both untouched. The regression suite asserts exactly that, because "we preserved the kerning" is the kind of claim that quietly stops being true

Function EditableHere(Flags: Integer): Boolean;
Begin
  Result:= ((Flags and PDF_TEXT_CHAR_CONTENT_LOCATION_VALID)<> 0)and
    ((Flags and (PDF_TEXT_CHAR_CONTENT_LOCATION_GENERATED or
      PDF_TEXT_CHAR_CONTENT_LOCATION_ACTUALTEXT or
      PDF_TEXT_CHAR_CONTENT_LOCATION_NESTED or
      PDF_TEXT_CHAR_CONTENT_LOCATION_CROSS_LAYER or
      PDF_TEXT_CHAR_CONTENT_LOCATION_TRANSCODED))= 0);
End;

// ...
If EditableHere(Flags) Then
Begin
  If Lib.ReplaceTextBlockCharSourceBytes(ListID, Block, CharPos, 'Z')= 1 Then
  Begin
    // Every location in the old list is now stale. Re-extract.
    Lib.ReleaseTextBlocks(ListID);
    ListID:= Lib.ExtractPageTextBlocks(3);
  End
  Else If Lib.LastErrorCode= PDFLIB_ERROR_TEXT_LOCATION_STALE Then
    // The layer changed under us since extraction
  Else If Lib.LastErrorCode= PDFLIB_ERROR_TEXT_LOCATION_READ_ONLY Then
    // A flag we failed to check, or a flag added by a later version
End;

Two operational details are worth internalizing. The call temporarily switches to the page the text list was extracted from and restores the previously selected page on success and on failure alike, so it does not silently move your cursor. And on success it clears the page element snapshots, which invalidates any handles you were holding from an earlier enumeration pass

Which characters cannot be edited?

Six categories, and the library names each one in the Flags bitmask instead of failing vaguely. This matters more than the happy path, because on real documents the unmappable cases are common and each one has a different reason

  • PDF_TEXT_CHAR_CONTENT_LOCATION_LIGATURE: several extracted UTF-16 positions expand from one source glyph. A /ToUnicode entry mapping one code to fi gives you two characters sharing one byte range, so treat them as a single source glyph and edit the range once
  • PDF_TEXT_CHAR_CONTENT_LOCATION_GENERATED: the character was synthesized during layout. Inferred word spaces are the usual case, and they have no source bytes at all, so SourceByteOffset comes back -1 and SourceByteLength 0
  • PDF_TEXT_CHAR_CONTENT_LOCATION_ACTUALTEXT: the text you read came from an /ActualText replacement. There is no unique reverse mapping from the substituted string to source bytes, so the location is diagnostic only
  • PDF_TEXT_CHAR_CONTENT_LOCATION_NESTED: the glyph is inside a Form XObject. The bytes are addressable, but the Form may be drawn by several pages, so editing it through the high-level API would be an edit you did not ask for
  • PDF_TEXT_CHAR_CONTENT_LOCATION_TRANSCODED: the operand was a hex string carrying a UTF-16BE byte order mark, which the existing extraction path decodes before font mapping. Offsets into the decoded result no longer address the original bytes, so the valid flag is cleared
  • PDF_TEXT_CHAR_CONTENT_LOCATION_CROSS_LAYER: the string operand and its text-showing operator live in two different streams

That last one deserves a sentence of its own, because engineers routinely assume it cannot happen. ISO 32000-1 §7.8.2 says the streams in a page /Contents array are concatenated, and the division between them is only required to fall at a lexical boundary. So BT /F1 16 Tf 220 340 Td (CrossLayer) in one stream and Tj ET in the next is a perfectly legal page. The mapping keeps the diagnostic position but marks it read-only, because the operator's instruction index belongs to a different layer than the operand's bytes and using one to address the other would corrupt the file

How does the library know the map is still valid?

Fingerprints, checked immediately before the write. Each extraction list records the source page plus, for every content layer, the layer length and two independent rolling hashes: an FNV-1a hash and a DJB2-style xor hash. Before ReplaceTextBlockCharSourceBytes parses anything it re-reads the target layer and compares all three values. Any byte change anywhere in that layer returns PDFLIB_ERROR_TEXT_LOCATION_STALE and the write does not happen. This is deliberately conservative: the check is per layer, not per instruction, so an unrelated edit elsewhere in the same content stream also invalidates your location. That is the correct trade: an offset into a stream that has shifted by even one byte is not a near miss, it is a silent corruption. The same discipline governs the rest of the editing surface, including the content-stream state tracker for CTM and clipping. After any successful replacement, drop the list and extract again

Read-only mapping through Direct Access

DAGetTextBlockCharContentLocation gives you the identical record for a page opened through the Direct Access path, with the identical flag vocabulary. It is diagnostic only, by construction: ReplaceTextBlockCharSourceBytes operates on the selected editable document, and Direct Access is a read path. The location data survives in the text block list after the file handle is closed, which makes it usable for offline auditing

FileHandle:= Lib.DAOpenFileReadOnly('audit.pdf', '');
Try
  PageRef:= Lib.DAFindPage(FileHandle, 1);
  DirectList:= Lib.DAExtractPageTextBlocks(FileHandle, PageRef, 3);
  Try
    Lib.DAGetTextBlockCharContentLocation(DirectList, Block, 1,
      ContentLayer, StreamObjectNumber, StreamGeneration,
      InstructionIndex, OperandIndex, ArrayElementIndex,
      SourceByteOffset, SourceByteLength, Flags);
    // Locations stay readable after DACloseFile
  Finally
    Lib.DAReleaseTextBlocks(DirectList);
  End;
Finally
  Lib.DACloseFile(FileHandle);
End;

Use it to answer questions rather than to change things. Which pages carry text you could never edit in place? How much of this corpus arrives with /ActualText overrides? Which vendor's output splits operators across content layers? Those are cheap queries once every character has an address, and they are worth running before you commit to a correction pipeline

Where point editing stops

Point editing is a scalpel, not a text engine. It changes bytes in place, so replacement text that is wider or narrower than the original will not reflow, will not rewrap, and will not update the kerns around it. Substituting one digit for another in a monospaced field is a good fit. Retyping a paragraph is not. And it is emphatically not a security tool: overwriting glyph bytes leaves the original bytes recoverable from the file's revision history, so anything with a confidentiality requirement belongs in true redaction that removes content instead of covering it. What you get in exchange for those limits is honesty. Every character either has a byte address you can act on or a named flag telling you why it does not, and the fingerprint check makes a stale map a hard error instead of a corrupted page. Character-to-content-byte mapping and in-place source byte replacement ship as part of the text extraction and content editing surface of PDF Library for Delphi, the native Object Pascal PDF library for Delphi, C++Builder and Lazarus