Rewrite PDF Content Streams Safely in Delphi

HotPDF walks every operator in a page's content stream and hands each one to your code through VisitLoadedPageContentOperators. For each operator you choose an action: keep it, delete it, replace it with your own bytes, stop the walk, or reject the whole edit. When every operator is kept, the rewritten stream is byte-for-byte identical to the original, which is the property that makes the mechanism safe to run over documents you did not create

Content stream editing is where PDF tools usually go wrong. A stream is a sequence of operators over a stack machine, not a document tree, and the naive approach of running a regular expression over the decoded bytes breaks the moment a string literal contains something that looks like an operator, or an inline image contains arbitrary binary. A visitor that has already tokenized the stream does not have that problem

HotPDF: PDF content stream operators such as BT, Tf and TJ flowing into the visitor event data with phase, operator name, operand count, byte span and nesting state, then returning one of five actions: keep, replace, delete, stop, or reject — Each visited operator yields exactly one decision, and a pass that keeps everything writes back untouched ranges for byte-identical output

What can you actually do with operator-level access?

Three families of task. Auditing: count how many times a page sets a color outside a known set, or find text drawn with a font that is not embedded. Removal: delete a watermark that was drawn as marked content, or strip a specific image invocation. And normalization: replace one operator form with an equivalent that a downstream consumer handles better

Each of those shares one requirement: you must be able to leave the other 99 percent of the stream untouched. HotPDF's syntax tree stores one copy of the source plus compact spans, and edits are sorted non-overlapping overlays. Writing the result copies untouched ranges directly and emits replacement bytes only where you changed something, so a no-op round trip cannot introduce drift

The visitor contract

Implement the interface or assign the event, then call the visit method with ApplyChanges set to false for an audit or true for a rewrite. The event data carries the phase, the operator name, its operand count, its byte span, the current nesting state and a borrowed view for reading operands:

uses
  HPDFDoc, HPDFContentStream;

procedure TAuditor.HandleOperator(Sender: TObject;
  var EventData: THPDFContentOperatorEventData);
var
  Bytes: TBytes;
begin
  if EventData.Phase <> covpOperator then
    Exit;

  // Count text-showing operators drawn inside marked content
  if (EventData.OperatorName = 'Tj') or
     (EventData.OperatorName = 'TJ') then
  begin
    Inc(FTextOperators);
    if EventData.State.MarkedContentDepth > 0 then
      Inc(FTextInsideMarkedContent);
  end;

  // Read an operand only when you need it
  if EventData.OperatorName = 'Do' then
    if EventData.View.GetOperandBytes(0, Bytes) then
      NoteXObjectUse(TEncoding.ANSI.GetString(Bytes));

  EventData.Action := coaKeep;
end;

procedure TAuditor.Run(const FileName: string);
var
  Pdf: THotPDF;
  Stats: THPDFContentOperatorVisitStatistics;
  I: Integer;
begin
  Pdf := THotPDF.Create(nil);
  try
    if Pdf.LoadFromFile(FileName) <= 0 then
      Exit;
    Pdf.OnContentOperator := HandleOperator;
    for I := 0 to Pdf.LoadedPageCount - 1 do
      Pdf.VisitLoadedPageContentOperators(I, False, Stats);   // audit only
    Writeln(Format('%d operators visited, max q/Q depth %d',
      [Stats.VisitedOperatorCount, Stats.MaximumGraphicsStateDepth]));
  finally
    Pdf.Free;
  end;
end;

Why is the view borrowed rather than mutable?

The callback never receives a modifiable syntax tree. It receives a view that can fetch the current operation's bytes or a specific operand on demand, and that is all. Every change goes through the action and replacement fields instead

That indirection is not ceremony. Direct tree mutation would let a visitor bypass the replacement budget and the overlapping-edit check, and overlapping edits in a byte-span overlay model produce output that is silently wrong rather than loudly broken. Routing changes through the action field means every edit is validated the same way regardless of what the visitor intended

Reading on demand also keeps the common case cheap. An audit that only counts operator names never materializes operand bytes at all, which matters on a content stream with hundreds of thousands of operations

The state field describes the input, always

Each event carries the graphics state depth, text object depth, marked content depth and compatibility section depth as they stood in the original input immediately before that operator executed. That is true even if your visitor deleted or replaced earlier operators

Anchoring state to the original input rather than to the edited output is a deliberate choice with a practical payoff: running the same audit twice over the same document produces identical results, and an audit that runs alongside edits does not shift its own frame of reference. If you need to know the depth in the output, you know what you deleted, so you can compute it; the reverse, reconstructing stable input positions from a moving output, is not possible

Deleting and replacing, with the rules that protect you

A replacement is raw content-stream bytes, and you are responsible for their syntax. The safeguards around them are the budget on total replacement bytes, the non-overlapping requirement, and the optional output validation that reparses the result before it is accepted:

Rewrite safeguards in Delphi: a visitor declaring Action and Replacement intent routed through the replacement budget, non-overlapping byte span, and output validation guards into the validated saved page — Every change funnels through the action field into the budget, overlap and reparse guards, so failed validation aborts instead of writing broken operators

procedure TWatermarkStripper.HandleOperator(Sender: TObject;
  var EventData: THPDFContentOperatorEventData);
begin
  EventData.Action := coaKeep;
  if EventData.Phase <> covpOperator then
    Exit;

  // Drop the operators that paint the watermark form
  if (EventData.OperatorName = 'Do') and OperandIsWatermark(EventData) then
    EventData.Action := coaDelete
  // Normalise a stray colour to the house grey
  else if EventData.OperatorName = 'rg' then
  begin
    EventData.Replacement := TEncoding.ANSI.GetBytes('0.2 0.2 0.2 rg');
    EventData.Action := coaReplace;
  end
  // Refuse the entire edit if we meet something we do not understand
  else if EventData.OperatorName = 'BI' then
    EventData.Action := coaReject;
end;

var
  Options: THPDFContentOperatorVisitOptions;
begin
  Options := THPDFContentOperatorVisitOptions.Default;
  Options.ValidateOutput := True;        // reparse before accepting
  Options.MaxReplacementBytes := 64 * 1024;

  Pdf.OnContentOperator := HandleOperator;
  if Pdf.VisitLoadedPageContentOperators(PageIndex, True, Options, nil, Stats) then
    if Stats.Modified then
      Pdf.SaveLoadedDocument('cleaned.pdf');
end;

The action set includes two exits for a reason. coaStop ends the walk and keeps what has been decided so far, which is right when you have found what you were looking for. coaReject abandons the whole visit and leaves the page untouched, which is right when you encounter a construct your rewrite rules do not cover. Inline images are the classic example: their binary payload is not something a color-normalizing visitor should be editing around

Reading the statistics, and where the walk stops

The status field distinguishes completion from stopping, rejection, invalid input, an exceeded limit, invalid output and a visitor error. Those are different problems: invalid output means your replacement bytes did not reparse, while invalid input means the source stream was already malformed and no edit was attempted

The counters make an audit self-checking. VisitedOperatorCount against KeptOperatorCount, DeletedOperatorCount and ReplacedOperatorCount must add up, and the maximum depth figures tell you whether the page nests deeply enough to be worth a closer look. SourceBytes against OutputBytes gives the size effect of your edit in one number

HotPDF: Audit statistics reconciliation where kept, deleted and replaced operator counts sum to the visited count, beside seven status endings from completed to visitor error, over a note that state depths describe the original input so audits repeat identically — Counters that must add up plus a status naming each possible ending make an audit self-checking on documents you did not create

For the common tasks there are higher-level routes that do not require writing a visitor at all. Removing content permanently, with the resource cleanup that real redaction needs, is covered in redaction and page stitching on loaded documents. Replacing visible text is covered in search and replace in an existing PDF. Reach for the operator visitor when your rule is structural rather than textual, or when you need an audit that no existing report answers

Content stream visiting, extraction and rewriting run against the same loaded document model in Delphi and C++Builder; the complete feature list is on the HotPDF Delphi PDF component page