PDF Text Advance and q/Q Clip Restore in a Delphi Renderer

A content stream that sets /F1 1 Tf, then 12 0 0 12 72 700 Tm, then shows (Hello world) Tj is perfectly legal. A hand-rolled renderer will often draw all eleven characters inside about five and a half points — a dark blot at the left margin where a 66-point word should be. The arithmetic behind that blot is one line long. ISO 32000-1 §9.4.4 defines each glyph advance in text space, so the displacement has to pass through the text matrix before it reaches the page, and a clip set inside q ... Q belongs to the graphics state just as much as the line width does

Why does text collapse into a clump when a PDF uses Tf 1?

Text collapses because the renderer adds a text-space distance straight to the translation component of Tm, which only works while text space and user space share the same scale. Plenty of real-world producers write the font size as 1 and carry the real size in the text matrix instead. Under /F1 1 Tf and 12 0 0 12 72 700 Tm, a glyph 500 units wide advances 0.5 in text space. Tm scales that to 6 points on the page. A renderer that executes Tm.e := Tm.e + Adv moves the pen 0.5 points. Every glyph then lands one twelfth of a character after the previous one, and a viewer that follows §9.4.4 shows the same file as normal body text

The shortcut survives for a long time because most generators people test with write /F1 12 Tf and an unscaled Tm, and in that case text space and user space really are the same size. Here is the naive version in full, with the content stream that breaks it written as a comment on top:

// Content stream from a producer that encodes size in Tm, not Tf:
//   BT
//   /F1 1 Tf
//   12 0 0 12 72 700 Tm
//   [(Hel) 30 (lo) -250 (world)] TJ
//   ET

// Naive advance: a text-space distance added to Tm.e as if it were user space
Adv := W * FontSize / 1000;                  // 0.5 for a 500-unit glyph
if (HorizScale <> 0) and (HorizScale <> 100) then
  Adv := Adv * HorizScale / 100;             // Th on the width only
Adv := Adv + CharSpace;                      // Tc not scaled by Th
if Code = 32 then
  Adv := Adv + WordSpace * FontSize / 1000;  // Tw wrongly scaled by Tfs
Tm.e := Tm.e + Adv;                          // ignores Tm.a, Tm.b, Tm.c, Tm.d

// Naive TJ adjustment: no Th, and again only Tm.e
Tm.e := Tm.e - NumValue * FontSize / 1000;

The Tm.e line is the loud bug. The others are quieter, and they show up on different files. Word spacing Tw is already expressed in unscaled text space units, so multiplying it by FontSize / 1000 turns a 3 Tw under Tf 12 into 0.036 units. A justified paragraph loses nearly all of its inter-word gap. Horizontal scaling Th must apply to Tc and Tw too, and to the TJ kerning number. The Code = 32 test also fires on any byte of value 32 inside a two-byte code, which spreads random gaps through CJK text. Text state mistakes like these rarely fail loudly: like the operand index and resource name traps that can zero Tc, Tw and Tz before a single glyph is drawn, they produce plausible pages from one producer and broken ones from the next

How does ISO 32000-1 §9.4.4 define the glyph advance?

ISO 32000-1 §9.4.4 computes the advance entirely in text space and applies it to the text matrix as a translation matrix, so the scaling, rotation and skew all come from Tm and never from the advance itself. For horizontal writing, tx = ((w0 − Tj/1000) × Tfs + Tc + Tw) × Th, where w0 is the glyph width in thousandths of an em, Tj is the TJ adjustment and Th is Tz divided by 100. The new matrix is [1 0 0 1 tx 0] × Tm. Written out, that product adds a·tx to e and b·tx to f, which is why a translation in text space can never be a plain addition to e unless a is 1 and b is 0. Per §9.3.3, Tw applies only to the single-byte character code 32

The HotPDF Delphi Component keeps that formula in one place. The unit HPDFContentStream exports HPDFTextGlyphDisplacement for the per-glyph advance and HPDFTextAdjustmentDisplacement for a TJ number, and both the page renderer and the text extraction interpreter call them, so a page that renders correctly also reports glyph positions that line up with what you see. If you write your own content-stream consumer on top of the library's parser, you can call the same two routines and supply the matrix step yourself:

uses
  HPDFContentStream;  // HPDFTextGlyphDisplacement, HPDFTextAdjustmentDisplacement

type
  // Your own text-matrix record, not a HotPDF type
  TTextMatrix = record
    a, b, c, d, e, f: Double;
  end;

// Your own helper: [1 0 0 1 Tx Ty] x Tm, moving through a, b, c and d
procedure TranslateInTextSpace(var M: TTextMatrix; Tx, Ty: Double);
begin
  M.e := M.e + M.a * Tx + M.c * Ty;
  M.f := M.f + M.b * Tx + M.d * Ty;
end;

// W0 is the glyph width in 1/1000 em; Tz is the Tz operand (100 = no scaling)
procedure AdvanceAfterGlyph(var Tm: TTextMatrix; W0: Double; Code: Cardinal;
  TwoByteCodes: Boolean; Tfs, Tc, Tw, Tz: Double);
var
  Tx, Ty: Double;
begin
  HPDFTextGlyphDisplacement(W0 * Tfs / 1000, False,
    (Code = 32) and not TwoByteCodes,  // Tw only on the single-byte space
    Tc, Tw, Tz, Tx, Ty);
  TranslateInTextSpace(Tm, Tx, Ty);
  // Tf 1, Tm 12 0 0 12 72 700, W0 = 500: Tx = 0.5 and Tm.e grows by 6
end;

procedure ApplyTJNumber(var Tm: TTextMatrix; Adjustment, Tfs, Tz: Double);
var
  Tx, Ty: Double;
begin
  HPDFTextAdjustmentDisplacement(Adjustment, Tfs, Tz, False, Tx, Ty);
  TranslateInTextSpace(Tm, Tx, Ty);  // Th applies to kerning as well
end;

Inside the renderer, the same matrix step drives every operator that moves the text position. Td, TD, T*, ' and " translate the line matrix through its coefficients, so 0 -1.2 Td under a 12× text matrix moves the baseline 14.4 points down instead of 1.2. Glyphs that are never painted take the identical path. Render mode 3, the invisible mode OCR text layers use, and text inside hidden optional content both advance Tm exactly like painted text. Giving them a private copy of the arithmetic is the classic mistake, because then any visible text that follows an invisible run in the same BT block starts from a slightly different position

Placing the glyph itself

Glyph placement follows the same logic. When no embedded outline is available and the renderer falls back to GDI TextOutW, it builds the full glyph matrix from CTM × Tm × rise × em scale, with Th included. The matrix goes in through SetWorldTransform in GM_ADVANCED mode, inside a SaveDC / RestoreDC pair. The GDI font is created at a fixed 1000-unit height and the world transform does all of the sizing. Rotated and skewed text therefore keeps its orientation instead of being drawn upright at a transformed origin. Stroked render modes reset the world transform before stroking, since a pen width measured against a 1000-unit design size would otherwise shrink to almost nothing. Vertical writing is the one deliberate asymmetry: a WMode 1 font advances down the y axis by its vertical metric, Tc and Tw apply along that axis, and Th does not apply at all

What does q/Q actually save in a PDF graphics state?

ISO 32000-1 §8.4.2 lists the current clipping path as part of the graphics state, so Q has to restore the clip exactly as it stood at the matching q. That sounds obvious. In a GDI-backed renderer it is easy to miss, because the CTM, colors, line parameters and text state live in the renderer's own state stack while the clip lives in the device context — outside that stack. Copying the numeric state restores everything except the clip. A W n inside a q ... Q block then keeps cropping every later operation on the page, and the symptom looks like missing content far away from the operator that caused it

Form XObjects open a second route to the same failure. §8.10 gives a form an implicit save and restore around its content and clips it to its /BBox, and real-world form content sometimes leaves its own q and Q unbalanced even though the specification requires them to pair up. HotPDF records the clip and saves the device context before running a form, and when the form finishes it throws away any clip frames the form left open and restores the device context, so every saved region is released exactly once. A stray Q inside the form that would pop the caller's state is refused rather than applied. With RenderFallbackPolicy set to rfpReport, that refusal appears in the diagnostics, which makes it easy to tell a sloppy producer from a rendering problem:

// Diags: THPDFRenderFallbackDiagnostics (declared in HPDFPageRenderer)
Pdf.RenderFallbackPolicy := rfpReport;
Bmp := Pdf.RenderLoadedPageToBitmap(PageIndex, 150);
try
  Diags := Pdf.GetLastRenderFallbackDiagnostics;
  for J := 0 to High(Diags) do
    if (Diags[J].Kind = rfkInvalidObject) and (Diags[J].OperatorName = 'Q') then
      Log.Add(Format('page %d: unbalanced Q refused - %s',
        [PageIndex + 1, string(Diags[J].Reason)]));
finally
  Bmp.Free;  // Free is safe on nil
end;

Capturing the clip only when a frame changes it

The straightforward fix is to read the GDI clip region on every q and put it back on every Q. It is correct. It is also expensive on exactly the pages that use q most, because generators love to wrap each small drawing in q 1 0 0 1 x y cm ... Q purely to move the origin. Each of those pairs would then create a GDI region, copy the clip into it and delete it again. When several pages render in parallel, the threads end up competing for GDI region objects instead of rasterizing, and the speedup from the parallel render pipeline shrinks accordingly

HotPDF defers the expensive part. A q only records the stack depth and the surface it belongs to, in a frame array that grows by doubling from 16 and is reused across the page, so a run of numeric transforms allocates no GDI object at all. The clip is captured the first time something in that frame is about to change it: path painting or n with a pending W or W*, a pattern fill, the clip left behind by text render modes 4 to 7, or entry into a form. On Q, a captured frame puts its region back and frees it, and a frame that never changed the clip pops only the numeric state. The common page then allocates no region at all, and a page that clips pays exactly one region per actual clip change. If you build your own GDI device for a PDF interpreter, the same shape fits in a few dozen lines:

// A sketch for your own GDI-backed device; none of this is HotPDF API
// needs Windows, SysUtils and Math in the uses clause
type
  TClipFrame = record
    Region: HRGN;
    Captured: Boolean;
  end;

procedure TMyDevice.PushState;            // q: no GDI call at all
begin
  if FCount = Length(FFrames) then
    SetLength(FFrames, Max(16, Length(FFrames) * 2));
  FFrames[FCount].Region := 0;
  FFrames[FCount].Captured := False;
  Inc(FCount);
end;

procedure TMyDevice.BeforeClipChange;     // call before W n, W f, text clip, form BBox
var
  R: HRGN;
  Res: Integer;
begin
  if (FCount = 0) or FFrames[FCount - 1].Captured then Exit;
  R := CreateRectRgn(0, 0, 0, 0);
  if R = 0 then RaiseLastOSError;
  Res := GetClipRgn(FDC, R);              // 1 copied, 0 no clip, -1 error
  if Res <= 0 then
  begin
    DeleteObject(R);
    R := 0;
    if Res < 0 then RaiseLastOSError;
  end;
  FFrames[FCount - 1].Region := R;
  FFrames[FCount - 1].Captured := True;
end;

procedure TMyDevice.PopState;             // Q
begin
  if FCount = 0 then Exit;                // unbalanced Q: ignore it
  Dec(FCount);
  if FFrames[FCount].Captured then
    SelectClipRgn(FDC, FFrames[FCount].Region);  // Region 0 removes the clip
  if FFrames[FCount].Region <> 0 then
    DeleteObject(FFrames[FCount].Region);
end;

Two details in that sketch carry most of the weight. GetClipRgn returning 0 is not an error: it means the device context had no clip at the time. Passing a zero region to SelectClipRgn on Q is precisely what removes a clip that did not exist at the matching q. Treating 0 as failure leaves the inner clip in place. The other detail is the Captured flag, which stops a second clip change in the same frame from overwriting the saved region with an already-narrowed one

Where the saved clip and the advance rule stop

The saved clip is a GDI region in device pixels. It is exact for the bitmap being rendered and meaningless for any other surface, so each frame also records the device context it was taken from, and the restore is skipped when the context has changed, for example while a transparency group renders into its own layer bitmap

On the text side, the advance rule places each glyph from the width it is given. When a simple font omits its /Widths array and no embedded font program is available, the advance can only be as good as the width fallback, and §9.4.4 has nothing to say about where that width comes from. Regression tests for this live on files that simple generated documents almost never contain: one with Tf 1 and a scaled Tm, one with nonzero Tz and Tw on justified text, and one with a W n clip inside q ... Q followed by content drawn outside the clipped area. A correct render of the third shows that outside content in full

From application code, none of this needs any setup: the call is the one shown in rendering a PDF page to a bitmap, and both rules apply to every page you render that way. Supported Delphi and C++Builder versions and licensing details are listed on the HotPDF Delphi PDF Component page