PDFium Component version 3.117.0 stops reporting justified paragraphs as whitespace-aligned tables by requiring every column boundary to be a vertical corridor with no text on any row it separates, skipping words already claimed by a ruled grid, and assembling cell text by vertical overlap instead of glyph-box centre distance. All three changes live inside ExtractTables and ExtractDocumentTables and need no option
The report that started this was unglamorous. A press-release page with no table on it came back from ExtractTables with a 5x4 whitespace table, confidence comfortably above the default MinConfidence of 0.5, and the cells held fragments of ordinary body text. An admission form did the same with its essay paragraphs and produced a 3x4 and a 5x3. Both documents were set justified. The obvious response is to tune the thresholds, and the useful lesson from this release is that tuning cannot fix it, because the rule being tuned was asking the wrong question
uses
PDFium;
// Regression check: list every whitespace table in a document so a page
// you know is prose-only can be confirmed clean
procedure ReportWhitespaceTables(Pdf: TPdf);
var
Options: TPdfTableExtractionOptions;
Tables: TPdfTables;
I: Integer;
begin
Options := TPdfTableExtractionOptions.Default; // MinColumnGap 12pt
Tables := Pdf.ExtractDocumentTables(Options);
for I := 0 to High(Tables) do
if Tables[I].DetectionMode = ptdmWhitespace then
Writeln(Format('page %d: %dx%d whitespace table, confidence %.2f, ' +
'first cell "%s"',
[Tables[I].PageNumber, Tables[I].RowCount, Tables[I].ColumnCount,
Tables[I].Confidence, Tables[I].Cells[0].Text]));
end;Why does justified text look like a table?
A justified paragraph looks like a table because a justified line is a row of words separated by gaps that the layout engine stretched, and once a stretched gap reaches MinColumnGap the detector has no row-local way to tell it from a column separator. The whitespace strategy in PDFium Component groups word boxes into visual rows, splits each row into word groups wherever the horizontal distance to the previous word is at least MinColumnGap (12 points by default), and accepts a table when at least two consecutive rows repeat at least MinColumns left-aligned group anchors within AlignmentTolerance, which is 3 points. That is the rule described in the table detection overview, and for a genuine aligned table it is exactly right
Now apply it to twenty lines of justified 10-point prose. Every line is stretched to the same right margin, so a line that ends with a long word pulls its interior spaces open, and in a paragraph with a few short lines some of those spaces cross 12 points. Two consecutive lines only need one stretched gap each, landing within 3 points of the same X position, to form a two-row, two-column candidate. Over enough lines this is not bad luck; it is a probability that approaches certainty, and the 5x4 on the press release was simply the run where four such gaps lined up on five lines
Every threshold trades one class of document against another. Raising MinColumnGap to 20 points loses the compact columns of dense financial reports, which is the exact case the default was already lowered for. Raising MinRows to 3 discards real two-row tables and merely lowers the odds for long paragraphs. Tightening AlignmentTolerance below 3 points breaks OCR-derived word boxes, whose left edges jitter by more than that. The row-level signal is genuinely ambiguous, so the fix has to come from a signal that rows do not carry on their own
What makes a column boundary real?
A real column boundary is a vertical strip of the page that stays empty across every row it separates. A table has one between each pair of columns by construction, because the cells were laid out against shared X positions. A justified paragraph stretches its word spaces at different horizontal positions on each line, so no strip survives the intersection of more than a line or two. PDFium Component now tests exactly that: after the candidate's word groups have been assigned to anchor columns, for each pair of adjacent columns it takes, on every row that has content in both cells, the interval from the rightmost edge of the left cell's words to the leftmost edge of the right cell's words, intersects those intervals across rows, and rejects the whole candidate if the intersection is narrower than MinColumnGap times 0.5, which is 6 points at the default
Two details matter. Rows where either cell is empty do not vote, so a table with a blank cell, or a header that spans fewer columns than the body, still passes. And the corridor width is derived from MinColumnGap rather than exposed as a separate option, because the two describe the same physical thing: the gap a designer leaves between columns. The logic is small enough to reproduce if you are building on raw word boxes rather than the table API, and the sample below mirrors the check inside the component:
uses
Math, PDFium;
type
TIndexList = array of Integer;
TCellIndexes = array of TIndexList; // Row * ColumnCount + Column
// Returns False when any adjacent column pair lacks a text-free vertical
// corridor at least MinColumnGap / 2 wide across the rows that use it
function HasTextFreeCorridors(const Words: TPdfWordBoxes;
const Cells: TCellIndexes; RowCount, ColumnCount: Integer;
MinColumnGap: Double): Boolean;
var
Col, Row, I, LeftCell, RightCell, Supported: Integer;
CorridorLeft, CorridorRight, RowLeft, RowRight: Double;
begin
for Col := 0 to ColumnCount - 2 do
begin
CorridorLeft := -MaxDouble;
CorridorRight := MaxDouble;
Supported := 0;
for Row := 0 to RowCount - 1 do
begin
LeftCell := Row * ColumnCount + Col;
RightCell := LeftCell + 1;
if (Length(Cells[LeftCell]) = 0) or (Length(Cells[RightCell]) = 0) then
Continue; // empty cells do not vote
RowLeft := -MaxDouble;
RowRight := MaxDouble;
for I in Cells[LeftCell] do
RowLeft := Max(RowLeft, Words[I].Rect.Right);
for I in Cells[RightCell] do
RowRight := Min(RowRight, Words[I].Rect.Left);
CorridorLeft := Max(CorridorLeft, RowLeft);
CorridorRight := Min(CorridorRight, RowRight);
Inc(Supported);
end;
if (Supported > 0) and
(CorridorRight - CorridorLeft < MinColumnGap * 0.5) then
Exit(False);
end;
Result := True;
end;Why were ruled tables extracted twice?
Ruled tables were extracted twice because the whitespace pass used to see every word on the page, including the words the ruled pass had already placed into a grid, and a clean ruled table is by construction also a perfectly aligned whitespace table. An overlap check already rejected a whitespace candidate whose bounds covered more than half of an existing table, but a candidate that combined the table's lower rows with a few aligned lines of text beneath it could fall under that ratio and survive as a second, slightly larger table that bled into its neighbour. ExtractTables now removes those words before the whitespace pass runs. A word is dropped when its centre point lies inside the bounds of any table the ruled pass produced; the centre is used rather than full containment so that a word straddling a border by a fraction of a point follows the table it visually belongs to. The whitespace strategy then works only on the free words, which also means a small unruled table sitting directly under a ruled one is detected on its own merits instead of being fused with the grid above it
Why did "Purpose of Request:" come out as "of Purpose Request:"?
The words came out reordered because the word boxes PDFium Component builds are unions of glyph bounding boxes, and "of" has no descender while "Purpose" and "Request:" do. FPDFText_GetCharBox returns the tight box of the glyph's ink in page space, not a box padded to the font's ascent and descent, and the word box is the union of its characters' boxes. A word without descenders is therefore shorter and its vertical centre sits higher, by 2 to 3 points on the form in question. The old cell-text routine sorted words by centre Y first, with a 1-point tolerance for "same line", and then by left edge; "of" cleared the tolerance, sorted as its own line above the others, and was emitted first
This is not a PDFium quirk so much as a consequence of how PDF positions text. ISO 32000-1 §9.2.2 and §9.4.4 define glyph placement as horizontal displacement along the baseline in text space, and the only vertical metrics the file carries are per-font: the Ascent, Descent, and FontBBox entries of the font descriptor in §9.8.1. Nothing in the file says two glyphs share a line; that has to be inferred from geometry, and the tight glyph boxes that make selection highlighting look right, as described in text line selection with PDFium char boxes, are the wrong input for a centre-distance comparison
The fix in version 3.117.0 changes the question from "how far apart are the centres" to "how much do the boxes overlap vertically". Cell text is assembled by first grouping the cell's words into visual lines, where a word joins a line when its vertical overlap with the line's running bounds is at least 25 per cent of the smaller of the two heights, then insertion-sorting each line by left edge, then joining lines with a line break. "Purpose" and "of" overlap over the full x-height, which is far more than 25 per cent of the shorter box, so they land on the same line and sort by X as intended
Group text lines by overlap, not by centre distance
The rule worth taking away from this bug is general: any PDF text-layout code that decides "same line" by comparing vertical centres against a fixed tolerance will fail on real fonts, and the failure is silent: nothing errors, words simply come out in the wrong order. Mixed descenders are the mildest trigger. A bold 12-point label beside 10-point values, a superscript footnote marker, a currency symbol drawn from a fallback font, and OCR word boxes with per-word height noise all move centres by more than any tolerance that still separates adjacent lines of 10-point text at 12-point leading. Overlap ratio is size-invariant: two boxes on one baseline overlap over their shared x-height whatever their ascenders and descenders do, and two boxes on adjacent lines overlap by nothing
The same rule is easy to apply outside table extraction. TPdf.PageWordBoxes returns every word on the active page with its page-space rectangle, so grouping a page into visual lines is a short loop:
uses
Math, PDFium;
function SameVisualLine(const A, B: TPdfRectangle): Boolean;
var
Overlap, MinHeight: Double;
begin
Overlap := Min(A.Top, B.Top) - Max(A.Bottom, B.Bottom);
MinHeight := Min(A.Top - A.Bottom, B.Top - B.Bottom);
Result := (MinHeight > 0) and (Overlap >= MinHeight * 0.25);
end;
procedure GroupPageIntoLines(Pdf: TPdf; out Lines: TArray<TPdfWordBoxes>);
var
Words: TPdfWordBoxes;
Bounds: TArray<TPdfRectangle>; // running union per line
I, J, Found: Integer;
begin
Words := Pdf.PageWordBoxes;
Lines := nil;
Bounds := nil;
for I := 0 to High(Words) do
begin
Found := -1;
for J := High(Lines) downto 0 do
if SameVisualLine(Bounds[J], Words[I].Rect) then
begin
Found := J;
Break;
end;
if Found < 0 then
begin
SetLength(Lines, Length(Lines) + 1);
SetLength(Bounds, Length(Bounds) + 1);
Found := High(Lines);
Bounds[Found] := Words[I].Rect;
end;
SetLength(Lines[Found], Length(Lines[Found]) + 1);
Lines[Found][High(Lines[Found])] := Words[I];
Bounds[Found].Left := Min(Bounds[Found].Left, Words[I].Rect.Left);
Bounds[Found].Right := Max(Bounds[Found].Right, Words[I].Rect.Right);
Bounds[Found].Top := Max(Bounds[Found].Top, Words[I].Rect.Top);
Bounds[Found].Bottom := Min(Bounds[Found].Bottom, Words[I].Rect.Bottom);
end;
// sort each line by Rect.Left before reading it; PageWordBoxes returns
// words in content-stream order, which is not guaranteed to be visual
end;What changes for existing callers, and where the limits are
The point of that snippet is the predicate, not the loop; for anything beyond a quick dump, start from the structured text model, which already carries blocks, lines, and a reading-order source, as covered in structured PDF text extraction with reading order. Existing table callers get all three corrections without touching their options. The corridor threshold is fixed at half of MinColumnGap, the whitespace strategy keeps its two-row floor even when MinRows is set to 1 (which the ruled strategy now accepts), and the ruled-first word filtering is unconditional whenever both strategies are enabled. On the 13-document sample set used for the release, the whitespace pass had previously returned 34 fragments and false positives alongside 9 ruled tables; after the release it returns none, and the count of ruled tables rose to 41, though most of that rise comes from the same release teaching the ruled detector to read borders drawn as filled rectangles, which is a separate story
The honest limits: the corridor test needs at least one row with content on both sides of a boundary to reject anything, so a two-row candidate whose two stretched gaps happen to fall within 6 points of each other still passes. That is a narrow coincidence rather than the near-certainty it was before, but prose-heavy documents with no genuine two-row tables can close it by setting MinRows to 3. Left-aligned ragged text was never the problem and is not affected. And PDF still has no table object; ISO 32000-1 §14.8.4.3 defines a Table structure element, but only Tagged PDF carries it, so for everything else the grid remains an inference from geometry, and the confidence value on each TPdfTable is there because inference deserves a score
Table extraction, structured text, and word boxes all read from the same page model in Delphi, C++Builder, and Lazarus; the complete API, including TPdfTableExtractionOptions and the TableExtractionLab demo that ships alongside it, is described on the PDFium Component for Delphi page