HotPDF Delphi Component, since v2.768.15, removes the duplicate characters that fake bold and overprinted logos leave in extracted PDF text. When a producer paints the same glyph twice a fraction of a point apart, ExtractLoadedPageText, ExtractLoadedPageTextLayout and ExtractLoadedPageTextColumns now read that character once, so "Bold words" no longer comes out as "Bold wordsBold words"
This is one of those bugs that looks like a decoder failure and is not. Every glyph decodes correctly; the text really is on the page twice. The fix has to decide which of two perfectly valid glyphs is the copy, and do it without eating legitimate repeated letters such as the double l in "fill" or the identical cells of a table. The rest of this article explains the rule HotPDF uses, the thresholds behind it, where it deliberately differs from other extraction tools, and a second, unrelated extraction bug from the same release train that also chopped text off line ends
Why does fake bold produce doubled text in PDF extraction?
Fake bold doubles extracted text because the producer draws the string twice and both copies are real text-showing operations. When a font family has no bold face, or the authoring tool simply does not bother, many generators emulate bold by showing the same string with Tj or TJ, nudging the text position right by a few tenths of a point, and showing it again. On screen the two copies merge into a heavier stroke. In the content stream there are two complete runs of character codes, and ISO 32000-1 §9.4 gives an extractor no flag that says "this one is decoration"
Overprinted logos hit the same problem from a different angle. A logotype may be painted once as a fill and again as a stroke or a slightly offset shadow, using the same font and the same characters. A naive extractor that walks the content stream in painting order emits the whole first run, then the whole second run, which is why the result reads "Bold wordsBold words" rather than "BBoolldd". Before v2.768.15, HotPDF text assembly kept both copies, and the duplicate broke the thing most callers actually care about: a search for "Bold words" across the page text could fail because the word boundary between the two runs was gone
Real text rendering mode 2 (fill then stroke, ISO 32000-1 §9.3.6) does not have this problem, because it is one text-showing operation that produces one set of glyphs. The duplicates only appear when the producer issues the text twice
How does HotPDF decide which glyph is the duplicate?
HotPDF treats two glyphs as one painted character when they carry the same Unicode value, have the same writing direction, are within 5% of each other in size, and sit almost on top of each other: no more than 0.1 of the glyph size apart along the baseline and no more than 0.2 of the glyph size apart across it. Of each such pair, the glyph painted first is kept and the later one is dropped. Those are the values in the v2.768.15 implementation, and each one is there for a reason
- Same Unicode, same direction. Only identical characters can be duplicates, and a horizontal glyph is never compared with a vertical (WMode 1) one
- Size within 5%. A large heading letter and a small caption letter that happen to overlap are not the same text, so a size mismatch rules a pair out
- 0.1 of the size along the baseline. Fake bold offsets are a few tenths of a point, a few percent of a normal text size. Two real adjacent letters, even a tightly kerned "ll", are separated by roughly a full advance width, an order of magnitude more than this tolerance
- 0.2 of the size across the baseline. Shadows and outline passes often shift down as well as right, so the vertical tolerance is looser, but still far below the distance to the next line
- First painted wins. Keeping the earlier glyph preserves the reading order of the first run, so the surviving text still flows the way the producer wrote it
The distances are measured in the glyph's own baseline frame rather than raw page X and Y, so rotated text and text on a slanted baseline use the same tolerances as ordinary horizontal lines. The size used is the glyph's size as painted on the page, derived from its quad, not the size operand given to Tf; that matters for documents that set Tf to 1 and scale everything through the text matrix, where a raw font size would make every tolerance useless
Spaces are never candidates. Code points up to U+0020, the no-break space U+00A0 and the ideographic space U+3000 are skipped, because blank glyphs carry no visible ink to duplicate and dropping one would merge words
Keeping the check fast on dense pages
The duplicate check in HotPDF runs in roughly n log n time, not n squared, because it sorts first and only compares glyphs inside a small window. Comparing every glyph with every other glyph would be correct and unusable: a dense page of small print can easily carry tens of thousands of glyphs, and the check runs on every text extraction call
The internal routine sorts the non-space glyphs by Unicode value, then by origin X, then by painting order, using a stable merge sort. After sorting, every potential duplicate of a glyph sits immediately after it in the order, so the scan only walks forward while the Unicode value matches and the X difference stays within 0.25 of the glyph size. The 0.25 window is deliberately wider than the 0.1 along-baseline tolerance, so a pair on a slightly slanted baseline still lands inside it before the precise baseline-frame test runs. One subtle case: if the scan from glyph A meets a matching glyph that was painted before A, A is the copy, so A is dropped and the scan stops, leaving the earlier glyph to continue the comparison with anything further along. That rule is what makes "first painted wins" hold for triple strikes as well as pairs
The comparison is on Unicode value, not font or character code. Two glyphs that map to the same Unicode through different fonts, for instance a fill pass in one embedded subset and a stroke pass in another, are still recognised as one character. The flip side is that a glyph whose code resolves to Unicode 0 has nothing reliable to compare, which is one more reason to keep an eye on the unresolved count covered in extracting text from loaded PDFs with HotPDF
Which HotPDF extraction calls remove duplicate glyphs?
All the text-producing calls in HotPDF remove duplicates, and the raw glyph call does not. The deduplication lives in THotPDF.ExtractVisibleLoadedPageGlyphs, the step that also discards glyphs outside the page's visible box, and the plain text, column, layout and semantic text extraction all read their glyphs from it. THotPDF.ExtractLoadedPageGlyphs still returns every glyph the content stream paints, overprints included, because callers who build highlighting, redaction or forensic tooling need the full record of what was drawn
For ordinary text work nothing changes in calling code. The same loop that extracted doubled text before v2.768.15 now extracts each word once
var
Pdf: THotPDF;
PageCount, I: Integer;
PageText: UnicodeString;
begin
Pdf := THotPDF.Create(nil);
try
PageCount := Pdf.LoadFromFile('brochure.pdf');
for I := 0 to PageCount - 1 do
if Pdf.ExtractLoadedPageText(I, PageText) then
if Pos('Bold words', PageText) > 0 then
Writeln('Found on page ', I + 1);
finally
Pdf.Free;
end;
end;Layout and column output benefit just as much, since a duplicated run used to push the rest of a line sideways on the character grid. ExtractLoadedPageTextColumns keeps its optional gap multiplier; the duplicate filter runs before column splitting, so it does not interact with that setting
var
Grid, Columns: UnicodeString;
begin
if Pdf.ExtractLoadedPageTextLayout(0, Grid) then
TFile.WriteAllText('page1-layout.txt', Grid, TEncoding.UTF8);
// Columns split where a gap exceeds 3x the median glyph advance
if Pdf.ExtractLoadedPageTextColumns(0, Columns, 3.0) then
TFile.WriteAllText('page1-columns.txt', Columns, TEncoding.UTF8);
end;If you want to know whether a page contains overprinted text at all, compare the two glyph calls. Be aware that the difference also counts glyphs that fall outside the crop box, so treat it as an upper bound on duplicates rather than an exact figure
var
AllGlyphs, Visible: THPDFGlyphArray;
begin
if Pdf.ExtractLoadedPageGlyphs(0, AllGlyphs) and
Pdf.ExtractVisibleLoadedPageGlyphs(0, Visible) then
Writeln(Format('%d painted, %d read as text (%d overprinted or off-page)',
[Length(AllGlyphs), Length(Visible),
Length(AllGlyphs) - Length(Visible)]));
end;The word spacing and line break rules that turn these glyphs into text are unchanged by the filter; they are covered in detail in how HotPDF places word spaces and line breaks in page text
Where the rule deliberately disagrees with other tools
HotPDF drops exact duplicates that some other extraction tools keep, and that difference is intentional. When the v2.768.15 change was checked against a reference corpus, one page footer painted twice at exactly the same position extracted once in HotPDF and twice in a reference extractor. A glyph painted on top of an identical glyph is invisible as a separate character, so reading it once is what the page actually shows
The honest limit runs the other way. Fake bold that offsets the second pass by more than 0.1 of the glyph size along the baseline is not caught, and neither is a shadow pass drawn at a noticeably different size. Those layouts are rare in practice, because the visual effect stops looking like bold once the offset gets that large, but if you see doubled text from a decorative heading, compare the glyph X values from ExtractLoadedPageGlyphs before assuming the filter is broken. The filter also does nothing for duplicates that are not geometrically co-located, such as a running header repeated in a hidden layer elsewhere on the page; that is a content question, not an overprint
Why were line endings cut off in CID font text?
A separate bug fixed in HotPDF v2.768.12 dropped the ends of lines in text set with CID-keyed (Type 0) fonts whose /W width array was not sorted by CID. ISO 32000-1 §9.7.4.3 defines the /W array format but does not require its entries to be in ascending CID order, and some generators write them in the order glyphs were first used in the document
The text extraction decoder looked widths up with a binary search, which silently assumes sorted input. On an unsorted array most lookups missed and fell back to the default width /DW, which is 1000 units, a full em. Latin text in such a font was therefore laid out as if every letter were full-width, lines ran far past the right edge of the page, and the glyphs that landed outside the visible box were discarded, so the ends of lines vanished from extracted text and from search. Nothing crashed and nothing was reported, which is what made it hard to spot
HotPDF now sorts the /W entries with a stable merge sort right after reading them, so where two entries overlap the one written first still wins, matching how the array reads in file order. The page renderer was never affected, because it keeps its own copy of the width data and scans it linearly, and the vertical width array /W2 is also scanned linearly. That split is worth remembering when a page looks perfect but extracts badly: rendering and extraction in a PDF library often share far less code than you would expect
FAQ
Does HotPDF remove repeated letters such as the double l in "fill"?
No. Two real adjacent letters are about one advance width apart, while HotPDF only merges identical characters that are within 0.1 of the glyph size along the baseline and 0.2 across it, so legitimate repeats are kept
How do I get every glyph, including overprinted duplicates?
Call THotPDF.ExtractLoadedPageGlyphs, which returns every glyph the page paints in painting order. THotPDF.ExtractVisibleLoadedPageGlyphs and all the text extraction calls return the deduplicated, visible set
Which HotPDF version fixes doubled text from fake bold?
HotPDF v2.768.15 added duplicate glyph removal to text, layout, column and semantic text extraction, and v2.768.12 fixed missing line ends in CID fonts with an unsorted /W array
Both fixes ship in the current HotPDF Delphi PDF component, which extracts text from loaded PDFs in Delphi and C++Builder alongside its PDF creation, editing and rendering features