A pipe that never reports end-of-file is an odd bug to chase in an OCR feature. One thread in your application reads the standard output of a helper program and waits for ReadFile to fail with ERROR_BROKEN_PIPE. Another thread starts tesseract.exe through CreateProcess with bInheritHandles set to True, because that is how you hand a child process a file for its standard error. The helper's pipe write end was inheritable, so Tesseract now owns a copy, and the reading thread hangs until Tesseract finishes its page — which, on a 300 DPI scan with two language models loaded, is not quick
An external recognizer brings more than the hang: what the child can see, what to do with half-valid output, how to turn pixel boxes into searchable PDF text. The built-in template-matching OCR engine covers machine-printed ASCII letters and digits on purpose. Accented names, Chinese contracts and mixed-language archives need trained language models, and Tesseract is the usual candidate because it is a command-line program you can install next to your executable
Plugging Tesseract into the HotPDF OCR text layer
HotPDF wraps a locally installed Tesseract executable as an IHPDFOCREngine through HPDFCreateTesseractOCREngine in the HPDFTesseractRecognition unit. You pass that engine to THotPDF.ApplyLoadedOCRTextLayer, which renders each requested page, runs Tesseract once per page, filters the words and writes an invisible Unicode text layer for every requested page in one transaction, or for none of them. Coordinate mapping, rotation, confidence filtering and the commit are the pipeline the built-in engine already uses; only recognition changes
The factory raises EArgumentException before any page is touched. The executable and tessdata directory must exist, the timeout must lie between 1 and 3,600,000 milliseconds, and the language may contain only ASCII letters, digits, _ and +, because it ends up on a command line: eng+chi_sim is legitimate, a value with quotes or spaces is an injection waiting to happen. A second overload takes a THPDFTesseractOptions record. Its Default gives --psm 3, --oem 3, 60 seconds per page and a cap of 16,777,216 pixels, and it rejects tpsOSDOnly and tpsAutoOnly, the two modes that return no words
uses
SysUtils, HPDFTypes, HPDFDoc, HPDFTesseractRecognition;
function MakeSearchable(const SourceFile, TargetFile: string;
Token: THPDFCancellationToken): Boolean;
var
Doc: THotPDF;
Engine: IHPDFOCREngine;
TessOptions: THPDFTesseractOptions;
Options: THPDFOCRTextLayerOptions;
Info: THPDFOCRTextLayerInfo;
begin
TessOptions := THPDFTesseractOptions.Default; // psm 3, oem 3, 60 s, 16,777,216 px
TessOptions.TimeoutMilliseconds := 120000; // applies to each page
// raises EArgumentException for a missing executable or tessdata folder,
// a bad language identifier, or options outside their ranges
Engine := HPDFCreateTesseractOCREngine(
'C:\OCR\Tesseract\tesseract.exe',
'C:\OCR\Tesseract\tessdata',
'eng+chi_sim', // models joined with '+'
TessOptions);
Doc := THotPDF.Create(nil);
try
Doc.AutoLaunch := False;
if Doc.LoadFromFile(SourceFile) < 1 then
raise Exception.Create('Cannot load ' + SourceFile);
Options := THPDFOCRTextLayerOptions.Default; // 300 DPI, MinimumConfidence 0.5
Options.CancellationToken := Token;
// an empty page list means every page; pages that already have text are skipped
Result := Doc.ApplyLoadedOCRTextLayer([], Engine, Options, Info);
if Result then
begin
Writeln(string(Info.EngineName), ': ', Info.AcceptedWordCount,
' words kept, ', Info.DroppedWordCount, ' below the threshold');
Doc.SaveLoadedDocument(TargetFile);
end
else
case Info.Status of
otlsCancelled: Writeln('Cancelled, document unchanged');
otlsEngineError: Writeln('Engine: ', string(Info.Diagnostic));
otlsBudgetExceeded: Writeln('Budget: ', string(Info.Diagnostic));
else
Writeln(string(Info.Diagnostic));
end;
finally
Doc.Free;
end;
end;Do the pixel arithmetic once. An A4 page at 300 DPI is about 2,481 by 3,508 pixels, roughly 8.7 million, and fits. An A3 page at the same resolution is about 3,508 by 4,961, close to 17.4 million, so it fails with a pixel-budget diagnostic and the run ends as otlsEngineError. The overload that takes only a timeout applies no pixel cap at all. For a service that accepts files from outside, keep the options overload and raise MaxPixels (up to 67,108,864) or lower THPDFOCRTextLayerOptions.DPI deliberately when large formats are expected
Per page, the engine creates a private HotPDF-OCR-{GUID} directory under the temp path, saves the bitmap as input.bmp and runs tesseract input.bmp output --tessdata-dir … -l … --dpi N --psm 3 -c tessedit_create_tsv=1, adding --oem when an engine mode is set. Paths are quoted with the Windows rules for backslashes and embedded quotes, and --dpi carries the render resolution so Tesseract never guesses it. Tesseract and its .traineddata files are not bundled; provisioning them is the application's job. On our own Linux test machines the recognizer runs from a user-level bundle of the binary, its shared-library closure and the tessdata folder, because relying on /usr/bin/tesseract only works where someone installed the system package
Why does one broken TSV row empty the whole page?
The Tesseract adapter fails the entire page on the first malformed row, because a partially parsed word list produces a text layer that disagrees with the image, and nothing in the saved PDF says which words went missing. The output must decode as strict UTF-8 with no embedded NUL. The first line must equal Tesseract's twelve-column header, level through text, once an optional byte order mark is stripped. Each row must split into exactly twelve fields, and the split stops after the eleventh tab, so a tab inside the recognized text lands in the text field instead of creating a thirteenth column, where the control-character check below then rejects it
Only level 5 rows are words; page, block, paragraph and line rows are skipped, as are words that are empty or pure whitespace. A word containing a control character (below U+0020, or U+007F to U+009F) fails the page. The rest need integer geometry, a confidence parsed with invariant formatting so a German-locale application still reads 93.5, a box fully inside the bitmap, and a confidence from 0 to 100. One valid word followed by one broken row yields zero words
One odd row therefore fails the whole run and leaves every page untouched. That is the right trade. A rerun is cheap; a searchable PDF where a search for an invoice number misses the one page that matters tends to surface months later, in somebody else's hands
The last step also meets a default you might not expect. Tesseract reports confidence from 0 to 100, the pipeline divides by 100, and THPDFOCRTextLayerOptions.MinimumConfidence defaults to 0.5, so any word below 50 lands in Info.DroppedWordCount instead of on the page. On a clean scan that floor is sensible. On a noisy fax it can remove a large share of the text, so compare Info.ReceivedWordCount with Info.AcceptedWordCount before lowering it
Running OCR again on a file that already has text
SkipPagesWithText is True by default, and that catches people the second time round. Suppose a batch was processed with eng and now needs eng+deu: every page already carries invisible text, so Info.SkippedPageCount equals the page count. HotPDF marks the content stream it writes as its own OCR layer, and ReplaceExistingOCRTextLayer uses that mark to recognize those pages again, dropping the old layer and keeping the original page content
Options := THPDFOCRTextLayerOptions.Default;
Options.ReplaceExistingOCRTextLayer := True; // swap our earlier layer
Options.UseOptionalContentGroup := True; // layer named 'OCR Text Layer'
Options.CancellationToken := Token;
if Doc.ApplyLoadedOCRTextLayer([], Engine, Options, Info) then
Writeln(Info.ReplacedPageCount, ' replaced, ',
Info.SkippedPageCount, ' skipped, ',
Info.RecognizedPageCount, ' recognized')
else
Writeln(string(Info.Diagnostic));A file OCR'd by a scanner driver carries no such mark, so its pages still count as pages with text. Turning SkipPagesWithText off would stack a second invisible layer on the first, and every search hit would show up twice. Leave those files alone
Which handles does the Tesseract process inherit?
The Tesseract child process inherits exactly two handles from HotPDF: one opened on NUL for standard input and output, and one for stderr.txt in the private directory. The pipe hang above comes from CreateProcess itself: bInheritHandles = True is required to pass standard handles, and on its own it passes every inheritable handle in the host, including pipes, files and events opened by unrelated code. HotPDF uses STARTUPINFOEX with an attribute list carrying PROC_THREAD_ATTRIBUTE_HANDLE_LIST and the EXTENDED_STARTUPINFO_PRESENT flag. bInheritHandles stays True, yet only the listed handles cross
The same pattern belongs in every other child process your application starts, since one launcher that inherits everything is enough to hang every pipe reader in the process:
// Pattern for your own CreateProcess calls (fragment: NullHandle, ErrorHandle,
// Exe, CommandLine, WorkDir and ProcessInfo are declared elsewhere).
// Older Delphi RTLs do not declare the attribute-list functions; add your own
// external declarations from kernel32 if the compiler cannot find them
// Check each BOOL result in production code
type
TStartupInfoExRec = record
StartupInfo: TStartupInfo;
AttributeList: Pointer;
end;
const
HANDLE_LIST_ATTRIBUTE = $00020002; // PROC_THREAD_ATTRIBUTE_HANDLE_LIST
EXTENDED_STARTUP_FLAG = $00080000; // EXTENDED_STARTUPINFO_PRESENT
var
Startup: TStartupInfoExRec;
HandleList: array[0..1] of THandle;
Size: NativeUInt;
begin
FillChar(Startup, SizeOf(Startup), 0);
HandleList[0] := NullHandle; // created with bInheritHandle = True
HandleList[1] := ErrorHandle; // likewise
Size := 0;
InitializeProcThreadAttributeList(nil, 1, 0, Size);
GetMem(Startup.AttributeList, Size);
InitializeProcThreadAttributeList(Startup.AttributeList, 1, 0, Size);
UpdateProcThreadAttribute(Startup.AttributeList, 0, HANDLE_LIST_ATTRIBUTE,
@HandleList[0], SizeOf(HandleList), nil, nil);
Startup.StartupInfo.cb := SizeOf(Startup);
// ... set dwFlags and hStdInput / hStdOutput / hStdError as usual ...
CreateProcess(PChar(Exe), PChar(CommandLine), nil, nil,
True, // still required with a handle list
CREATE_NO_WINDOW or EXTENDED_STARTUP_FLAG,
nil, PChar(WorkDir), Startup.StartupInfo, ProcessInfo);
// after the child exits: DeleteProcThreadAttributeList, FreeMem, CloseHandle
end;Bridge scripts add one more Windows detail. A Python launcher can start a second interpreter process, and terminating the process you created leaves that one running. When the engine is given a script path, as the RapidOCR adapter below is, HotPDF starts the child suspended, assigns it to a job object with JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE and then resumes it, so closing the job ends the whole tree. The same containment idea drives isolating PDF image codecs in worker processes, where the child is untrusted; here the child is trusted, but your process is not the only code holding handles in it
Cancellation, timeouts and the all-or-nothing commit
IHPDFOCREngine.Recognize returns one Boolean, and False covers both "Tesseract failed" and "the user pressed Cancel". The adapter polls the token and its timeout every 25 milliseconds while Tesseract runs, and on cancellation it cleans up and returns False. Taken at face value that would read as otlsEngineError, so ApplyLoadedOCRTextLayer checks the token first whenever Recognize returns False. Recognition and content construction finish for every requested page before the document transaction opens, so a cancellation on page 40 of 50 reports otlsCancelled and leaves all 50 pages as loaded. The other limits follow the same style:
- The timeout runs from the start of each
Recognizecall, so the 60,000 ms default is a per-page limit - A child still running at timeout or cancellation is terminated and waited on for up to 5 seconds; the private directory is removed in a
finallyblock, with deletion retried for up to a second because output files can stay locked briefly after the process ends output.tsvis capped at 64 MiB andstderr.txtat 1 MiB, checked while the child runs and after it exits- Words and UTF-16 code units per page are capped by what remains of
MaxWordsPerPage,MaxTotalWordsandMaxTextCodeUnits; going over fails the run instead of truncating the list - A non-zero exit code comes back with up to 4,096 characters of
stderr.txt, the quickest way to learn that a.traineddatafile is missing
What the invisible text layer looks like inside the page
HotPDF writes the words with text rendering mode 3 (neither fill nor stroke, ISO 32000-1 §9.3.6), so the scan stays visible while search and copy work on the words. The content stream opens BT with 3 Tr; each word gets a Tm matrix at its baseline, a font size of box height in pixels times 72 over the render DPI, and a Tz scale that stretches the run to the box width. That stretch keeps a search highlight on the word in the image
The TSV has boxes but no baselines, so each baseline is placed one fifth of the box height above the bottom edge. Between neighboring words on a left-to-right line in space-separated scripts such as Latin or Cyrillic, a stretched space fills the gap, so copied text keeps its word breaks; Chinese gets none. Glyphs go through a shared unembedded Type0 font with Identity-H encoding and a generated ToUnicode CMap, one CID per distinct Unicode scalar, which is how Chinese, accented Latin and supplementary-plane characters survive copy and search. One run holds up to 65,535 distinct scalars, and because ISO 19005 requires embedded fonts, PDF/A output needs a separately embedded conforming font. To check a result, save, reload and run the ordinary path from extracting text from a loaded PDF in Delphi; if the expected words come back on the expected pages, the layer is real
RapidOCR and the native DLL on the same contract
For Simplified Chinese scans, HPDFCreateRapidOCREngine(PythonExecutable, BridgeScript, ModelDirectory, TimeoutMilliseconds) reuses the same process runner and TSV parser. The bridge script path follows the Python executable and the language is fixed to chi_sim. HotPDF ships the bridge as tools/OCR/rapidocr_tsv.py; it needs the rapidocr and onnxruntime packages, local ONNX models and a font file, refuses automatic downloads, and writes Tesseract-shaped TSV; a THPDFRapidOCROptions overload sets model files, thread counts and a pixel cap. The generic factory behind both adapters, HPDFCreateLocalTSVOCREngine in HPDFLocalTSVRecognition, accepts any program that takes the Tesseract-style arguments and writes the twelve-column TSV, and it inherits the handle list, timeout, cancellation and budgets
HPDFCreateTesseractDLLOCREngine(LibraryPath, TessDataDirectory, Language) loads the Tesseract C API in-process instead, reports Tesseract (native DLL), checks up front that every requested .traineddata file exists, and maps cancellation and timeout onto Tesseract's own progress monitor. It saves a process start and a BMP round trip per page; the executable keeps the recognizer's crashes and memory growth out of your address space. For a desktop tool working on the user's own scans the DLL is fine. For a service that ingests documents from strangers, the command-line engine is the safer choice. Either way recognition runs on Windows, one page at a time, on the rendered bitmap, so deskew and clean up difficult scans before they reach the engine
The OCR adapters, the text-layer writer, the renderer feeding them and the extraction that checks the output all live in one native VCL component for Delphi and C++Builder. If your capture or archiving application needs searchable PDF output, the HotPDF Delphi PDF component covers everything except the OCR engine you install yourself