Technical Article

HotPDF In-Process RapidOCR: Native DLL OCR in Delphi

TerminateThread looks like the obvious way to stop an OCR call that has been grinding on one scanned page for longer than your UI is willing to wait. The Windows documentation explains why it is not: if the target thread holds the process heap lock, that lock is never released, and the next allocation on any other thread can block forever. Once OCR runs inside your own process, the only safe stop is a cooperative one, at checkpoints between inference stages, and that is the price of loading the ONNX models once and keeping them warm

The alternative most Delphi teams try first is a child process. A Python worker per page can be killed at any moment, which is attractive, but every call starts an interpreter, imports its runtime, and loads detection and recognition models before it reads a pixel. Suppose an archive job has 500 pages: that start-up work runs 500 times, and the deployment now includes a Python environment sitting next to a native executable

Searchable PDF from scans with HPDFCreateRapidOCRDLLOCREngine

HotPDF runs RapidOCR in-process through HPDFCreateRapidOCRDLLOCREngine, declared in the HPDFRapidOCRRecognition unit. The factory loads HotPDFRapidOCR.dll, initialises the ONNX detection, optional angle classification, and recognition models once, and returns an IHPDFOCREngine. That engine plugs into THotPDF.ApplyLoadedOCRTextLayer exactly like the other OCR engines: the document renders each page, the DLL runs CPU inference on it, and the result is committed as an invisible Unicode text layer. No Python, no child process. The adapter ships for Delphi, C++Builder, and Windows FPC/Lazarus builds

Validation happens in the factory, before any page is touched. Every model and dictionary file must resolve, HPDFRapidOCRAbiVersion must return 1, and the required exports must exist before HPDFRapidOCRCreate builds the models. Configuration mistakes raise EArgumentException; a model that refuses to load raises EInvalidOperation with the DLL's own diagnostic, such as a dictionary that does not match the recognition model

HotPDF RapidOCR DLL factory validation sequence for HPDFCreateRapidOCRDLLOCREngine: paths and model files must exist, HPDFRapidOCRAbiVersion must return 1, required exports must resolve, and HPDFRapidOCRCreate must initialise the models, with EArgumentException or EInvalidOperation raised eagerly before any recognition runs, the latter carrying the native diagnostic text
validation is eager on purpose: configuration problems raise before any model initialises, so a bad path or ABI never reaches a recognition deadline
uses
  SysUtils, HPDFTypes, HPDFDoc, HPDFRapidOCRRecognition;

procedure MakeSearchable(const SourceFile, TargetFile: string);
var
  Doc: THotPDF;
  Engine: IHPDFOCREngine;
  Options: THPDFOCRTextLayerOptions;
  Info: THPDFOCRTextLayerInfo;
begin
  // Models load here, outside any recognition deadline.
  // Relative names in THPDFRapidOCRDLLOptions.Default resolve
  // against the model directory
  Engine := HPDFCreateRapidOCRDLLOCREngine(
    'C:\OCR\Win64\HotPDFRapidOCR.dll', 'C:\OCR\models');
  Doc := THotPDF.Create(nil);
  try
    Doc.AutoLaunch := False;
    if Doc.LoadFromFile(SourceFile) < 1 then
      raise Exception.Create('Cannot load ' + SourceFile);
    Options := THPDFOCRTextLayerOptions.Default;  // 300 DPI, MinimumConfidence 0.5
    // empty page list = every page; SkipPagesWithText is True by default
    if not Doc.ApplyLoadedOCRTextLayer([], Engine, Options, Info) then
      raise Exception.Create(string(Info.Diagnostic));
    Writeln(string(Info.EngineName), ': ', Info.AcceptedWordCount,
      ' lines accepted, ', Info.DroppedWordCount, ' dropped');
    Doc.SaveLoadedDocument(TargetFile);
  finally
    Doc.Free;
  end;
end;

THPDFRapidOCRDLLOptions.Default names ch_PP-OCRv3_det_infer.onnx, ch_PP-OCRv3_rec_infer.onnx, ch_ppocr_mobile_v2.0_cls_infer.onnx, and ppocr_keys_v1.txt, with one CPU thread, a 16,777,216-pixel input limit, and a 60,000 ms deadline per recognition call. For other scripts, THPDFRapidOCRDLLOptions.ForLanguage keeps those defaults and swaps in <profile>/recognition.onnx plus <profile>/dictionary.txt for one of nine profiles, from Traditional Chinese and Cyrillic to Arabic and Devanagari. Each engine reads one language; pick the profile per page or per document. Why the model and the dictionary always travel as a pair is covered in RapidOCR multilingual models and CTC dictionaries in HotPDF

The engine reports itself as RapidOCR (native DLL) in Info.EngineName. Log that string. When the same service also runs the external Tesseract OCR process adapter or the built-in template-matching OCR engine, it is the quickest way to tell which recogniser produced a bad page

Why does the C ABI only speak int32_t and UTF-8 bytes?

The HotPDFRapidOCR.dll boundary uses fixed-width integers, raw pointers, and explicit byte lengths because Delphi, C++Builder, and Free Pascal share nothing with MSVC beyond the C calling convention. A std::string, a std::vector, or a C++ exception carries a memory layout and an unwinding model that belong to one compiler and one runtime library, and letting any of them cross into Pascal code ends in a corrupted stack or a heap block freed by the wrong allocator

ABI version 1 is a short list of rules. Every export is cdecl and returns an int32_t status: 1 for success, 0 for failure. Every function that can fail takes a caller-owned diagnostic buffer and its size in bytes, and the DLL writes a NUL-terminated UTF-8 message truncated to fit. Inside the DLL, each export body sits in a try with both catch (const std::exception &) and catch (...). An ONNX Runtime error, an OpenCV assertion, or a malformed dictionary becomes status 0 plus text

ExportRoleWhen the adapter resolves it
HPDFRapidOCRAbiVersionReturns 1; any other value is rejectedRight after the DLL loads
HPDFRapidOCRCreateLoads detection, optional classification, recognition models and the dictionaryIn the factory
HPDFRapidOCRRecognizeRuns one bitmap and emits one callback per text lineIn the factory
HPDFRapidOCRDestroyFrees the model instanceIn the factory
HPDFRapidOCRSetReadingDirectionOptional right-to-left row orderOnly when RightToLeft is set

The reading-direction export is resolved lazily on purpose, so a DLL built without it still serves every left-to-right request. The adapter loads the library with LoadLibraryEx and search flags that cover the DLL's own folder plus the default safe directories; ONNX Runtime or OpenCV dependencies placed beside HotPDFRapidOCR.dll are found without editing PATH. Model and dictionary paths cross the boundary as UTF-8, and the DLL converts them with MultiByteToWideChar in strict mode before opening files through wide-character APIs. A model directory under C:\Users\Łukasz or a Cyrillic user name opens normally

One rule lives in the build files rather than the header. The DLL links ONNX Runtime and OpenCV statically, and the CMake project defaults to the static release CRT (/MT). Mix in static libraries compiled against /MD and you get link errors if you are lucky and two independent heaps if you are not. Provision libraries built for the same CRT mode as the DLL

What happens between a TBitmap and a text line?

HotPDF hands the DLL an independent top-down BGR copy of the rendered page, and the DLL hands back one callback per recognised line. The text in that callback is borrowed: the pointer is valid only until the callback returns, so the adapter copies it on the spot

On Delphi the snapshot comes from a private TBitmap forced to pf24bit and read with GetDIBits using a negative biHeight. That gives top-down rows padded to four-byte alignment, and the stride travels to the DLL as an explicit argument. On FPC the adapter reads through CreateIntfImage. The caller's bitmap is never modified. Size checks run before the copy is allocated: at most 32,767 pixels per side, and no more than MaxPixels in total, which defaults to 16,777,216 and can be raised to 67,108,864

HotPDF RapidOCR DLL pipeline from bitmap to text layer: the adapter snapshots the page as top-down pf24bit BGR, the DLL pads, detects, orders and recognises crops, delivers one callback per line with borrowed UTF-8 text, box and confidence, and the adapter validates each line before the text-layer commit
pixels cross the ABI once as a snapshot, lines come back one callback at a time, and nothing reaches the searchable layer until every check passes

Inside the DLL the image gets a 50-pixel white border, text regions are detected with the longer side scaled to at most 1,024 pixels, and the boxes are sorted into horizontal rows. With the angle classifier enabled, a crop is turned 180 degrees only when the upside-down score exceeds 0.9, which keeps a weak prediction from flipping short words. Each line then arrives as a const char*, a byte count, an integer box in original-image pixels, and the mean character confidence. The adapter is strict about what it accepts:

  • UTF-8 is decoded with MB_ERR_INVALID_CHARS, so a malformed byte sequence fails the page instead of planting replacement characters in a searchable layer
  • C0 and C1 control characters are rejected; whitespace-only lines are skipped
  • The box must lie inside the bitmap, and the confidence must be a finite value from 0 to 1
  • Text counts against the request's MaxTextCodeUnits, with a hard ceiling of 1,048,576 UTF-16 units per call; characters outside the Basic Multilingual Plane cost two units
  • A Pascal exception inside the callback is caught there, stored, and turned into a 0 return, which makes the DLL stop; the stored message becomes Info.Diagnostic

Two of these details matter when you tune the output. The unit is a line, not a word: each line takes one MaxWords slot, Info.AcceptedWordCount and Info.DroppedWordCount count lines, and a search hit highlights the whole line box. And MinimumConfidence is compared against the line's mean character confidence, so one smudged character among twenty clean ones rarely sinks the line. Raising the threshold to 0.7 or 0.8 drops whole lines, never single characters. A blank page succeeds with zero lines, and any failure discards partial results so the multi-page commit stays all-or-nothing

One engine, one model copy, one caller at a time

Each RapidOCR DLL engine owns exactly one model instance for its whole lifetime, and Recognize calls on that engine are serialised by a critical section. Holding the IHPDFOCREngine reference is what keeps the models in memory, so batch work should create the engine once and reuse it across every document

procedure OcrBatch(const Files: TStrings; const OutputDir: string);
var
  Models: THPDFRapidOCRDLLOptions;
  Engine: IHPDFOCREngine;
  Doc: THotPDF;
  Options: THPDFOCRTextLayerOptions;
  Info: THPDFOCRTextLayerInfo;
  I: Integer;
begin
  Models := THPDFRapidOCRDLLOptions.ForLanguage('ru-RU');
  Models.UseAngleClassifier := False;    // upright scans: no classifier model loaded
  Models.Threads := 4;                   // 1..64, clamped to active processors
  Models.TimeoutMilliseconds := 120000;  // per Recognize call, cooperative
  Engine := HPDFCreateRapidOCRDLLOCREngine(
    'C:\OCR\Win64\HotPDFRapidOCR.dll', 'C:\OCR\models', Models);
  Options := THPDFOCRTextLayerOptions.Default;
  for I := 0 to Files.Count - 1 do
  begin
    Doc := THotPDF.Create(nil);
    try
      Doc.AutoLaunch := False;
      if (Doc.LoadFromFile(Files[I]) > 0) and
        Doc.ApplyLoadedOCRTextLayer([], Engine, Options, Info) then
        Doc.SaveLoadedDocument(IncludeTrailingPathDelimiter(OutputDir) +
          ExtractFileName(Files[I]))
      else
        Writeln(Files[I], ': ', string(Info.Diagnostic));
    finally
      Doc.Free;
    end;
  end;
end;  // last reference released: models destroyed, then the DLL unloaded

Threads sets both the intra-op and inter-op thread counts of every ONNX session in the engine, and the DLL clamps it to the active processor count. Two threads sharing one engine never run inference in parallel; the second one waits. That wait is not a blind EnterCriticalSection. The adapter tries the lock with TryEnterCriticalSection, sleeps 25 ms between attempts, and checks the cancellation token and the deadline each time, so a queued request can still be cancelled

The deadline clock starts when Recognize is entered, before the lock is taken. Time spent queued behind another page counts against TimeoutMilliseconds. With four worker threads on one engine and a 60-second deadline, the fourth page can time out without ever reaching the model. Give each worker its own engine instead. It costs one copy of the models in memory per engine, and that memory buys predictable timeouts and real parallelism; sharing one engine only makes sense when requests are rare enough that they almost never queue

Why can't a native OCR call be killed mid-inference?

A native RapidOCR call cannot be killed mid-inference because it runs on your thread, inside your process, in the middle of an ONNX Runtime session that has no interruption API for a running Run. Cancellation in the DLL adapter is cooperative. The DLL calls an abort callback before and after detection, after classification of each crop, and after each recognised line, and stops at the first checkpoint where the callback returns 0. A single Run that has already started finishes first

The forceful options are worse than waiting. TerminateThread would leave the CRT heap lock, ONNX Runtime's thread pool, and any OpenCV state in whatever condition they happened to be in. Calling FreeLibrary while a call is still executing unmaps code that is on the stack. Neither can be made safe, so the adapter never attempts them. An expired deadline comes back as otlsEngineError with a timed-out diagnostic; a cancelled token comes back as otlsCancelled:

// Token is created by the caller and shared with the UI thread,
// which calls Token.Cancel when the user presses Stop
function OcrWithStop(Doc: THotPDF; const Engine: IHPDFOCREngine;
  Token: THPDFCancellationToken): Boolean;
var
  Options: THPDFOCRTextLayerOptions;
  Info: THPDFOCRTextLayerInfo;
begin
  Options := THPDFOCRTextLayerOptions.Default;
  Options.CancellationToken := Token;
  Result := Doc.ApplyLoadedOCRTextLayer([], Engine, Options, Info);
  if not Result then
    case Info.Status of
      otlsCancelled:
        // stopped at the next stage or line checkpoint; document unchanged
        Writeln('Cancelled');
      otlsEngineError:
        // includes a cooperative deadline expiry and native diagnostics
        Writeln('Engine: ', string(Info.Diagnostic));
      otlsBudgetExceeded:
        // a page, pixel, word or content limit in Options was reached
        Writeln('Budget: ', string(Info.Diagnostic));
    else
      Writeln(string(Info.Diagnostic));
    end;
end;

This is the real trade-off between the process adapters and the in-process DLL, and neither side wins every row:

HotPDF OCR adapter trade-offs: process adapters start a worker and load models on every page but can be killed and contain crashes, while the in-process RapidOCR DLL loads models once, stops only at cooperative checkpoints, shares the address space, and deploys as a DLL with its models and dictionary
pick per workload: a page-at-a-time desktop app benefits from the warm DLL, while a server ingesting untrusted scans should pay for the process wall
  • Start-up cost: the Tesseract and Python RapidOCR adapters launch a process and load models for every page; the DLL loads models once per engine
  • Stopping: a child process can be terminated outright, and the Python worker runs inside a kill-on-close Job Object so its whole process tree goes with it; the DLL stops at stage and line checkpoints
  • Fault containment: a crash in tesseract.exe fails one page; an access violation inside the DLL takes your process down
  • Deployment: process adapters need an installed program or a Python environment; the DLL needs itself, its models, and its dictionary, matched to the application's bitness
  • Memory: a process adapter releases everything when the child exits; a DLL engine keeps its models resident until the last interface reference goes away

Pick the DLL for an interactive desktop application that OCRs a page at a time and needs a responsive Stop button that returns within one inference stage. For a server ingesting untrusted scans around the clock, pay the per-page start-up cost and keep the process boundary; a crafted image that crashes a recogniser should cost one page, and with a child process it does

Building HotPDFRapidOCR.dll and choosing models

HotPDFRapidOCR.dll is built from the C++ sources in Native/RapidOCR with MSVC, C++17, a Windows SDK, and CMake 3.20 or later. The tools/Build-HotPDFRapidOCR.ps1 helper takes the native source, ONNX Runtime, and OpenCV directories plus -Platform Win32 or Win64, and writes the result to Lib/Native/RapidOCR/<Platform>/. Build both if you ship both. A 32-bit Delphi application cannot load a 64-bit DLL, and the static libraries you provision must match the target architecture as well as the CRT mode

Models have their own compatibility rules. The detector is a DB text detector. The recogniser accepts CTC models in NCHW layout with a fixed input height of 32 or 48, and uses 48 when the model declares a dynamic height. The model format has to be one that the statically linked ONNX Runtime understands; newer exports such as PP-OCRv5 can require a newer runtime, and an unreadable model fails at initialisation with a diagnostic. tools/Install-RapidOCRModels.ps1 installs PP-OCRv3 and PP-OCRv4 packs verified against pinned SHA256 hashes, with a shared detector and classifier under the file names Default expects. Stay on those packs unless you control the runtime build too

The dictionary must be UTF-8 without a BOM, in exactly the model's character order, and its entry count must match the model's output classes. CRLF line endings are fine. When a model embeds its own character list in metadata, model initialisation compares every entry and its position against the dictionary file, so a dictionary that merely has the right length still fails. The DLL never downloads a missing model

HotPDF bundles the native RapidOCR adapter with the process-based OCR adapters, the page renderer that feeds them, and the invisible Unicode text-layer writer, all inside one native VCL component for Delphi and C++Builder. For capture and archiving software that has to produce searchable PDF on machines with no Python installed, the HotPDF Delphi PDF component leaves only the DLL and its model files to deploy