A batch render freezes halfway through because the executor in PDFium Component does not consider a task finished until its reply has been dispatched. Under padSynchronize that reply runs on the main thread. If the main thread blocks without pumping CheckSynchronize, the worker waits on the main thread while the main thread waits on idle
The debugger picture is unmistakable once you have seen it. Pause the frozen process and the main thread sits inside a wait on the idle event, several frames below your own batch loop. Switch to any worker thread and it sits inside TThread.Synchronize, holding a finished result it cannot hand over. Nothing is spinning, no CPU is burning, the process is simply parked. This article is about why that state exists at all, and about three neighbouring rules that decide whether a Delphi worker pool over PDFium behaves or bites: what QueueCapacity actually bounds, what order shutdown has to cancel in, and what parallelism does not buy you where PDFium object ownership is concerned
Why does WaitForIdle hang the main thread?
It hangs because idle in TPdfAsyncExecutor is defined to include reply dispatch, not just worker completion. The running count is incremented in DequeueTask when a worker picks a task up, and it is decremented in TaskFinished, which the worker calls only after TPdfAsyncTaskOperation.Execute has returned. That method runs the worker body, records the outcome, and then dispatches the reply according to TPdfAsyncDispatchMode. With padSynchronize the dispatch is a TThread.Synchronize call, so Execute does not return until the main thread has run it
Delphi puts the second half of that contract on you. TThread.Synchronize appends the method to a global queue and blocks the calling thread on an event; something on the main thread has to call CheckSynchronize before that event is ever signalled. The VCL message loop does this for you between messages, which is exactly why the bug is invisible during interactive use and appears the moment you write a blocking batch loop. A blocking main thread is a main thread that has left the message loop, and a main thread outside the message loop is not draining anybody
uses
System.Classes, FPdfAsync, PDFium;
// The shape that deadlocks: a synchronized reply plus a blocking main thread
Task := Executor.Submit(RenderPageWorker, PageRendered, papNormal,
padSynchronize);
Task.WaitFor(High(Cardinal)); // the main thread now parks in a kernel wait
// Meanwhile TPdfAsyncTaskOperation.Execute has reached:
// TThread.Synchronize(AWorkerThread, DispatchReply);
// which enqueues DispatchReply and waits for the main thread to drain it.
// The main thread is draining nothing, so both sides wait forever.Task completion and executor idle are two different milestones
They are separated on purpose, and knowing which one you are waiting on is the whole fix. IPdfAsyncTask.WaitFor is satisfied the instant the worker result is decided: Complete writes the final TPdfAsyncTaskState and sets the done event before any reply is considered. TPdfAsyncExecutor.WaitForIdle is satisfied later, once queued and running counts are both zero, and running does not drop until the reply has landed. So a task can be patsSucceeded and observable through Snapshot while the executor is still legitimately busy
// TPdfAsyncExecutor.WaitForIdle already pumps for you: it waits on the idle
// event in short slices and calls CheckSynchronize(0) between them.
if not Executor.WaitForIdle(30000) then
ReportBatchTimeout;
// Any hand-rolled main-thread wait has to do the same thing explicitly.
function WaitForTaskOnMainThread(const ATask: IPdfAsyncTask;
ATimeoutMs: Cardinal): Boolean;
var
StartedAt: UInt64;
begin
StartedAt := PdfAsyncTick;
repeat
if ATask.WaitFor(10) then
Exit(True);
CheckSynchronize(0); // release any pending padSynchronize reply
Result := PdfAsyncTickDelta(StartedAt, PdfAsyncTick) < ATimeoutMs;
until not Result;
end;One consequence worth internalising: a reply that raises does not rewrite history. DispatchReply catches the exception and stores it in ReplyErrorMessage, leaving State, CancellationReason and ErrorMessage exactly as the worker determined them. A UI callback that blows up while painting a thumbnail therefore never turns a successful render into a failed one, and your telemetry keeps reporting what the render engine actually did. If you want the callback-shaped API around a single operation rather than a pool, background rendering with cancellable futures covers that path
Does QueueCapacity bound running workers too?
No. QueueCapacity in PDFium Component counts queued tasks only, never the ones already executing on a worker. That is deliberate: capacity is meant to express real backpressure on the waiting line, and folding the fixed concurrency slots into the same number would count them twice. With four workers and a capacity of eight you can have twelve tasks in flight, and GetStats reports the split honestly through QueuedCount and RunningCount
var
Stats: TPdfAsyncExecutorStats;
Task: IPdfAsyncTask;
begin
// TrySubmit never raises: it returns False when the waiting line is full or
// the executor is already shutting down, and bumps RejectedCount.
if not Executor.TrySubmit(RenderPageWorker, PageRendered, Task, papHigh,
padSynchronize) then
begin
Stats := Executor.GetStats;
// QueuedCount is what QueueCapacity bounds. RunningCount is bounded by
// WorkerCount and is never charged against the capacity.
LogBackpressure(Stats.QueuedCount, Stats.RunningCount,
Stats.RejectedCount);
Exit;
end;The four lanes of TPdfAsyncPriority are strict, not weighted. DequeueTask walks from papCritical down to papLow and takes the first non-empty lane, preserving FIFO order within each. That gives an interactive request a clean way to jump ahead of a batch that has not started yet, but it never interrupts work already running, and a caller that keeps feeding papCritical can starve papLow indefinitely. Reserve the top two lanes for things a human is visibly waiting on, and leave bulk export on papNormal or below. Use Submit when a full queue is a programming error worth an EPdfAsyncQueueFull, and TrySubmit when it is a normal condition you intend to handle
Why does Shutdown cancel outside the executor lock?
Because cancelling inside it would invert the lock order and hang the shutdown you are trying to perform. Shutdown(True) takes the executor lock, flips the shutdown flag, and appends every pending task to a local snapshot array through AppendSnapshot. Then it releases the lock and only afterwards walks the snapshot calling Cancel on each entry. Cancelling a task fires user callbacks registered on its token source, and those callbacks are ordinary application code: they may query GetStats, submit compensating work, or wait for idle. Every one of those re-enters the executor lock, and a callback invoked while that lock is held would deadlock against itself
The token source obeys the same discipline one level down. CancelWithReason takes the source lock, decides the single winning canceller, writes Reason, CancellationMessage and CancelledAtTick, and only then flips the cancelled flag atomically. Publish before flip is what makes the metadata safe to read: any thread that observes IsCancelled as True is guaranteed to find a complete reason behind it, and later callers lose the race, return False, and cannot overwrite the first reason. The registered callbacks are snapshotted and cleared inside the lock but invoked outside it, each wrapped so one failing handler cannot suppress the rest. Tasks that are already running are never killed; they end cooperatively when their worker body next calls ThrowIfCancelled, which is why Shutdown finishes with a pumping WaitForIdle before joining threads
Do parallel workers relax PDFium object ownership?
They do not, and this is the boundary most likely to be misread. TPdfAsyncExecutor schedules work; it makes no claim about the thread affinity of anything you touch inside that work. A live TPdf instance does not become concurrently accessible because two workers happen to call into it, and the internal render lock is a guard against overlapping render calls, not a licence to share a document across threads. Parallel rendering or export means one TPdf per worker, created and destroyed inside the job
type
TPageRenderJob = class
private
FFileName: string;
FPageIndex: Integer;
public
procedure Run(const AToken: IPdfCancellationToken);
end;
procedure TPageRenderJob.Run(const AToken: IPdfCancellationToken);
var
LocalPdf: TPdf; // one document instance per worker, never shared
Bmp: TBitmap;
begin
LocalPdf := TPdf.Create(nil);
try
LocalPdf.FileName := FFileName;
LocalPdf.Active := True;
LocalPdf.PageNumber := FPageIndex;
AToken.ThrowIfCancelled;
Bmp := LocalPdf.RenderPage(0, 0, 1024, 1448);
try
HandOffBitmap(FPageIndex, Bmp); // ownership moves to the reply stage
finally
Bmp.Free;
end;
finally
LocalPdf.Free;
end;
end;The cost is real and worth naming: every worker pays its own parse and its own page cache, so memory scales with worker count rather than with document count. That is the price of a model where a worker can be cancelled or crash without corrupting anyone else. If your workers do share a viewer-side document instead, the locking rules around that are covered in the render lock and the calls that miss it, and the single-document cancellable path is in cancellable progressive rendering
Extending a published interface without breaking the vtable
IPdfCancellationToken and IPdfCancellationTokenSource are COM-style interfaces that external binaries may already consume, so appending a method to either would shift every later slot in the vtable and silently misroute calls compiled against the old layout. The diagnostic, waiting, removable-callback and atomic-cancel capabilities therefore live in IPdfCancellationTokenEx and IPdfCancellationTokenSourceEx, which inherit rather than modify. New and Run keep their original semantics for existing callers; new code reaches for NewEx, NewTimeout and RunEx when it wants CancelWithReason, WaitForCancellation or a managed IPdfCancellationRegistration. Inheritance is the only safe way to grow a published interface, and it costs one extra type per generation
NewTimeout deserves an honest note. Each timeout source owns a lightweight thread that waits on the cancellation event or the deadline, whichever comes first. For a handful or a few dozen deadlines that is simple, low-latency and identical across Delphi, Lazarus and C++Builder. For thousands of short deadlines it is the wrong shape, and you should drive cancellation from one application-level timer instead of holding thousands of waiting threads
None of these rules are exotic once written down, but every one of them is a production incident when it is not. Wait on the right milestone and let something pump the synchronize queue, read QueueCapacity as a bound on the waiting line only, cancel outside your locks, and give every worker its own document. The async layer described here ships as part of the Delphi PDFium Component, alongside the rendering, text and form APIs it schedules