Technical Article

PDFium Progressive Download and Cancel in Delphi (FPDFAvail)

A non-null document handle with a page count of zero is what FPDFAvail_GetDocument can hand back when it is called on a PDF that was never linearised, even after FPDFAvail_IsDocAvail has reported PDF_DATA_AVAIL. The handle looks open, nothing complains, and the viewer shows an empty document. PDFium's availability API reads the linearised layout of ISO 32000-1 Annex F; any other file has to be complete and opened through the ordinary custom-document path. That routing decision is half of what opening a PDF during download involves. The other half is the user who closes the tab at a quarter of the file and opens the same link again a moment later

The availability loop behind TPdfProgressiveDocument

In the PDFium Component the whole session lives in TPdfProgressiveDocument, a TPdf subclass that keeps a PDFium availability provider alive while a random-access stream fills up. BeginProgressiveLoad(AStream, AFileSize, AOwnsStream, AInitialAvailableByteCount) takes the backing stream and the logical size of the remote file (the Content-Length, not the current length of your local store), wires an IsDataAvail callback and an AddSegment callback, and calls FPDFAvail_Create. Once OpenProgressiveDocument succeeds, the object behaves like any other active TPdf: pages, text, rendering and saving all work through the usual members

Every parse step is a question to your code. When PDFium wants a range, the component answers yes if the range lies inside the contiguous prefix described by AvailableByteCount, or inside a range already completed through the RangeRequests scheduler. An OnDataAvailable handler then gets the last word, which is how a sparse cache answers for ranges neither of those can describe. CheckDocumentAvailability returns one of three TPdfDataAvailability values (pdaAvailable, pdaNotAvailable, pdaError) and hands back the byte ranges PDFium asked for as a sorted, merged TPdfDownloadRanges array. Those ranges are already queued on the scheduler at rrpImmediate priority by the time you see them, so the hint array is for logging and progress display; the work queue is RangeRequests

// FetchRange is your transport (HTTP Range GET, socket, blob reader):
// it writes Size bytes at Offset into Store and returns how many arrived
function FetchRange(Store: TStream; Offset, Size: UInt64): UInt64; forward;

procedure OpenWhileDownloading(Pdf: TPdfProgressiveDocument; Store: TStream;
  RemoteSize: UInt64);
const
  MaxRounds = 64;
var
  Hints: TPdfDownloadRanges;
  State: TPdfDataAvailability;
  Request: TPdfRangeRequest;
  Round: Integer;
begin
  Pdf.BeginProgressiveLoad(Store, RemoteSize, False);
  State := pdaNotAvailable;
  for Round := 1 to MaxRounds do
  begin
    State := Pdf.CheckDocumentAvailability(Hints);
    if State <> pdaNotAvailable then
      Break;
    // The hints are already queued; write the bytes first, then complete
    while Pdf.RangeRequests.TryDequeue(Request) do
      Pdf.RangeRequests.CompleteRequest(Request,
        FetchRange(Store, Request.Offset, Request.Size));
  end;
  if State <> pdaAvailable then
    raise EPdfError.Create('The document could not be discovered');
  Pdf.OpenProgressiveDocument;
end;

Two details in that loop carry weight. The round cap exists because a dead link makes CheckDocumentAvailability ask for the same ranges on every call, and an unbounded loop turns a network failure into a hung UI. The ordering matters because the scheduler serializes its own state with a critical section but does nothing for TStream.Position on the backing store. A transport thread has to write the response bytes into the stream before it calls CompleteRequest, since PDFium may read the range the moment the completion is published. Concurrent writers need positioned I/O or a lock of their own

The availability loop of TPdfProgressiveDocument in PDFium Component: BeginProgressiveLoad creates the FPDFAvail provider, CheckDocumentAvailability hands back sorted merged download hints queued at rrpImmediate priority, the transport writes bytes into the store before CompleteRequest publishes each range to PDFium, and the loop is capped at 64 rounds because a dead link keeps asking for the same ranges
Write the bytes, then complete the request: the moment a completion is published PDFium may read that range, and nothing protects the stream position for you

Why does AvailableByteCount refuse to move backwards?

AvailableByteCount only grows. Assigning a smaller value raises EPdfError with "Available byte count cannot move backwards", because once the availability callback has told PDFium that a range exists, the parser may already have read and cached objects from it. Taking those bytes away afterwards would leave PDFium holding objects whose source the availability answers now deny. The same setter rejects values larger than LogicalFileSize and raises "No progressive load is active" outside a session

That last check decides where bytes you already hold belong. If a previous visit left the first 300 KB of the file in a cache, pass that count as AInitialAvailableByteCount; the component records it before FPDFAvail_Create runs, so PDFium's first look at the header already sees it. With an initial count of zero, Linearization reads plnUnknown until PDFium has seen the header; that is the normal starting state

Raising the prefix also cancels work. Any in-flight request that overlaps the newly announced bytes is now redundant, so the scheduler drops it and fires OnCancelRequest for it; a late CompleteRequest for that request returns False. A store that fills out of order should not try to express itself through the prefix at all. Complete the ranges through the scheduler, or answer per range in OnDataAvailable. An exception raised inside that handler cannot cross the C callback boundary, so the component catches it and reports the range as missing, and PDFium simply asks again on the next check

When can a partially downloaded PDF actually open?

Only a linearised PDF opens before the whole file has arrived. OpenProgressiveDocument checks the Linearization property (plnUnknown, plnNotLinearised, plnLinearised) and picks the loader. A linearised file opens through FPDFAvail_GetDocument as soon as the first-page section and the hint tables are present. A non-linearised file goes through FPDF_LoadCustomDocument on the same file-access record, and for such a file PDFium only reports the document as available once every byte is there. That routing is what keeps the zero-page handle from the opening paragraph out of your viewer

How OpenProgressiveDocument routes a partial download in PDFium Component: a linearised file opens through FPDFAvail_GetDocument once the first-page section and hint tables arrive, a non-linearised file needs FPDF_LoadCustomDocument and every byte, and LoadAvailablePage checks form availability with FPDFAvail_IsFormAvail before the page check, avoiding the non-null zero page handle trap
Only linearised files gain a head start; on anything else FPDFAvail_GetDocument can return an open-looking document with zero pages, which is exactly what the routing prevents

So the file on the server decides how fast the first page appears. If time to first paint matters and you control the publishing side, linearise the files there. TPdf.SaveAsLinearised writes the Annex F layout from the component itself, and the Fast Web View and hint table walkthrough covers what that layout contains. For files you do not control, a non-linearised PDF still loads through the same loop; it just reaches pdaAvailable at 100 percent instead of early

LoadAvailablePage takes a 1-based page number and keeps the order PDFium expects. Before the first page check it runs CheckFormAvailability, which wraps FPDFAvail_IsFormAvail, and only then calls FPDFAvail_IsPageAvail. A result of pfaNotPresent is the normal answer for a document without an AcroForm and blocks nothing. When the page is ready, LoadAvailablePage makes it the active page, so a viewer can render page 1 of a linearised brochure while later pages are still in transit. The page wait is the same shape as the first listing: call LoadAvailablePage, drain RangeRequests on pdaNotAvailable, stop on pdaAvailable or pdaError, and keep a round cap. FirstAvailablePageNumber reports the page the linearisation dictionary designates as first, already converted from PDFium's zero-based index. For a non-linearised file every page reports pdaAvailable straight away, because the document could only open once the file was complete

Retries, short reads and priorities in the range scheduler

The FPDFAvail_* API itself knows nothing about networks. It says which bytes the parser needs right now and asks again later; there is no notion of a request, a retry or a cancel. TPdfRangeRequestScheduler supplies those, and four of its rules shape how you write the transport

  • A short response is treated as progress. CompleteRequest(Request, BytesReceived) with fewer bytes than requested publishes the prefix and queues the remainder. The short response also counts as a failed attempt, so the remainder is still bound by MaxRetryCount and is abandoned once that runs out; zero bytes is a plain failed attempt
  • MaxRetryCount defaults to 2, so one range gets three attempts in total. FailRequest returns True when it re-queued the range and False once the range is abandoned; Statistics.AbandonedRequestCount tracks how many gave up
  • MaxRequestSize splits large ranges into chunks at dequeue time, and MergeGap lets two pending ranges merge across a small unknown gap, but never across bytes that are already completed or in flight
  • Hints from CheckDocumentAvailability and LoadAvailablePage arrive at rrpImmediate. Form hints from a direct CheckFormAvailability call arrive at rrpNormal; when LoadAvailablePage runs that check itself and the form data is missing, the hints are queued at rrpImmediate. TryDequeue always serves the highest priority first and the lowest offset within a priority

The priority rule pays off when you prefetch. Queue the whole file at rrpBackground right after the document opens, and the scheduler subtracts everything already completed or in flight, so only the missing bytes go on the wire. When the user then jumps to page 40 and that page's hint overlaps a background chunk already downloading, the scheduler pulls the background request, fires OnCancelRequest for it, puts the non-overlapping tails back at background priority without charging them a retry, and queues the overlap as fresh immediate work. The jump wins the connection. Without that, the page request would wait behind a 256 KB chunk it does not care about

procedure TDownloadForm.QueueRestOfFile;
var
  Whole: TPdfDownloadRanges;
begin
  FPdf.RangeRequests.MaxRequestSize := 256 * 1024;  // chunk size is your call
  SetLength(Whole, 1);
  Whole[0].Offset := 0;
  Whole[0].Size := FPdf.LogicalFileSize;
  FPdf.RangeRequests.EnqueueHints(Whole, rrpBackground);
end;

// Called on a worker thread when one HTTP Range response finishes.
// WriteAt is your own positioned write into the backing store
procedure TDownloadForm.RangeFinished(const Request: TPdfRangeRequest;
  const Body: TBytes; Succeeded: Boolean);
begin
  if Succeeded and (Length(Body) > 0) then
  begin
    WriteAt(FStore, Request.Offset, Body);              // bytes first
    FPdf.RangeRequests.CompleteRequest(Request, Length(Body));
  end
  else if not FPdf.RangeRequests.FailRequest(Request) then
    TThread.Queue(nil, ShowRangeGaveUp);  // abandoned, or from a cancelled session
end;

Both CompleteRequest and FailRequest are safe to call from the worker thread; the scheduler takes its own lock. A request that was superseded or belongs to a cancelled session comes back as False without touching anything, so the worker does not need to know why its range went stale

What does CancelProgressiveLoad release, and in what order?

CancelProgressiveLoad tears a session down in four steps, always in the same order: cancel the range scheduler, close the document, destroy the availability handle with FPDFAvail_Destroy, then release the callback records and the stream adapter. Cancelling the scheduler first advances its generation counter, drops every pending and in-flight request, and fires OnCancelRequest for each in-flight one. A transport completion that lands afterwards carries the old generation, and CompleteRequest returns False. The document closes before the availability handle and the stream adapter go away because PDFium can still read through the file-access provider while it closes a document

The fixed teardown order of CancelProgressiveLoad in PDFium Component: cancel the range scheduler first so late completions hit the bumped generation counter and return False, close the document before the file-access adapter disappears, destroy the availability handle with FPDFAvail_Destroy, and only then dispose the callback records and free the stream adapter
One idempotent method cleans up a failed start, a user cancel and the destructor alike; with a worker thread writing into the store, stream ownership stays with you
procedure TDownloadForm.FormCreate(Sender: TObject);
begin
  FPdf := TPdfProgressiveDocument.Create(nil);
  // The scheduler lives as long as FPdf, so wire it once
  FPdf.RangeRequests.OnCancelRequest := RangeCancelled;
end;

procedure TDownloadForm.RangeCancelled(Sender: TObject; RequestId: UInt64;
  Attempt: Cardinal);
begin
  FTransport.Abort(RequestId);   // your code: close that socket or request
end;

procedure TDownloadForm.CancelButtonClick(Sender: TObject);
begin
  FPdf.CancelProgressiveLoad;
  // ProgressiveLoading = False, Active = False, AvailableByteCount = 0
end;

The method is idempotent and serves as the single cleanup path for three situations: a BeginProgressiveLoad that fails halfway through construction, an explicit user cancel, and the destructor. BeginProgressiveLoad calls it too, so restarting the same object on a new URL is fine while the earlier session never got as far as opening. Once OpenProgressiveDocument has succeeded the object is active, and BeginProgressiveLoad raises the usual "Cannot perform this operation on an active" error; call CancelProgressiveLoad first. OnCancelRequest is also the only place where your sockets get closed, since the scheduler has no idea what a socket is. Exceptions raised inside it are swallowed per request, so one failing abort cannot block the rest

Who writes the store decides who owns it. With a worker thread writing into the backing stream, pass AOwnsStream = False and free the stream yourself after the worker has stopped. Handing ownership over means the cancel frees the stream while a write already in progress may still be on its way. Keep ownership whenever a thread other than the caller touches the store; AOwnsStream = True only fits a fully synchronous transport like the first listing

Features that wait for the whole file

Some features need the original file bytes, not the parsed document, and they refuse an incomplete progressive source outright. ReadXmpPacket raises "PDF/A metadata is unavailable until progressive loading completes". ValidatePades returns a result whose SourceStatus is pvssIncomplete, and ValidatePdfMac reports unsupported with a message asking for the complete source. BeginIncrementalSave fails as well, because an incremental update has to append to the exact bytes that were loaded

The completeness test behind all of them is AvailableByteCount reaching LogicalFileSize. A transport that delivers everything through the scheduler can therefore hold every byte and still be told the source is incomplete, because the contiguous prefix never moved. Publish the prefix once the scheduler confirms full coverage:

procedure TDownloadForm.CheckSignaturesWhenComplete;
var
  Report: TPadesValidationResult;
begin
  if FPdf.RangeRequests.IsRangeAvailable(0, FPdf.LogicalFileSize) then
    FPdf.AvailableByteCount := FPdf.LogicalFileSize;
  Report := FPdf.ValidatePades;
  if Report.SourceStatus = pvssIncomplete then
    LabelStatus.Caption := 'Signatures are checked when the download finishes'
  else
    ShowSignatureReport(Report);   // your UI
end;

One size limit applies before any of this starts. On Windows FPDF_FILEACCESS stores the file length in a 32-bit field, so BeginProgressiveLoad rejects a logical size above 4 GiB with an EPdfError instead of letting PDFium parse a truncated length

For loading a complete local file on demand through a stream adapter, see streaming large PDFs on demand with PDFium; for a PDF that sits inside a larger buffer, see byte range loading for embedded PDFs. Stopping a slow render of a page that has already arrived is a different mechanism, covered in cancellable progressive page rendering. TPdfProgressiveDocument, its range scheduler and SaveAsLinearised are part of the PDFium Component for Delphi and C++Builder