Freeing a HotPDF object graph exactly once in Delphi

A Delphi object model mirroring a parsed PDF file usually ends up as one flat list of every object plus a pointer graph hanging off it. Walk that list calling Free on each entry and the run dies part way through with an access violation inside a destructor, on an object an earlier Free had already released through a different parent. Swap the loop for List.Clear and the crash disappears — but the process now grows by roughly one document every time you reload. Both symptoms share one cause: the list is flat whilst ownership is a graph. A teardown that works decides which pointers own their targets, collects that closure once, cuts every owning edge, and only then frees each node, exactly once

Why does freeing every registered object crash?

In HotPDF the flat list is the document's object registry, and the classes it holds disagree about who destroys what. Taken on its own, each destructor is perfectly reasonable. Put them together over a registry holding entries from several levels of the same ownership chain, and a per-entry Free releases some memory twice and some never

Three destructors in HPDFObjs.pas and HPDFDoc.pas set this up. THPDFDictionaryObject.Destroy walks its Items and frees a value only when IsIndirect is False, on the assumption that indirect children belong to the registry. THPDFArrayObject.Destroy frees every item it holds, indirect or otherwise. THPDFIndirectObject — the wrapper used for objects pulled out of object streams (ISO 32000-1 §7.5.7) — frees its InternalObject body. Now picture a registry holding an indirect dictionary, an array listing that same dictionary in one slot, and a wrapper whose body is also registered as a root in its own right. Free the array first and the dictionary has gone before the registry reaches it. Free the wrapper and then the body, and the second call runs a destructor over memory belonging to nobody. Free only the dictionary, and every indirect child it skipped stays allocated until the process exits

Reordering the registry cannot rescue the situation. Whichever order you choose, some pair of entries ends up on the wrong side of an ownership edge, because a list has no way of saying that entry 12 owns entry 40

Owning edges and references in a PDF object graph

An owning edge is a pointer whose target the source must itself destroy; everything else is a reference, and the teardown follows only the first kind. HotPDF has exactly four kinds of owning edge: the Items of a THPDFDictionaryObject, the Items of a THPDFArrayObject, the InternalObject behind a THPDFIndirectObject, and both fields of a THPDFStreamObject — its Dictionary and its Stream payload

References deserve just as much care, because following one turns a graph walk into an endless loop or a read of freed memory. A THPDFLink holds an object number and generation — precisely the indirect reference of ISO 32000-1 §7.3.10, a name for an object stored elsewhere in the file. Resolving that number through the registry lands on a node another edge already owns, so the teardown never dereferences links at all. The FParent back-pointer kept by dictionaries and arrays runs in the opposite direction: the parent already owns the child, and climbing that pointer would only revisit a node the walk has passed. Both kinds stay untouched for the whole teardown

Collect, detach, free: the teardown in three passes

HotPDF releases a document's objects in three passes inside the protected THotPDF.CloseIndirectObjects method, and the order of those passes is the whole trick. Pass one collects: a worklist starts with every registry entry, and for each node the walk appends the targets of its owning edges, skipping anything it has seen already. The seen-set is an open-addressing array of raw pointers with linear probing that doubles once half full, so nothing is allocated per node — which starts to matter once a document carries hundreds of thousands of objects. Stream payloads go into a second list, being TStream descendants rather than THPDFObject nodes

The two listings below are simplified excerpts of the component's internal implementation, shown so you can see the passes in the library's own object types. Application code never calls any of this; LoadFromFile, LoadFromStream and the destructor run it on your behalf. In the excerpt the address hash is reduced to a shift and a mask, and the step doubling the seen-set table once it fills up is left out

// Simplified excerpt of the internal teardown, not a callable API
procedure Collect(Value: TObject; Payload: boolean);
var
  Slot: Integer;
begin
  if Value = nil then Exit;
  // (the real routine doubles and rehashes the table once it is half full)
  Slot := Integer(NativeUInt(Value) shr 4) and (Length(Seen) - 1);
  while Seen[Slot] <> nil do
  begin
    if Seen[Slot] = Pointer(Value) then Exit;   // already collected
    Slot := (Slot + 1) and (Length(Seen) - 1);
  end;
  Seen[Slot] := Pointer(Value);
  Inc(SeenCount);
  if Payload then Streams.Add(Value) else Nodes.Add(Value);
end;

// Pass one: seed with the registry, then follow only owning edges
for I := 0 to IndirectObjects.Count - 1 do
  Collect(TObject(IndirectObjects[I]), False);
I := 0;
while I < Nodes.Count do
begin
  Obj := THPDFObject(Nodes[I]);
  if Obj is THPDFIndirectObject then
    Collect(THPDFIndirectObject(Obj).InternalObject, False)
  else if Obj is THPDFStreamObject then
  begin
    Collect(THPDFStreamObject(Obj).Dictionary, False);
    Collect(THPDFStreamObject(Obj).Stream, True);
  end
  else if Obj is THPDFDictionaryObject then
    for J := 0 to THPDFDictionaryObject(Obj).Items.Count - 1 do
      Collect(PHPDFDictionaryItem(THPDFDictionaryObject(Obj).Items[J])^.Value, False)
  else if Obj is THPDFArrayObject then
    for J := 0 to THPDFArrayObject(Obj).Items.Count - 1 do
      Collect(TObject(THPDFArrayObject(Obj).Items[J]), False);
  // THPDFLink values and FParent pointers are never followed
  Inc(I);
end;

Pass two cuts. Every owning edge is set to nil before a single destructor runs. A wrapper receives MarkAsFreed, which clears InternalObject and sets the IsFreed flag its destructor checks first. A stream object has Dictionary and Stream assigned nil, each dictionary item has its Value cleared, and each array slot is overwritten with nil. Pass three clears the registry, frees every node in Nodes and then every payload in Streams

// Simplified excerpt of the internal teardown, continued; not a callable API
// Pass two: detach every owning edge before freeing anything
for I := 0 to Nodes.Count - 1 do
begin
  Obj := THPDFObject(Nodes[I]);
  if Obj is THPDFIndirectObject then
    THPDFIndirectObject(Obj).MarkAsFreed
  else if Obj is THPDFStreamObject then
  begin
    THPDFStreamObject(Obj).Dictionary := nil;
    THPDFStreamObject(Obj).Stream := nil;
  end
  else if Obj is THPDFDictionaryObject then
    for J := 0 to THPDFDictionaryObject(Obj).Items.Count - 1 do
      PHPDFDictionaryItem(THPDFDictionaryObject(Obj).Items[J])^.Value := nil
  else if Obj is THPDFArrayObject then
    for J := 0 to THPDFArrayObject(Obj).Items.Count - 1 do
      THPDFArrayObject(Obj).Items[J] := nil;
end;

// Pass three: each unique node and payload is freed exactly once
IndirectObjects.Clear;
for I := 0 to Nodes.Count - 1 do TObject(Nodes[I]).Free;
for I := 0 to Streams.Count - 1 do TObject(Streams[I]).Free;
FreeAndNil(IndirectObjects);

The cut is what lets the existing destructors stay exactly as they are: with no edges left, each destructor finds nothing to recurse into and destroys only itself. The other route to the same result is to teach every destructor about the registry, so that a dictionary, an array and a wrapper each ask whether some other path will free their children. That spreads graph knowledge across four classes and every class added thereafter, and one wrong answer anywhere brings the double free straight back. Keeping the whole ownership model in a single teardown routine is the safer arrangement

Run the awkward cases through it and none of them needs a special branch. A dictionary shared by two stream objects is collected once, detached from both and freed once. A cycle in which an array lists its own parent dictionary terminates because the seen-set refuses the second visit. A wrapper and its body registered as two roots are two distinct pointers in the set, so both are freed, and the wrapper's destructor leaves the body alone because MarkAsFreed has already removed that edge. One TMemoryStream assigned as the payload of two stream objects sits in Streams exactly once

What has to stop before the graph comes down?

Anything borrowing objects from the graph has to stop first. HotPDF can render pages on a worker thread through PrefetchLoadedPages, and it caches bitmaps and display lists compiled from the document's objects; a worker or cache entry that outlived the graph would read freed memory. The teardown therefore opens with CancelLoadedPagePrefetch and invalidates the rendered page cache before touching the registry. LoadFromFile, LoadFromStream and the THotPDF destructor all route through it, so the order is identical whether you replace a document or free the component — and the rules for reusing one THotPDF across documents lean on exactly that

The notification contract matters just as much. InvalidateRenderedPageCache is the method that raises OnLoadedDocumentModified with PageIndex = -1, and a viewer reloading a file has no business receiving an edit notification for the old document's internal cleanup. Inside the teardown the handler is saved, set to nil around that call and restored in a finally block. A reload still produces one PageIndex = -1 call, raised by the load itself as it resets the cache for the new document, and that is the one a viewer should repaint on. In the destructor the caches have already been disposed of by the time the graph closes, so the teardown skips the invalidation there entirely

procedure TViewerForm.FormCreate(Sender: TObject);
begin
  FPdf := THotPDF.Create(Self);
  FPdf.OnLoadedDocumentModified := PdfModified;
end;

procedure TViewerForm.PdfModified(Sender: TObject; PageIndex: Integer);
begin
  if PageIndex = -1 then
    RepaintAllPages           // your own viewer method
  else
    RepaintPage(PageIndex);   // your own viewer method
end;

procedure TViewerForm.OpenDocument(const FileName: string);
begin
  // LoadFromFile cancels a running prefetch and releases the previous
  // document's objects before parsing; PdfModified sees one -1 call
  FPdf.LoadFromFile(FileName);
  FPdf.PrefetchLoadedPages('1-10', 150);
end;

No explicit CancelLoadedPagePrefetch call is needed in OpenDocument, because loading performs it. Calling it yourself remains the right move when you are about to modify the loaded document whilst a prefetch might be running; the declaration comment in HPDFDoc.pas asks for exactly that. For worker-pool rendering on a larger scale, the parallel render pipeline article covers how background work is throttled

How do you tell a leak from allocator retention?

Watch the memory manager's count of live allocations across repeated loads, and ignore the process working set. The Delphi memory manager keeps freed memory pooled for reuse, so a process holding steady at a high footprint after closing a document has not necessarily leaked anything; a process whose live large-block count climbs on every reload has. The loop below loads the same file twenty times on one instance and prints the large-block figures after each pass

procedure ReportLargeBlocks(const Tag: string);
var
  State: TMemoryManagerState;
begin
  GetMemoryManagerState(State);
  Writeln(Tag, ': ', State.AllocatedLargeBlockCount, ' large blocks, ',
    State.TotalAllocatedLargeBlockSize, ' bytes');
end;

var
  Pdf: THotPDF;
  I: Integer;
begin
  Pdf := THotPDF.Create(nil);
  try
    for I := 1 to 20 do
    begin
      Pdf.LoadFromFile('sample.pdf');
      ReportLargeBlocks('pass ' + IntToStr(I));
    end;
  finally
    Pdf.Free;
  end;
  ReportLargeBlocks('after Free');
end.

Read the output as a shape. Numbers that settle after the first pass are retention or ordinary per-document state. Numbers rising by the same step every pass are a leak, and for a PDF the step is usually the size of the larger stream payloads — page content and embedded images — since those are the allocations big enough to land in the large-block figures. Use a file with a dozen or more pages so the step stands out against the noise. Growth in proportion to page count is something allocator retention cannot explain. To find which objects hold memory whilst a document is open, the object dependency graph with retained bytes answers that side of the question

Reusing the pattern in your own Delphi object model

Nothing here is specific to PDF. Any Delphi object model in which destructors own children inconsistently, one child is reachable from several parents, or back-pointers sit alongside forward pointers will crash or leak under a per-object Free. The fix has the same shape every time: classify each pointer field as owning or reference, collect the closure of owning edges through a pointer set that tolerates revisits, cut every edge, destroy the flat list. Cutting is the step people skip — skip it, and every destructor ends up needing to know about every other one

The pattern comes with boundaries worth setting down before you copy it. The pointer set uses the object address as identity, which is sound only because no destructor runs during collection, so no address can be freed and handed to a fresh allocation mid-walk. The walk sees only the edge kinds it was written for; a new class owning a child through some other field must be added to both the collect and the detach pass. And because links are resolved rather than followed, an object referenced only by a link must sit in the registry to be reached. The HotPDF parser registers every object it reads, and a hand-built graph in your own code has to keep the same rule

Test it by counting destructor calls rather than bytes. Build the nasty graph by hand: a dictionary shared by two streams, an array holding both that dictionary and its own parent, one payload assigned to both streams, a root registered twice, and a wrapper whose numeric body is registered as well. Tear it down and expect exactly one destruction per unique object — one payload, two streams, two dictionaries, one array, one wrapper, one number. In your own classes a class-level counter bumped in each destructor is enough for the job (an instrumentation hook you add yourself), and a teardown that merely clears the list reports zero

In HotPDF all of this sits behind LoadFromFile, LoadFromStream and the destructor, so application code sees only the result: closing or reloading a document hands its memory back. HotPDF is a native VCL PDF library for Delphi and C++Builder that ships with full source, so the teardown described here can be read line by line in HPDFDoc.pas; the API reference and a trial build are on the HotPDF PDF component for Delphi page