Freeing a PDF Object Graph Exactly Once in Delphi: HotPDF

A Delphi object model that mirrors a parsed PDF file usually ends up as one flat list of every object plus a pointer graph hanging off it. Walk that list calling Free on each entry and the run dies partway through with an access violation inside a destructor, on an object that an earlier Free already released through a different parent. Replace the loop with List.Clear and the crash goes away, but the process now grows by roughly one document every time you reload. Both symptoms have one cause: the list is flat and ownership is a graph. The teardown that works decides which pointers own their targets, collects that closure once, cuts every owning edge and only then frees each node, exactly once

Why does freeing every registered object crash?

In HotPDF the flat list is the document's object registry, and the classes it holds disagree about who destroys what. Each destructor is reasonable on its own terms. Put them together over a registry that contains entries from several levels of the same ownership chain, and a per-entry Free releases some memory twice and some memory never

Three destructors in HPDFObjs.pas and HPDFDoc.pas set this up. THPDFDictionaryObject.Destroy walks its Items and frees a value only when IsIndirect is False, on the assumption that indirect children belong to the registry. THPDFArrayObject.Destroy frees every item it holds, indirect or not. THPDFIndirectObject, the wrapper used for objects pulled out of object streams (ISO 32000-1 §7.5.7), frees its InternalObject body. Now picture a registry holding an indirect dictionary, an array that lists the same dictionary in one slot, and a wrapper whose body is also registered as a root of its own. Free the array first and the dictionary is gone before the registry reaches it. Free the wrapper and then the body, and the second call runs a destructor on memory that no longer belongs to anyone. Free only the dictionary and every indirect child it skipped stays allocated until process exit

Reordering the registry cannot rescue this. Whatever order you pick, some pair of entries lands on the wrong side of an ownership edge, because a list has no way to say that entry 12 owns entry 40

Owning edges and references in a PDF object graph

An owning edge is a pointer whose target the source is responsible for destroying. Everything else is a reference, and the teardown follows only the first kind. In HotPDF there are exactly four owning edge kinds: the Items of a THPDFDictionaryObject, the Items of a THPDFArrayObject, the InternalObject behind a THPDFIndirectObject, and both fields of a THPDFStreamObject — its Dictionary and its Stream payload

References deserve the same care, because following one turns a graph walk into an endless loop or a read of freed memory. A THPDFLink holds an object number and generation, which is exactly the indirect reference of ISO 32000-1 §7.3.10: a name for an object stored somewhere else in the file. Resolving that number through the registry lands on a node that another edge already owns, so the teardown never dereferences links at all. The FParent back-pointer that dictionaries and arrays keep runs in the opposite direction. The parent already owns the child, and climbing that pointer would only revisit a node the walk has passed. Both kinds stay untouched for the entire teardown

Collect, detach, free: the teardown in three passes

HotPDF releases a document's objects in three passes inside the protected THotPDF.CloseIndirectObjects method, and the order of those passes is the whole trick. Pass one collects. A worklist starts with every registry entry, and for each node the walk appends the targets of its owning edges, skipping anything it has already seen. The seen-set is an open-addressing array of raw pointers with linear probing that doubles once it is half full, so nothing gets allocated per node, which starts to matter when a document carries hundreds of thousands of objects. Stream payloads go into a second list because they are TStream descendants rather than THPDFObject nodes

The two listings that follow are simplified excerpts of the component's internal implementation, shown so you can see the passes in the library's own object types. Application code never calls any of this; LoadFromFile, LoadFromStream and the destructor run it for you. In the excerpt the address hash is reduced to a shift and a mask, and the step that doubles the seen-set table when it fills up is left out

// Simplified excerpt of the internal teardown, not a callable API
procedure Collect(Value: TObject; Payload: boolean);
var
  Slot: Integer;
begin
  if Value = nil then Exit;
  // (the real routine doubles and rehashes the table once it is half full)
  Slot := Integer(NativeUInt(Value) shr 4) and (Length(Seen) - 1);
  while Seen[Slot] <> nil do
  begin
    if Seen[Slot] = Pointer(Value) then Exit;   // already collected
    Slot := (Slot + 1) and (Length(Seen) - 1);
  end;
  Seen[Slot] := Pointer(Value);
  Inc(SeenCount);
  if Payload then Streams.Add(Value) else Nodes.Add(Value);
end;

// Pass one: seed with the registry, then follow only owning edges
for I := 0 to IndirectObjects.Count - 1 do
  Collect(TObject(IndirectObjects[I]), False);
I := 0;
while I < Nodes.Count do
begin
  Obj := THPDFObject(Nodes[I]);
  if Obj is THPDFIndirectObject then
    Collect(THPDFIndirectObject(Obj).InternalObject, False)
  else if Obj is THPDFStreamObject then
  begin
    Collect(THPDFStreamObject(Obj).Dictionary, False);
    Collect(THPDFStreamObject(Obj).Stream, True);
  end
  else if Obj is THPDFDictionaryObject then
    for J := 0 to THPDFDictionaryObject(Obj).Items.Count - 1 do
      Collect(PHPDFDictionaryItem(THPDFDictionaryObject(Obj).Items[J])^.Value, False)
  else if Obj is THPDFArrayObject then
    for J := 0 to THPDFArrayObject(Obj).Items.Count - 1 do
      Collect(TObject(THPDFArrayObject(Obj).Items[J]), False);
  // THPDFLink values and FParent pointers are never followed
  Inc(I);
end;

Pass two cuts. Every owning edge is set to nil before a single destructor runs. A wrapper gets MarkAsFreed, which clears InternalObject and sets the IsFreed flag its destructor checks first. A stream object has Dictionary and Stream assigned nil, each dictionary item has its Value cleared, and each array slot is overwritten with nil. Pass three clears the registry, frees every node in Nodes and then every payload in Streams

// Simplified excerpt of the internal teardown, continued; not a callable API
// Pass two: detach every owning edge before freeing anything
for I := 0 to Nodes.Count - 1 do
begin
  Obj := THPDFObject(Nodes[I]);
  if Obj is THPDFIndirectObject then
    THPDFIndirectObject(Obj).MarkAsFreed
  else if Obj is THPDFStreamObject then
  begin
    THPDFStreamObject(Obj).Dictionary := nil;
    THPDFStreamObject(Obj).Stream := nil;
  end
  else if Obj is THPDFDictionaryObject then
    for J := 0 to THPDFDictionaryObject(Obj).Items.Count - 1 do
      PHPDFDictionaryItem(THPDFDictionaryObject(Obj).Items[J])^.Value := nil
  else if Obj is THPDFArrayObject then
    for J := 0 to THPDFArrayObject(Obj).Items.Count - 1 do
      THPDFArrayObject(Obj).Items[J] := nil;
end;

// Pass three: each unique node and payload is freed exactly once
IndirectObjects.Clear;
for I := 0 to Nodes.Count - 1 do TObject(Nodes[I]).Free;
for I := 0 to Streams.Count - 1 do TObject(Streams[I]).Free;
FreeAndNil(IndirectObjects);

The cut is what lets the existing destructors stay exactly as they are. With no edges left, each destructor finds nothing to recurse into and destroys only itself. The other route to the same result is to teach every destructor about the registry, so that a dictionary, an array and a wrapper each ask whether some other path will free their children. That spreads graph knowledge across four classes and every class added later, and one wrong answer anywhere brings the double free back. Keeping the whole ownership model in a single teardown routine is the safer arrangement

Run the awkward cases through it and none of them needs a special branch. A dictionary shared by two stream objects is collected once, detached from both and freed once. A cycle in which an array lists its own parent dictionary ends because the seen-set refuses the second visit. A wrapper and its body registered as two roots are two distinct pointers in the set, so both get freed, and the wrapper's destructor leaves the body alone because MarkAsFreed has already removed that edge. One TMemoryStream assigned as the payload of two stream objects sits in Streams exactly once

What has to stop before the graph comes down?

Anything that borrows objects from the graph has to stop first. HotPDF can render pages on a worker thread through PrefetchLoadedPages, and it caches bitmaps and display lists compiled from the document's objects; a worker or a cache entry that outlived the graph would read freed memory. The teardown therefore opens with CancelLoadedPagePrefetch and invalidates the rendered page cache before it touches the registry. LoadFromFile, LoadFromStream and the THotPDF destructor all route through it, so the order is identical whether you replace a document or free the component, and the rules for reusing one THotPDF across documents lean on that

The notification contract matters just as much. InvalidateRenderedPageCache is the method that raises OnLoadedDocumentModified with PageIndex = -1, and a viewer that reloads a file has no business receiving an edit notification for the old document's internal cleanup. Inside the teardown the handler is saved, set to nil around that call and restored in a finally block. A reload still produces one PageIndex = -1 call, raised by the load itself when it resets the cache for the new document, and that is the one a viewer should repaint on. In the destructor the caches are already disposed by the time the graph closes, so the teardown skips the invalidation there entirely

procedure TViewerForm.FormCreate(Sender: TObject);
begin
  FPdf := THotPDF.Create(Self);
  FPdf.OnLoadedDocumentModified := PdfModified;
end;

procedure TViewerForm.PdfModified(Sender: TObject; PageIndex: Integer);
begin
  if PageIndex = -1 then
    RepaintAllPages           // your own viewer method
  else
    RepaintPage(PageIndex);   // your own viewer method
end;

procedure TViewerForm.OpenDocument(const FileName: string);
begin
  // LoadFromFile cancels a running prefetch and releases the previous
  // document's objects before parsing; PdfModified sees one -1 call
  FPdf.LoadFromFile(FileName);
  FPdf.PrefetchLoadedPages('1-10', 150);
end;

No explicit CancelLoadedPagePrefetch call is needed in OpenDocument, because loading performs it. Calling it yourself is still the right move when you are about to modify the loaded document while a prefetch might be running; the declaration comment in HPDFDoc.pas asks for exactly that. For worker-pool rendering at larger scale, the parallel render pipeline article covers how background work is throttled

How do you tell a leak from allocator retention?

Watch the memory manager's count of live allocations across repeated loads, and ignore the process working set. The Delphi memory manager keeps freed memory pooled for reuse, so a process that holds steady at a high footprint after closing a document has not necessarily leaked anything. A process whose live large-block count climbs on every reload has. The loop below loads the same file twenty times on one instance and prints the large-block figures after each pass

procedure ReportLargeBlocks(const Tag: string);
var
  State: TMemoryManagerState;
begin
  GetMemoryManagerState(State);
  Writeln(Tag, ': ', State.AllocatedLargeBlockCount, ' large blocks, ',
    State.TotalAllocatedLargeBlockSize, ' bytes');
end;

var
  Pdf: THotPDF;
  I: Integer;
begin
  Pdf := THotPDF.Create(nil);
  try
    for I := 1 to 20 do
    begin
      Pdf.LoadFromFile('sample.pdf');
      ReportLargeBlocks('pass ' + IntToStr(I));
    end;
  finally
    Pdf.Free;
  end;
  ReportLargeBlocks('after Free');
end.

Read the output as a shape. Numbers that settle after the first pass are retention or ordinary per-document state. Numbers that rise by the same step every pass are a leak, and for a PDF the step is usually the size of the larger stream payloads, page content and embedded images, since those are the allocations big enough to land in the large-block figures. Use a file with a dozen or more pages so the step is easy to see against the noise. Growth in proportion to page count is something allocator retention cannot explain. To find which objects hold memory while a document is open, the object dependency graph with retained bytes answers that side of the question

Reusing the pattern in your own Delphi object model

Nothing here is specific to PDF. Any Delphi object model where destructors own children inconsistently, where one child is reachable from several parents, or where back-pointers sit next to forward pointers will crash or leak under a per-object Free. The fix has the same shape every time: classify each pointer field as owning or reference, collect the closure of owning edges through a pointer set that tolerates revisits, cut every edge, destroy the flat list. The cutting step is the one people skip. Skip it and every destructor ends up needing to know about every other one

The pattern comes with boundaries worth writing down before you copy it. The pointer set uses the object address as identity, which is sound only because no destructor runs during collection, so no address can be freed and handed to a fresh allocation mid-walk. The walk sees only the edge kinds it was written for; a new class that owns a child through some other field has to be added to both the collect and the detach pass. And because links are resolved instead of followed, an object referenced only by a link must be in the registry to be reached. The HotPDF parser registers every object it reads, and a hand-built graph in your own code has to keep the same rule

Test it by counting destructor calls instead of bytes. Build the nasty graph by hand: a dictionary shared by two streams, an array holding both that dictionary and its own parent, one payload assigned to both streams, a root registered twice, and a wrapper whose numeric body is registered as well. Tear it down and expect exactly one destruction per unique object — one payload, two streams, two dictionaries, one array, one wrapper, one number. In your own classes a class-level counter bumped in each destructor is enough for this (an instrumentation hook you add yourself), and a teardown that only clears the list reports zero

In HotPDF all of this sits behind LoadFromFile, LoadFromStream and the destructor, so application code only sees the result: closing or reloading a document hands its memory back. HotPDF is a native VCL PDF library for Delphi and C++Builder that ships with full source, so the teardown described here can be read line by line in HPDFDoc.pas; the API reference and a trial build are on the HotPDF PDF component for Delphi page