Cross-Document PDF Object Copy in Delphi: Cycle Crashes

Merge two PDFs by hand, move a single page object into the target document, and the copy walks straight into an access violation. PDFlibPas fixes this in CopyForeignObject: it deep copies one indirect object plus its whole reference closure, and resolves cyclic back-references such as /Parent to null instead of recursing

Why does copying one page across documents crash?

Because a PDF page tree is only a tree if you read it downwards. Walk it the way a recursive copier does, following every value in every dictionary, and the page dictionary hands you /Parent, which points back at the /Pages node you arrived from, and that node hands you /Kids, which points back at the page. ISO 32000-1 §7.7.3 makes /Parent required on every page tree node except the root, so this is not a malformed file you can reject — it is the normal shape of every document you will ever be handed

The second half of the problem is numbering. Indirect objects are identified by an object number that is local to one file (ISO 32000-1 §7.3.10), so an object dragged from document A into document B has to be renumbered, and every reference to it inside the copied closure has to be renumbered the same way, or two references that used to point at one shared font now point at two unrelated things. That renumbering is the same job a fast merge does at the byte level, and it is worth reading the two side by side: byte-level reference shifting for fast PDF merge solves it by translating whole files, while an object-level copy has to solve it one edge at a time

What PDFlibPas CopyForeignObject actually copies

TPDFlib.CopyForeignObject(SourceDocumentID, ObjectNumber) clones one indirect object and everything reachable from it — nested dictionaries, arrays, strings, names, numbers, and streams with their dictionaries intact — into the currently selected document, and returns a non-zero handle to the new indirect reference. Source object numbers are remapped through a live map held for the duration of the call, so an object reached twice in the closure is cloned once and shared twice. It returns zero, without raising, when the source document ID is unknown, when the source is the selected document itself, or when ObjectNumber is below 1

var
  Lib: TPDFlib;
  SourceDoc, TargetDoc, Handle: Integer;
begin
  Lib := TPDFlib.Create;
  try
    TargetDoc := Lib.NewDocument;
    if Lib.LoadFromFile('source.pdf', '') <> 1 then
      Exit;                              // LoadFromFile returns 1 on success
    SourceDoc := Lib.SelectedDocument;   // the load selected what it loaded
    Lib.SelectDocument(TargetDoc);       // copy targets the selected document
    Handle := Lib.CopyForeignObject(SourceDoc, 12);
    if Handle = 0 then
      raise Exception.Create('cross-document copy rejected');
  finally
    Lib.Free;
  end;
end;

Two details bite people on the first run. LoadFromFile answers 1 or 0, not a document ID, so the handle you need comes from SelectedDocument straight after the load; and the copy always writes into whatever SelectDocument last made current, never into the document you loaded from. Internally the recursion also carries a hard depth cap of 64, which is a backstop against pathological nesting, not the mechanism that handles cycles — the cycle handling is separate and deliberate

Why does reserving a Nil mapping not break the cycle?

Because Nil in the mapping table means two different things at once, and the code cannot tell them apart. The obvious defence against a cycle is to add the map entry before recursing into the object, so that anything looping back finds the entry and stops. But the entry cannot hold the real target yet — the target does not exist until the closure below it has been written — so it holds Nil, and the lookup that is supposed to catch the back-edge reads Nil and concludes the object was never mapped

// Broken: a reserved Nil target is indistinguishable from "not mapped yet"
NewRef := FindMapped(SrcRef.ObjNum);
if not Assigned(NewRef) then
begin
  SetLength(Map, Length(Map) + 1);
  Map[High(Map)].SourceObjNum := SrcRef.ObjNum;
  Map[High(Map)].Target := nil;          // reserved, still Nil
  NewRef := NewObjRef(CloneObject(SrcInd.Obj, Depth + 1));
  Map[High(Map)].Target := NewRef;       // only backfilled on the way out
end;

Follow that through the page loop. The clone of the page reaches /Parent, recurses into the /Pages node, which reaches /Kids, which recurses back into the page — whose reserved entry still reads Nil, so it is cloned a second time, and a third, each level pushing a fresh frame and a fresh half-built object. What you observe is not a clean stack overflow either: the outer frames are sitting on references whose targets were never assigned, so the first write through one of those slots is an access violation somewhere that looks nothing like the page copy that caused it

The fix: an explicit in-progress state

The repair is to stop overloading Nil and ask the question directly. A map entry whose target is still unassigned means this object is currently being cloned, and an InProgress predicate tests exactly that before the ordinary lookup runs. When it is true, the edge is a cycle back into an ancestor of the current clone, and PDFlibPas emits a null object for it rather than following it

// A map entry with a Nil target marks an in-progress clone
function InProgress(Num: Integer): Boolean;
var
  I: Integer;
begin
  Result := False;
  for I := 0 to High(Map) do
    if (Map[I].SourceObjNum = Num) and (not Assigned(Map[I].Target)) then
      Exit(True);
end;

// ... inside CloneObject, for an indirect reference:
if InProgress(SrcRef.ObjNum) then
  Exit(FStructure.NewNull);              // cyclic back-edge, do not recurse
NewRef := FindMapped(SrcRef.ObjNum);
if not Assigned(NewRef) then
begin
  SrcInd := SourceDoc.FindObj(SrcRef.ObjNum, SrcRef.GenNum);
  if (not Assigned(SrcInd)) or (not Assigned(SrcInd.Obj)) then
    Exit(FStructure.NewNull);            // dangling source reference
  SetLength(Map, Length(Map) + 1);
  Map[High(Map)].SourceObjNum := SrcRef.ObjNum;
  Map[High(Map)].Target := nil;          // reserve, then recurse
  NewRef := NewObjRef(CloneObject(SrcInd.Obj, Depth + 1));
  Map[High(Map)].Target := NewRef;       // backfill
end;
Exit(NewRef);

This is safe to generalise only because of a structural fact about PDF: cycles in the object graph appear on back-links, not on the content edges. /Parent in the page tree and /Prev in an outline chain point upwards or backwards at something already visited; the closure of a font, an image XObject, or a form XObject runs downwards and terminates. So a copy of a font descriptor, a colour space, or a shading dictionary is unaffected by the null substitution — nothing in those closures ever hits InProgress. The cost, stated plainly, is that the cyclic edge does not survive the copy. A page dictionary cloned this way arrives with /Parent as a null object, which ISO 32000-1 §7.3.9 makes equivalent to an absent entry, so the copied page is a valid object that belongs to no page tree until you link it into the target /Pages node and fix /Count yourself. A copied outline item loses its /Prev the same way and needs the sibling chain rebuilt. That is the honest trade: CopyForeignObject gives you a correct closure and leaves the structural re-parenting to the caller, which is the same boundary replacing pages while preserving object numbers works within

Why the map entry has to be reserved before NewObjRef

An obvious alternative would sidestep the whole in-progress dance: allocate an empty shell object first, register its real number in the map, then fill the shell in once the children are cloned. That does not work here, because TPDFIndObj.Obj is read-only and its content cannot be replaced after construction — there is no shell to fill. The number and the content are decided together by NewObjRef, which means the map entry must be created before the recursive call and completed after it, and the interval between those two moments is precisely what InProgress has to cover. One consequence worth knowing before you diff output: because NewObjRef runs after the child closure is written, numbering in the target comes out bottom-up, and object numbers will not mirror the source order. Nothing in the file format cares, but a byte comparison against a hand-built expectation will. If a run leaves objects you decided not to link into anything, they are unreferenced rather than corrupt, and mark-and-sweep collection of unreachable PDF objects is the tool that clears them out before saving

The regression that covers this needs one detail that surprises people writing tests against TPDFlib: the constructor already holds a default document, so DocumentCount starts at 1 and a two-document fixture must assert >= 2, not = 2. Alongside the successful copy, the test pins the three rejections — an unknown source ID, the selected document as its own source, and an object number of zero — all returning 0 rather than raising, because a merge loop is a bad place to discover that a guard clause throws

Where this fits in a merge pipeline

Object-level copying is the primitive you reach for when whole-file merging is too coarse: lifting one font programme out of a template, pulling a single form XObject into a stamping document, or moving an annotation with its appearance streams across files without dragging the rest of the page along. PDFlibPas exposes it as a single call against loaded documents, and you can see how it sits with the rest of the low-level object API in the PDFlibPas Delphi PDF Library reference