CollateDocumentsEx in the PDFlibPas Delphi PDF library merges several open documents into one interleaved document. It appends GroupSize pages from each source per round, accepts a per-source page range list, and treats a descending range such as 3-1 as a reversal of that source. One call turns a front stack and a reversed back stack into reading order
The scenario behind that API is mundane and extremely common. A sheet-fed scanner with a single-sided path runs the whole stack face down, then the operator flips the stack and runs it again. You end up with two PDFs: fronts in order, backs in reverse order. The output the user wants is one file, page 1 front, page 1 back, page 2 front, and so on. This article is about the ordering problem and the resource duplication trap that sits underneath it. If your concern is raw concatenation throughput instead, see fast PDF merge by byte-level ref shifting; if the inputs are too large to hold in memory at all, see merging and splitting gigabyte PDFs with direct access
The scanner produces two stacks, one of them backwards
Collation is not merging. A merge concatenates page ranges; a collation interleaves them, and the interleaving pattern is a property of the physical device that produced the input. Get the pattern wrong and the file is not slightly wrong, it is unreadable: every second page belongs to a different sheet. Three variables describe almost every real case: how many sources are in the rotation, how many pages come from each source per round, and whether any source needs to be read backwards. CollateDocuments covers the first two with a plain array of document handles and a GroupSize integer. CollateDocumentsEx adds the third by accepting a semicolon-separated list of page ranges, one segment per source, where an empty segment means all pages of that source and a descending range reverses it. Both functions append onto the end of the currently selected document and return 1 on success, 0 on any rejection
Why does the naive collate multiply the file size?
Because the import map that maps source object numbers to target object numbers is rebuilt on every copy call, and anything reachable from more than one chunk gets imported once per chunk. Inside PDFlibPas, TPDFDocument.CopyPagesFromDoc resets its NewIndObjList at the top of every invocation. That list is the only memory the copier has of what it already brought across. Call it once with a ten-page range and a font shared by all ten pages is embedded once. Call it ten times with one page each and that same font is embedded ten times. This matters far more for scans than for text documents, because a scanned page is a single large image XObject and the shared objects are the ones with real weight: an embedded ICC profile, a shared /DecodeParms chain, a stamp or watermark form XObject applied to every sheet, the OCR text layer font. The obvious way to write a round-robin collate is a loop over rounds, and that loop is exactly the pathological case
// Do not do this. Each CopyPageRanges call rebuilds the import map,
// so anything the two sources share internally is imported once per
// round instead of once per source.
var
RoundIndex: Integer;
begin
for RoundIndex := 1 to 12 do
begin
PDF.CopyPageRanges(Fronts, IntToStr(RoundIndex));
PDF.CopyPageRanges(Backs, IntToStr(13 - RoundIndex));
end;
end;Twelve rounds, two sources, twenty-four import maps. Nothing warns you. The page order is correct, every page renders, and the only symptom is a file several times larger than the sum of its inputs. On a 300-page batch job the multiplier is not a rounding error, it is the difference between an archive that fits the retention budget and one that does not
Import once, then reorder the page tree
The fix is to separate the two concerns that the naive loop had fused together. Copying decides which objects exist in the target; ordering decides where the pages sit in the page tree. CollateDocumentsEx copies each source exactly once, in a single CopyPagesFromDoc call with that source complete range, so each source gets one import map and shared resources are written once. Only after every source has landed does the interleaving happen, and it happens entirely through TPDFPageTree.MovePage
Page moves are free in the sense that matters here. ISO 32000-1 §7.7.3 defines the page tree as a balanced structure of node dictionaries whose /Kids arrays hold indirect references, with /Count carrying the leaf total at each node. Relocating a page means removing one indirect reference from one /Kids array, inserting it into another, adjusting both /Count values, and repointing the page /Parent. No content stream is touched, no resource is duplicated, no object is created. The page object keeps its object number, which is also why the object numbers stay stable the way they do in page replacement that preserves object numbers. There is one further detail that a naive page move gets wrong and MovePage does not. ISO 32000-1 §7.7.3.4 lets /Resources, /MediaBox, /CropBox and /Rotate be inherited from an ancestor node rather than stated on the page. A page that inherits its resources from node A and is then moved under node B silently inherits something different, or nothing at all. MovePage therefore resolves the inherited value and writes it onto the page dictionary before the relocation, so the page carries its own attributes across the move
What does the reordering pass actually do?
It runs a selection sort against insert-at semantics. The desired block-relative order is computed first: walk the sources in rotation, take up to GroupSize indices from each, skip a source that is exhausted, repeat until every page is placed. That produces a permutation over the appended block. Applying it is the awkward part, because MovePage is an insert, not a swap, so every move shifts everything between the old and the new position by one
The implementation keeps a Current array modelling where each appended page currently sits, scans forward from position K for the page that belongs at K, issues the move, then slides the array entries to mirror what the move did to the tree. It is O(n squared) in array operations and zero in object copies, which is the correct trade for this workload: a 500-page collate is a quarter of a million integer shuffles and not one byte of duplicated image data. Descending ranges and repeated pages need no special handling in this pass because PLParsePageRangeList is called with sorting disabled and duplicates allowed, so the requested order survives parsing intact
Reversed ranges and the one-call duplex merge
With reversal expressed as a range, the flatbed double-pass case collapses into a single call. The fronts want their natural order and the backs want 12-1, and the empty first segment before the semicolon says the first source contributes all of its pages
var
PDF: TPDFlib;
Target, Fronts, Backs: Integer;
begin
PDF := TPDFlib.Create;
try
Target := PDF.NewDocument;
if PDF.LoadFromFile('fronts.pdf', '') <> 1 then
Exit;
Fronts := PDF.SelectedDocument;
if PDF.LoadFromFile('backs.pdf', '') <> 1 then
Exit;
Backs := PDF.SelectedDocument;
PDF.SelectDocument(Target);
// fronts 1..12 in order, backs scanned in reverse: F1 B12 F2 B11 ...
if PDF.CollateDocumentsEx([Fronts, Backs], ';12-1', 1) = 1 then
PDF.SaveToFile('duplex.pdf');
finally
PDF.Free;
end;
end;Two behaviours in that snippet are worth stating explicitly. The collated pages are appended to the selected document, so a document created with NewDocument contributes its initial blank page ahead of them and you should delete it if you do not want it. And the sources may be uneven: with GroupSize 2 over a three-page and a five-page source, the rounds come out A1 A2 B1 B2, then A3 B3 B4 once A is nearly spent, then B5 alone, because an exhausted source is simply skipped rather than padded
Rollback, form fields, and what does not come along
Every argument is validated before the target is touched. A missing document handle, the selected document listed as its own source, a GroupSize below one, a segment count that does not match the source count, a range naming a page the source does not have: all of these return 0 with the target unchanged. Failure during copying is the harder case, and it is handled through the public DeletePages rather than the raw PageTree.DeletePages. The reason is specific. The copy runs with MergeFormData enabled, so the source form fields have already been appended to the target /AcroForm /Fields array by the time a later source fails. Deleting the pages at the page-tree level would strip the widget pages and leave those field references dangling; the public path unlinks the field, outline and article-thread references alongside the pages
if PDF.CollateDocumentsEx([Fronts, Backs], ';12-1', 1) = 0 then
// Nothing was appended and the target is byte-identical to before.
// 412 is the copy failure; 0 means the arguments were rejected
// during validation, before any page was touched.
Log(Format('collate rejected, LastErrorCode=%d', [PDF.LastErrorCode]));Be honest with your users about the boundaries. The collate carries pages, their annotations and their form fields, and it merges the AcroForm field list, the calculation order array and the default resources dictionary. It does not carry source bookmarks: the outline tree of a scanned front stack is almost always empty, so nothing is lost in the duplex case, but if you collate two authored documents their outlines stay behind and you rebuild the navigation yourself. Named destinations that lived only in the source catalogue are in the same position. Plan for that before you promise a customer a lossless collate
PDFlibPas ships the collation functions together with the rest of its page assembly surface, so the scanner workflow, the range-based extraction and the large-file paths all sit behind one component in Delphi and C++Builder. The full API reference and a trial build are on the losLab Delphi PDF library product page