An /Indexed lookup table stores components of its base color space, not RGB triples. A DeviceGray base packs one byte per entry, DeviceCMYK four, Lab three that need range scaling, and ICCBased whatever the profile /N declares. HotPDF normalizes all of them at parse time into the public RGB palette that THPDFColorSpace exposes
That sentence is the whole bug class. Almost every palette renderer starts life assuming a three-byte stride, because the overwhelming majority of real-world /Indexed spaces sit on /DeviceRGB and the assumption holds for years. Then a scanned invoice arrives with [/Indexed /DeviceCMYK 15 <...>], the reader walks the lookup string three bytes at a time, and every index past zero lands in the middle of somebody else color. The output is not blank and it does not crash. It is plausible-looking garbage, which is far worse, because nobody files a bug against colors that merely look off
Why does an Indexed palette come out wrong?
The stride is wrong because the stride is not a constant. ISO 32000-1 §8.6.6.3 defines the lookup table as m × (hival + 1) bytes, where m is the number of components in the base space, and it says nothing about that base being RGB. The same section forbids only two bases outright: /Indexed itself and /Pattern. Everything else is fair game, including all four CIE-based families from §8.6.5.2 through §8.6.5.5
So the real work is not reading bytes, it is resolving the base first and asking it how wide an entry is. ResolveIndexedPalette in HotPDF does exactly that: it resolves the base into a THPDFColorSpace, takes ComponentCount from the resolved record, and rejects the array outright if the base came back as csfIndexed, csfPattern, or csfUnsupported. Only then does it size the byte budget as PaletteCount * Count, which for the legal maximum of hival 255 and a four-component base tops out at 1024 bytes. Two details in that sizing are where a hostile file will probe you. HotPDF clamps hival into 0..255 before using it, per the §8.6.6.3 limit, so a lookup array claiming hival 100000 does not turn into a gigabyte allocation; and it truncates the budget to whatever the lookup data actually contains, then stops filling the palette at the first entry whose bytes run past the end. A short table yields a short palette, not a read past the buffer
Normalizing four different base spaces into one RGB table
HotPDF converts palettes to RGB during parsing rather than at draw time, and that decision drives the rest of the design. The public THPDFColorSpace.Palette field is a TBytes of RGB triples with length 3 * PaletteCount, and it has been that shape since long before CIE-based bases were supported. Rewriting the record layout to carry native base components would have broken every consumer of the field. Converting at parse time keeps the layout intact and, as a bonus, routes Gray, CMYK, Lab, CalGray, CalRGB and ICCBased entries through the same HPDFResolveColor path the renderer already uses for direct color operators
The scaling step is where the base spaces stop being interchangeable. A lookup byte is always 0..255, but what that byte means depends on the component range the base declares. Device and calibrated components map onto 0..1. Lab does not: the L* component runs 0..100, while a* and b* run over the /Range the space declares. HotPDF pulls those bounds from the AMin, AMax, BMin and BMax fields of THPDFCIEParams, which the Lab parser fills from the dictionary /Range entry and defaults to −108..101 when the entry is absent, so do not hard-code a symmetric range from memory. ICCBased bases add a third source of truth: the component count comes from the profile stream /N, surfaced as ICCN, and an ICCBased stream may itself carry a /Range array that HotPDF reads through HPDFReadNumericArrayUpTo into the same per-component bounds table. A three-component ICC palette and a three-component Lab palette therefore walk the lookup string with identical strides and completely different arithmetic, which is exactly the outcome a fixed RGB stride cannot produce
var
CS: THPDFColorSpace;
Entry: Integer;
R, G, B: Byte;
begin
// ColorSpaceObj is the /ColorSpace entry from an image XObject or a
// resource dictionary; it may be a name, an array, or an indirect ref
CS := HPDFResolveColorSpace(ColorSpaceObj);
if CS.Family <> csfIndexed then
Exit;
// Palette is already RGB regardless of what the base space was
for Entry := 0 to CS.PaletteCount - 1 do
begin
if (Entry * 3 + 2) >= Length(CS.Palette) then
Break; // short lookup table, stop cleanly
R := CS.Palette[Entry * 3];
G := CS.Palette[Entry * 3 + 1];
B := CS.Palette[Entry * 3 + 2];
UseColor(Entry, R, G, B);
end;
end;When the lookup table lives in a stream
ISO 32000-1 §8.6.6.3 allows the lookup table to be a string or a stream, and the stream form is the one that bites. A stream can carry /FlateDecode, it can be an indirect object shared with unrelated parts of the file, and its TStream may be positioned anywhere at the moment your parser reaches it. HotPDF handles the compressed case by delegating to the decoded-bytes callback registered through HPDFSetColorStreamReader, the same callback the Type 0 sampled-function engine uses
When no reader is registered, the fallback is deliberately narrow rather than best-effort. HotPDF checks the stream dictionary for /Filter and refuses the palette entirely if one is present, instead of feeding compressed bytes into the lookup and producing a confidently wrong table. Without a filter it reads raw bytes, capped at the byte budget computed from hival and the base component count. The whole stream branch sits inside a try..finally that restores the original Stream.Position on every exit path, including the refusals, because the object is shared and the next consumer will not thank you for a moved cursor. LookupStreamUsesDecodedReaderAndRestoresPosition in the regression suite pins both halves of that behavior
// Register both callbacks before resolving any colour space, otherwise
// indirect bases and compressed lookup streams will simply be refused
HPDFSetColorSpaceResolver(MyResolveIndirectObject);
HPDFSetColorStreamReader(MyDecodedStreamReader);
try
CS := HPDFResolveColorSpace(ColorSpaceObj);
if (CS.Family = csfIndexed) and (Length(CS.Palette) = 0) then
LogWarning('lookup table unreadable: filtered stream or no reader');
finally
HPDFSetColorStreamReader(nil);
HPDFSetColorSpaceResolver(nil);
end;What stops an Indexed color space from recursing forever?
Rejecting /Indexed as a direct base is necessary and not remotely sufficient. The interesting attack goes sideways. Take a valid [/Indexed /Separation ...], and recall that §8.6.6.4 lets a Separation name an alternate space and §8.6.6.5 lets DeviceN do the same. Make that alternate an indirect reference pointing back at the original Indexed array. Nothing in the chain violates the direct-base rule at any single hop: the base is a Separation, the Separation alternate is an array, and the array only happens to be the one you started from. Resolve it naively and the parser walks the loop until the stack gives out. The tint-transform machinery those spaces rely on is a topic of its own, covered in the article on rendering Separation and DeviceN spot colors; here they matter only as a link in the cycle
HotPDF closes this at the single entry point every path already funnels through. HPDFResolveColorSpace wraps the internal resolver in a depth counter bounded by HPDFColorSpaceMaxDepth, which is 64. Cross that bound and the guard sets a flag, and the flag is what matters: on unwind, any nesting level that saw it set discards its own partially resolved record and returns csfUnsupported, so the whole graph is rejected rather than half of it retained, and both counters reset when the outermost call returns. Those counters are declared as threadvar, not unit globals, and that is not incidental tidiness: GResolveObj and GReadStream live in thread-local storage for the same reason, because pages render in parallel. A shared global would let one thread deep in a nested DeviceN chain trip the limit for another thread parsing a perfectly ordinary [/Indexed /DeviceRGB 255 <...>], producing a nondeterministic, load-dependent failure — the least debuggable kind there is. Sixty-four is comfortable headroom for legitimate content, which rarely nests past four or five levels, so treat csfUnsupported from a structurally valid-looking array as worth logging: in practice it means a cycle or a depth no real producer emits. IndirectAlternateCycleIsUnsupported, parameterized over Separation and DeviceN, and ExcessivePatternDepthIsUnsupported cover both cases
function TryResolveImageSpace(Obj: THPDFObject;
out CS: THPDFColorSpace): Boolean;
begin
CS := HPDFResolveColorSpace(Obj);
// csfUnsupported here covers a genuinely unknown family, a cycle
// through an indirect alternate, and depth beyond 64 levels
Result := CS.Family <> csfUnsupported;
if not Result then
CS := HPDFDefaultColorSpace; // DeviceGray, the PDF initial space
end;One API boundary worth documenting for your callers
HotPDF applies the ICC conversion configuration in force at parse time when it builds an Indexed palette, and later calls to HPDFSetColorSpaceRenderingIntent do not regenerate a palette that is already resolved. That function walks the ICCBased and alternate transforms of the record you hand it and rebuilds those pipelines; the flattened RGB bytes in Palette are a parse-time artifact and stay as they were. This is a consequence of normalizing early, not an oversight, and it is worth writing into your own API docs rather than leaving for a customer to discover. If your application lets a user switch rendering intent mid-session, resolve the color spaces again after the switch; if it sets intent once during pipeline setup, which is the overwhelmingly common case, there is nothing to do. The same ordering applies to the proofing profiles installed through HPDFSetICCColorPipeline: configure them before you start resolving, not after
The mirror-image bug on the writing side
Everything above is about reading. The symmetric defect on the writing side is easier to ship and much easier to miss. When HotPDF builds an /Indexed array from a Windows 8-bit bitmap it calls GetPaletteEntries, and that API fills only as many entries as the palette actually holds and returns that count. The original code tested the result for = 0 and then iterated a fixed 256 entries over a stack array. For a 16-color bitmap that wrote 240 entries of uninitialized stack memory straight into the lookup string of a PDF you then hand to a customer, with hival hard-coded to 255 so every reader dutifully treated the residue as real color data. It is a correctness bug and an information-disclosure bug wearing the same clothes. The fix is unglamorous and complete: zero the array before the call, then honor the returned count for both the loop bound and hival. If you generate /Indexed images anywhere in your own code, go check that hival now
Two adjacent pieces round out the picture: Indexed palettes and Type 0 sampled functions share the decoded-stream reader and the same clamping discipline, discussed in the piece on Type 0 function color lookup tables, and the filter rules governing lookup streams are the same ones that govern image data in extracting loaded images through decode filters. The color-space resolver described here ships as part of the HotPDF Delphi Component for Delphi and C++Builder, with the full THPDFColorSpace reference on the product page