HotXLS Excel Library for Delphi and C++Builder reads and writes the Compound File Binary container behind every legacy .xls file in pure Object Pascal. The TlxCompoundFile class implements the [MS-CFB] version 3 layout directly against a TStream — header, DIFAT, FAT chains, MiniFAT and the directory tree — with no ole32.dll and no COM IStorage anywhere in the path
That sounds like plumbing, and for twenty years it was plumbing somebody else owned. Every Delphi codebase that touched a .xls file reached for StgOpenStorage, got back an IStorage, and pulled the Workbook stream out of it. Three lines, worked fine, nobody thought about it again — until the day the same code had to run somewhere Windows was not
Why does StgOpenStorage stop working on a server?
The COM structured-storage API fails in exactly the deployment shapes modern Delphi code lives in, for reasons that have nothing to do with the file format. StgOpenStorage is a Win32 entry point in ole32.dll: it wants a path on a filesystem, it wants COM initialized on the calling thread, and it wants to be on Windows. The path requirement hurts first, because a REST endpoint receiving an uploaded workbook has the bytes in a buffer, not on disk — so you write the buffer to a temp file, open it, read it back, delete it, and now own a temp-file lifecycle to get wrong under load. ILockBytes is the documented escape hatch, but wiring a custom implementation over a TMemoryStream is more COM interop than most teams want. The initialization requirement bites second, usually in a service worker thread nobody called CoInitialize on, and the platform requirement ends the conversation the moment the target is Linux under FPC, a container image, or macOS. HotXLS therefore keeps the classic lxOLE path built on StgOpenStorage as the default, since it is battle-tested and existing callers should not have to change; TlxCompoundFile is the opt-in alternative for everyone else
What the header and the FAT chains actually say
The first 512 bytes of a compound file answer every structural question you need before reading a byte of payload. [MS-CFB] §2.2 fixes the header signature at offset 0 as the eight bytes D0 CF 11 E0 A1 B1 1A E1, and lxIsCompoundStream checks exactly that, restoring the stream position afterwards so a caller can sniff without disturbing anything. Four more fields decide the geometry: byte order at 0x1C must be 0xFFFE, which doubles as a cheap second signature check; sector shift at 0x1E gives the sector size as 1 shl SectorShift, so version 3 uses shift 9 for 512-byte sectors and version 4 uses shift 12 for 4096; mini sector shift at 0x20 is 6, making mini sectors 64 bytes; and the mini stream cutoff at 0x38 is 4096. The address arithmetic that follows is the most common place to go wrong. Sector 0 begins immediately after the header, so sector N starts at byte offset 512 + N * SectorSize — note the literal 512, not SectorSize. On a version 3 file the two are identical and the bug hides forever; on a version 4 file it silently reads the wrong sector, which is why HotXLS keeps this in one function, SidToOffset
A compound file is a FAT filesystem inside a file, so reading it means walking linked lists of sector IDs where FAT[n] holds the ID following sector n. Three sentinels terminate or annotate a chain — ENDOFCHAIN, FATSECT for a sector belonging to the FAT itself, and DIFSECT for a DIFAT sector — and all three read as negative signed 32-bit integers, which keeps the loop conditions simple. Finding the FAT needs one more indirection: the DIFAT is the array of sector IDs saying where the FAT sectors live, and its first 109 entries sit in the header at offset 0x4C. TlxCompoundFile walks those 109, stops at the first negative entry, and concatenates each FAT sector into one flat Integer array. That is 109 FAT sectors at 128 entries each on a 512-byte sector, so 13,952 addressable sectors, so roughly 6.8 MiB of container before the DIFAT must spill into a chain of its own
The second allocation table exists because 512-byte sectors waste most of their space on small streams. Any stream below the 4096-byte cutoff is not stored in sectors at all: it lives inside the mini stream, itself an ordinary stream hanging off the root directory entry, subdivided into 64-byte mini sectors and chained through a parallel MiniFAT rooted at header offset 0x3C. Open a real .xls and the Workbook stream sits on the normal FAT while the summary-information streams sit down in mini-sector space, which is why an implementation covering only the FAT path appears to work right up until it needs document metadata. The directory is the third structure and the one that makes the container navigable: each entry is exactly 128 bytes, four per 512-byte sector, carrying a UTF-16 name in the first 64 bytes, its byte length at 0x40, the object type at 0x42 (1 = storage, 2 = stream, 5 = root), tree links at 0x44, 0x48 and 0x4C, the start sector at 0x74 and the 32-bit stream size at 0x78. That name length counts bytes including the terminating null, so the character count is NameLen div 2 - 1, and getting it off by one is how you end up with a stream named Workboo
Pulling a Workbook stream out of a memory buffer
TlxCompoundFile.OpenStream hides all of the above behind one call that takes a stream name and returns a TlxCfbStream holding the fully materialized bytes. The whole sequence — sniff, load, extract — runs against a TBytesStream with nothing ever touching disk
uses
Classes, SysUtils, lxCompoundFile;
function ExtractBiffPayload(const Blob: TBytes): TBytes;
var
Src: TBytesStream;
Cfb: TlxCompoundFile;
Wb: TlxCfbStream;
begin
SetLength(Result, 0);
Src:= TBytesStream.Create(Blob);
try
if not lxIsCompoundStream(Src) then
Exit; // not a CFB container at all
Cfb:= TlxCompoundFile.Create;
try
Cfb.LoadFromStream(Src); // header, FAT, directory, MiniFAT
Wb:= Cfb.OpenStream('Workbook'); // BIFF8
if Wb = nil then
Wb:= Cfb.OpenStream('Book'); // BIFF5 / BIFF7
if Wb <> nil then
try
Result:= Wb.Data;
finally
Wb.Free;
end;
finally
Cfb.Free;
end;
finally
Src.Free;
end;
end;Two details there are worth calling out. LoadFromStream takes an AOwnsStream flag defaulting to False, so the caller keeps responsibility for the source stream — deliberate, because the common case is a stream the application already owns. And OpenStream returns a TlxCfbStream owning its own copy of the bytes, exposed through Data, Size, Read, Seek and CopyTo. That copy is a real cost on a large workbook, and it is the honest price of a design where the returned object stays valid after the container is freed. When a workbook is large enough that a full in-memory copy is the wrong shape entirely, the streaming direct reader for oversized spreadsheets is the better entry point
Why does an encrypted XLSX look like an XLS file?
Because it is one, at the container level — and this is the practical payoff of owning that layer. Open an encrypted .xlsx in a hex editor and the first eight bytes are D0 CF 11 E0 A1 B1 1A E1, byte for byte identical to a 1997-vintage .xls, because [MS-OFFCRYPTO] encryption does not encrypt the ZIP package in place: it wraps the whole package inside a CFB container as a stream named EncryptedPackage, beside an EncryptionInfo stream describing the algorithm. The signature therefore identifies the container and says nothing about the payload. Telling a BIFF workbook apart from an encrypted OOXML package means reading the directory, which after LoadFromStream is a scan over EntryCount and Entries, or a pair of HasStream probes
type
TCfbPayload = (cpUnknown, cpBiffWorkbook, cpEncryptedOoxml);
function ClassifyContainer(AStream: TStream): TCfbPayload;
var
Cfb: TlxCompoundFile;
E: TlxCfbEntry;
I: Integer;
begin
Result:= cpUnknown;
Cfb:= TlxCompoundFile.Create;
try
Cfb.LoadFromStream(AStream);
for I:= 0 to Cfb.EntryCount - 1 do
begin
E:= Cfb.Entries(I);
if E.EntryType <> cfbStream then
Continue;
if E.Name = 'EncryptedPackage' then
Result:= cpEncryptedOoxml
else if (E.Name = 'Workbook') or (E.Name = 'Book') then
Result:= cpBiffWorkbook;
end;
finally
Cfb.Free;
end;
end;Directory names deserve a warning of their own: the summary-information streams carry a leading 0x05 control character in their names, so a comparison written against a plain display string will never match them and a naive log line renders them as garbage. Everything downstream of this classification — deriving the key, checking the password verifier — is a separate problem, covered in the notes on why Excel rejects a workbook encrypted with the wrong cipher mode. The container layer only tells you which door you are standing in front of
Writing a container that Excel will actually open
The writing side of TlxCompoundFile is deliberately narrower than the reading side, and understanding why saves an argument with the spec. [MS-CFB] permits an enormous space of valid containers: multi-level storages, properly balanced red-black directory trees, mini streams, DIFAT chains. Excel emits a small corner of that space and reads a somewhat larger one. HotXLS writes a corner smaller still — the minimum Excel demonstrably loads. Every stream goes on the normal FAT with no mini-stream path, which costs disk space and buys correctness: a 300-byte summary stream that Excel would have packed into five 64-byte mini sectors instead occupies a full 512-byte sector, and for a workbook that is noise next to maintaining a second allocation table, a second chain walk and the root-entry stream backing it on the write path. Directory entries form a flat sibling chain under the root with every node colored black, and the emission order is fixed: header placeholder, stream data sectors, directory sectors, FAT sectors, then a seek back to rewrite the header with the sector IDs that are only known at the end. The FAT sizes itself through a short fixed-point loop, because adding FAT sectors can push the sector count high enough to require another FAT sector
procedure SaveAsCompoundFile(const Dest: string; const BiffBytes: TBytes);
var
FS: TFileStream;
Cfb: TlxCompoundFile;
begin
FS:= TFileStream.Create(Dest, fmCreate);
try
Cfb:= TlxCompoundFile.Create;
try
Cfb.CreateNew(FS); // v3 header, 512-byte sectors
Cfb.AddStream('Workbook', BiffBytes);
Cfb.Save; // data -> dir -> FAT -> header
finally
Cfb.Free;
end;
finally
FS.Free;
end;
end;Where the implementation stops
Three boundaries are worth stating plainly, because a container reader that quietly mishandles an edge case is worse than one that raises. TlxCompoundFile reads the 109 DIFAT entries resident in the header and does not follow the DIFAT chain at 0x44 beyond them, capping a readable container at roughly 6.8 MiB on 512-byte sectors — comfortably above the real .xls files HotXLS meets in the field, but a hard ceiling nonetheless, and the writer enforces the same limit explicitly rather than emitting a container it cannot describe. Second, version 4 containers with 4096-byte sectors are accommodated by the sector-size arithmetic but are not what the code is tuned for, and the 64-bit stream size is not consulted: HotXLS reads the low 32 bits at offset 0x78 and leaves the high half alone, which is correct for version 3 and only for version 3. Third, entry lookup is a flat scan by name across the directory list rather than a walk down the red-black tree from a parent storage, so nested storages resolve by name collision rather than by path — every stream a .xls file needs sits at the top level, which is what makes the simpler design defensible, but code expecting to address SomeStorage/SomeStream will not find it
None of that changes what the unit is for. Owning the container layer turns .xls handling into ordinary Object Pascal: parseable from a byte array, testable without a filesystem, portable to whatever platform the compiler targets, and free of a COM apartment. It also retires the sniffing shortcuts, because identifying a workbook now means reading its directory rather than its first eight bytes — the same discipline behind listing sheet names without opening the whole workbook
TlxCompoundFile ships as part of the HotXLS Excel Component for Delphi and C++Builder, alongside the BIFF and OOXML layers that sit on top of it; the product page carries the full unit reference and the supported compiler matrix