Two threads share one TFileStream over an .xlsx package. Thread A sets Position to the first compressed byte of sheet1.xml, thread B sets it to the first byte of sheet2.xml a moment later, and A's Read returns a buffer that belongs to the other member. A's inflater is now decoding the middle of somebody else's deflate stream; zlib usually stops with Z_DATA_ERROR a few blocks in, and if it happens to produce output, the member's CRC32 will not match. Deflate is not the problem here. The race lives entirely in the stream's position cursor, and one lock around the seek-and-read pair is enough to let many workers inflate members of the same package at once
The reason anyone wants that is memory. Suppose a workbook is 80 MB on disk and its worksheet XML expands five to ten times when inflated. An open path that extracts each worksheet into a buffer before parsing it pays for those inflated bytes on top of the workbook it is building, and the peak arrives before a single cell exists. This article covers the package-level concurrency that removes the extraction step. The allocator ceiling above it is covered in the article on parallel XLSX parsing and the memory manager, and the read-once API that never builds a workbook at all is in the streaming direct reader walkthrough
The staging design, and what it holds in memory
The cautious way to parallelize an XLSX open is a three-phase pipeline where only the middle phase runs on workers. Phase A walks the sheet list on the calling thread, creates each worksheet, reads its relationship part, and copies the whole inflated worksheet XML into a private TMemoryStream. Phase B fans the XML parsing out over a thread pool. Phase C returns to the archive, still on the calling thread, for the small satellite parts: comments, threaded comments, drawings, charts and tables
The argument for that shape sounds airtight. A ZIP archive and its inflate state are not thread-safe, so keep every archive touch on one thread and skip locking altogether. The bill is a workbook with eight busy sheets holding eight fully inflated worksheet buffers at the same moment, and those buffers are the largest transient objects anywhere in the open path. With the 80 MB package assumed above, that is several hundred megabytes of XML text that exists only so a parser can read it once
HotXLS keeps Phases A and C but drops the copy. Each worker in Phase B opens its own entry stream on the shared package and hands it straight to the XML reader, so the inflated worksheet never exists as one block. That only works because the "not thread-safe" argument turns out to be about two different things
Can two threads inflate from one ZIP stream?
Yes, provided each thread owns its inflater and the shared byte source is serialized at the read. The cautious argument folds two kinds of state into one sentence. Inflate state really cannot be shared: a zlib z_stream carries the 32 KB sliding window that RFC 1951 defines, the current Huffman tables and the bit position inside one compressed member, and two threads pushing bytes through the same one produce garbage. The byte source is a different matter. A file stream has exactly one piece of mutable state worth protecting, its position
The ZIP container makes the split legal. PKWARE's APPNOTE (section 4.3) lays a package out as independent members, each with its own local file header, its own compressed data at its own offset, and its own CRC32 and sizes recorded in the central directory. No dictionary spans members the way a solid 7z block does, so entry N can be inflated without reading a byte of entry M. Give every worker a private inflater over its own byte range and the only collision left is the seek. In HotXLS that collision is handled by TZipReadGate in lxZipArchive.pas, and the whole class fits on one screen
type
TZipReadGate = class
private
FBaseStream: TStream;
FLock: TXLSCriticalSection; // lxPlatform wrapper over a critical section
public
constructor Create(ABaseStream: TStream);
destructor Destroy; override;
function ReadAt(AOffset: Int64; var Buffer; Count: Longint): Longint;
end;
function TZipReadGate.ReadAt(AOffset: Int64; var Buffer;
Count: Longint): Longint;
begin
if Count <= 0 then
begin
Result := 0;
Exit;
end;
XLSEnterCriticalSection(FLock);
try
FBaseStream.Position := AOffset;
Result := FBaseStream.Read(Buffer, Count);
finally
XLSLeaveCriticalSection(FLock);
end;
end;What the read gate locks, and what stays outside it
TZipReadGate.ReadAt guards one indivisible operation, positioning the shared stream and reading from it. TZipArchive.OpenArchive builds the gate once the central directory has parsed, for both the file-name and the TStream overloads, and freeing the archive releases it. Archives created for writing never get one, because nothing reads from them concurrently
Everything else stays outside the lock because it is already private or already frozen. Each entry stream that TZipArchive.OpenFile returns keeps its own position inside its own member and converts it to an absolute package offset only at the moment it calls ReadAt. The inflater wrapped around it is created per entry with a window size of -15, which zlib reads as raw deflate with no zlib header, and is never handed to another thread. The central directory, local headers included, is fully parsed before any worker starts, so looking up an entry by name is a read-only hash lookup by the time concurrency begins. Lookups are case-insensitive and reject names containing .. segments
One consequence matters when you call TXLSXWorkbook.Open(Stream) with your own stream. The gate is wrapped around that stream, and its lock coordinates the library's workers with each other. It knows nothing about your code. If another thread in your application reads or repositions the same TStream while Open is running, the cursor race from the first paragraph is back, so hand the stream over and leave it alone until Open returns
How much does one lock cost under contention?
Less than "a global lock on the archive" suggests, because of how much work each acquisition covers. The inflater refills its input in 64 KB chunks of compressed bytes, so one trip through the gate feeds 64 KB of deflate data to a worker that then decodes and parses it without holding anything. At the five-to-ten-times expansion assumed earlier, that refill becomes roughly 320 to 640 KB of worksheet markup. The lock itself covers a positioned read that is usually served from the operating system's file cache
The ratio inverts in three places, all of them about the input or the machine. Members stored without compression go through the gate once per read call with no inflate work between acquisitions; most writers deflate worksheet parts, but a package produced by a tool that stores everything will serialize harder. A cold file on slow or network storage widens the critical section, since the read inside it becomes a real transfer. And past a handful of workers the gate is not the first wall anyway: worksheet parsing allocates heavily, and the Delphi memory manager serializes allocations across threads well before the read gate saturates
That last point decides the thread count. TXLSXWorkbook.ParallelParseThreads defaults to 0, which starts one worker per logical processor, never more than the number of sheets. On a laptop with four to eight logical processors that default is fine. On a 32-thread build server opening workbooks with dozens of sheets, set an explicit small cap such as 4: the extra workers mostly queue on the allocator and each one holds its own inflater and buffers. To find the right number for your files, open one representative workbook in a loop with the property set to 1, 2, 4 and 8, and compare wall time against peak working set in Task Manager or Process Explorer
Reading an entry to the end is what fires the CRC check
Every entry stream the archive returns checks itself, but only at the end. It accumulates a running CRC32 as bytes pass through, and when its position reaches the uncompressed size recorded in the central directory it compares both values and probes for one extra byte. The failures surface as exceptions with literal messages: 'ZIP entry uncompressed size mismatch', 'ZIP entry CRC32 mismatch', 'ZIP entry exceeds its declared size', and 'ZIP entry ended before its declared size' for a member that runs out early
An XML reader stops at the closing </worksheet> tag. Anything after it, often just a trailing newline, stays unread, the position never reaches the declared size, and the comparison never runs. The staging design got the check for free, since copying an entry into a buffer reads every byte. A parser reading the entry stream directly does not, so each HotXLS worker drains whatever the parser left with a 32 KB scratch buffer before it frees the stream, and freeing the outermost stream releases the inflater and the sub-stream beneath it
The same rule applies when your own code uses TZipArchive from lxZipArchive, for example to sanity-check an incoming package before handing it to the workbook loader. A function that reads only the first few kilobytes of each part to sniff its root element proves nothing about integrity. Read to the end:
uses
Classes, SysUtils, lxZipArchive;
// Reads one package part to its declared end so the size and CRC32 checks run.
// PartIsIntact is a helper you write yourself; it is not part of the library
function PartIsIntact(Zip: TZipArchive; const PartName: WideString;
out Problem: string): Boolean;
var
Stream: TStream;
Buffer: array [0..32767] of Byte;
begin
Result := False;
Problem := '';
Stream := Zip.OpenFile(PartName); // nil when the part does not exist
if Stream = nil then
begin
Problem := 'part not found';
Exit;
end;
try
try
while Stream.Read(Buffer, SizeOf(Buffer)) > 0 do
;
Result := True;
except
on E: Exception do
Problem := E.Message; // e.g. 'ZIP entry CRC32 mismatch'
end;
finally
Stream.Free;
end;
end;
var
Zip: TZipArchive;
Problem: string;
begin
Zip := TZipArchive.Create;
try
Zip.OpenArchive('quarterly-consolidation.xlsx');
if not PartIsIntact(Zip, 'xl/worksheets/sheet1.xml', Problem) then
Writeln('sheet1.xml: ', Problem);
finally
Zip.Free;
end;
end;The second trap in that code is rewinding. Entry streams are forward-only: reading Size works, because the declared size is known up front, but setting Position := 0 after a peek raises 'ZIP entry stream is forward-only'. If a component insists on seeking, open the entry a second time with OpenFile, which is cheap and gives you a fresh stream at offset zero. Several threads may each call OpenFile on one archive opened for reading, since that is exactly what the workbook loader does, as long as nothing frees the archive while their streams are alive
What still runs serially, and the switch that turns it off
Phase A still runs, minus the extraction. It creates every worksheet and reads its relationships on the calling thread, which is what leaves every shared map immutable once workers start. The shared string table and the cell-format tables are complete before Phase B begins and are never written during it. Phase C still walks the sheets in order afterward for comments, drawings, charts and tables, skipping any sheet whose part is missing from the package
uses
SysUtils, lxHandleX;
var
Wb: TXLSXWorkbook;
begin
Wb := TXLSXWorkbook.Create;
try
Wb.ParallelParse := True; // default; False parses one sheet at a time
Wb.ParallelParseThreads := 4; // 0 = one worker per logical processor
Wb.SelectedSheets.Add('Q3 Detail'); // optional: parse cells for this sheet only
if Wb.Open('quarterly-consolidation.xlsx') <> 1 then
raise Exception.Create('Workbook open failed');
// The loaded workbook is identical whichever path parsed it
finally
Wb.Free;
end;
end;Setting ParallelParse to False before Open runs the same per-sheet job with a thread count of one, and the pool degenerates to a plain loop on the calling thread. A workbook with a single worksheet takes that path regardless. Keep the switch in mind when diagnosing an open problem: it takes threading out of the picture in one line, and both paths run the same parsing code, so the workbook you get back does not depend on the setting
Errors keep the serial shape too. When a worker raises, no further sheets are dispatched, every worker joins, and the exception from the lowest sheet index is re-raised on the calling thread, so a corrupt sheet3.xml still surfaces as one exception from Open. With RecoveryMode set to True, a worksheet that fails to parse is skipped instead and recorded as a warning in the workbook diagnostics. Sheets left out of SelectedSheets never open their entry stream at all, and MetadataOnly skips the parallel phase entirely. Broader tuning of the open path is covered in the guide to large workbook performance in Delphi
The read gate, the parallel open phase and the entry-level ZIP access shown here are part of the HotXLS Delphi Excel component for Delphi and C++Builder, which ships with full source, so you can read lxZipArchive.pas alongside the TXLSXWorkbook open properties described on the product page