Technical Article

BIFF8 XLUnicodeString Decoding in Delphi: cch and fHigh

HotXLS decodes a BIFF8 XLUnicodeString by reading cch and the fHigh flag first, then choosing the reader that matches the encoding: TXLSBlob.GetWideString with a byte count of cch * 2 when fHigh is 1, and TXLSBlob.GetString when fHigh is 0. Pair them the other way round and the record yields an empty or half-length string, never an exception

That is what makes this class of bug expensive. A chart opens, the series draw correctly, the axes are right, and one trendline caption is simply blank. Nothing in the log, nothing in the exception handler, no corrupt-file dialogue. The file was fine the whole time; the reader asked for the wrong number of bytes and got exactly what it asked for

Why does a BIFF8 string come back empty?

A BIFF8 string comes back empty because a length guard rejected the payload before the read ever happened, or because the reader stopped at the first NUL it found. Both paths are silent by construction. In HotXLS the guard is usually an explicit DataLength check in the record handler, and it has to be computed per encoding: a 16-bit payload needs 8 + cch * 2 bytes for an SXViewLink body, but an 8-bit payload only needs 8 + cch. Apply the wide-character arithmetic to an 8-bit record and every short name fails the gate. The NUL behaviour is the second trap, because TXLSBlob.GetString and TXLSBlob.GetWideString both scan the decoded result for a terminator and truncate there, returning an empty string when the terminator lands in position one. Read a 16-bit body with half the byte count and you keep the first cch div 2 characters; read an 8-bit body through the wide reader and the byte pairs form arbitrary code points. Only an over-long read is loud: TXLSBlob.EnsureReadable raises Blob read exceeds data size when the request runs past the blob. Under-reading has no such alarm

GetWideString counts bytes, not characters

TXLSBlob.GetWideString(Index, Count) takes Count in bytes. Internally it does a SetString over a PWideChar with Count div SizeOf(WideChar), so passing a character count silently halves the string. The BIFF8 record layouts, meanwhile, express string length in characters. Every 16-bit call site therefore has to carry the * 2 conversion itself, and every 8-bit call site has to omit it. This is the same encoding boundary that shows up when you write text out again rather than read it, which is worth reading alongside Unicode-safe spreadsheet export in Delphi if your pipeline moves strings in both directions

// 16-bit XLUnicodeStringNoCch: cch characters, cch * 2 bytes
Name := Data.GetWideString(Start, cch * 2);        // correct
Name := Data.GetWideString(Start, cch);            // half the text, no error

// 8-bit XLUnicodeStringNoCch: cch characters, cch bytes
Name := WideString(Data.GetString(Start, cch));    // correct
Name := Data.GetWideStringWithZero(Start, cch);    // still a wide reader

The convention holds everywhere the byte stream is walked by hand. When HotXLS stitches a long String record ($0207, [MS-XLS] 2.4.268) back together from its Continue records ($003C), the wide branch computes segCh from the segment length and then calls GetWideString(3, segCh * 2), because the record body starts at offset 3 and the count is still bytes. The rich-text reader does the same thing from offset 1 on the first Continue segment. [MS-XLS] 2.5.293 guarantees the break sits on a double-byte character boundary when fHighByte is 1, so no partial-character bookkeeping is needed, but the byte arithmetic is still yours to get right

What does GetWideStringWithZero actually do?

TXLSBlob.GetWideStringWithZero is a wide-character reader that keeps embedded NULs. The WithZero suffix marks NUL retention, not character width: internally it runs the same SetString over a PWideChar with Count div SizeOf(WideChar) as GetWideString, minus the terminator scan. The one-byte counterpart is TXLSBlob.GetStringWithZero, which returns an AnsiString. Nothing in the name says which is which, and that ambiguity has cost this codebase real bugs. The specific misreading is worth naming, because it looks so plausible: GetWideString needs cch * 2, so GetWideStringWithZero must be the one that takes cch directly. It does take cch without complaint, it does return a WideString, and the compiler is happy. It also returns half the characters, assembled from the wrong byte pairs. The correct 8-bit path is TXLSBlob.GetString with a plain cch byte count, cast to WideString at the assignment. HotXLS 2.376.0 fixed exactly that misuse in two chart decoders

SXViewLink and the per-encoding length gate

SXViewLink ($0858, [MS-XLS] 2.4.316) is the cleanest worked example, because it packs both asymmetries into one eight-byte header. The layout is rt(2), unused(2), reserved(2), cch(1), fHigh(1), followed by an XLUnicodeStringNoCch body: fHigh = 1 means cch * 2 bytes of UTF-16, fHigh = 0 means cch bytes of one-byte characters, and cch is capped at 255 because the length field is a single byte. HotXLS writes the record into the chart globals ahead of Units, next to PivotChartBits ($0859, [MS-XLS] 2.4.196), when a chart sheet links to a PivotTable view; the record-level view of that machinery is covered in writing BIFF8 PivotTable records in Delphi

// SXViewLink ([MS-XLS] 2.4.316): rt(2) unused(2) reserved(2) cch(1)
// then an XLUnicodeStringNoCch - fHigh(1) followed by the characters
PivCch := Item.FData.GetByte(6);
if (PivCch > 0) and (Item.FData.GetByte(7) <> 0) and
   (Item.FData.DataLength >= LongWord(8 + PivCch * 2)) then
begin
  Result.PivotSourceName := Item.FData.GetWideString(8, PivCch * 2);
  Result.IsPivotChart := True;
end
else if (PivCch > 0) and (Item.FData.GetByte(7) = 0) and
   (Item.FData.DataLength >= LongWord(8 + PivCch)) then
begin
  Result.PivotSourceName := WideString(Item.FData.GetString(8, PivCch));
  Result.IsPivotChart := True;
end;

The two branches are not cosmetic. An earlier version gated both encodings behind the wide-character expression 8 + cch * 2, so an Excel-written 8-bit view name failed the guard and the decoder returned an empty PivotSourceName with IsPivotChart left false. The pivot link vanished from the model without a single diagnostic. The identical mistake sat in the Trendline decoder ($2050, [MS-XLS] 2.4.328), where the name field follows 28 bytes of numeric payload with a two-byte cch at offset 28, fHigh at offset 30, and the characters at offset 31; trendline captions that Excel had written with 8-bit characters decoded as empty strings. Both were fixed in the same release. And the 8-bit case is not a legacy curiosity confined to Excel 2.0 through 4.0 files: current Excel still writes 8-bit BIFF8 payloads whenever every character fits in one byte

How do you decode a new BIFF8 record safely in Delphi?

Where the field really is a standard XLUnicodeString, use TXLSBlob.GetBiffString instead of hand-rolling the branch. It reads the length field, reads the option byte, dispatches to the matching reader, and advances the cursor past the body. The two Boolean parameters are the part to read carefully: is8bit describes the width of the length field, not the width of the characters, and iswide says whether an fHigh option byte is present at all. BIFF versions below $0600 have neither

var
  Offset: LongWord;
begin
  Offset := 6;  // in SXViewLink the cch byte starts here
  // is8bit = the length field is one byte wide
  // iswide = an fHigh option byte follows the length field
  Name := Data.GetBiffString(Offset, True, True);
  // Offset now points at the first byte after the string body

Hand-rolled branches still earn their place when the handler has to survive truncated or hostile input, because GetBiffString leans on EnsureReadable raising rather than on a bounds check you control. That is why the HotXLS chart decoder gates on DataLength and returns nothing instead of throwing: a malformed third-party workbook should cost you one caption, not the whole document. The trade-off is deliberate, and it is exactly why the encoding-specific guard has to be right, since the guard is the thing that converts a bad read into silence

One last piece of process, learned the hard way in the same release. Assert on a saved and reopened workbook, not on the in-memory model you just built. The 2.376.0 batch also turned up an SXEx emitter ([MS-XLS] 2.4.282) that declared a 24-byte body and wrote only 22, misaligning every record after the PivotTable view, including the worksheet EOF and any chart sheet substream that followed. The existing pivot tests never caught it because they all asserted against memory. String decoding has the same property: a round trip through the file is the only test that actually exercises the byte counts

If you work with classic XLS internals in Delphi or C++Builder and would rather not maintain a BIFF8 record reader of your own, the encoding rules above are already implemented and regression-tested in the HotXLS Delphi spreadsheet component, which reads and writes XLS and XLSX without Excel or any OLE automation