Uniscribe does more work than most callers realise. ScriptItemize performs bidirectional analysis and script segmentation in one pass, and ScriptLayout produces the visual order of the resulting runs. HarfBuzz, the portable replacement people reach for, does neither: it shapes a single run whose direction and script have already been decided by somebody else. So the hard part of taking a Windows PDF text pipeline to Linux or macOS is not binding a shaping engine. It is supplying the bidirectional algorithm that Uniscribe was quietly providing, and in the PDFium component that is what FPdfBidi is for
The unit implements UAX #9 directly: rules P2 and P3 for paragraph direction, X1 through X10 for explicit embeddings and isolates, W1 through W7 for weak types, N0 through N2 for neutrals and brackets, I1 and I2 for implicit levels, and L1 and L2 for the final reordering. Two functions carry it: PdfResolveBidiLevels returns one embedding level per UTF-16 code unit, and PdfBidiVisualOrder turns those levels into the permutation that places code units left to right
What the algorithm gives you, and what it does not
It gives you numbers. Even levels are left-to-right, odd levels are right-to-left, and the level of each character encodes the nesting of directional runs that character sits inside. From those numbers L2 derives a permutation. What the algorithm deliberately does not do is decide which font to use, form ligatures, or reorder glyphs within a cluster; those are shaping concerns and belong to the stage after this one
uses
FPdfBidi;
var
Levels: TPdfBidiLevels;
Order: TPdfBidiOrder;
ParagraphLevel: Byte;
Text, Visual: WideString;
I: Integer;
begin
Text := SourceLine;
// pbdAuto applies P2-P3: the first strong character decides
if PdfResolveBidiLevels(Text, pbdAuto, Levels, ParagraphLevel) then
begin
Order := PdfBidiVisualOrder(Text, Levels);
SetLength(Visual, Length(Order));
for I := 0 to High(Order) do
Visual[I + 1] := Text[Order[I] + 1];
// Visual now reads left to right; Levels[] still says which
// runs are RTL so a shaper can be handed correct directions
end;
end;The character-class table is generated, not written
Every code point has a Bidi_Class property, and the algorithm consults it constantly, so the table is the foundation everything else stands on. It is generated from the Unicode Character Database rather than maintained by hand: field five of UnicodeData.txt gives the assigned classes, and the @missing declarations in DerivedBidiClass.txt give the defaults for code points the database does not assign, which is how unallocated blocks correctly default to R, AL, ET or BN rather than to L
The compression trick is to emit only the ranges whose class is not L. Anything falling outside every range is L, which is both the Unicode default and the class of the overwhelming majority of code points. That takes a table that would otherwise run to thousands of entries down to 745 ranges and about 6.7 KB. The operational consequence is worth stating: when you move to a new Unicode version, rerun the generator. Hand-editing the include file will work, and it will also silently diverge from the database on the next upgrade
L2 must reorder code points, not UTF-16 code units
This is the mistake that produces genuinely corrupted output, and the first implementation made it. L2 says to reverse contiguous runs at each level from the highest down to the lowest odd level. Written against a UTF-16 string, "reverse a run" naturally means reversing the code units in it. For characters in the Basic Multilingual Plane that is fine. For an RTL character in an astral plane, such as those in the Cypriot or Old South Arabian blocks near U+10800, it is not: the character is a surrogate pair, reversing the run puts the low surrogate before the high one, and the string now contains two unpaired surrogates instead of one character. Nothing downstream can recover it
The fix is to do L2 on code-point units. The implementation merges code units into code-point units, performs the reversals on those units, and expands the result back to code-unit indices at the end. That is why PdfBidiVisualOrder takes the text and not only the levels array: it cannot tell where the surrogate boundaries are from levels alone. The same surrogate-pair discipline runs through the text APIs generally, as described in the emoji, CJK and surrogate pair article
The descent through levels must include levels that do not occur
The second mistake is subtler and produces no crash, just text that is not reordered. L2 says to start at the highest level present and work down to the lowest odd level. A natural optimisation is to collect the set of levels that actually occur and iterate over that set. It is wrong
Consider a line of Latin text inside a right-to-left embedding. The paragraph level is 0, the embedding pushes the Latin characters to level 2, and no character sits at level 1. Iterating over occurring levels finds only 0 and 2, and there is no odd level at all, so the loop performs no reversal. That answer is correct, but for a reason the optimisation does not know: a reversal at level 2 followed by a reversal at level 1 would cancel exactly, so performing neither is the right outcome. Change the input slightly, so that both level 1 and level 3 characters exist but level 2 does not, and the set-based loop skips the level-2 reversal that the algorithm requires
// Correct: walk every level from the maximum down to the lowest odd
// level, including levels no character actually has
Level := MaxLevel;
while Level >= LowestOddLevel do
begin
ReverseRunsAtOrAbove(Level); // no-op when no run qualifies
Dec(Level);
end;Written as a plain decrementing loop the behaviour falls out for free, and the no-op iterations cost nothing measurable. This is a case where the obvious optimisation is not slightly wrong, it is wrong in an input-dependent way that a small test corpus will never reveal
Brackets: BD16 with a pragmatic table
Rule N0 and the BD16 bracket-pair algorithm exist so that a parenthesis in mixed-direction text resolves to the direction of what it encloses rather than to whatever happens to be adjacent. That needs a table of bracket pairs. The implementation carries the pairs in general use rather than the full contents of the Unicode bracket file: ASCII, CJK, fullwidth, mathematical and ornamental brackets
An unlisted bracket is not an error. It resolves as an ordinary neutral through N1 and N2, which is exactly the behaviour every implementation had before Unicode 6.3 introduced N0. So the boundary is "less refined for rare brackets", not "incorrect". One detail does need explicit handling: the canonical equivalence between the angle brackets at U+2329 and U+232A and those at U+3008 and U+3009 has to be folded when matching pairs, or an opening bracket written one way will fail to pair with a closing bracket written the other
How you test thirty interacting rules
Not with a large corpus, at least not first. The productive approach was sixteen hand-verified cases, each chosen to exercise a specific rule and each checked against the levels UAX #9 says it should produce: paragraph direction detection under P2 and P3, the weak-type rules W2, W3 and W7, the implicit level rules I1 and I2, explicit embedding via X2 and X7, isolates via X5a and X6a, the L1 reset of trailing whitespace and separators, an N0 bracket case, and one case with an astral character to lock down the surrogate handling
Sixteen cases with known-correct expected levels catch more than sixteen hundred cases with plausible-looking output, because the failure mode of a bidirectional implementation is text that reads almost right. Once those pass, a corpus is useful for finding table gaps and performance problems, which are different classes of defect
Inside the PDFium component the levels feed two consumers. On the writing side they tell the shaping backend the direction of each run, which is the input HarfBuzz requires. On the reading side they inform selection geometry and reading order, since a click in RTL text has to map to a logical position rather than a visual one; that mapping is covered in the visual line selection article and the reading-order model in structured text blocks and reading order. Platform support details for the component are on the PDFium Delphi component product page