From: Lady <redacted>
Date: Mon, 1 Apr 2024 20:45:42 +0000 (-0400)
Subject: Improve (fix) the T·S·V parser
X-Git-Tag: 0.7.0~5
X-Git-Url: https://git.ladys.computer/Shushe/commitdiff_plain/872c70923f2c19e4c9abad7ea1c2f5be1ffd3e65?hp=872c70923f2c19e4c9abad7ea1c2f5be1ffd3e65

Improve (fix) the T·S·V parser

Although `exslstr:tokenize()` is fast, it should not be used when
splitting the columns of a T·S·V file, as it will collapse empty
columns. Introduce a new transform in `lib/` for splitting, and import
it into the T·S·V parser.

This transform was largely copied from Caudex
<https://git.ladys.computer/Caudex/blob/0.1.1:/lib/split.xslt> and is
likely to be useful downstream as well.

Continue using `exslstr:tokenize()` for splitting the _rows_ of the
T·S·V, as empty rows _should_ be collapsed.
---