]> Lady’s Gitweb - Shushe/blob - README.markdown
Reduce the strength of magic matches
[Shushe] / README.markdown
1 # ⛩️📰 书社
2
3 <b>A make·file for X·M·L.</b>
4
5 <dfn>⛩️📰 书社</dfn> aims to make it easy to generate websites with
6 X·S·L·T and G·N·U Make.
7 It is consequently only a good choice for people who like X·S·L·T and
8 G·N·U Make and wish it were easier to make websites with them.
9
10 It makes things easier by :⁠—
11
12 - Automatically identifying source files and characterizing them by
13 type (X·M·L, text, or asset).
14
15 - Parsing supported text types into X·M·L trees.
16
17 - Enabling easy inclusion of source files within each other.
18
19 It aims to do this with zero dependencies beyond the programs already
20 installed on your computer.
21 (On Linux machines, you may need to install `libxml2-utils` to get the
22 commandline programs from `libxml2`.)
23
24 **Note:**
25 ⛩️📰 书社 requires functionality present in G·N·U Make 3.81 (or later)
26 and will not work in previous versions, or other implementations of
27 Make.
28 Compatibility with later versions of G·N·U Make is assumed, but not
29 tested.
30
31 ## Nomenclature
32
33 <i lang="cmn-Hans">书社</i> is a Chinese word meaning “publishing
34 house”.
35
36 The first character, <i lang="cmn-Hans">书</i>, is the simplified form
37 of “document”.
38
39 The second character, <i lang="cmn-Hans">社</i>, contemporarily means
40 “association”, but historically referred to the god of the soil and
41 related altars or festivities.
42 In Japanese, it is an alternate spelling for <i lang="ja">やしろ</i>,
43 the word for “Shinto shrine”.
44
45 The name <i lang="cmn-Hans">书社</i> was chosen to play on this pun, as
46 it is intended as a publishing program for webshrines.
47
48 In Ascii environments, ⛩️📰 书社 should be written `Shushe`, following
49 the pinyin transliteration.
50
51 ## Basic Usage
52
53 Place source files in `sources/` and run `make install` to compile
54 the result to `public/`.
55 Compilation involves the following steps :⁠—
56
57 1. ⛩️📰 书社 compiles all of the magic files in `magic/` into a single
58 file, `build/magic.mgc`.
59
60 2. ⛩️📰 书社 processes all of the parsers in `parsers/` and determines
61 the list of supported plaintext types.
62
63 3. ⛩️📰 书社 identifies all of the source files and includes and uses
64 `build/magic.mgc` to classify them by media type.
65
66 4. ⛩️📰 书社 parses all plaintext and X·M·L source files and includes
67 and then builds a dependency tree between them.
68
69 5. ⛩️📰 书社 uses the dependency tree to establish prerequisites for
70 each output file.
71
72 6. ⛩️📰 书社 compiles each output file to `build/public`.
73
74 7. ⛩️📰 书社 copies the output files to `public`.
75
76 You can use `make list` to list each identified source file or include
77 alongside its computed type and dependencies.
78 As this is a Make‐based program, steps will only be run if the
79 corresponding buildfile or output file is older than its
80 prerequisites.
81
82 ## Namespaces
83
84 The ⛩️📰 书社 namespace is `urn:fdc:ladys.computer:20231231:Shu1She4`.
85
86 This document uses a few namespace prefixes, with the following
87 meanings :⁠—
88
89 | Prefix | Expansion |
90 | -------: | :----------------------------------------- |
91 | `html:` | `http://www.w3.org/1999/xhtml` |
92 | `xlink:` | `http://www.w3.org/1999/xlink` |
93 | `xslt:` | `http://www.w3.org/1999/XSL/Transform` |
94 | `书社:` | `urn:fdc:ladys.computer:20231231:Shu1She4` |
95
96 ## Setup and Configuration
97
98 ⛩️📰 书社 depends on the following programs to run.
99 In every case, you may supply your own implementation by overriding the
100 corresponding (allcaps) variable (e·g, set `MKDIR` to supply your own
101 `mkdir` implementation).
102
103 - `awk`
104 - `cat`
105 - `cp`
106 - `date`
107 - `echo`
108 - `file`
109 - `find`
110 - `git` (optional; set `GIT=` to disable)
111 - `mkdir` (requires support for `-p`)
112 - `mv`
113 - `od` (requires support for `-t x1`)
114 - `printf`
115 - `rm`
116 - `sed`
117 - `sleep`
118 - `stat`
119 - `test`
120 - `touch`
121 - `tr` (requires support for `-d`)
122 - `uuencode` (requires support for `-m` and `-r`)
123 - `xargs` (requires support for `-0`)
124 - `xmlcatalog` (provided by `libxml2`)
125 - `xmllint` (provided by `libxml2`)
126 - `xsltproc` (provided by `libxslt`)
127
128 The following additional variables can be used to control the behaviour
129 of ⛩️📰 书社 :⁠—
130
131 - **`SRCDIR`:**
132 The location of the source files (default: `sources`).
133 Multiple source directories can be provided, so long as the same
134 file subpath doesn’t exist in more than one of them.
135
136 - **`INCLUDEDIR`:**
137 The location of source includes (default: `sources/includes`).
138 This can be inside of `SRCDIR`, but needn’t be.
139 Multiple include directories can be provided, so long as the same
140 file subpath doesn’t exist in more than one of them.
141
142 - **`BUILDDIR`:**
143 The location of the (temporary) build directory (default: `build`).
144 `make clean` will delete this, and it is recommended that it not be
145 used for programs aside from ⛩️📰 书社.
146
147 - **`DESTDIR`:**
148 The location of directory to output files to (default: `public`).
149 `make install` will overwrite files in this directory which
150 correspond to those in `SRCDIR`.
151 It *will not* touch other files, including those generated from files
152 in `SRCDIR` which have since been deleted.
153
154 Files are first compiled to `$(BUILDDIR)/public` before they are
155 copied to `DESTDIR`, so this folder is relatively quick and
156 inexpensive to re·create.
157 It’s reasonable to simply delete it before every `make install` to
158 ensure stale content is removed.
159
160 - **`THISDIR`:**
161 The location of the ⛩️📰 书社 `GNUmakefile`.
162 This should be set automatically when calling Make and shouldn’t ever
163 need to be set manually.
164 This variable is used to find the ⛩️📰 书社 `lib/` folder, which is
165 expected to be in the same location.
166
167 - **`MAGICDIR`:**
168 The location of the magic files to use (default: `$(THISDIR)/magic`).
169
170 - **`FINDRULES`:**
171 Rules to use with `find` when searching for source files.
172 The default ignores files that start with a period or hyphen‐minus
173 and those which contain a hash, buck, percent, asterisk, colon,
174 semi, eroteme, bracket, backslash, or pipe.
175
176 - **`FINDINCLUDERULES`:**
177 Rules to use with `find` when searching for includes (default:
178 `$(FINDRULES)`).
179
180 - **`PARSERS`:**
181 A white·space‐separated list of parsers to use (default:
182 `$(THISDIR)/parsers/*.xslt`).
183
184 - **`TRANSFORMS`:**
185 A white·space‐separated list of transforms to use (default:
186 `$(THISDIR)/transforms/*.xslt`).
187
188 - **`XMLTYPES`:**
189 A white·space‐separated list of media types to consider X·M·L
190 (default: `application/xml text/xml`).
191
192 - **`VERBOSE`:**
193 If this variable has a value, every recipe instruction will be
194 printed when it runs (default: empty).
195 This is helpful for debugging, but typically too noisy for general
196 usage.
197
198 ## Source Files
199
200 Source files may be placed in `SRCDIR` in any manner; the file
201 structure used there will match the output.
202 The type of source files is *not* determined by file extension, but
203 rather by magic number; this means that files **must** begin with
204 something recognizable.
205 Supported magic numbers include :⁠—
206
207 - `<?xml` for `application/xml` files
208 - `#!js` for `text/javascript` files
209 - `@charset "` for `text/css` files
210 - `#!tsv` for `text/tab-separated-values` files
211 - `%%` for `text/record-jar` files (unregistered; see
212 [[draft-phillips-record-jar-01][]])
213
214 Text formats with associated X·S·L·T parsers are wrapped in a H·T·M·L
215 `<script>` element whose `@type` gives its media type, and then
216 passed to the parser to process.
217 Source files whose media type does not have an associated X·S·L·T
218 parser are considered “assets” and will not be transformed.
219
220 **☡ For compatibility with this program, source file·names must not
221 contain Ascii white·space, colons (`:`), semis (`;`), pipes (`|`),
222 bucks (`$`), percents (`%`), hashes (`#`), asterisks (`*`), brackets
223 (`[` or `]`), erotemes (`?`), backslashes (`\`), or control
224 characters, and must not begin with a hyphen‐minus (`-`).**
225 The former characters have the potential to conflict with make syntax,
226 and a leading hyphen‐minus is confusable for a command‐line argument.
227
228 ## Parsers
229
230 Parsers are used to convert plaintext files into X·M·L trees, as well
231 as convert plaintext formats which are already included inline in
232 existing source X·M·L documents.
233 ⛩️📰 书社 comes with some parsers; namely :⁠—
234
235 - **`parsers/plain.xslt`:**
236 Wraps `text/plain` contents in a `<html:pre class="plain">` element.
237
238 - **`parsers/record-jar.xslt`:**
239 Converts `text/record-jar` contents into a
240 `<html:div class="record-jar">` of `<html:dl>` elements (one for
241 each record).
242
243 - **`parsers/tsv.xslt`:**
244 Converts `text/tab-separated-values` contents into an
245 `<html:table class="tsv">` element.
246
247 New ⛩️📰 书社 parsers which target plaintext formats should have an
248 `<xslt:template>` element with no `@name` or `@mode` and whose
249 `@match` attribute…
250
251 - Starts with an appropriately‐namespaced qualified name for a
252 `<html:script>` element.
253
254 - Follows this with the string `[@type=`.
255
256 - Follows this with a quoted string giving a media type supported by
257 the parser.
258 Media type parameters are *not* supported.
259
260 - Follows this with the string `]`.
261
262 For example, the trivial `text/plain` parser is defined as follows :⁠—
263
264 ```xml
265 <?xml version="1.0"?>
266 <transform
267 xmlns="http://www.w3.org/1999/XSL/Transform"
268 xmlns:html="http://www.w3.org/1999/xhtml"
269 xmlns:书社="urn:fdc:ladys.computer:20231231:Shu1She4"
270 version="1.0"
271 >
272 <书社:id>example:text/plain</书社:id>
273 <template match="html:script[@type='text/plain']">
274 <html:pre><value-of select="."/></html:pre>
275 </template>
276 </transform>
277 ```
278
279 ⛩️📰 书社 will scan the provided parsers for this pattern to determine
280 the set of allowed plaintext file types.
281 Multiple such `<xslt:template>` elements may be provided in a single
282 parser, for example if the parser supports multiple media types.
283 Alternatively, you can set the `@书社:supported-media-types` attribute
284 on the root element of the parser to override media type support
285 detection.
286
287 Even when `@书社:supported-media-types` is set, it is a requirement
288 that each parser transform any `<html:script>` elements with a
289 `@type` which matches their registered types into something else.
290 Otherwise the parser will be stuck in an endless loop.
291 The result tree of applying the transform to the `<html:script>`
292 element will be reparsed (in case any new `<html:script>` elements
293 were added in its subtree), and a `@书社:parsed-by` attribute will be
294 added to each toplevel element in the result.
295 The value of this attribute will be the value of the `<书社:id>`
296 toplevel element in the parser.
297
298 It is possible for parsers to support zero plaintext types.
299 This is useful when targeting specific dialects of X·M·L; parsers in
300 this sense operate on the same basic principles as transforms
301 (described below).
302 The major distinction between X·M·L parsers and transforms is where in
303 the process the transformation happens:
304 Parsers are applied *prior* to embedding (and can be used to generate
305 embeds); transforms are applied *after*.
306
307 It is **strongly recommended** that auxillary templates in parsers be
308 namespaced (by `@name` or `@mode`) whenever possible, to avoid
309 conflicts between parsers.
310
311 ## Embedding
312
313 Documents can be embedded in other documents using a `<书社:link>`
314 element with `@xlink:show="embed"`.
315 The `@xlink:href`s of these elements should have the format
316 `about:shushe?source=<path>`, where `<path>` provides the path to the
317 file within `SRCDIR`.
318 Includes, which do not generate outputs of their own but may still be
319 freely embedded, instead use the format
320 `about:shushe?include=<path>`, where `<path>` provides the path
321 within `INCLUDEDIR`.
322
323 Embeds are replaced with the parsed contents of a file, unless the file
324 is an asset, in which case an `<html:object>` element is produced
325 instead (with the contents of the asset file provided as a base64
326 `data:` u·r·i).
327
328 Embedding takes place after parsing but before transformation, so
329 parsers are able to generate their own embeds.
330 ⛩️📰 书社 is able to detect the transitive embed dependencies of files
331 and update them accordingly; it will signal an error if the
332 dependencies are recursive.
333
334 ## Output Redirection
335
336 By default, ⛩️📰 书社 installs files to the same location in `DESTDIR`
337 as they were placed in their `SRCDIR`.
338 This behaviour can be customized by setting the `@书社:destination`
339 attribute on the root element, whose value can give a different path.
340 This attribute is read after parsing, but before transformation (where
341 it is silently dropped).
342
343 ## Transforms
344
345 Transforms are used to convert X·M·L files into their final output,
346 after all necessary parsing and embedding has taken place.
347 ⛩️📰 书社 comes with some transforms; namely :⁠—
348
349 - **`transforms/attributes.xslt`:**
350 Applies transforms to the children of any `<书社:apply-attributes>`
351 elements, and then applies the attributes of the
352 `<书社:apply-attributes>` to each result child, replacing the
353 element with the result.
354 This is useful in combination with image embeds to apply alt‐text to
355 the resulting `<html:img>`.
356
357 - **`transforms/asset.xslt`:**
358 Converts `<html:object>` elements which correspond to recognized
359 media types into the appropriate H·T·M·L elements, and deletes
360 `<html:style>` elements from the body of the document and moves
361 them to the head.
362
363 - **`transforms/metadata.xslt`:**
364 Provides basic `<html:head>` metadata.
365 This metadata is generated from `<html:meta>` elements with one of
366 the following `@itemprop` attributes :⁠—
367
368 - **`urn:fdc:ladys.computer:20231231:Shu1She4:title`:**
369 Provides the title of the page.
370
371 ⛩️📰 书社 automatically encapsulates embeds so that their metadata
372 does not propogate up to the embedding document.
373 To undo this behaviour, remove the `@itemscope` and `@itemtype`
374 attributes from the embed during the transformation phase.
375
376 The following are recommendations on effective creation of
377 transforms :⁠—
378
379 - Make template matchers as specific as possible.
380 It is likely an error if two transforms have templates which match
381 the same element (unless the templates have different priority).
382
383 - Namespace templates (with `@name` or `@mode`) whenever possible.
384
385 - Set `@exclude-result-prefixes` on the root `xslt:transform` element
386 to reduce the number of declared namespaces in the final result.
387
388 ## Global Params
389
390 The following params are made available globally in parsers and
391 transforms :⁠—
392
393 - **`BUILDTIME`:**
394 The current time.
395
396 - **`SRCREV`:**
397 The tag or hash of the current commit in the working directory (if
398 `GIT` is defined and `./.git` exists).
399
400 - **`SRCTIME`:**
401 The time at which the source file was last modified.
402
403 - **`VERSION`:**
404 The tag or hash of the current commit in `THISDIR` (if `GIT` is
405 defined and `$(THISDIR)/.git` exists).
406
407 The following params are only available in transforms :⁠—
408
409 - **`CATALOG`:**
410 The path of the catalog file (within `BUILDDIR`).
411
412 - **`PATH`:**
413 The path of the output file (within `DESTDIR`).
414
415 ## Output Wrapping
416
417 ⛩️📰 书社 will wrap the final output of the transforms in appropriate
418 `<html:html>` and `<html:body>` elements, so it is not necessary for
419 transforms to do this explicitly.
420 After performing the initial transform, ⛩️📰 书社 will match the root
421 node of the result in the following modes to fill in areas of the
422 wrapper :⁠—
423
424 - **`书社:header`:**
425 The result of matching in this mode is prepended into the
426 `<html:body>` of the output (before the transformation result).
427
428 - **`书社:footer`:**
429 The result of matching in this mode is appended into the
430 `<html:body>` of the output (after the transformation result).
431
432 - **`书社:metadata`:**
433 The result of matching in this mode is inserted into the
434 `<html:head>` of the output.
435
436 In addition to being called with the transform result, each of these
437 modes will additionally be called with a `<xslt:include>` element
438 corresponding to each transform.
439 If a transform has a `<书社:id>` top‐level element whose value is an
440 i·r·i, its `<xslt:include>` element will have a corresponding
441 `@书社:id` attribute.
442 This mechanism can be used to allow transforms to insert content
443 without matching any elements in the result; for example, the
444 following transform adds a link to a stylesheet to the `<html:head>`
445 of every page :⁠—
446
447 ```xml
448 <?xml version="1.0"?>
449 <transform
450 xmlns="http://www.w3.org/1999/XSL/Transform"
451 xmlns:html="http://www.w3.org/1999/xhtml"
452 xmlns:xslt="http://www.w3.org/1999/XSL/Transform"
453 xmlns:书社="urn:fdc:ladys.computer:20231231:Shu1She4"
454 exclude-result-prefixes="书社"
455 version="1.0"
456 >
457 <书社:id>example:add-stylesheet-links.xslt</书社:id>
458 <template match="xslt:include[@书社:id='example:add-stylesheet-links.xslt']" mode="书社:metadata">
459 <html:link rel="stylesheet" type="text/css" href="/style.css"/>
460 </template>
461 </transform>
462 ```
463
464 Output wrapping can be entirely disabled by adding a
465 `@书社:disable-output-wrapping` attribute to the top‐level element in
466 the result tree.
467
468 ## License
469
470 Source files are licensed under the terms of the <cite>Mozilla Public
471 License, version 2.0</cite>.
472 For more information, see [LICENSE](./LICENSE).
473
474 [draft-phillips-record-jar-01]: <https://datatracker.ietf.org/doc/html/draft-phillips-record-jar-01>
This page took 0.072641 seconds and 5 git commands to generate.