]> Lady’s Gitweb - Shushe/blob - README.markdown
Make percent‐decoding awk script portable
[Shushe] / README.markdown
1 # ⛩️📰 书社
2
3 <b>A make·file for X·M·L.</b>
4
5 <dfn>⛩️📰 书社</dfn> aims to make it easy to generate websites with
6 X·S·L·T and G·N·U Make.
7 It is consequently only a good choice for people who like X·S·L·T and
8 G·N·U Make and wish it were easier to make websites with them.
9
10 It makes things easier by :⁠—
11
12 - Automatically identifying source files and characterizing them by
13 type (X·M·L, text, or asset).
14
15 - Parsing supported text types into X·M·L trees.
16
17 - Enabling easy inclusion of source files within each other.
18
19 It aims to do this with zero dependencies beyond the programs already
20 installed on your computer.
21 (On Linux machines, you may need to install `libxml2-utils` to get the
22 commandline programs from `libxml2`.)
23
24 **Note:**
25 ⛩️📰 书社 requires functionality present in G·N·U Make 3.81 (or later)
26 and will not work in previous versions, or other implementations of
27 Make.
28 Compatibility with later versions of G·N·U Make is assumed, but not
29 tested.
30
31 ## Nomenclature
32
33 <i lang="cmn-Hans">书社</i> is a Chinese word meaning “publishing
34 house”.
35
36 The first character, <i lang="cmn-Hans">书</i>, is the simplified form
37 of “document”.
38
39 The second character, <i lang="cmn-Hans">社</i>, contemporarily means
40 “association”, but historically referred to the god of the soil and
41 related altars or festivities.
42 In Japanese, it is an alternate spelling for <i lang="ja">やしろ</i>,
43 the word for “Shinto shrine”.
44
45 The name <i lang="cmn-Hans">书社</i> was chosen to play on this pun, as
46 it is intended as a publishing program for webshrines.
47
48 In Ascii environments, ⛩️📰 书社 should be written `Shushe`, following
49 the pinyin transliteration.
50
51 ## Basic Usage
52
53 Place source files in `sources/` and run `make install` to compile
54 the result to `public/`.
55 Compilation involves the following steps :⁠—
56
57 1. ⛩️📰 书社 compiles all of the magic files in `magic/` into a single
58 file, `build/magic.mgc`.
59
60 2. ⛩️📰 书社 processes all of the parsers in `parsers/` and determines
61 the list of supported plaintext types.
62
63 3. ⛩️📰 书社 identifies all of the source files and includes and uses
64 `build/magic.mgc` to classify them by media type.
65
66 4. ⛩️📰 书社 parses all plaintext and X·M·L source files and includes
67 and then builds a dependency tree between them.
68
69 5. ⛩️📰 书社 uses the dependency tree to establish prerequisites for
70 each output file.
71
72 6. ⛩️📰 书社 compiles each output file to `build/public`.
73
74 7. ⛩️📰 书社 copies the output files to `public`.
75
76 You can use `make list` to list each identified source file or include
77 alongside its computed type and dependencies.
78 As this is a Make‐based program, steps will only be run if the
79 corresponding buildfile or output file is older than its
80 prerequisites.
81
82 ## Namespaces
83
84 The ⛩️📰 书社 namespace is `urn:fdc:ladys.computer:20231231:Shu1She4`.
85
86 This document uses a few namespace prefixes, with the following
87 meanings :⁠—
88
89 | Prefix | Expansion |
90 | -------: | :----------------------------------------- |
91 | `html:` | `http://www.w3.org/1999/xhtml` |
92 | `xlink:` | `http://www.w3.org/1999/xlink` |
93 | `xslt:` | `http://www.w3.org/1999/XSL/Transform` |
94 | `书社:` | `urn:fdc:ladys.computer:20231231:Shu1She4` |
95
96 ## Setup and Configuration
97
98 ⛩️📰 书社 depends on the following programs to run.
99 In every case, you may supply your own implementation by overriding the
100 corresponding (allcaps) variable (e·g, set `MKDIR` to supply your own
101 `mkdir` implementation).
102
103 - `awk`
104 - `cat`
105 - `cp`
106 - `date`
107 - `echo`
108 - `file`
109 - `find`
110 - `git` (optional; set `GIT=` to disable)
111 - `ln`
112 - `mkdir` (requires support for `-p`)
113 - `mv`
114 - `od` (requires support for `-t x1`)
115 - `printf`
116 - `rm`
117 - `sed`
118 - `sleep`
119 - `stat`
120 - `test`
121 - `touch`
122 - `tr` (requires support for `-d`)
123 - `uuencode` (requires support for `-m` and `-r`)
124 - `xargs` (requires support for `-0`)
125 - `xmlcatalog` (provided by `libxml2`)
126 - `xmllint` (provided by `libxml2`)
127 - `xsltproc` (provided by `libxslt`)
128
129 The following additional variables can be used to control the behaviour
130 of ⛩️📰 书社 :⁠—
131
132 - **`SRCDIR`:**
133 The location of the source files (default: `sources`).
134 Multiple source directories can be provided, so long as the same
135 file subpath doesn’t exist in more than one of them.
136
137 - **`INCLUDEDIR`:**
138 The location of source includes (default: `sources/includes`).
139 This can be inside of `SRCDIR`, but needn’t be.
140 Multiple include directories can be provided, so long as the same
141 file subpath doesn’t exist in more than one of them.
142
143 - **`BUILDDIR`:**
144 The location of the (temporary) build directory (default: `build`).
145 `make clean` will delete this, and it is recommended that it not be
146 used for programs aside from ⛩️📰 书社.
147
148 - **`DESTDIR`:**
149 The location of directory to output files to (default: `public`).
150 `make install` will overwrite files in this directory which
151 correspond to those in `SRCDIR`.
152 It *will not* touch other files, including those generated from files
153 in `SRCDIR` which have since been deleted.
154
155 Files are first compiled to `$(BUILDDIR)/public` before they are
156 copied to `DESTDIR`, so this folder is relatively quick and
157 inexpensive to re·create.
158 It’s reasonable to simply delete it before every `make install` to
159 ensure stale content is removed.
160
161 - **`THISDIR`:**
162 The location of the ⛩️📰 书社 `GNUmakefile`.
163 This should be set automatically when calling Make and shouldn’t ever
164 need to be set manually.
165 This variable is used to find the ⛩️📰 书社 `lib/` folder, which is
166 expected to be in the same location.
167
168 - **`MAGIC`:**
169 A white·space‐separated list of magic files to use (default:
170 `$(THISDIR)/magic/*`).
171
172 - **`EXTRAMAGIC`:**
173 The value of this variable is appended to `MAGIC` by default, to
174 enable additional magic files without overriding the existing ones.
175
176 - **`FINDRULES`:**
177 Rules to use with `find` when searching for source files.
178 The default ignores files that start with a period or hyphen‐minus,
179 those which end with a cloparen, and those which contain a hash,
180 buck, percent, asterisk, colon, semi, eroteme, bracket, backslash,
181 or pipe.
182
183 - **`EXTRAFINDRULES`:**
184 The value of this variable is appended to `FINDRULES` by default, to
185 enable additional rules without overriding the existing ones.
186
187 - **`FINDINCLUDERULES`:**
188 Rules to use with `find` when searching for includes (default:
189 `$(FINDRULES)`).
190
191 - **`EXTRAFINDINCLUDERULES`:**
192 The value of this variable is appended to `FINDINCLUDERULES` by
193 default, to enable additional rules without overriding the existing
194 ones.
195
196 - **`PARSERS`:**
197 A white·space‐separated list of parsers to use (default:
198 `$(THISDIR)/parsers/*.xslt`).
199
200 - **`EXTRAPARSERS`:**
201 The value of this variable is appended to `PARSERS` by default, to
202 enable additional parsers without overriding the existing ones.
203
204 - **`TRANSFORMS`:**
205 A white·space‐separated list of transforms to use (default:
206 `$(THISDIR)/transforms/*.xslt`).
207
208 - **`EXTRATRANSFORMS`:**
209 The value of this variable is appended to `TRANSFORMS` by default, to
210 enable additional transforms without overriding the existing ones.
211
212 - **`XMLTYPES`:**
213 A white·space‐separated list of media types to consider X·M·L
214 (default: `application/xml text/xml`).
215
216 - **`VERBOSE`:**
217 If this variable has a value, every recipe instruction will be
218 printed when it runs (default: empty).
219 This is helpful for debugging, but typically too noisy for general
220 usage.
221
222 ## Source Files
223
224 Source files may be placed in `SRCDIR` in any manner; the file
225 structure used there will match the output.
226 The type of source files is *not* determined by file extension, but
227 rather by magic number; this means that files **must** begin with
228 something recognizable.
229 Supported magic numbers include :⁠—
230
231 - `<?xml` for `application/xml` files
232 - `#!js` for `text/javascript` files
233 - `@charset "` for `text/css` files
234 - `#!tsv` for `text/tab-separated-values` files
235 - `%%` for `text/record-jar` files (unregistered; see
236 [[draft-phillips-record-jar-01][]])
237
238 Text formats with associated X·S·L·T parsers are wrapped in a H·T·M·L
239 `<script>` element whose `@type` gives its media type, and then
240 passed to the parser to process.
241 Source files whose media type does not have an associated X·S·L·T
242 parser are considered “assets” and will not be transformed.
243
244 **☡ For compatibility with this program, source file·names must not
245 contain Ascii white·space, colons (`:`), semis (`;`), pipes (`|`),
246 bucks (`$`), percents (`%`), hashes (`#`), asterisks (`*`), brackets
247 (`[` or `]`), erotemes (`?`), backslashes (`\`), or control
248 characters, must not begin with a hyphen‐minus (`-`), and must not
249 end with a cloparen (`)`).**
250 The former characters have the potential to conflict with make syntax,
251 a leading hyphen‐minus is confusable for a command‐line argument, and
252 a trailing cloparen [activates a bug in G·N·U Make
253 3.81](https://stackoverflow.com/questions/17148468/capturing-filenames-including-parentheses-with-gnu-makes-wildcard-function#comment24825307_17148894).
254
255 ## Parsers
256
257 Parsers are used to convert plaintext files into X·M·L trees, as well
258 as convert plaintext formats which are already included inline in
259 existing source X·M·L documents.
260 ⛩️📰 书社 comes with some parsers; namely :⁠—
261
262 - **`parsers/plain.xslt`:**
263 Wraps `text/plain` contents in a `<html:pre class="plain">` element.
264
265 - **`parsers/record-jar.xslt`:**
266 Converts `text/record-jar` contents into a
267 `<html:div class="record-jar">` of `<html:dl>` elements (one for
268 each record).
269
270 - **`parsers/tsv.xslt`:**
271 Converts `text/tab-separated-values` contents into an
272 `<html:table class="tsv">` element.
273
274 New ⛩️📰 书社 parsers which target plaintext formats should have an
275 `<xslt:template>` element with no `@name` or `@mode` and whose
276 `@match` attribute…
277
278 - Starts with an appropriately‐namespaced qualified name for a
279 `<html:script>` element.
280
281 - Follows this with the string `[@type=`.
282
283 - Follows this with a quoted string giving a media type supported by
284 the parser.
285 Media type parameters are *not* supported.
286
287 - Follows this with the string `]`.
288
289 For example, the trivial `text/plain` parser is defined as follows :⁠—
290
291 ```xml
292 <?xml version="1.0"?>
293 <transform
294 xmlns="http://www.w3.org/1999/XSL/Transform"
295 xmlns:html="http://www.w3.org/1999/xhtml"
296 xmlns:书社="urn:fdc:ladys.computer:20231231:Shu1She4"
297 version="1.0"
298 >
299 <书社:id>example:text/plain</书社:id>
300 <template match="html:script[@type='text/plain']">
301 <html:pre><value-of select="."/></html:pre>
302 </template>
303 </transform>
304 ```
305
306 ⛩️📰 书社 will scan the provided parsers for this pattern to determine
307 the set of allowed plaintext file types.
308 Multiple such `<xslt:template>` elements may be provided in a single
309 parser, for example if the parser supports multiple media types.
310 Alternatively, you can set the `@书社:supported-media-types` attribute
311 on the root element of the parser to override media type support
312 detection.
313
314 Even when `@书社:supported-media-types` is set, it is a requirement
315 that each parser transform any `<html:script>` elements with a
316 `@type` which matches their registered types into something else.
317 Otherwise the parser will be stuck in an endless loop.
318 The result tree of applying the transform to the `<html:script>`
319 element will be reparsed (in case any new `<html:script>` elements
320 were added in its subtree), and a `@书社:parsed-by` attribute will be
321 added to each toplevel element in the result.
322 The value of this attribute will be the value of the `<书社:id>`
323 toplevel element in the parser.
324
325 It is possible for parsers to support zero plaintext types.
326 This is useful when targeting specific dialects of X·M·L; parsers in
327 this sense operate on the same basic principles as transforms
328 (described below).
329 The major distinction between X·M·L parsers and transforms is where in
330 the process the transformation happens:
331 Parsers are applied *prior* to embedding (and can be used to generate
332 embeds); transforms are applied *after*.
333
334 It is **strongly recommended** that auxillary templates in parsers be
335 namespaced (by `@name` or `@mode`) whenever possible, to avoid
336 conflicts between parsers.
337
338 ## Embedding
339
340 Documents can be embedded in other documents using a `<书社:link>`
341 element with `@xlink:show="embed"`.
342 The `@xlink:href`s of these elements should have the format
343 `about:shushe?source=<path>`, where `<path>` provides the path to the
344 file within `SRCDIR`.
345 Includes, which do not generate outputs of their own but may still be
346 freely embedded, instead use the format
347 `about:shushe?include=<path>`, where `<path>` provides the path
348 within `INCLUDEDIR`.
349
350 Embeds are replaced with the parsed contents of a file, unless the file
351 is an asset, in which case an `<html:object>` element is produced
352 instead (with the contents of the asset file provided as a base64
353 `data:` u·r·i).
354
355 Embedding takes place after parsing but before transformation, so
356 parsers are able to generate their own embeds.
357 ⛩️📰 书社 is able to detect the transitive embed dependencies of files
358 and update them accordingly; it will signal an error if the
359 dependencies are recursive.
360
361 ## Output Redirection
362
363 By default, ⛩️📰 书社 installs files to the same location in `DESTDIR`
364 as they were placed in their `SRCDIR`.
365 This behaviour can be customized by setting the `@书社:destination`
366 attribute on the root element, whose value can give a different path.
367 This attribute is read after parsing, but before transformation (where
368 it is silently dropped).
369
370 ## Transforms
371
372 Transforms are used to convert X·M·L files into their final output,
373 after all necessary parsing and embedding has taken place.
374 ⛩️📰 书社 comes with some transforms; namely :⁠—
375
376 - **`transforms/attributes.xslt`:**
377 Applies transforms to the children of any `<书社:apply-attributes>`
378 elements, and then applies the attributes of the
379 `<书社:apply-attributes>` to each result child, replacing the
380 element with the result.
381 This is useful in combination with image embeds to apply alt‐text to
382 the resulting `<html:img>`.
383
384 - **`transforms/asset.xslt`:**
385 Converts `<html:object>` elements which correspond to recognized
386 media types into the appropriate H·T·M·L elements, and deletes
387 `<html:style>` elements from the body of the document and moves
388 them to the head.
389
390 - **`transforms/metadata.xslt`:**
391 Provides basic `<html:head>` metadata.
392 This metadata is generated from `<html:meta>` elements with one of
393 the following `@itemprop` attributes :⁠—
394
395 - **`urn:fdc:ladys.computer:20231231:Shu1She4:title`:**
396 Provides the title of the page.
397
398 ⛩️📰 书社 automatically encapsulates embeds so that their metadata
399 does not propogate up to the embedding document.
400 To undo this behaviour, remove the `@itemscope` and `@itemtype`
401 attributes from the embed during the transformation phase.
402
403 The following are recommendations on effective creation of
404 transforms :⁠—
405
406 - Make template matchers as specific as possible.
407 It is likely an error if two transforms have templates which match
408 the same element (unless the templates have different priority).
409
410 - Namespace templates (with `@name` or `@mode`) whenever possible.
411
412 - Set `@exclude-result-prefixes` on the root `xslt:transform` element
413 to reduce the number of declared namespaces in the final result.
414
415 ## Global Params
416
417 The following params are made available globally in parsers and
418 transforms :⁠—
419
420 - **`BUILDTIME`:**
421 The current time.
422
423 - **`SRCREV`:**
424 The tag or hash of the current commit in the working directory (if
425 `GIT` is defined and `./.git` exists).
426
427 - **`SRCTIME`:**
428 The time at which the source file was last modified.
429
430 - **`VERSION`:**
431 The tag or hash of the current commit in `THISDIR` (if `GIT` is
432 defined and `$(THISDIR)/.git` exists).
433
434 The following params are only available in transforms :⁠—
435
436 - **`CATALOG`:**
437 The path of the catalog file (within `BUILDDIR`).
438
439 - **`PATH`:**
440 The path of the output file (within `DESTDIR`).
441
442 ## Output Wrapping
443
444 ⛩️📰 书社 will wrap the final output of the transforms in appropriate
445 `<html:html>` and `<html:body>` elements, so it is not necessary for
446 transforms to do this explicitly.
447 After performing the initial transform, ⛩️📰 书社 will match the root
448 node of the result in the following modes to fill in areas of the
449 wrapper :⁠—
450
451 - **`书社:header`:**
452 The result of matching in this mode is prepended into the
453 `<html:body>` of the output (before the transformation result).
454
455 - **`书社:footer`:**
456 The result of matching in this mode is appended into the
457 `<html:body>` of the output (after the transformation result).
458
459 - **`书社:metadata`:**
460 The result of matching in this mode is inserted into the
461 `<html:head>` of the output.
462
463 In addition to being called with the transform result, each of these
464 modes will additionally be called with a `<xslt:include>` element
465 corresponding to each transform.
466 If a transform has a `<书社:id>` top‐level element whose value is an
467 i·r·i, its `<xslt:include>` element will have a corresponding
468 `@书社:id` attribute.
469 This mechanism can be used to allow transforms to insert content
470 without matching any elements in the result; for example, the
471 following transform adds a link to a stylesheet to the `<html:head>`
472 of every page :⁠—
473
474 ```xml
475 <?xml version="1.0"?>
476 <transform
477 xmlns="http://www.w3.org/1999/XSL/Transform"
478 xmlns:html="http://www.w3.org/1999/xhtml"
479 xmlns:xslt="http://www.w3.org/1999/XSL/Transform"
480 xmlns:书社="urn:fdc:ladys.computer:20231231:Shu1She4"
481 exclude-result-prefixes="书社"
482 version="1.0"
483 >
484 <书社:id>example:add-stylesheet-links.xslt</书社:id>
485 <template match="xslt:include[@书社:id='example:add-stylesheet-links.xslt']" mode="书社:metadata">
486 <html:link rel="stylesheet" type="text/css" href="/style.css"/>
487 </template>
488 </transform>
489 ```
490
491 Output wrapping can be entirely disabled by adding a
492 `@书社:disable-output-wrapping` attribute to the top‐level element in
493 the result tree.
494
495 ## License
496
497 Source files are licensed under the terms of the <cite>Mozilla Public
498 License, version 2.0</cite>.
499 For more information, see [LICENSE](./LICENSE).
500
501 [draft-phillips-record-jar-01]: <https://datatracker.ietf.org/doc/html/draft-phillips-record-jar-01>
This page took 0.123864 seconds and 5 git commands to generate.