2 SPDX-FileCopyrightText: 2024 Lady <https://www.ladys.computer/about/#lady>
3 SPDX-License-Identifier: CC0-1.0
7 <b>A make·file for X·M·L.</b>
9 <dfn>⛩️📰 书社</dfn> aims to make it easy to generate websites with
10 X·S·L·T and G·N·U Make.
11 It is consequently only a good choice for people who like X·S·L·T and
12 G·N·U Make and wish it were easier to make websites with them.
14 It makes things easier by :—
16 - Automatically identifying source files and characterizing them by
17 type (X·M·L, text, or asset).
19 - Parsing supported text types into X·M·L trees.
21 - Enabling easy inclusion of source files within each other.
23 It aims to do this with zero dependencies beyond the programs already
24 installed on your computer†.
26 † Assuming an operating system with a fairly featureful, and
27 Posix‐compliant, development setup (e·g, macOS).
28 In fact, on Linux you will probably need to install a few programs:
29 `libxml2-utils`, `xsltproc`, `sharutils`, and `pax`.
33 <i lang="cmn-Hans">书社</i> is a Chinese word meaning “publishing
36 The first character, <i lang="cmn-Hans">书</i>, is the simplified form
39 The second character, <i lang="cmn-Hans">社</i>, contemporarily means
40 “association”, but historically referred to the god of the soil and
41 related altars or festivities.
42 In Japanese, it is an alternate spelling for <i lang="ja">やしろ</i>,
43 the word for “Shinto shrine”.
45 The name <i lang="cmn-Hans">书社</i> was chosen to play on this pun, as
46 it is intended as a publishing program for webshrines.
48 In Ascii environments, ⛩️📰 书社 should be written `Shushe`, following
49 the pinyin transliteration.
53 In most cases, ⛩️📰 书社 aims to require only functionality which is
54 present in all Posix‐compliant operating systems.
55 There are a few exceptions.
56 Details on particular programs are given below; if a program is not
57 listed, it is assumed that any Posix‐compliant implementation will
62 This is a Posix utility, but ⛩️📰 书社 currently depends on
63 unspecified behaviour.
64 When the G·N·U version of `stat` is being used, then the G·N·U version
65 of `date` is also expected.
69 This is a Posix utility, but ⛩️📰 书社 currently depends on
70 unspecified behaviour.
71 It requires support for the following additional options :—
73 - **`-C`**, when supplied with `-m`, must be useable to compile a
74 `.mgc` magicfile for use with future invocations of `file`.
76 - **`--files-from`** must be useable to provide a file that `file`
77 should read file·names from, and `-` must be useable in this
78 context to specify the standard input.
80 - **`--mime-type`** must cause `file` to print the internet media type
81 of the file with no charset parameter.
83 - **`--separator`** must be useable to set the separator that `file`
84 uses to separate file names from types.
86 These options are implemented by the
87 [Fine Free File Command](https://darwinsys.com/file/), which is used
88 by most operating systems.
92 This is not a Posix utility.
93 Usage of `git` is optional, but recommended (and activated by default).
94 To disable it, set `GIT=`.
98 This is a Posix utility, but ⛩️📰 书社 currently depends on
99 unspecified behaviour.
100 ⛩️📰 书社 requires specifically the G·N·U version of `make`, and
101 depends on functionality present in version 3.81 or later.
102 It is not expected to work in previous versions, or with other
103 implementations of Make.
107 This is a Posix utility, but not included in the Linux Standard Base or
108 installed by default in many distributions.
109 Only `ustar` format support is required.
113 This is not a Posix utility, and nor is it particularly portable.
114 To get around incompatibilities, ⛩️📰 书社 attempts to recognize G·N·U
115 `stat` by searching for the string `GNU` when invoked with the
116 `--version` option, and falls back to B·S·D behaviour otherwise.
118 ### `uudecode` and `uuencode`
120 These are Posix utilities, but not included in the Linux Standard Base
121 or installed by default in many distributions.
122 The G·N·U [Sharutils](https://www.gnu.org/software/sharutils/) package
123 can be installed to access them.
125 ### `xmlcatalog` and `xmllint`
127 These are not a Posix utilities.
128 They is a part of `libxml2`, but may need to be installed separately
129 (e·g by the name `libxml2-utils`).
133 This is not a Posix utility.
134 It is a part of `libxslt`, but may need to be installed separately.
138 Place source files in `sources/` and run `make install` to compile
139 the result to `public/`.
140 Compilation involves the following steps :—
142 1. ⛩️📰 书社 compiles all of the magic files in `magic/` into a single
143 file, `build/magic.mgc`.
145 2. ⛩️📰 书社 processes all of the parsers in `parsers/` and determines
146 the list of supported plaintext types.
148 3. ⛩️📰 书社 identifies all of the source files and includes and uses
149 `build/magic.mgc` to classify them by media type.
151 4. ⛩️📰 书社 parses all plaintext and X·M·L source files and includes
152 and then builds a dependency tree between them.
154 5. ⛩️📰 书社 uses the dependency tree to establish prerequisites for
157 6. ⛩️📰 书社 compiles each output file to `build/result`.
159 7. ⛩️📰 书社 copies most output files from `build/result` to
160 `build/public`, but it does some additional processing instead on
161 those which indicate a non‐X·M·L desired final output form.
163 8. ⛩️📰 书社 copies the final resulting files to `public`.
165 You can use `make list` to list each identified source file or include
166 alongside its computed type and dependencies.
167 As this is a Make‐based program, steps will only be run if the
168 corresponding buildfile or output file is older than its
173 The ⛩️📰 书社 name·space is `urn:fdc:ladys.computer:20231231:Shu1She4`.
175 This document uses a few name·space prefixes, with the following
178 | Prefix | Expansion |
179 | ---------: | :-------------------------------------------- |
180 | `catalog:` | `urn:oasis:names:tc:entity:xmlns:xml:catalog` |
181 | `exsl:` | `http://exslt.org/common` |
182 | `exslstr:` | `http://exslt.org/strings` |
183 | `html:` | `http://www.w3.org/1999/xhtml` |
184 | `svg:` | `http://www.w3.org/2000/svg` |
185 | `xlink:` | `http://www.w3.org/1999/xlink` |
186 | `xslt:` | `http://www.w3.org/1999/XSL/Transform` |
187 | `书社:` | `urn:fdc:ladys.computer:20231231:Shu1She4` |
189 ## Setup and Configuration
191 ⛩️📰 书社 depends on the following programs to run.
192 In every case, you may supply your own implementation by overriding the
193 corresponding (allcaps) variable (e·g, set `MKDIR` to supply your own
194 `mkdir` implementation).
204 - `git` (optional; set `GIT=` to disable)
211 - `pax` (only when generating archives)
216 - `stat` (BSD *or* GNU)
223 - `xmlcatalog` (provided by `libxml2`)
224 - `xmllint` (provided by `libxml2`)
225 - `xsltproc` (provided by `libxslt`)
227 The following additional variables can be used to control the behaviour
231 The location of the source files (default: `sources`).
232 Multiple source directories can be provided, so long as the same
233 file subpath doesn’t exist in more than one of them.
236 The location of source includes (default: `sources/includes`).
237 This can be inside of `SRCDIR`, but needn’t be.
238 Multiple include directories can be provided, so long as the same
239 file subpath doesn’t exist in more than one of them.
242 The location of the (temporary) build directory (default: `build`).
243 `make clean` will delete this, and it is recommended that it not be
244 used for programs aside from ⛩️📰 书社.
247 The location of directory to output files to (default: `public`).
248 `make install` will overwrite files in this directory which
249 correspond to those in `SRCDIR`.
250 It *will not* touch other files, including those generated from files
251 in `SRCDIR` which have since been deleted.
253 Files are first compiled to `$(BUILDDIR)/public` before they are
254 copied to `DESTDIR`, so this folder is relatively quick and
255 inexpensive to re·create.
256 It’s reasonable to simply delete it before every `make install` to
257 ensure stale content is removed.
260 The location of the ⛩️📰 书社 `GNUmakefile`.
261 This should be set automatically when calling Make and shouldn’t ever
262 need to be set manually.
263 This variable is used to find the ⛩️📰 书社 `lib/` folder, which is
264 expected to be in the same location.
267 A white·space‐separated list of magic files to use (default:
268 `$(THISDIR)/magic/*`).
271 The value of this variable is appended to `MAGIC` by default, to
272 enable additional magic files without overriding the existing ones.
275 Rules to use with `find` when searching for source files.
276 The default ignores files that start with a period or hyphen‐minus,
277 those which end with a cloparen, and those which contain a hash,
278 buck, percent, asterisk, colon, semi, eroteme, bracket, backslash,
281 - **`EXTRAFINDRULES`:**
282 The value of this variable is appended to `FINDRULES` by default, to
283 enable additional rules without overriding the existing ones.
285 - **`FINDINCLUDERULES`:**
286 Rules to use with `find` when searching for includes (default:
289 - **`EXTRAFINDINCLUDERULES`:**
290 The value of this variable is appended to `FINDINCLUDERULES` by
291 default, to enable additional rules without overriding the existing
295 A white·space‐separated list of parsers to use (default:
296 `$(THISDIR)/parsers/*.xslt`).
298 - **`EXTRAPARSERS`:**
299 The value of this variable is appended to `PARSERS` by default, to
300 enable additional parsers without overriding the existing ones.
303 A white·space‐separated list of transforms to use (default:
304 `$(THISDIR)/transforms/*.xslt`).
306 - **`EXTRATRANSFORMS`:**
307 The value of this variable is appended to `TRANSFORMS` by default, to
308 enable additional transforms without overriding the existing ones.
311 A white·space‐separated list of media types to consider X·M·L
312 (default: `application/xml text/xml`).
315 The current version of ⛩️📰 书社 (default: derived from the current
316 git tag/branch/commit).
319 The current version of the source files (default: derived from the
320 current git tag/branch/commit).
323 If this variable has a value, every recipe instruction will be
324 printed when it runs (default: empty).
325 This is helpful for debugging, but typically too noisy for general
330 Source files may be placed in `SRCDIR` in any manner; the file
331 structure used there will match the output.
332 The type of source files is *not* determined by file extension, but
333 rather by magic number; this means that files **must** begin with
334 something recognizable.
335 Supported magic numbers include :—
337 - `<?xml` for `application/xml` files
338 - `#!js` for `text/javascript` files
339 - `@charset "` for `text/css` files
340 - `#!tsv` for `text/tab-separated-values` files
341 - `%%` for `text/record-jar` files (unregistered; see
342 [[draft-phillips-record-jar-01][]])
344 Text formats with associated X·S·L·T parsers are wrapped in a H·T·M·L
345 `<script>` element whose `@type` gives its media type, and then
346 passed to the parser to process.
347 Source files whose media type does not have an associated X·S·L·T
348 parser are considered “assets” and will not be transformed.
350 **☡ For compatibility with this program, source file·names must not
351 contain Ascii white·space, colons (`:`), semis (`;`), pipes (`|`),
352 bucks (`$`), percents (`%`), hashes (`#`), asterisks (`*`), brackets
353 (`[` or `]`), erotemes (`?`), backslashes (`\`), or control
354 characters, must not begin with a hyphen‐minus (`-`), must not end
355 with a cloparen (`)`), and must not contain quoted braces (`"{` or
357 The former characters have the potential to conflict with make syntax,
358 a leading hyphen‐minus is confusable for a command‐line argument, a
359 trailing cloparen [activates a bug in G·N·U Make
360 3.81](https://stackoverflow.com/questions/17148468/capturing-filenames-including-parentheses-with-gnu-makes-wildcard-function#comment24825307_17148894),
361 and quoted braces are used internally by the program.
365 Parsers are used to convert plaintext files into X·M·L trees, as well
366 as convert plaintext formats which are already included inline in
367 existing source X·M·L documents.
368 ⛩️📰 书社 comes with some parsers; namely :—
370 - **`parsers/plain.xslt`:**
371 Wraps `text/plain` contents in a `<html:pre>` element.
373 - **`parsers/record-jar.xslt`:**
374 Converts `text/record-jar` contents into a `<html:div>` of
375 `<html:dl>` elements (one for each record).
377 - **`parsers/tsv.xslt`:**
378 Converts `text/tab-separated-values` contents into an `<html:table>`
381 New ⛩️📰 书社 parsers which target plaintext formats should have an
382 `<xslt:template>` element with no `@name` or `@mode` and whose
385 - Starts with an appropriately‐name·spaced qualified name for a
386 `<html:script>` element.
388 - Follows this with the string `[@type=`.
390 - Follows this with a quoted string giving a media type supported by
392 Media type parameters are *not* supported.
394 - Follows this with the string `]`.
396 For example, the trivial `text/plain` parser is defined as follows :—
399 <?xml version="1.0"?>
401 xmlns="http://www.w3.org/1999/XSL/Transform"
402 xmlns:html="http://www.w3.org/1999/xhtml"
403 xmlns:书社="urn:fdc:ladys.computer:20231231:Shu1She4"
406 <书社:id>example:text/plain</书社:id>
407 <template match="html:script[@type='text/plain']">
408 <html:pre><value-of select="."/></html:pre>
413 ⛩️📰 书社 will scan the provided parsers for this pattern to determine
414 the set of allowed plaintext file types.
415 Multiple such `<xslt:template>` elements may be provided in a single
416 parser, for example if the parser supports multiple media types.
417 Alternatively, you can set the `@书社:supported-media-types` attribute
418 on the root element of the parser to override media type support
421 Even when `@书社:supported-media-types` is set, it is a requirement
422 that each parser transform any `<html:script>` elements with a
423 `@type` which matches their registered types into something else.
424 Otherwise the parser will be stuck in an endless loop.
425 The result tree of applying the transform to the `<html:script>`
426 element will be reparsed (in case any new `<html:script>` elements
427 were added in its subtree), and a `@书社:parsed-by` attribute will be
428 added to each toplevel element in the result.
429 The value of this attribute will be the value of the `<书社:id>`
430 toplevel element in the parser.
432 It is possible for parsers to support zero plaintext types.
433 This is useful when targeting specific dialects of X·M·L; parsers in
434 this sense operate on the same basic principles as transforms
436 The major distinction between X·M·L parsers and transforms is where in
437 the process the transformation happens:
438 Parsers are applied *prior* to embedding (and can be used to generate
439 embeds); transforms are applied *after*.
441 It is **strongly recommended** that auxillary templates in parsers be
442 name·spaced (by `@name` or `@mode`) whenever possible, to avoid
443 conflicts between parsers.
445 ### Attributes added during parsing
447 ⛩️📰 书社 will add a few attributes to the output of the parsing step,
450 - A `@书社:cksum` attribute on toplevel result elements, giving the
451 `cksum` checksum of the corresponding source file.
453 - For the elements which result from parsing plaintext `<html:script>`
456 - A `@书社:parsed-by` attribute, giving a space‐separated list of
457 parsers which parsed the node.
458 (Generally, this will be a list of one, but it is possible for the
459 result of a parse to be another plaintext node, which may be
460 parsed by a different parser.)
462 - A `@书社:media-type` attribute, giving the identified media type of
467 Documents can be embedded in other documents using a `<书社:link>`
468 element with `@xlink:show="embed"`.
469 The `@xlink:href`s of these elements should have the format
470 `about:shushe?source=<path>`, where `<path>` provides the path to the
471 file within `SRCDIR`.
472 Includes, which do not generate outputs of their own but may still be
473 freely embedded, instead use the format
474 `about:shushe?include=<path>`, where `<path>` provides the path
477 Embeds are replaced with the parsed contents of a file, unless the file
478 is an asset, in which case an `<html:object>` element is produced
479 instead (with the contents of the asset file provided as a base64
481 Embed replacements will be given a `@书社:identifier` attribute whose
482 value will match the `@xlink:href` of the embed.
484 Embedding takes place after parsing but before transformation, so
485 parsers are able to generate their own embeds.
486 ⛩️📰 书社 is able to detect the transitive embed dependencies of files
487 and update them accordingly; it will signal an error if the
488 dependencies are recursive.
490 ## Output Redirection
492 By default, ⛩️📰 书社 installs files to the same location in `DESTDIR`
493 as they were placed in their `SRCDIR`.
494 This behaviour can be customized by setting the `@书社:destination`
495 attribute on the root element, whose value can give a different path.
496 This attribute is read after parsing, but before transformation (where
497 it is silently dropped).
501 Transforms are used to convert X·M·L files into their final output,
502 after all necessary parsing and embedding has taken place.
503 ⛩️📰 书社 comes with some transforms; namely :—
505 - **`transforms/asset.xslt`:**
506 Converts `<html:object>` elements which correspond to recognized
507 media types into the appropriate H·T·M·L elements, and deletes
508 `<html:style>` elements from the body of the document and moves
511 - **`transforms/metadata.xslt`:**
512 Provides basic `<html:head>` metadata.
513 This metadata is generated from `<html:meta>` elements with one of
514 the following `@itemprop` attributes :—
516 - **`urn:fdc:ladys.computer:20231231:Shu1She4:title`:**
517 Provides the title of the page.
519 ⛩️📰 书社 automatically encapsulates H·T·M·L embeds so that their
520 metadata does not propogate up to the embedding document.
521 To undo this behaviour, remove the `@itemscope` and `@itemtype`
522 attributes from the embed during the transformation phase.
524 - **`transforms/serialization.xslt`:**
525 Replaces `<书社:serialize-xml>` elements with the (escaped)
526 serialized X·M·L of their contents.
527 This replacement happens during the application phase, after most
528 other transformations have taken place.
530 If a `@with-namespaces` attribute is provided, any name·space nodes
531 on the toplevel serialized elements whose U·R·I’s correspond to the
532 definitions of the provided prefixes, as defined for the
533 `<书社:serialize-xml>` element, will be declared using name·space
534 attributes on the serialized elements.
535 Otherwise, only name·space nodes which _differ_ from the definitions
536 on the `<书社:serialize-xml>` element will be declared.
537 The string `#default` may be used to represent the default
539 Multiple prefixes may be provided, separated by white·space.
541 When it comes to name·spaces used internally by ⛩️📰 书社, the
542 prefix used by ⛩️📰 书社 may be declared _in addition to_ the
543 prefix(es) used in the source document(s).
544 It is not possible to selectively only declare one prefix for a
545 name·space to the exclusion of others.
547 `<书社:raw-output>` elements may be used inside of
548 `<书社:serialize-xml>` elements to inject raw output into the
551 The following are recommendations on effective creation of
554 - Make template matchers as specific as possible.
555 It is likely an error if two transforms have templates which match
556 the same element (unless the templates have different priority).
558 - Name·space templates (with `@name` or `@mode`) whenever possible.
560 - Set `@exclude-result-prefixes` on the root `xslt:transform` element
561 to reduce the number of declared name·spaces in the final result.
565 The following params are made available globally in parsers and
572 The checksum of the source file (⅌ `cksum`).
575 The ⛩️📰 书社 identifier of the source file (a u·r·i beginning with
579 The value of the `SRCREV` variable (if present).
582 The time at which the source file was last modified.
585 The value of the `THISREV` variable (if present).
587 The following params are only available in transforms :—
590 The path of the catalog file (within `BUILDDIR`).
593 The path of the output file (within `DESTDIR`).
597 ⛩️📰 书社 will wrap the final output of the transforms in appropriate
598 `<html:html>` and `<html:body>` elements, so it is not necessary for
599 transforms to do this explicitly.
600 After performing the initial transform, ⛩️📰 书社 will match the root
601 node of the result in the following modes to fill in areas of the
605 The result of matching in this mode is prepended into the
606 `<html:body>` of the output (before the transformation result).
609 The result of matching in this mode is appended into the
610 `<html:body>` of the output (after the transformation result).
613 The result of matching in this mode is inserted into the
614 `<html:head>` of the output.
616 In addition to being called with the transform result, each of these
617 modes will additionally be called with a `<xslt:include>` element
618 corresponding to each transform.
619 If a transform has a `<书社:id>` top‐level element whose value is an
620 i·r·i, its `<xslt:include>` element will have a corresponding
622 This mechanism can be used to allow transforms to insert content
623 without matching any elements in the result; for example, the
624 following transform adds a link to a stylesheet to the `<html:head>`
628 <?xml version="1.0"?>
630 xmlns="http://www.w3.org/1999/XSL/Transform"
631 xmlns:html="http://www.w3.org/1999/xhtml"
632 xmlns:xslt="http://www.w3.org/1999/XSL/Transform"
633 xmlns:书社="urn:fdc:ladys.computer:20231231:Shu1She4"
634 exclude-result-prefixes="书社"
637 <书社:id>example:add-stylesheet-links.xslt</书社:id>
638 <template match="xslt:include[@书社:id='example:add-stylesheet-links.xslt']" mode="书社:metadata">
639 <html:link rel="stylesheet" type="text/css" href="/style.css"/>
644 Output wrapping can be entirely disabled by adding a
645 `@书社:disable-output-wrapping` attribute to the top‐level element in
647 It will not be performed on outputs whose root elements are
648 `<书社:archive>`, `<书社:base64-binary>`, or `<书社:raw-text>`
651 ## Applying Attributes
653 The `<书社:apply-attributes>` element will apply any attributes on the
654 element to the element(s) it wraps.
655 It is especially useful in combination with embeds.
657 The `<书社:apply-attributes-to-root>` element will apply any attributes
658 on the element to the root node of the final transformation result.
659 It is especially useful in combination with output wrapping.
661 In both cases, attributes from various sources are combined with
662 white·space between them.
663 Attribute application takes place after all ordinary transforms have
666 Both elements ignore attributes in the `xml:` name·space, except for
667 `@xml:lang`, which ignores all but the first definition (including
668 any already present on the root element).
669 On H·T·M·L and S·V·G elements, `@lang` has the same behaviour as
672 ## Other Kinds of Output
674 There are a few special elements in the `书社:` name·space which, if
675 they appear as the toplevel element in a transformation result, cause
676 ⛩️📰 书社 to produce something other than an X·M·L file.
679 - **`<书社:archive>`:**
680 Each child element with a `@书社:archived-as` attribute will be
681 archived as a separate file in a resulting tarball (this attribute
682 gives the file name).
683 These elements will be processed the same as the root elements of any
684 other file (e·g, they will be wrapped; they can themselves specify
685 non X·M·L output types, ⁊·c).
686 Other child elements will be ignored.
688 If the `<书社:archive>` element is given an `@书社:expanded`
689 attribute, rather than producing a tarball ⛩️📰 书社 will output
690 the directory which expanding the tarball would produce.
691 This mechanism can be used to generate multiple files from a single
692 source, provided all of the files are contained with·in the same
695 - **`<书社:base64-binary>`:**
696 The text nodes in the transformation result will, after removing all
697 Ascii whitespace, be treated as a Base·64 string, which is then
700 - **`<书社:raw-text>`:**
701 A plaintext (U·T·F‐8) file will be produced from the text nodes in
702 the transformation result.
706 It is possible to have a single source file produce multiple output
707 files via `<书社:page>` elements, whose `@name` gives the name of the
709 If a parsed document has a `@书社:destination` which contains `%s`,
710 the `%s` will be replaced with the `@name` for each `<书社:page>` (and
711 removed for the main output).
712 Otherwise, the `@name` is inserted before the first period of the
713 filename (or at the end of the filename for those with no period).
714 If `<书社:page>`s do not have a `@name`, they are numbered
716 The destination of pages must be in the same directory as their parent.
718 Pagination essentially forms a limited convenience for the more
719 sophisticated technique of creating an archive with ⛩️📰 书社 and
721 Pages are, from Make’s point of view, untracked side·effects of
722 installing the main output, meaning they cannot be targeted directly
723 and will not appear in `make list` or `make listout`.
724 They are intended solely for the like of indices and feeds, for which
725 convenience and necessity outweigh their flaws.
729 This repository conforms to [REUSE][].
731 Most source files are licensed under the terms of the <cite>Mozilla
732 Public License, version 2.0</cite>.
734 [REUSE]: <https://reuse.software/spec/>
735 [draft-phillips-record-jar-01]: <https://datatracker.ietf.org/doc/html/draft-phillips-record-jar-01>