2 SPDX-FileCopyrightText: 2024 Lady <https://www.ladys.computer/about/#lady>
3 SPDX-License-Identifier: CC0-1.0
7 <b>A make·file for X·M·L.</b>
9 <dfn>⛩️📰 书社</dfn> aims to make it easy to generate websites with
10 X·S·L·T and G·N·U Make.
11 It is consequently only a good choice for people who like X·S·L·T and
12 G·N·U Make and wish it were easier to make websites with them.
14 It makes things easier by :—
16 - Automatically identifying source files and characterizing them by
17 type (X·M·L, text, or asset).
19 - Parsing supported text types into X·M·L trees.
21 - Enabling easy inclusion of source files within each other.
23 It aims to do this with zero dependencies beyond the programs already
24 installed on your computer†.
26 † Assuming an operating system with a fairly featureful, and
27 Posix‐compliant, development setup (e·g, Macintosh ≥ version 10.8).
28 In fact, on Linux you will probably need to install a few programs:
29 `libxml2-utils`, `xsltproc`, `sharutils`, and `pax`.
33 <i lang="cmn-Hans">书社</i> is a Chinese word meaning “publishing
36 The first character, <i lang="cmn-Hans">书</i>, is the simplified form
39 The second character, <i lang="cmn-Hans">社</i>, contemporarily means
40 “association”, but historically referred to the god of the soil and
41 related altars or festivities.
42 In Japanese, it is an alternate spelling for <i lang="ja">やしろ</i>,
43 the word for “Shinto shrine”.
45 The name <i lang="cmn-Hans">书社</i> was chosen to play on this pun, as
46 it is intended as a publishing program for webshrines.
48 In Ascii environments, ⛩️📰 书社 should be written `Shushe`, following
49 the pinyin transliteration.
53 In most cases, ⛩️📰 书社 aims to require only functionality which is
54 present in all Posix‐compliant (`POSIX.1-2001`) operating systems.
55 There are a few exceptions.
56 Details on particular programs are given below; if a program is not
57 listed, it is assumed that any Posix‐compliant implementation will
62 This is a Posix utility, but ⛩️📰 书社 depends on functionality
63 introduced after `POSIX.1-2001` (the `-u` option, introduced in
65 Macintosh systems somewhat interestingly implement this option
66 correctly in legacy mode (`COMMAND_MODE=legacy`) but incorrectly by
67 default (despite claiming `POSIX.1-2008` conformance for this
69 [Note this erroneous comment claiming nanosecond & timezone are
70 extensions rather than standardized.][rdar-92753335]
71 Despite this, the default Macintosh implementation will still work with
72 ⛩️📰 书社, with the caveat that the timestamp will only include a
73 fractional component when a Posix⹀compliant (e·g, Macintosh legacy or
74 G·N·U) implementation is used.
78 This is a Posix utility, but it was considered optional in
79 `POSIX.1-2001` (altho it was made mandatory in `POSIX.1-2008`) and
80 ⛩️📰 书社 currently depends on unspecified behaviour.
81 It requires support for the following additional options :—
83 - **`-C`**, when supplied with `-m`, must be useable to compile a
84 `.mgc` magicfile for use with future invocations of `file`.
86 - **`--files-from`** must be useable to provide a file that `file`
87 should read file·names from, and `-` must be useable in this
88 context to specify the standard input.
90 - **`--mime-type`** must cause `file` to print the internet media type
91 of the file with no charset parameter.
93 - **`--separator`** must be useable to set the separator that `file`
94 uses to separate file names from types.
96 These options are implemented by the
97 [Fine Free File Command](https://darwinsys.com/file/), which is used
98 by most operating systems.
102 This is not a Posix utility.
103 Usage of `git` is optional, but recommended (and activated by default).
104 To disable it, set `GIT=`.
108 This is a Posix utility, but it is considered an optional Software
109 Development utility and ⛩️📰 书社 currently depends on unspecified
111 ⛩️📰 书社 requires specifically the G·N·U version of `make`, and
112 depends on functionality present in version 3.81 or later.
113 It is not expected to work in previous versions, or with other
114 implementations of Make.
118 This is a Posix utility, but it is not included in the Linux Standard
119 Base or installed by default in many distributions.
120 ⛩️📰 书社 only requires support for the `ustar` format.
122 ### `uudecode` and `uuencode`
124 These are Posix utilities, but they were considered optional in
125 `POSIX.1-2001` (altho they are made mandatory in `POSIX.1-2008`) and
126 they are not included in the Linux Standard Base or installed by
127 default in many distributions.
128 The G·N·U [Sharutils](https://www.gnu.org/software/sharutils/) package
129 provides one implementation.
131 ### `xmlcatalog` and `xmllint`
133 These are not a Posix utilities.
134 They are a part of `libxml2`, but may need to be installed separately
135 on some platforms (e·g by the name `libxml2-utils`).
139 This is not a Posix utility.
140 It is a part of `libxslt`, but may need to be installed separately on
145 Place source files in `sources/` and run `make install` to compile
146 the result to `public/`.
147 Compilation involves the following steps :—
149 1. ⛩️📰 书社 compiles all of the magic files in `magic/` into a single
150 file, `build/magic.mgc`.
152 2. ⛩️📰 书社 processes all of the parsers in `parsers/` and determines
153 the list of supported plaintext types.
155 3. ⛩️📰 书社 identifies all of the source files and includes and uses
156 `build/magic.mgc` to classify them by media type.
158 4. ⛩️📰 书社 parses all plaintext and X·M·L source files and includes
159 and then builds a dependency tree between them.
161 5. ⛩️📰 书社 uses the dependency tree to establish prerequisites for
164 6. ⛩️📰 书社 compiles each output file to `build/result`.
166 7. ⛩️📰 书社 copies most output files from `build/result` to
167 `build/public`, but it does some additional processing instead on
168 those which indicate a non‐X·M·L desired final output form.
170 8. ⛩️📰 书社 copies the final resulting files to `public`.
172 You can use `make list` to list each identified source file or include
173 alongside its computed type and dependencies.
174 As this is a Make‐based program, steps will only be run if the
175 corresponding buildfile or output file is older than its
180 The ⛩️📰 书社 name·space is `urn:fdc:ladys.computer:20231231:Shu1She4`.
182 This document uses a few name·space prefixes, with the following
185 | Prefix | Expansion |
186 | ---------: | :-------------------------------------------- |
187 | `catalog:` | `urn:oasis:names:tc:entity:xmlns:xml:catalog` |
188 | `exsl:` | `http://exslt.org/common` |
189 | `exslstr:` | `http://exslt.org/strings` |
190 | `html:` | `http://www.w3.org/1999/xhtml` |
191 | `svg:` | `http://www.w3.org/2000/svg` |
192 | `xlink:` | `http://www.w3.org/1999/xlink` |
193 | `xslt:` | `http://www.w3.org/1999/XSL/Transform` |
194 | `书社:` | `urn:fdc:ladys.computer:20231231:Shu1She4` |
196 ## Setup and Configuration
198 ⛩️📰 书社 depends on the following programs to run.
199 In every case, you may supply your own implementation by overriding the
200 corresponding (allcaps) variable (e·g, set `MKDIR` to supply your own
201 `mkdir` implementation).
212 - `git` (optional; set `GIT=` to disable)
218 - `pax` (only when generating archives)
229 - `xmlcatalog` (provided by `libxml2`)
230 - `xmllint` (provided by `libxml2`)
231 - `xsltproc` (provided by `libxslt`)
233 The following additional variables can be used to control the behaviour
237 The location of the source files (default: `sources`).
238 Multiple source directories can be provided, so long as the same
239 file subpath doesn’t exist in more than one of them.
242 The location of source includes (default: `sources/includes`).
243 This can be inside of `SRCDIR`, but needn’t be.
244 Multiple include directories can be provided, so long as the same
245 file subpath doesn’t exist in more than one of them.
248 The location of the (temporary) build directory (default: `build`).
249 `make clean` will delete this, and it is recommended that it not be
250 used for programs aside from ⛩️📰 书社.
253 The location of directory to output files to (default: `public`).
254 `make install` will overwrite files in this directory which
255 correspond to those in `SRCDIR`.
256 It *will not* touch other files, including those generated from files
257 in `SRCDIR` which have since been deleted.
259 Files are first compiled to `$(BUILDDIR)/public` before they are
260 copied to `DESTDIR`, so this folder is relatively quick and
261 inexpensive to re·create.
262 It’s reasonable to simply delete it before every `make install` to
263 ensure stale content is removed.
266 The location of the ⛩️📰 书社 `GNUmakefile`.
267 This should be set automatically when calling Make and shouldn’t ever
268 need to be set manually.
269 This variable is used to find the ⛩️📰 书社 `lib/` folder, which is
270 expected to be in the same location.
273 A white·space‐separated list of magic files to use (default:
274 `$(THISDIR)/magic/*`).
277 The value of this variable is appended to `MAGIC` by default, to
278 enable additional magic files without overriding the existing ones.
281 Rules to use with `find` when searching for source files.
282 The default ignores files that start with a period or hyphen‐minus,
283 those which end with a cloparen, and those which contain a hash,
284 buck, percent, asterisk, colon, semi, eroteme, bracket, backslash,
286 It is important that these rules not produce any output, as anything
287 printed to `stdout` will be considered a result of the find.
289 - **`EXTRAFINDRULES`:**
290 The value of this variable is appended to `FINDRULES` by default, to
291 enable additional rules without overriding the existing ones.
293 - **`FINDINCLUDERULES`:**
294 Rules to use with `find` when searching for includes (default:
297 - **`EXTRAFINDINCLUDERULES`:**
298 The value of this variable is appended to `FINDINCLUDERULES` by
299 default, to enable additional rules without overriding the existing
303 A white·space‐separated list of parsers to use (default:
304 `$(THISDIR)/parsers/*.xslt`).
306 - **`EXTRAPARSERS`:**
307 The value of this variable is appended to `PARSERS` by default, to
308 enable additional parsers without overriding the existing ones.
311 A white·space‐separated list of transforms to use (default:
312 `$(THISDIR)/transforms/*.xslt`).
314 - **`EXTRATRANSFORMS`:**
315 The value of this variable is appended to `TRANSFORMS` by default, to
316 enable additional transforms without overriding the existing ones.
319 A white·space‐separated list of media types or media type suffixes to
320 consider X·M·L (default: `application/xml text/xml +xml`).
323 A program to run on (unspecial) X·M·L files after they are
324 transformed (default: `xmllint --nonet --nsclean`).
325 This variable can be used for postprocessing.
328 The current version of ⛩️📰 书社 (default: derived from the current
329 git tag/branch/commit).
332 The current version of the source files (default: derived from the
333 current git tag/branch/commit).
336 If this variable has a value, informative messages will not be
337 printed (default: empty).
338 Informative messages print to stderr, not stdout, so disabling them
339 usually shouldn’t be necessary.
340 This does not (cannot) disable messages from Make itself, for which
341 the `-s`, `--silent` ∕ `--quiet` Make option is more likely to be
345 If this variable has a value, every recipe instruction will be
346 printed when it runs (default: empty).
347 This is helpful for debugging, but typically too noisy for general
352 Source files may be placed in `SRCDIR` in any manner; the file
353 structure used there will match the output.
354 The type of source files is *not* determined by file extension, but
355 rather by magic number; this means that files **must** begin with
356 something recognizable.
357 Supported magic numbers include :—
359 - `<?xml` for `application/xml` files
360 - `#!js` for `text/javascript` files
361 - `@charset "` for `text/css` files
362 - `#!tsv` for `text/tab-separated-values` files
363 - `%%` for `text/record-jar` files (unregistered; see
364 [[draft-phillips-record-jar-01][]])
366 Text formats with associated X·S·L·T parsers are wrapped in a H·T·M·L
367 `<script>` element whose `@type` gives its media type, and then
368 passed to the parser to process.
369 Source files whose media type does not have an associated X·S·L·T
370 parser are considered “assets” and will not be transformed.
372 **☡ For compatibility with this program, source file·names must not
373 contain Ascii white·space, colons (`:`), semis (`;`), pipes (`|`),
374 bucks (`$`), percents (`%`), hashes (`#`), asterisks (`*`), brackets
375 (`[` or `]`), erotemes (`?`), backslashes (`\`), or control
376 characters, must not begin with a hyphen‐minus (`-`), and must not end
377 with a cloparen (`)`).**
378 The former characters have the potential to conflict with make syntax,
379 a leading hyphen‐minus is confusable for a commandline argument, and a
380 trailing cloparen [activates a bug in G·N·U Make
381 3.81](https://stackoverflow.com/questions/17148468/capturing-filenames-including-parentheses-with-gnu-makes-wildcard-function#comment24825307_17148894).
385 Parsers are used to convert plaintext files into X·M·L trees, as well
386 as convert plaintext formats which are already included inline in
387 existing source X·M·L documents.
388 ⛩️📰 书社 comes with some parsers; namely :—
390 - **`parsers/plain.xslt`:**
391 Wraps `text/plain` contents in a `<html:pre>` element.
393 - **`parsers/record-jar.xslt`:**
394 Converts `text/record-jar` contents into a `<html:div>` of
395 `<html:dl>` elements (one for each record).
397 - **`parsers/tsv.xslt`:**
398 Converts `text/tab-separated-values` contents into an `<html:table>`
401 New ⛩️📰 书社 parsers which target plaintext formats should have an
402 `<xslt:template>` element with no `@name` or `@mode` and whose
405 - Starts with an appropriately‐name·spaced qualified name for a
406 `<html:script>` element.
408 - Follows this with the string `[@type=`.
410 - Follows this with a quoted string giving a media type supported by
412 Media type parameters are *not* supported.
414 - Follows this with the string `]`.
416 For example, the trivial `text/plain` parser is defined as follows :—
419 <?xml version="1.0"?>
421 xmlns="http://www.w3.org/1999/XSL/Transform"
422 xmlns:html="http://www.w3.org/1999/xhtml"
423 xmlns:书社="urn:fdc:ladys.computer:20231231:Shu1She4"
426 <书社:id>example:text/plain</书社:id>
427 <template match="html:script[@type='text/plain']">
428 <html:pre><value-of select="."/></html:pre>
433 ⛩️📰 书社 will scan the provided parsers for this pattern to determine
434 the set of allowed plaintext file types.
435 Multiple such `<xslt:template>` elements may be provided in a single
436 parser, for example if the parser supports multiple media types.
437 Alternatively, you can set the `@书社:supported-media-types` attribute
438 on the root element of the parser to override media type support
441 Even when `@书社:supported-media-types` is set, it is a requirement
442 that each parser transform any `<html:script>` elements with a
443 `@type` which matches their registered types into something else.
444 Otherwise the parser will be stuck in an endless loop.
445 The result tree of applying the transform to the `<html:script>`
446 element will be reparsed (in case any new `<html:script>` elements
447 were added in its subtree), and a `@书社:parsed-by` attribute will be
448 added to each toplevel element in the result.
449 The value of this attribute will be the value of the `<书社:id>`
450 toplevel element in the parser.
452 It is possible for parsers to support zero plaintext types.
453 This is useful when targeting specific dialects of X·M·L; parsers in
454 this sense operate on the same basic principles as transforms
456 The major distinction between X·M·L parsers and transforms is where in
457 the process the transformation happens:
458 Parsers are applied *prior* to embedding (and can be used to generate
459 embeds); transforms are applied *after*.
461 It is **strongly recommended** that auxillary templates in parsers be
462 name·spaced (by `@name` or `@mode`) whenever possible, to avoid
463 conflicts between parsers.
465 ### Attributes added during parsing
467 ⛩️📰 书社 will add a few attributes to the output of the parsing step,
470 - A `@书社:cksum` attribute on toplevel result elements, giving the
471 `cksum` checksum of the corresponding source file.
473 - For the elements which result from parsing plaintext `<html:script>`
476 - A `@书社:parsed-by` attribute, giving a space‐separated list of
477 parsers which parsed the node.
478 (Generally, this will be a list of one, but it is possible for the
479 result of a parse to be another plaintext node, which may be
480 parsed by a different parser.)
482 - A `@书社:media-type` attribute, giving the identified media type of
487 Documents can be embedded in other documents using a `<书社:link>`
488 element with `@xlink:show="embed"`.
489 The `@xlink:href`s of these elements should have the format
490 `about:shushe?source=<path>`, where `<path>` provides the path to the
491 file within `SRCDIR`.
492 Includes, which do not generate outputs of their own but may still be
493 freely embedded, instead use the format
494 `about:shushe?include=<path>`, where `<path>` provides the path
497 Embeds are replaced with the parsed contents of a file, unless the file
498 is an asset, in which case an `<html:object>` element is produced
499 instead (with the contents of the asset file provided as a base64
501 Embed replacements will be given a `@书社:identifier` attribute whose
502 value will match the `@xlink:href` of the embed.
504 Embedding takes place after parsing but before transformation, so
505 parsers are able to generate their own embeds.
506 ⛩️📰 书社 is able to detect the transitive embed dependencies of files
507 and update them accordingly; it will signal an error if the
508 dependencies are recursive.
510 ## Output Redirection
512 By default, ⛩️📰 书社 installs files to the same location in `DESTDIR`
513 as they were placed in their `SRCDIR`.
514 This behaviour can be customized by setting the `@书社:destination`
515 attribute on the root element, whose value can give a different path.
516 This attribute is read after parsing, but before transformation (where
517 it is silently dropped).
521 Transforms are used to convert X·M·L files into their final output,
522 after all necessary parsing and embedding has taken place.
523 ⛩️📰 书社 comes with some transforms; namely :—
525 - **`transforms/asset.xslt`:**
526 Converts `<html:object>` elements which correspond to recognized
527 media types into the appropriate H·T·M·L elements, and deletes
528 `<html:style>` elements from the body of the document and moves
531 - **`transforms/metadata.xslt`:**
532 Provides basic `<html:head>` metadata.
533 This metadata is generated from `<html:meta>` elements with one of
534 the following `@itemprop` attributes :—
536 - **`urn:fdc:ladys.computer:20231231:Shu1She4:title`:**
537 Provides the title of the page.
539 ⛩️📰 书社 automatically encapsulates H·T·M·L embeds so that their
540 metadata does not propogate up to the embedding document.
541 To undo this behaviour, remove the `@itemscope` and `@itemtype`
542 attributes from the embed during the transformation phase.
544 - **`transforms/serialization.xslt`:**
545 Replaces `<书社:serialize-xml>` elements with the (escaped)
546 serialized X·M·L of their contents.
547 This replacement happens during the application phase, after most
548 other transformations have taken place.
550 If a `@with-namespaces` attribute is provided, any name·space nodes
551 on the toplevel serialized elements whose U·R·I’s correspond to the
552 definitions of the provided prefixes, as defined for the
553 `<书社:serialize-xml>` element, will be declared using name·space
554 attributes on the serialized elements.
555 Otherwise, only name·space nodes which _differ_ from the definitions
556 on the `<书社:serialize-xml>` element will be declared.
557 The string `#default` may be used to represent the default
559 Multiple prefixes may be provided, separated by white·space.
561 When it comes to name·spaces used internally by ⛩️📰 书社, the
562 prefix used by ⛩️📰 书社 may be declared _in addition to_ the
563 prefix(es) used in the source document(s).
564 It is not possible to selectively only declare one prefix for a
565 name·space to the exclusion of others.
567 `<书社:raw-output>` elements may be used inside of
568 `<书社:serialize-xml>` elements to inject raw output into the
571 The following are recommendations on effective creation of
574 - Make template matchers as specific as possible.
575 It is likely an error if two transforms have templates which match
576 the same element (unless the templates have different priority).
578 - Name·space templates (with `@name` or `@mode`) whenever possible.
580 - Set `@exclude-result-prefixes` on the root `xslt:transform` element
581 to reduce the number of declared name·spaces in the final result.
585 The following params are made available globally in parsers and
592 The checksum of the source file (⅌ `cksum`).
595 The ⛩️📰 书社 identifier of the source file (a u·r·i beginning with
599 The value of the `SRCREV` variable (if present).
602 The time at which the source file was last modified.
605 The value of the `THISREV` variable (if present).
607 The following params are only available in transforms :—
610 The path of the catalog file (within `BUILDDIR`).
613 The path of the output file (within `DESTDIR`).
617 ⛩️📰 书社 will wrap the final output of the transforms in appropriate
618 `<html:html>` and `<html:body>` elements, so it is not necessary for
619 transforms to do this explicitly.
620 After performing the initial transform, ⛩️📰 书社 will match the root
621 node of the result in the following modes to fill in areas of the
625 The result of matching in this mode is prepended into the
626 `<html:body>` of the output (before the transformation result).
629 The result of matching in this mode is appended into the
630 `<html:body>` of the output (after the transformation result).
633 The result of matching in this mode is inserted into the
634 `<html:head>` of the output.
636 In addition to being called with the transform result, each of these
637 modes will additionally be called with a `<xslt:include>` element
638 corresponding to each transform.
639 If a transform has a `<书社:id>` top‐level element whose value is an
640 i·r·i, its `<xslt:include>` element will have a corresponding
642 This mechanism can be used to allow transforms to insert content
643 without matching any elements in the result; for example, the
644 following transform adds a link to a stylesheet to the `<html:head>`
648 <?xml version="1.0"?>
650 xmlns="http://www.w3.org/1999/XSL/Transform"
651 xmlns:html="http://www.w3.org/1999/xhtml"
652 xmlns:xslt="http://www.w3.org/1999/XSL/Transform"
653 xmlns:书社="urn:fdc:ladys.computer:20231231:Shu1She4"
654 exclude-result-prefixes="书社"
657 <书社:id>example:add-stylesheet-links.xslt</书社:id>
658 <template match="xslt:include[@书社:id='example:add-stylesheet-links.xslt']" mode="书社:metadata">
659 <html:link rel="stylesheet" type="text/css" href="/style.css"/>
664 Output wrapping can be entirely disabled by adding a
665 `@书社:disable-output-wrapping` attribute to the top‐level element in
667 It will not be performed on outputs whose root elements are
668 `<书社:archive>`, `<书社:base64-binary>`, or `<书社:raw-text>`
671 ## Applying Attributes
673 The `<书社:apply-attributes>` element will apply any attributes on the
674 element to the element(s) it wraps.
675 It is especially useful in combination with embeds.
677 The `<书社:apply-attributes-to-root>` element will apply any attributes
678 on the element to the root node of the final transformation result.
679 It is especially useful in combination with output wrapping.
681 In both cases, attributes from various sources are combined with
682 white·space between them.
683 Attribute application takes place after all ordinary transforms have
686 Both elements ignore attributes in the `xml:` name·space, except for
687 `@xml:lang`, which ignores all but the first definition (including
688 any already present on the root element).
689 On H·T·M·L and S·V·G elements, `@lang` has the same behaviour as
692 ## Other Kinds of Output
694 There are a few special elements in the `书社:` name·space which, if
695 they appear as the toplevel element in a transformation result, cause
696 ⛩️📰 书社 to produce something other than an X·M·L file.
699 - **`<书社:archive>`:**
700 Each child element with a `@书社:archived-as` attribute will be
701 archived as a separate file in a resulting tarball (this attribute
702 gives the file name).
703 These elements will be processed the same as the root elements of any
704 other file (e·g, they will be wrapped; they can themselves specify
705 non X·M·L output types, ⁊·c).
706 Other child elements will be ignored.
708 If the `<书社:archive>` element is given an `@书社:expanded`
709 attribute, rather than producing a tarball ⛩️📰 书社 will output
710 the directory which expanding the tarball would produce.
711 This mechanism can be used to generate multiple files from a single
712 source, provided all of the files are contained with·in the same
715 - **`<书社:base64-binary>`:**
716 The text nodes in the transformation result will, after removing all
717 Ascii whitespace, be treated as a Base·64 string, which is then
720 - **`<书社:raw-text>`:**
721 A plaintext (U·T·F‐8) file will be produced from the text nodes in
722 the transformation result.
726 This repository conforms to [REUSE][].
728 Most source files are licensed under the terms of the <cite>Mozilla
729 Public License, version 2.0</cite>.
731 [REUSE]: <https://reuse.software/spec/>
732 [draft-phillips-record-jar-01]: <https://datatracker.ietf.org/doc/html/draft-phillips-record-jar-01>
733 [rdar-92753335]: <https://github.com/apple-oss-distributions/patch_cmds/blob/5084833f90df1b0e0924ea56f94c0199b3b8bbc6/diff/diffreg.c#L1800-L1808>