+<!--
+SPDX-FileCopyrightText: 2024 Lady <https://www.ladys.computer/about/#lady>
+SPDX-License-Identifier: CC0-1.0
+-->
# ⛩️📰 书社
-<b>An X·S·L·T‐based static site generator.</b>
+<b>A make·file for X·M·L.</b>
<dfn>⛩️📰 书社</dfn> aims to make it easy to generate websites with
X·S·L·T and G·N·U Make.
- Enabling easy inclusion of source files within each other.
It aims to do this with zero dependencies beyond the programs already
- installed on your computer.
+ installed on your computer†.
+
+† Assuming an operating system with a fairly featureful, and
+ Posix‐compliant, development setup (e·g, Macintosh ≥ version 10.8).
+In fact, on Linux you will probably need to install a few programs:
+ `libxml2-utils`, `xsltproc`, `sharutils`, and `pax`.
## Nomenclature
In Ascii environments, ⛩️📰 书社 should be written `Shushe`, following
the pinyin transliteration.
+## Prerequisites
+
+In most cases, ⛩️📰 书社 aims to require only functionality which is
+ present in all Posix‐compliant (`POSIX.1-2001`) operating systems.
+There are a few exceptions.
+Details on particular programs are given below; if a program is not
+ listed, it is assumed that any Posix‐compliant implementation will
+ work.
+
+### `diff`
+
+This is a Posix utility, but ⛩️📰 书社 depends on functionality
+ introduced after `POSIX.1-2001` (the `-u` option, introduced in
+ `POSIX.1-2008`).
+Macintosh systems somewhat interestingly implement this option
+ correctly in legacy mode (`COMMAND_MODE=legacy`) but incorrectly by
+ default (despite claiming `POSIX.1-2008` conformance for this
+ utility).
+[Note this erroneous comment claiming nanosecond & timezone are
+ extensions rather than standardized.][rdar-92753335]
+Despite this, the default Macintosh implementation will still work with
+ ⛩️📰 书社, with the caveat that the timestamp will only include a
+ fractional component when a Posix⹀compliant (e·g, Macintosh legacy or
+ G·N·U) implementation is used.
+
+### `file`
+
+This is a Posix utility, but it was considered optional in
+ `POSIX.1-2001` (altho it was made mandatory in `POSIX.1-2008`) and
+ ⛩️📰 书社 currently depends on unspecified behaviour.
+It requires support for the following additional options :—
+
+- **`-C`**, when supplied with `-m`, must be useable to compile a
+ `.mgc` magicfile for use with future invocations of `file`.
+
+- **`--files-from`** must be useable to provide a file that `file`
+ should read file·names from, and `-` must be useable in this
+ context to specify the standard input.
+
+- **`--mime-type`** must cause `file` to print the internet media type
+ of the file with no charset parameter.
+
+- **`--separator`** must be useable to set the separator that `file`
+ uses to separate file names from types.
+
+These options are implemented by the
+ [Fine Free File Command](https://darwinsys.com/file/), which is used
+ by most operating systems.
+
+### `git`
+
+This is not a Posix utility.
+Usage of `git` is optional, but recommended (and activated by default).
+To disable it, set `GIT=`.
+
+### `make`
+
+This is a Posix utility, but it is considered an optional Software
+ Development utility and ⛩️📰 书社 currently depends on unspecified
+ behaviour.
+⛩️📰 书社 requires specifically the G·N·U version of `make`, and
+ depends on functionality present in version 3.81 or later.
+It is not expected to work in previous versions, or with other
+ implementations of Make.
+
+### `pax`
+
+This is a Posix utility, but it is not included in the Linux Standard
+ Base or installed by default in many distributions.
+⛩️📰 书社 only requires support for the `ustar` format.
+
+### `uudecode` and `uuencode`
+
+These are Posix utilities, but they were considered optional in
+ `POSIX.1-2001` (altho they are made mandatory in `POSIX.1-2008`) and
+ they are not included in the Linux Standard Base or installed by
+ default in many distributions.
+The G·N·U [Sharutils](https://www.gnu.org/software/sharutils/) package
+ provides one implementation.
+
+### `xmlcatalog` and `xmllint`
+
+These are not a Posix utilities.
+They are a part of `libxml2`, but may need to be installed separately
+ on some platforms (e·g by the name `libxml2-utils`).
+
+### `xsltproc`
+
+This is not a Posix utility.
+It is a part of `libxslt`, but may need to be installed separately on
+ some platforms.
+
## Basic Usage
Place source files in `sources/` and run `make install` to compile
5. ⛩️📰 书社 uses the dependency tree to establish prerequisites for
each output file.
-6. ⛩️📰 书社 compiles each output file to `build/public`.
+6. ⛩️📰 书社 compiles each output file to `build/result`.
+
+7. ⛩️📰 书社 copies most output files from `build/result` to
+ `build/public`, but it does some additional processing instead on
+ those which indicate a non‐X·M·L desired final output form.
-7. ⛩️📰 书社 copies the output files to `public`.
+8. ⛩️📰 书社 copies the final resulting files to `public`.
You can use `make list` to list each identified source file or include
alongside its computed type and dependencies.
corresponding buildfile or output file is older than its
prerequisites.
-## Namespaces
+## Name·spaces
-The ⛩️📰 书社 namespace is `urn:fdc:ladys.computer:20231231:Shu1She4`.
+The ⛩️📰 书社 name·space is `urn:fdc:ladys.computer:20231231:Shu1She4`.
-This document uses a few namespace prefixes, with the following
+This document uses a few name·space prefixes, with the following
meanings :—
-| Prefix | Expansion |
-| -------: | :----------------------------------------- |
-| `html:` | `http://www.w3.org/1999/xhtml` |
-| `xlink:` | `http://www.w3.org/1999/xlink` |
-| `xslt:` | `http://www.w3.org/1999/XSL/Transform` |
-| `书社:` | `urn:fdc:ladys.computer:20231231:Shu1She4` |
+| Prefix | Expansion |
+| ---------: | :-------------------------------------------- |
+| `catalog:` | `urn:oasis:names:tc:entity:xmlns:xml:catalog` |
+| `exsl:` | `http://exslt.org/common` |
+| `exslstr:` | `http://exslt.org/strings` |
+| `html:` | `http://www.w3.org/1999/xhtml` |
+| `svg:` | `http://www.w3.org/2000/svg` |
+| `xlink:` | `http://www.w3.org/1999/xlink` |
+| `xslt:` | `http://www.w3.org/1999/XSL/Transform` |
+| `书社:` | `urn:fdc:ladys.computer:20231231:Shu1She4` |
## Setup and Configuration
corresponding (allcaps) variable (e·g, set `MKDIR` to supply your own
`mkdir` implementation).
+- `awk`
- `cat`
+- `cd`
+- `cksum`
- `cp`
- `date`
-- `echo`
+- `diff`
- `file`
- `find`
-- `mkdir` (requires support for `-p`)
+- `git` (optional; set `GIT=` to disable)
+- `grep`
+- `ln`
+- `mkdir`
- `mv`
+- `od`
+- `pax` (only when generating archives)
- `printf`
- `rm`
- `sed`
- `sleep`
-- `stat`
- `test`
- `touch`
-- `tr` (requires support for `-d`)
-- `uuencode` (requires support for `-m` and `-r`)
+- `tr`
+- `uuencode`
+- `uudecode`
+- `xargs`
- `xmlcatalog` (provided by `libxml2`)
- `xmllint` (provided by `libxml2`)
- `xsltproc` (provided by `libxslt`)
- **`SRCDIR`:**
The location of the source files (default: `sources`).
+ Multiple source directories can be provided, so long as the same
+ file subpath doesn’t exist in more than one of them.
- **`INCLUDEDIR`:**
- The location of the source files (default: `sources/includes`).
+ The location of source includes (default: `sources/includes`).
This can be inside of `SRCDIR`, but needn’t be.
+ Multiple include directories can be provided, so long as the same
+ file subpath doesn’t exist in more than one of them.
- **`BUILDDIR`:**
The location of the (temporary) build directory (default: `build`).
+ `make clean` will delete this, and it is recommended that it not be
+ used for programs aside from ⛩️📰 书社.
- **`DESTDIR`:**
The location of directory to output files to (default: `public`).
+ `make install` will overwrite files in this directory which
+ correspond to those in `SRCDIR`.
+ It *will not* touch other files, including those generated from files
+ in `SRCDIR` which have since been deleted.
+
+ Files are first compiled to `$(BUILDDIR)/public` before they are
+ copied to `DESTDIR`, so this folder is relatively quick and
+ inexpensive to re·create.
+ It’s reasonable to simply delete it before every `make install` to
+ ensure stale content is removed.
- **`THISDIR`:**
The location of the ⛩️📰 书社 `GNUmakefile`.
This variable is used to find the ⛩️📰 书社 `lib/` folder, which is
expected to be in the same location.
-- **`MAGICDIR`:**
- The location of the magic files to use (default: `$(THISDIR)/magic`).
+- **`MAGIC`:**
+ A white·space‐separated list of magic files to use (default:
+ `$(THISDIR)/magic/*`).
-- **`FINDOPTS`:**
- Options to pass to `find` when searching for source files (default:
- `-LE`).
+- **`EXTRAMAGIC`:**
+ The value of this variable is appended to `MAGIC` by default, to
+ enable additional magic files without overriding the existing ones.
- **`FINDRULES`:**
- Rules to use with `find` when searching for source files (default:
- `-flags -nohidden -and -not -name '.*'`).
+ Rules to use with `find` when searching for source files.
+ The default ignores files that start with a period or hyphen‐minus,
+ those which end with a cloparen, and those which contain a hash,
+ buck, percent, asterisk, colon, semi, eroteme, bracket, backslash,
+ or pipe.
+ It is important that these rules not produce any output, as anything
+ printed to `stdout` will be considered a result of the find.
+
+- **`EXTRAFINDRULES`:**
+ The value of this variable is appended to `FINDRULES` by default, to
+ enable additional rules without overriding the existing ones.
+
+- **`FINDINCLUDERULES`:**
+ Rules to use with `find` when searching for includes (default:
+ `$(FINDRULES)`).
+
+- **`EXTRAFINDINCLUDERULES`:**
+ The value of this variable is appended to `FINDINCLUDERULES` by
+ default, to enable additional rules without overriding the existing
+ ones.
- **`PARSERS`:**
A white·space‐separated list of parsers to use (default:
`$(THISDIR)/parsers/*.xslt`).
+- **`EXTRAPARSERS`:**
+ The value of this variable is appended to `PARSERS` by default, to
+ enable additional parsers without overriding the existing ones.
+
- **`TRANSFORMS`:**
A white·space‐separated list of transforms to use (default:
`$(THISDIR)/transforms/*.xslt`).
+- **`EXTRATRANSFORMS`:**
+ The value of this variable is appended to `TRANSFORMS` by default, to
+ enable additional transforms without overriding the existing ones.
+
- **`XMLTYPES`:**
- A white·space‐separated list of media types to consider X·M·L
- (default: `application/xml text/xml`).
+ A white·space‐separated list of media types or media type suffixes to
+ consider X·M·L (default: `application/xml text/xml +xml`).
+
+- **`FINALIZE`:**
+ A program to run on (unspecial) X·M·L files after they are
+ transformed (default: `xmllint --nonet --nsclean`).
+ This variable can be used for postprocessing.
+
+- **`THISREV`:**
+ The current version of ⛩️📰 书社 (default: derived from the current
+ git tag/branch/commit).
+
+- **`SRCREV`:**
+ The current version of the source files (default: derived from the
+ current git tag/branch/commit).
+
+- **`QUIET`:**
+ If this variable has a value, informative messages will not be
+ printed (default: empty).
+ Informative messages print to stderr, not stdout, so disabling them
+ usually shouldn’t be necessary.
+ This does not (cannot) disable messages from Make itself, for which
+ the `-s`, `--silent` ∕ `--quiet` Make option is more likely to be
+ useful.
- **`VERBOSE`:**
If this variable has a value, every recipe instruction will be
Source files whose media type does not have an associated X·S·L·T
parser are considered “assets” and will not be transformed.
-For compatibility with this program, source filenames should conform to
- the following rules :—
-
-- They should not start with a hyphen‐minus.
- This is to prevent confusion between filenames and options on the
- commandline.
-
-- They should not contain spaces, colons, percent signs, backticks,
- question marks, hashes, or backslashes.
-
-In general, filenames should be such that they do not require
- percent‐encoding in the path component of an i·r·i.
+**☡ For compatibility with this program, source file·names must not
+ contain Ascii white·space, colons (`:`), semis (`;`), pipes (`|`),
+ bucks (`$`), percents (`%`), hashes (`#`), asterisks (`*`), brackets
+ (`[` or `]`), erotemes (`?`), backslashes (`\`), or control
+ characters, must not begin with a hyphen‐minus (`-`), and must not end
+ with a cloparen (`)`).**
+The former characters have the potential to conflict with make syntax,
+ a leading hyphen‐minus is confusable for a commandline argument, and a
+ trailing cloparen [activates a bug in G·N·U Make
+ 3.81](https://stackoverflow.com/questions/17148468/capturing-filenames-including-parentheses-with-gnu-makes-wildcard-function#comment24825307_17148894).
## Parsers
- **`parsers/plain.xslt`:**
Wraps `text/plain` contents in a `<html:pre>` element.
+- **`parsers/record-jar.xslt`:**
+ Converts `text/record-jar` contents into a `<html:div>` of
+ `<html:dl>` elements (one for each record).
+
- **`parsers/tsv.xslt`:**
Converts `text/tab-separated-values` contents into an `<html:table>`
element.
-New ⛩️📰 书社 parsers should have a `<xslt:template>` element with no
- `@name` or `@mode` and whose `@match` attribute…
+New ⛩️📰 书社 parsers which target plaintext formats should have an
+ `<xslt:template>` element with no `@name` or `@mode` and whose
+ `@match` attribute…
-- Starts with an appropriately‐namespaced qualified name for a
+- Starts with an appropriately‐name·spaced qualified name for a
`<html:script>` element.
- Follows this with the string `[@type=`.
<transform
xmlns="http://www.w3.org/1999/XSL/Transform"
xmlns:html="http://www.w3.org/1999/xhtml"
+ xmlns:书社="urn:fdc:ladys.computer:20231231:Shu1She4"
version="1.0"
>
+ <书社:id>example:text/plain</书社:id>
<template match="html:script[@type='text/plain']">
<html:pre><value-of select="."/></html:pre>
</template>
on the root element of the parser to override media type support
detection.
-It is **strongly recommended** that all templates in parsers other than
- those described above be namespaced (by `@name` or `@mode`), to avoid
- conflicts between templates in multiple parsers.
+Even when `@书社:supported-media-types` is set, it is a requirement
+ that each parser transform any `<html:script>` elements with a
+ `@type` which matches their registered types into something else.
+Otherwise the parser will be stuck in an endless loop.
+The result tree of applying the transform to the `<html:script>`
+ element will be reparsed (in case any new `<html:script>` elements
+ were added in its subtree), and a `@书社:parsed-by` attribute will be
+ added to each toplevel element in the result.
+The value of this attribute will be the value of the `<书社:id>`
+ toplevel element in the parser.
+
+It is possible for parsers to support zero plaintext types.
+This is useful when targeting specific dialects of X·M·L; parsers in
+ this sense operate on the same basic principles as transforms
+ (described below).
+The major distinction between X·M·L parsers and transforms is where in
+ the process the transformation happens:
+Parsers are applied *prior* to embedding (and can be used to generate
+ embeds); transforms are applied *after*.
+
+It is **strongly recommended** that auxillary templates in parsers be
+ name·spaced (by `@name` or `@mode`) whenever possible, to avoid
+ conflicts between parsers.
+
+### Attributes added during parsing
+
+⛩️📰 书社 will add a few attributes to elements which result from
+ parsing plaintext `<html:script>` elements.
+These include :—
+
+- A `@书社:parsed-by` attribute, giving a space‐separated list of
+ parsers which parsed the node.
+ (Generally, this will be a list of one, but it is possible for the
+ result of a parse to be another plaintext node, which may be parsed
+ by a different parser.)
+
+- A `@书社:media-type` attribute, giving the identified media type of
+ the plaintext node.
+
+## Output Redirection
+
+By default, ⛩️📰 书社 installs files to the same location in `DESTDIR`
+ as they were placed in their `SRCDIR`.
+This behaviour can be customized by setting the `@书社:destination`
+ attribute on the root element, whose value can give a different path.
+This attribute is read after parsing, but before transformation (where
+ it is silently dropped).
## Embedding
is an asset, in which case an `<html:object>` element is produced
instead (with the contents of the asset file provided as a base64
`data:` u·r·i).
+Embed replacements will be given a `@书社:identifier` attribute whose
+ value will match the `@xlink:href` of the embed.
Embedding takes place after parsing but before transformation, so
parsers are able to generate their own embeds.
and update them accordingly; it will signal an error if the
dependencies are recursive.
+### Attributes added during expansion
+
+⛩️📰 书社 will add a few attributes to toplevel result elements, both
+ in the main document and any embedded documents, during the expansion
+ phase prior to the main transformation.
+These include :—
+
+- A `@书社:cksum` attribute giving the `cksum` checksum of the
+ corresponding source file.
+
+- A `@书社:mtime` attribute giving the last modified time of the
+ corresponding source file.
+
+- A `@书社:identifier` attribute giving the ⛩️📰 书社 identifier
+ (i·e, starting with `about:shushe?`) of the corresponding source
+ file.
+
+- For elements in the `html` namespace, an `itemscope` attribute and an
+ `itemtype` attribute with a value of
+ `urn:fdc:ladys.computer:20231231:Shu1She4:document` (for the main
+ document) or `urn:fdc:ladys.computer:20231231:Shu1She4:embed` (for
+ embedded documents).
+ These attributes are used to scope any nested `<html:meta>` elements
+ with `@itemprop` attributes to their containing documents.
+
## Transforms
Transforms are used to convert X·M·L files into their final output,
- **`transforms/metadata.xslt`:**
Provides basic `<html:head>` metadata.
- This metadata is generated from `<html:meta>` elements with one o.
+ This metadata is generated from `<html:meta>` elements with one of
the following `@itemprop` attributes :—
- **`urn:fdc:ladys.computer:20231231:Shu1She4:title`:**
Provides the title of the page.
- ⛩️📰 书社 automatically encapsulates embeds so that their metadata
- does not propogate up to the embedding document.
+ ⛩️📰 书社 automatically encapsulates H·T·M·L embeds so that their
+ metadata does not propogate up to the embedding document.
To undo this behaviour, remove the `@itemscope` and `@itemtype`
attributes from the embed during the transformation phase.
+- **`transforms/serialization.xslt`:**
+ Replaces `<书社:serialize-xml>` elements with the (escaped)
+ serialized X·M·L of their contents.
+ This replacement happens during the application phase, after most
+ other transformations have taken place.
+
+ If a `@with-namespaces` attribute is provided, any name·space nodes
+ on the toplevel serialized elements whose U·R·I’s correspond to the
+ definitions of the provided prefixes, as defined for the
+ `<书社:serialize-xml>` element, will be declared using name·space
+ attributes on the serialized elements.
+ Otherwise, only name·space nodes which _differ_ from the definitions
+ on the `<书社:serialize-xml>` element will be declared.
+ The string `#default` may be used to represent the default
+ name·space.
+ Multiple prefixes may be provided, separated by white·space.
+
+ When it comes to name·spaces used internally by ⛩️📰 书社, the
+ prefix used by ⛩️📰 书社 may be declared _in addition to_ the
+ prefix(es) used in the source document(s).
+ It is not possible to selectively only declare one prefix for a
+ name·space to the exclusion of others.
+
+ `<书社:raw-output>` elements may be used inside of
+ `<书社:serialize-xml>` elements to inject raw output into the
+ serialized X·M·L.
+
The following are recommendations on effective creation of
transforms :—
It is likely an error if two transforms have templates which match
the same element (unless the templates have different priority).
-- Namespace templates (with `@name` or `@mode`) whenever possible.
+- Name·space templates (with `@name` or `@mode`) whenever possible.
- Set `@exclude-result-prefixes` on the root `xslt:transform` element
- to reduce the number of declared namespaces in the final result.
+ to reduce the number of declared name·spaces in the final result.
+
+## Global Params
+
+The following params are made available globally in parsers and
+ transforms :—
-The params `$buildtime`, `$srctime`, and `$path` are available within
- transforms and are initialized to the current time, the time that the
- source file was last modified, and the path of the output file within
- $(DESTDIR).
+- **`BUILDTIME`:**
+ The current time.
+
+- **`IDENTIFIER`:**
+ The ⛩️📰 书社 identifier of the source file (a u·r·i beginning with
+ `about:shushe`).
+
+- **`SRCREV`:**
+ The value of the `SRCREV` variable (if present).
+
+- **`THISREV`:**
+ The value of the `THISREV` variable (if present).
+
+The following params are only available in transforms :—
+
+- **`PATH`:**
+ The path of the output file (within `DESTDIR`).
## Output Wrapping
modes will additionally be called with a `<xslt:include>` element
corresponding to each transform.
If a transform has a `<书社:id>` top‐level element whose value is an
- i·r·i, its `<xslt:import>` element will have a corresponding
+ i·r·i, its `<xslt:include>` element will have a corresponding
`@书社:id` attribute.
This mechanism can be used to allow transforms to insert content
without matching any elements in the result; for example, the
Output wrapping can be entirely disabled by adding a
`@书社:disable-output-wrapping` attribute to the top‐level element in
the result tree.
+It will not be performed on outputs whose root elements are
+ `<书社:archive>`, `<书社:base64-binary>`, or `<书社:raw-text>`
+ (described below).
+
+## Applying Attributes
+
+The `<书社:apply-attributes>` element will apply any attributes on the
+ element to the element(s) it wraps.
+It is especially useful in combination with embeds.
+
+The `<书社:apply-attributes-to-root>` element will apply any attributes
+ on the element to the root node of the final transformation result.
+It is especially useful in combination with output wrapping.
+
+In both cases, attributes from various sources are combined with
+ white·space between them.
+Attribute application takes place after all ordinary transforms have
+ completed.
+
+Both elements ignore attributes in the `xml:` name·space, except for
+ `@xml:lang`, which ignores all but the first definition (including
+ any already present on the root element).
+On H·T·M·L and S·V·G elements, `@lang` has the same behaviour as
+ `@xml:lang`.
+
+## Other Kinds of Output
+
+There are a few special elements in the `书社:` name·space which, if
+ they appear as the toplevel element in a transformation result, cause
+ ⛩️📰 书社 to produce something other than an X·M·L file.
+They are :—
+
+- **`<书社:archive>`:**
+ Each child element with a `@书社:archived-as` attribute will be
+ archived as a separate file in a resulting tarball (this attribute
+ gives the file name).
+ These elements will be processed the same as the root elements of any
+ other file (e·g, they will be wrapped; they can themselves specify
+ non X·M·L output types, ⁊·c).
+ Other child elements will be ignored.
+
+ If the `<书社:archive>` element is given an `@书社:expanded`
+ attribute, rather than producing a tarball ⛩️📰 书社 will output
+ the directory which expanding the tarball would produce.
+ This mechanism can be used to generate multiple files from a single
+ source, provided all of the files are contained with·in the same
+ directory.
+
+- **`<书社:base64-binary>`:**
+ The text nodes in the transformation result will, after removing all
+ Ascii whitespace, be treated as a Base·64 string, which is then
+ decoded.
+
+- **`<书社:raw-text>`:**
+ A plaintext (U·T·F‐8) file will be produced from the text nodes in
+ the transformation result.
## License
-Source files are licensed under the terms of the <cite>Mozilla Public
- License, version 2.0</cite>.
-For more information, see [LICENSE](./LICENSE).
+This repository conforms to [REUSE][].
+
+Most source files are licensed under the terms of the <cite>Mozilla
+ Public License, version 2.0</cite>.
-[draft-phillips-record-jar-01]: <https://datatracker.ietf.org/doc/html/draft-phillips-record-jar-01>
\ No newline at end of file
+[REUSE]: <https://reuse.software/spec/>
+[draft-phillips-record-jar-01]: <https://datatracker.ietf.org/doc/html/draft-phillips-record-jar-01>
+[rdar-92753335]: <https://github.com/apple-oss-distributions/patch_cmds/blob/5084833f90df1b0e0924ea56f94c0199b3b8bbc6/diff/diffreg.c#L1800-L1808>
Lady’s Gitweb / Shushe / blobdiff