<!--
-SPDX-FileCopyrightText: 2024 Lady <https://www.ladys.computer/about/#lady>
+SPDX-FileCopyrightText: 2024, 2025 Lady <https://www.ladys.computer/about/#lady>
SPDX-License-Identifier: CC0-1.0
-->
-# ⛩️📰 书社
+# ⛩📰 书社
<b>A make·file for X·M·L.</b>
-<dfn>⛩️📰 书社</dfn> aims to make it easy to generate websites with
+<dfn>⛩📰 书社</dfn> aims to make it easy to generate websites with
X·S·L·T and G·N·U Make.
It is consequently only a good choice for people who like X·S·L·T and
G·N·U Make and wish it were easier to make websites with them.
The name <i lang="cmn-Hans">书社</i> was chosen to play on this pun, as
it is intended as a publishing program for webshrines.
-In Ascii environments, ⛩️📰 书社 should be written `Shushe`, following
+In Ascii environments, ⛩📰 书社 should be written `Shushe`, following
the pinyin transliteration.
## Prerequisites
-In most cases, ⛩️📰 书社 aims to require only functionality which is
+In most cases, ⛩📰 书社 aims to require only functionality which is
present in all Posix‐compliant (`POSIX.1-2001`) operating systems.
There are a few exceptions.
Details on particular programs are given below; if a program is not
### `diff`
-This is a Posix utility, but ⛩️📰 书社 depends on functionality
+This is a Posix utility, but ⛩📰 书社 depends on functionality
introduced after `POSIX.1-2001` (the `-u` option, introduced in
`POSIX.1-2008`).
Macintosh systems somewhat interestingly implement this option
[Note this erroneous comment claiming nanosecond & timezone are
extensions rather than standardized.][rdar-92753335]
Despite this, the default Macintosh implementation will still work with
- ⛩️📰 书社, with the caveat that the timestamp will only include a
+ ⛩📰 书社, with the caveat that the timestamp will only include a
fractional component when a Posix⹀compliant (e·g, Macintosh legacy or
G·N·U) implementation is used.
This is a Posix utility, but it was considered optional in
`POSIX.1-2001` (altho it was made mandatory in `POSIX.1-2008`) and
- ⛩️📰 书社 currently depends on unspecified behaviour.
+ ⛩📰 书社 currently depends on unspecified behaviour.
It requires support for the following additional options :—
- **`-C`**, when supplied with `-m`, must be useable to compile a
### `make`
This is a Posix utility, but it is considered an optional Software
- Development utility and ⛩️📰 书社 currently depends on unspecified
+ Development utility and ⛩📰 书社 currently depends on unspecified
behaviour.
-⛩️📰 书社 requires specifically the G·N·U version of `make`, and
+⛩📰 书社 requires specifically the G·N·U version of `make`, and
depends on functionality present in version 3.81 or later.
It is not expected to work in previous versions, or with other
implementations of Make.
This is a Posix utility, but it is not included in the Linux Standard
Base or installed by default in many distributions.
-⛩️📰 书社 only requires support for the `ustar` format.
+⛩📰 书社 only requires support for the `ustar` format.
### `uudecode` and `uuencode`
the result to `public/`.
Compilation involves the following steps :—
-1. ⛩️📰 书社 compiles all of the magic files in `magic/` into a single
+1. ⛩📰 书社 compiles all of the magic files in `magic/` into a single
file, `build/magic.mgc`.
-2. ⛩️📰 书社 processes all of the parsers in `parsers/` and determines
+2. ⛩📰 书社 processes all of the parsers in `parsers/` and determines
the list of supported plaintext types.
-3. ⛩️📰 书社 identifies all of the source files and includes and uses
+3. ⛩📰 书社 identifies all of the source files and includes and uses
`build/magic.mgc` to classify them by media type.
-4. ⛩️📰 书社 parses all plaintext and X·M·L source files and includes
+4. ⛩📰 书社 parses all plaintext and X·M·L source files and includes
and then builds a dependency tree between them.
-5. ⛩️📰 书社 uses the dependency tree to establish prerequisites for
+5. ⛩📰 书社 uses the dependency tree to establish prerequisites for
each output file.
-6. ⛩️📰 书社 compiles each output file to `build/result`.
+6. ⛩📰 书社 compiles each output file to `build/result`.
-7. ⛩️📰 书社 copies most output files from `build/result` to
+7. ⛩📰 书社 copies most output files from `build/result` to
`build/public`, but it does some additional processing instead on
those which indicate a non‐X·M·L desired final output form.
-8. ⛩️📰 书社 copies the final resulting files to `public`.
+8. ⛩📰 书社 copies the final resulting files to `public`.
You can use `make list` to list each identified source file or include
alongside its computed type and dependencies.
## Name·spaces
-The ⛩️📰 书社 name·space is `urn:fdc:ladys.computer:20231231:Shu1She4`.
+The ⛩📰 书社 name·space is `urn:fdc:ladys.computer:20231231:Shu1She4`.
This document uses a few name·space prefixes, with the following
meanings :—
| `exsl:` | `http://exslt.org/common` |
| `exslstr:` | `http://exslt.org/strings` |
| `html:` | `http://www.w3.org/1999/xhtml` |
+| `rdf:` | `http://www.w3.org/1999/02/22-rdf-syntax-ns#` |
| `svg:` | `http://www.w3.org/2000/svg` |
| `xlink:` | `http://www.w3.org/1999/xlink` |
| `xslt:` | `http://www.w3.org/1999/XSL/Transform` |
## Setup and Configuration
-⛩️📰 书社 depends on the following programs to run.
+⛩📰 书社 depends on the following programs to run.
In every case, you may supply your own implementation by overriding the
corresponding (allcaps) variable (e·g, set `MKDIR` to supply your own
`mkdir` implementation).
- `xsltproc` (provided by `libxslt`)
The following additional variables can be used to control the behaviour
- of ⛩️📰 书社 :—
+ of ⛩📰 书社 :—
- **`SRCDIR`:**
The location of the source files (default: `sources`).
Multiple include directories can be provided, so long as the same
file subpath doesn’t exist in more than one of them.
+- **`DATADIR`:**
+ If set to the location of a directory, ⛩📰 书社 will run a two‐stage build.
+ In the first stage, only files in `SRCDIR` which match `FINDDATARULES` (see below) will be built, with files in `DATADIR` serving as includes.
+ In the second stage, the remaining files in `SRCDIR` will be built, with the files built during the first stage, in addition to any files in `INCLUDEDIR`, serving as includes.
+ Files built during the first stage are copied into `DESTDIR` alongside those from the second stage when installing.
+
+ This functionality is intended for sites where the bulk of the site can be built from a few data files which are expensive to create.
+
- **`BUILDDIR`:**
The location of the (temporary) build directory (default: `build`).
`make clean` will delete this, and it is recommended that it not be
- used for programs aside from ⛩️📰 书社.
+ used for programs aside from ⛩📰 书社.
- **`DESTDIR`:**
The location of directory to output files to (default: `public`).
ensure stale content is removed.
- **`THISDIR`:**
- The location of the ⛩️📰 书社 `GNUmakefile`.
+ The location of the ⛩📰 书社 `GNUmakefile`.
This should be set automatically when calling Make and shouldn’t ever
need to be set manually.
- This variable is used to find the ⛩️📰 书社 `lib/` folder, which is
+ This variable is used to find the ⛩📰 书社 `lib/` folder, which is
expected to be in the same location.
- **`MAGIC`:**
default, to enable additional rules without overriding the existing
ones.
+- **`DATAOPTS`:**
+ Additional options to use when calling Make during the first stage of a two‐stage build using `DATADIR`.
+
+ This can be used to override variables which are only applicable during the second stage.
+ Note that when supplying this variable on the shell, it will need to be double‐quoted.
+
+- **`DATAEXT`:**
+ A list of file extensions which signify “data” files during a two‐stage build using `DATADIR`.
+
+- **`FINDDATARULES`:**
+ Rules to use with `find` when searching for data files.
+ By default, these rules are derived from `DATAEXT`.
+
+- **`EXTRAFINDDATARULES`:**
+ The value of this variable is appended to `FINDDATARULES` by
+ default, to enable additional rules without overriding the existing
+ ones.
+
+- **`FINDFILTERONLY`:**
+ A semicolon‐separated list of regular expressions, at least one of which the paths for sources and includes are required to match, unless empty (default: empty).
+
+- **`FINDFILTEROUT`:**
+ A semicolon‐separated list of regular expressions, each of which matches paths that should _not_ be considered sources or includes (default: empty).
+
+- **`FINDINCLUDEFILTERONLY`:**
+ A semicolon‐separated list of regular expressions, at least one of which the paths for includes are required to match, unless empty (default: empty).
+
+ Note that only paths which already match `FINDFILTERONLY` are considered.
+
+- **`FINDINCLUDEFILTEROUT`:**
+ A semicolon‐separated list of regular expressions, each of which matches paths that should _not_ be considered includes, but may still be considered sources (default: empty).
+
+- **`FINDFILTERONLYEXTENDED`:**
+ If non·empty, `FINDFILTERONLY` is an extended regular expression; otherwise, it is basic (default: empty).
+
+- **`FINDFILTEROUTEXTENDED`:**
+ If non·empty, `FINDFILTEROUT` is an extended regular expression; otherwise, it is basic (default: matches `FINDFILTERONLYEXTENDED`).
+
+- **`FINDINCLUDEFILTERONLYEXTENDED`:**
+ If non·empty, `FINDINCLUDEFILTERONLY` is an extended regular expression; otherwise, it is basic (default: matches `FINDFILTERONLYEXTENDED`).
+
+- **`FINDINCLUDEFILTEROUTEXTENDED`:**
+ If non·empty, `FINDINCLUDEFILTEROUT` is an extended regular expression; otherwise, it is basic (default: `1` if either `FINDFILTEROUTEXTENDED` or `FINDINCLUDEFILTERONLYEXTENDED` is non·empty).
+
- **`PARSERS`:**
A white·space‐separated list of parsers to use (default:
`$(THISDIR)/parsers/*.xslt`).
The value of this variable is appended to `PARSERS` by default, to
enable additional parsers without overriding the existing ones.
+- **`PARSERLIBS`:**
+ A white·space‐separated list of parser dependencies (default:
+ `$(THISDIR)/lib/split.xslt`).
+
+- **`EXTRAPARSERLIBS`:**
+ The value of this variable is appended to `PARSERLIBS` by default, to
+ enable additional parser dependencies without overriding the
+ existing ones.
+
- **`TRANSFORMS`:**
A white·space‐separated list of transforms to use (default:
`$(THISDIR)/transforms/*.xslt`).
The value of this variable is appended to `TRANSFORMS` by default, to
enable additional transforms without overriding the existing ones.
+- **`TRANSFORMLIBS`:**
+ A white·space‐separated list of transform dependencies (default:
+ `$(THISDIR)/lib/serialize.xslt`).
+
+- **`EXTRATRANSFORMLIBS`:**
+ The value of this variable is appended to `TRANSFORMLIBS` by default,
+ to enable additional transform dependencies without overriding the
+ existing ones.
+
- **`XMLTYPES`:**
A white·space‐separated list of media types or media type suffixes to
consider X·M·L (default: `application/xml text/xml +xml`).
This variable can be used for postprocessing.
- **`THISREV`:**
- The current version of ⛩️📰 书社 (default: derived from the current
+ The current version of ⛩📰 书社 (default: derived from the current
git tag/branch/commit).
- **`SRCREV`:**
Parsers are used to convert plaintext files into X·M·L trees, as well
as convert plaintext formats which are already included inline in
existing source X·M·L documents.
-⛩️📰 书社 comes with some parsers; namely :—
+⛩📰 书社 comes with some parsers; namely :—
- **`parsers/plain.xslt`:**
Wraps `text/plain` contents in a `<html:pre>` element.
Converts `text/tab-separated-values` contents into an `<html:table>`
element.
-New ⛩️📰 书社 parsers which target plaintext formats should have an
+New ⛩📰 书社 parsers which target plaintext formats should have an
`<xslt:template>` element with no `@name` or `@mode` and whose
`@match` attribute…
</transform>
```
-⛩️📰 书社 will scan the provided parsers for this pattern to determine
+⛩📰 书社 will scan the provided parsers for this pattern to determine
the set of allowed plaintext file types.
Multiple such `<xslt:template>` elements may be provided in a single
parser, for example if the parser supports multiple media types.
### Attributes added during parsing
-⛩️📰 书社 will add a few attributes to elements which result from
+⛩📰 书社 will add a few attributes to elements which result from
parsing plaintext `<html:script>` elements.
These include :—
- A `@书社:media-type` attribute, giving the identified media type of
the plaintext node.
+### Parsed metadata
+
+It is possible to extract metadata from a document at the same time as
+ it is being parsed.
+This is done by creating result elements in the `书社:about` mode;
+ these should be R·D·F property elements which apply to the conceptual
+ entity that is the document being parsed.
+
+During transformation, metadata for the file with identifier `$FILE`
+ can be read from the children of
+ `$书社:about//*[@rdf:about=$FILE]/nie:interpretedAs/*`.
+
## Output Redirection
-By default, ⛩️📰 书社 installs files to the same location in `DESTDIR`
+By default, ⛩📰 书社 installs files to the same location in `DESTDIR`
as they were placed in their `SRCDIR`.
This behaviour can be customized by setting the `@书社:destination`
attribute on the root element, whose value can give a different path.
This attribute is read after parsing, but before transformation (where
it is silently dropped).
+Multiple destinations can be provided if the same file should be output to multiple places.
+The file is retransformed each time, with the value of the `DESTINATION` global param set appropriately.
+
## Embedding
Documents can be embedded in other documents using a `<书社:link>`
Embedding takes place after parsing but before transformation, so
parsers are able to generate their own embeds.
-⛩️📰 书社 is able to detect the transitive embed dependencies of files
+⛩📰 书社 is able to detect the transitive embed dependencies of files
and update them accordingly; it will signal an error if the
dependencies are recursive.
### Attributes added during expansion
-⛩️📰 书社 will add a few attributes to toplevel result elements, both
+⛩📰 书社 will add a few attributes to toplevel result elements, both
in the main document and any embedded documents, during the expansion
phase prior to the main transformation.
These include :—
- A `@书社:mtime` attribute giving the last modified time of the
corresponding source file.
-- A `@书社:identifier` attribute giving the ⛩️📰 书社 identifier
+- A `@书社:identifier` attribute giving the ⛩📰 书社 identifier
(i·e, starting with `about:shushe?`) of the corresponding source
file.
Transforms are used to convert X·M·L files into their final output,
after all necessary parsing and embedding has taken place.
-⛩️📰 书社 comes with some transforms; namely :—
+⛩📰 书社 comes with some transforms; namely :—
- **`transforms/asset.xslt`:**
Converts `<html:object>` elements which correspond to recognized
media types into the appropriate H·T·M·L elements, and deletes
`<html:style>` elements from the body of the document and moves
them to the head.
+ This conversion happens during the finalization phase, after the main
+ transformation.
+
+- **`transforms/expansion.xslt`:**
+ Performs embedding, as described above.
- **`transforms/metadata.xslt`:**
Provides basic `<html:head>` metadata.
- **`urn:fdc:ladys.computer:20231231:Shu1She4:title`:**
Provides the title of the page.
- ⛩️📰 书社 automatically encapsulates H·T·M·L embeds so that their
+ ⛩📰 书社 automatically encapsulates H·T·M·L embeds so that their
metadata does not propogate up to the embedding document.
To undo this behaviour, remove the `@itemscope` and `@itemtype`
attributes from the embed during the transformation phase.
- **`transforms/serialization.xslt`:**
Replaces `<书社:serialize-xml>` elements with the (escaped)
serialized X·M·L of their contents.
- This replacement happens during the application phase, after most
+ This replacement happens during the finalization phase, after most
other transformations have taken place.
If a `@with-namespaces` attribute is provided, any name·space nodes
name·space.
Multiple prefixes may be provided, separated by white·space.
- When it comes to name·spaces used internally by ⛩️📰 书社, the
- prefix used by ⛩️📰 书社 may be declared _in addition to_ the
+ When it comes to name·spaces used internally by ⛩📰 书社, the
+ prefix used by ⛩📰 书社 may be declared _in addition to_ the
prefix(es) used in the source document(s).
It is not possible to selectively only declare one prefix for a
name·space to the exclusion of others.
The current time.
- **`IDENTIFIER`:**
- The ⛩️📰 书社 identifier of the source file (a u·r·i beginning with
+ The ⛩📰 书社 identifier of the source file (a u·r·i beginning with
`about:shushe`).
- **`SRCREV`:**
- **`THISREV`:**
The value of the `THISREV` variable (if present).
-The following params are only available in transforms :—
+In transforms, the following params are additionally available :—
+
+- **`DESTINATION`:**
+ The destination being targeted by this transform.
-- **`PATH`:**
- The path of the output file (within `DESTDIR`).
+- **`书社:about`:**
+ R·D·F metadata about all of the documents ⛩📰 书社 knows about.
+ Use `$书社:about//*[@rdf:about=$IDENTIFIER]` to get the metadata for
+ the current document.
+
+- **`书社:source`:**
+ The parsed source document being transformed, prior to any expansion.
+
+- **`书社:expansion`:**
+ The document after the all embeds have been expanded.
+ Unavailable during the `书社:expand` stage.
+
+- **`书社:result`:**
+ The document after the main set of transformations have been applied.
+ Only available during the `书社:finalize` stage, where it is used to
+ apply output wrapping and other clean·up.
## Output Wrapping
-⛩️📰 书社 will wrap the final output of the transforms in appropriate
- `<html:html>` and `<html:body>` elements, so it is not necessary for
- transforms to do this explicitly.
-After performing the initial transform, ⛩️📰 书社 will match the root
- node of the result in the following modes to fill in areas of the
+Provided at least one toplevel result element belongs to the H·T·M·L
+ namespace, ⛩📰 书社 will wrap the final output of the transforms in
+ appropriate `<html:html>` and `<html:body>` elements, so it is not
+ necessary for transforms to do this explicitly.
+If a toplevel result element _is_ a `<html:html>` and `<html:body>`
+ element, it will be merged with the one that ⛩📰 书社 creates.
+Consequently, wrapping the result in a `<html:body>` element can be
+ used to enable wrapping for non‐H·T·M·L content, when desired.
+
+As a part of this process, after performing the initial transform
+ ⛩📰 书社 will match in the following modes to fill in areas of the
wrapper :—
- **`书社:header`:**
The result of matching in this mode is inserted into the
`<html:head>` of the output.
-In addition to being called with the transform result, each of these
- modes will additionally be called with a `<xslt:include>` element
- corresponding to each transform.
-If a transform has a `<书社:id>` top‐level element whose value is an
- i·r·i, its `<xslt:include>` element will have a corresponding
- `@书社:id` attribute.
-This mechanism can be used to allow transforms to insert content
- without matching any elements in the result; for example, the
- following transform adds a link to a stylesheet to the `<html:head>`
- of every page :—
+The document being matched will contain the full transform result
+ prior to wrapping as well as an `<书社:id>` element for each
+ transform.
+The latter elements can be matched to enable transforms to provide
+ content _without_ matching any elements in the result; for example,
+ the following transform adds a link to a stylesheet to the
+ `<html:head>` of every page :—
```xml
<?xml version="1.0"?>
version="1.0"
>
<书社:id>example:add-stylesheet-links.xslt</书社:id>
- <template match="xslt:include[@书社:id='example:add-stylesheet-links.xslt']" mode="书社:metadata">
+ <template match="书社:id[string(.)='example:add-stylesheet-links.xslt']" mode="书社:metadata">
<html:link rel="stylesheet" type="text/css" href="/style.css"/>
</template>
</transform>
the result tree.
It will not be performed on outputs whose root elements are
`<书社:archive>`, `<书社:base64-binary>`, or `<书社:raw-text>`
- (described below).
+ (described below), or on result trees which do not contain a toplevel
+ element in the H·T·M·L namespace.
## Applying Attributes
In both cases, attributes from various sources are combined with
white·space between them.
-Attribute application takes place after all ordinary transforms have
- completed.
+Attribute application takes place after each stage of the
+ transformation, including after the initial embedding phase.
Both elements ignore attributes in the `xml:` name·space, except for
`@xml:lang`, which ignores all but the first definition (including
There are a few special elements in the `书社:` name·space which, if
they appear as the toplevel element in a transformation result, cause
- ⛩️📰 书社 to produce something other than an X·M·L file.
+ ⛩📰 书社 to produce something other than an X·M·L file.
They are :—
- **`<书社:archive>`:**
Other child elements will be ignored.
If the `<书社:archive>` element is given an `@书社:expanded`
- attribute, rather than producing a tarball ⛩️📰 书社 will output
+ attribute, rather than producing a tarball ⛩📰 书社 will output
the directory which expanding the tarball would produce.
This mechanism can be used to generate multiple files from a single
source, provided all of the files are contained with·in the same
Lady’s Gitweb / Shushe / blobdiff