Add a basic record-jar parser

[Shushe] / README.markdown

# ⛩️📰 书社

<b>An X·S·L·T‐based static site generator.</b>

<dfn>⛩️📰 书社</dfn> aims to make it easy to generate websites with
  X·S·L·T and G·N·U Make.
It is consequently only a good choice for people who like X·S·L·T and
  G·N·U Make and wish it were easier to make websites with them.

It makes things easier by :⁠—

- Automatically identifying source files and characterizing them by
    type (X·M·L, text, or asset).

- Parsing supported text types into X·M·L trees.

- Enabling easy inclusion of source files within each other.

It aims to do this with zero dependencies beyond the programs already
  installed on your computer.

## Nomenclature

<i lang="cmn-Hans">书社</i> is a Chinese word meaning “publishing
  house”.

The first character, <i lang="cmn-Hans">书</i>, is the simplified form
  of “document”.

The second character, <i lang="cmn-Hans">社</i>, contemporarily means
  “association”, but historically referred to the god of the soil and
  related altars or festivities.
In Japanese, it is an alternate spelling for <i lang="ja">やしろ</i>,
  the word for “Shinto shrine”.

The name <i lang="cmn-Hans">书社</i> was chosen to play on this pun, as
  it is intended as a publishing program for webshrines.

In Ascii environments, ⛩️📰 书社 should be written `Shushe`, following
  the pinyin transliteration.

## Basic Usage

Place source files in `sources/` and run `make install` to compile
  the result to `public/`.
Compilation involves the following steps :⁠—

1. ⛩️📰 书社 compiles all of the magic files in `magic/` into a single
    file, `build/magic.mgc`.

2. ⛩️📰 书社 processes all of the parsers in `parsers/` and determines
    the list of supported plaintext types.

3. ⛩️📰 书社 identifies all of the source files and includes and uses
    `build/magic.mgc` to classify them by media type.

4. ⛩️📰 书社 parses all plaintext and X·M·L source files and includes
    and then builds a dependency tree between them.

5. ⛩️📰 书社 uses the dependency tree to establish prerequisites for
    each output file.

6. ⛩️📰 书社 compiles each output file to `build/public`.

7. ⛩️📰 书社 copies the output files to `public`.

You can use `make list` to list each identified source file or include
  alongside its computed type and dependencies.
As this is a Make‐based program, steps will only be run if the
  corresponding buildfile or output file is older than its
  prerequisites.

## Namespaces

The ⛩️📰 书社 namespace is `urn:fdc:ladys.computer:20231231:Shu1She4`.

This document uses a few namespace prefixes, with the following
  meanings :⁠—

|   Prefix | Expansion                                  |
| -------: | :----------------------------------------- |
|  `html:` | `http://www.w3.org/1999/xhtml`             |
| `xlink:` | `http://www.w3.org/1999/xlink`             |
|  `xslt:` | `http://www.w3.org/1999/XSL/Transform`     |
|  `书社:` | `urn:fdc:ladys.computer:20231231:Shu1She4` |

## Setup and Configuration

⛩️📰 书社 depends on the following programs to run.
In every case, you may supply your own implementation by overriding the
  corresponding (allcaps) variable (e·g, set `MKDIR` to supply your own
  `mkdir` implementation).

- `cat`
- `cp`
- `echo`
- `file`
- `find`
- `mkdir` (requires support for `-p`)
- `mv`
- `printf`
- `rm`
- `sed`
- `sleep`
- `test`
- `touch`
- `tr` (requires support for `-d`)
- `uuencode` (requires support for `-m` and `-r`)
- `xmlcatalog` (provided by `libxml2`)
- `xmllint` (provided by `libxml2`)
- `xsltproc` (provided by `libxslt`)

The following additional variables can be used to control the behaviour
  of ⛩️📰 书社 :⁠—

- **`SRCDIR`:**
  The location of the source files (default: `sources`).

- **`INCLUDEDIR`:**
  The location of the source files (default: `sources/includes`).
  This can be inside of `SRCDIR`, but needn’t be.

- **`BUILDDIR`:**
  The location of the (temporary) build directory (default: `build`).

- **`DESTDIR`:**
  The location of directory to output files to (default: `public`).

- **`THISDIR`:**
  The location of the ⛩️📰 书社 `GNUmakefile`.
  This should be set automatically when calling Make and shouldn’t ever
    need to be set manually.
  This variable is used to find the ⛩️📰 书社 `lib/` folder, which is
    expected to be in the same location.

- **`MAGICDIR`:**
  The location of the magic files to use (default: `$(THISDIR)/magic`).

- **`FINDOPTS`:**
  Options to pass to `find` when searching for source files (default:
    `-LE`).

- **`FINDRULES`:**
  Rules to use with `find` when searching for source files (default:
    `-flags -nohidden -and -not -name '.*'`).

- **`PARSERS`:**
  A white·space‐separated list of parsers to use (default:
    `$(THISDIR)/parsers/*.xslt`).

- **`TRANSFORMS`:**
  A white·space‐separated list of transforms to use (default:
    `$(THISDIR)/transforms/*.xslt`).

- **`XMLTYPES`:**
  A white·space‐separated list of media types to consider X·M·L
    (default: `application/xml text/xml`).

- **`VERBOSE`:**
  If this variable has a value, every recipe instruction will be
    printed when it runs (default: empty).
  This is helpful for debugging, but typically too noisy for general
    usage.

## Source Files

Source files may be placed in `SRCDIR` in any manner; the file
  structure used there will match the output.
The type of source files is *not* determined by file extension, but
  rather by magic number; this means that files **must** begin with
  something recognizable.
Supported magic numbers include :⁠—

- `<?xml` for `application/xml` files
- `#!js` for `text/javascript` files
- `@charset "` for `text/css` files
- `#!tsv` for `text/tab-separated-values` files
- `%%` for `text/record-jar` files (unregistered; see
    [[draft-phillips-record-jar-01][]])

Text formats with associated X·S·L·T parsers are wrapped in a H·T·M·L
  `<script>` element whose `@type` gives its media type, and then
  passed to the parser to process.
Source files whose media type does not have an associated X·S·L·T
  parser are considered “assets” and will not be transformed.

For compatibility with this program, source filenames should conform to
  the following rules :⁠—

- They should not start with a hyphen‐minus.
  This is to prevent confusion between filenames and options on the
    commandline.

- They should not contain spaces, colons, percent signs, backticks,
    question marks, hashes, or backslashes.

In general, filenames should be such that they do not require
  percent‐encoding in the path component of an i·r·i.

## Parsers

Parsers are used to convert plaintext files into X·M·L trees, as well
  as convert plaintext formats which are already included inline in
  existing source X·M·L documents.
⛩️📰 书社 comes with some parsers; namely :⁠—

- **`parsers/plain.xslt`:**
  Wraps `text/plain` contents in a `<html:pre>` element.

- **`parsers/tsv.xslt`:**
  Converts `text/tab-separated-values` contents into an `<html:table>`
    element.

New ⛩️📰 书社 parsers should have a `<xslt:template>` element with no
  `@name` or `@mode` and whose `@match` attribute…

- Starts with an appropriately‐namespaced qualified name for a
    `<html:script>` element.

- Follows this with the string `[@type=`.

- Follows this with a quoted string giving a media type supported by
    the parser.
  Media type parameters are *not* supported.

- Follows this with the string `]`.

For example, the trivial `text/plain` parser is defined as follows :⁠—

```xml
<?xml version="1.0"?>
<transform
  xmlns="http://www.w3.org/1999/XSL/Transform"
  xmlns:html="http://www.w3.org/1999/xhtml"
  version="1.0"
>
  <template match="html:script[@type='text/plain']">
    <html:pre><value-of select="."/></html:pre>
  </template>
</transform>
```

⛩️📰 书社 will scan the provided parsers for this pattern to determine
  the set of allowed plaintext file types.
Multiple such `<xslt:template>` elements may be provided in a single
  parser, for example if the parser supports multiple media types.

It is **strongly recommended** that all templates in parsers other than
  those described above be namespaced (by `@name` or `@mode`), to avoid
  conflicts between templates in multiple parsers.

## Embedding

Documents can be embedded in other documents using a `<书社:link>`
  element with `@xlink:show="embed"`.
The `@xlink:href`s of these elements should have the format
  `about:shushe?source=<path>`, where `<path>` provides the path to the
  file within `SRCDIR`.
Includes, which do not generate outputs of their own but may still be
  freely embedded, instead use the format
  `about:shushe?include=<path>`, where `<path>` provides the path
  within `INCLUDEDIR`.

Embeds are replaced with the parsed contents of a file, unless the file
  is an asset, in which case an `<html:object>` element is produced
  instead (with the contents of the asset file provided as a base64
  `data:` u·r·i).

Embedding takes place after parsing but before transformation, so
  parsers are able to generate their own embeds.
⛩️📰 书社 is able to detect the transitive embed dependencies of files
  and update them accordingly; it will signal an error if the
  dependencies are recursive.

## Transforms

Transforms are used to convert X·M·L files into their final output,
  after all necessary parsing and embedding has taken place.
⛩️📰 书社 comes with some transforms; namely :⁠—

- **`transforms/asset.xslt`:**
  Converts `<html:object>` elements which correspond to recognized
    media types into the appropriate H·T·M·L elements, and deletes
    `<html:style>` elements from the body of the document and moves
    them to the head.

- **`transforms/metadata.xslt`:**
  Provides basic `<html:head>` metadata.
  This metadata is generated from `<html:meta>` elements with one o.
    the following `@itemprop` attributes :⁠—

  - **`urn:fdc:ladys.computer:20231231:Shu1She4:title`:**
    Provides the title of the page.

  ⛩️📰 书社 automatically encapsulates embeds so that their metadata
    does not propogate up to the embedding document.
  To undo this behaviour, remove the `@itemscope` and `@itemtype`
    attributes from the embed during the transformation phase.

The following are recommendations on effective creation of
  transforms :⁠—

- Make template matchers as specific as possible.
  It is likely an error if two transforms have templates which match
    the same element (unless the templates have different priority).

- Namespace templates (with `@name` or `@mode`) whenever possible.

- Set `@exclude-result-prefixes` on the root `xslt:transform` element
    to reduce the number of declared namespaces in the final result.

## Output Wrapping

⛩️📰 书社 will wrap the final output of the transforms in appropriate
  `<html:html>` and `<html:body>` elements, so it is not necessary for
  transforms to do this explicitly.
After performing the initial transform, ⛩️📰 书社 will match the root
  node of the result in the following modes to fill in areas of the
  wrapper :⁠—

- **`书社:header`:**
  The result of matching in this mode is prepended into the
    `<html:body>` of the output (before the transformation result).

- **`书社:footer`:**
  The result of matching in this mode is appended into the
    `<html:body>` of the output (after the transformation result).

- **`书社:metadata`:**
  The result of matching in this mode is inserted into the
    `<html:head>` of the output.

In addition to being called with the transform result, each of these
  modes will additionally be called with a `<xslt:include>` element
  corresponding to each transform.
If a transform has a `<书社:id>` top‐level element whose value is an
  i·r·i, its `<xslt:import>` element will have a corresponding
  `@书社:id` attribute.
This mechanism can be used to allow transforms to insert content
  without matching any elements in the result; for example, the
  following transform adds a link to a stylesheet to the `<html:head>`
  of every page :⁠—

```xml
<?xml version="1.0"?>
<transform
  xmlns="http://www.w3.org/1999/XSL/Transform"
  xmlns:html="http://www.w3.org/1999/xhtml"
  xmlns:xslt="http://www.w3.org/1999/XSL/Transform"
  xmlns:书社="urn:fdc:ladys.computer:20231231:Shu1She4"
  exclude-result-prefixes="书社"
  version="1.0"
>
  <书社:id>example:add-stylesheet-links.xslt</书社:id>
  <template match="xslt:include[@书社:id='example:add-stylesheet-links.xslt']" mode="书社:metadata">
    <html:link rel="stylesheet" type="text/css" href="/style.css"/>
  </template>
</transform>
```

Output wrapping can be entirely disabled by adding a
  `@书社:disable-output-wrapping` attribute to the top‐level element in
  the result tree.

## License

Source files are licensed under the terms of the <cite>Mozilla Public
  License, version 2.0</cite>.
For more information, see [LICENSE](./LICENSE).

[draft-phillips-record-jar-01]: <https://datatracker.ietf.org/doc/html/draft-phillips-record-jar-01>