X-Git-Url: https://git.ladys.computer/Shushe/blobdiff_plain/03f9773767ae7811289561abe5162e8b04578991..5c2f4a2d2c9913196c6092f0f72a8c4ff6e76752:/README.markdown?ds=sidebyside diff --git a/README.markdown b/README.markdown index 573fb86..af78725 100644 --- a/README.markdown +++ b/README.markdown @@ -1,8 +1,12 @@ -# ⛩️📰 书社 + +# ⛩📰 书社 A make·file for X·M·L. -⛩️📰 书社 aims to make it easy to generate websites with +⛩📰 书社 aims to make it easy to generate websites with X·S·L·T and G·N·U Make. It is consequently only a good choice for people who like X·S·L·T and G·N·U Make and wish it were easier to make websites with them. @@ -17,16 +21,12 @@ It makes things easier by :⁠— - Enabling easy inclusion of source files within each other. It aims to do this with zero dependencies beyond the programs already - installed on your computer. -(On Linux machines, you may need to install `libxml2-utils` to get the - commandline programs from `libxml2`.) + installed on your computer†. -**Note:** -⛩️📰 书社 requires functionality present in G·N·U Make 3.81 (or later) - and will not work in previous versions, or other implementations of - Make. -Compatibility with later versions of G·N·U Make is assumed, but not - tested. +† Assuming an operating system with a fairly featureful, and + Posix‐compliant, development setup (e·g, Macintosh ≥ version 10.8). +In fact, on Linux you will probably need to install a few programs: + `libxml2-utils`, `xsltproc`, `sharutils`, and `pax`. ## Nomenclature @@ -45,33 +45,129 @@ In Japanese, it is an alternate spelling for やしろ, The name 书社 was chosen to play on this pun, as it is intended as a publishing program for webshrines. -In Ascii environments, ⛩️📰 书社 should be written `Shushe`, following +In Ascii environments, ⛩📰 书社 should be written `Shushe`, following the pinyin transliteration. +## Prerequisites + +In most cases, ⛩📰 书社 aims to require only functionality which is + present in all Posix‐compliant (`POSIX.1-2001`) operating systems. +There are a few exceptions. +Details on particular programs are given below; if a program is not + listed, it is assumed that any Posix‐compliant implementation will + work. + +### `diff` + +This is a Posix utility, but ⛩📰 书社 depends on functionality + introduced after `POSIX.1-2001` (the `-u` option, introduced in + `POSIX.1-2008`). +Macintosh systems somewhat interestingly implement this option + correctly in legacy mode (`COMMAND_MODE=legacy`) but incorrectly by + default (despite claiming `POSIX.1-2008` conformance for this + utility). +[Note this erroneous comment claiming nanosecond & timezone are + extensions rather than standardized.][rdar-92753335] +Despite this, the default Macintosh implementation will still work with + ⛩📰 书社, with the caveat that the timestamp will only include a + fractional component when a Posix⹀compliant (e·g, Macintosh legacy or + G·N·U) implementation is used. + +### `file` + +This is a Posix utility, but it was considered optional in + `POSIX.1-2001` (altho it was made mandatory in `POSIX.1-2008`) and + ⛩📰 书社 currently depends on unspecified behaviour. +It requires support for the following additional options :⁠— + +- **`-C`**, when supplied with `-m`, must be useable to compile a + `.mgc` magicfile for use with future invocations of `file`. + +- **`--files-from`** must be useable to provide a file that `file` + should read file·names from, and `-` must be useable in this + context to specify the standard input. + +- **`--mime-type`** must cause `file` to print the internet media type + of the file with no charset parameter. + +- **`--separator`** must be useable to set the separator that `file` + uses to separate file names from types. + +These options are implemented by the + [Fine Free File Command](https://darwinsys.com/file/), which is used + by most operating systems. + +### `git` + +This is not a Posix utility. +Usage of `git` is optional, but recommended (and activated by default). +To disable it, set `GIT=`. + +### `make` + +This is a Posix utility, but it is considered an optional Software + Development utility and ⛩📰 书社 currently depends on unspecified + behaviour. +⛩📰 书社 requires specifically the G·N·U version of `make`, and + depends on functionality present in version 3.81 or later. +It is not expected to work in previous versions, or with other + implementations of Make. + +### `pax` + +This is a Posix utility, but it is not included in the Linux Standard + Base or installed by default in many distributions. +⛩📰 书社 only requires support for the `ustar` format. + +### `uudecode` and `uuencode` + +These are Posix utilities, but they were considered optional in + `POSIX.1-2001` (altho they are made mandatory in `POSIX.1-2008`) and + they are not included in the Linux Standard Base or installed by + default in many distributions. +The G·N·U [Sharutils](https://www.gnu.org/software/sharutils/) package + provides one implementation. + +### `xmlcatalog` and `xmllint` + +These are not a Posix utilities. +They are a part of `libxml2`, but may need to be installed separately + on some platforms (e·g by the name `libxml2-utils`). + +### `xsltproc` + +This is not a Posix utility. +It is a part of `libxslt`, but may need to be installed separately on + some platforms. + ## Basic Usage Place source files in `sources/` and run `make install` to compile the result to `public/`. Compilation involves the following steps :⁠— -1. ⛩️📰 书社 compiles all of the magic files in `magic/` into a single +1. ⛩📰 书社 compiles all of the magic files in `magic/` into a single file, `build/magic.mgc`. -2. ⛩️📰 书社 processes all of the parsers in `parsers/` and determines +2. ⛩📰 书社 processes all of the parsers in `parsers/` and determines the list of supported plaintext types. -3. ⛩️📰 书社 identifies all of the source files and includes and uses +3. ⛩📰 书社 identifies all of the source files and includes and uses `build/magic.mgc` to classify them by media type. -4. ⛩️📰 书社 parses all plaintext and X·M·L source files and includes +4. ⛩📰 书社 parses all plaintext and X·M·L source files and includes and then builds a dependency tree between them. -5. ⛩️📰 书社 uses the dependency tree to establish prerequisites for +5. ⛩📰 书社 uses the dependency tree to establish prerequisites for each output file. -6. ⛩️📰 书社 compiles each output file to `build/public`. +6. ⛩📰 书社 compiles each output file to `build/result`. -7. ⛩️📰 书社 copies the output files to `public`. +7. ⛩📰 书社 copies most output files from `build/result` to + `build/public`, but it does some additional processing instead on + those which indicate a non‐X·M·L desired final output form. + +8. ⛩📰 书社 copies the final resulting files to `public`. You can use `make list` to list each identified source file or include alongside its computed type and dependencies. @@ -79,55 +175,63 @@ As this is a Make‐based program, steps will only be run if the corresponding buildfile or output file is older than its prerequisites. -## Namespaces +## Name·spaces -The ⛩️📰 书社 namespace is `urn:fdc:ladys.computer:20231231:Shu1She4`. +The ⛩📰 书社 name·space is `urn:fdc:ladys.computer:20231231:Shu1She4`. -This document uses a few namespace prefixes, with the following +This document uses a few name·space prefixes, with the following meanings :⁠— -| Prefix | Expansion | -| -------: | :----------------------------------------- | -| `html:` | `http://www.w3.org/1999/xhtml` | -| `xlink:` | `http://www.w3.org/1999/xlink` | -| `xslt:` | `http://www.w3.org/1999/XSL/Transform` | -| `书社:` | `urn:fdc:ladys.computer:20231231:Shu1She4` | +| Prefix | Expansion | +| ---------: | :-------------------------------------------- | +| `catalog:` | `urn:oasis:names:tc:entity:xmlns:xml:catalog` | +| `exsl:` | `http://exslt.org/common` | +| `exslstr:` | `http://exslt.org/strings` | +| `html:` | `http://www.w3.org/1999/xhtml` | +| `svg:` | `http://www.w3.org/2000/svg` | +| `xlink:` | `http://www.w3.org/1999/xlink` | +| `xslt:` | `http://www.w3.org/1999/XSL/Transform` | +| `书社:` | `urn:fdc:ladys.computer:20231231:Shu1She4` | ## Setup and Configuration -⛩️📰 书社 depends on the following programs to run. +⛩📰 书社 depends on the following programs to run. In every case, you may supply your own implementation by overriding the corresponding (allcaps) variable (e·g, set `MKDIR` to supply your own `mkdir` implementation). - `awk` - `cat` +- `cd` +- `cksum` - `cp` - `date` -- `echo` +- `diff` - `file` - `find` - `git` (optional; set `GIT=` to disable) +- `grep` - `ln` -- `mkdir` (requires support for `-p`) +- `mkdir` - `mv` -- `od` (requires support for `-t x1`) +- `od` +- `pax` (only when generating archives) - `printf` - `rm` - `sed` - `sleep` -- `stat` - `test` - `touch` -- `tr` (requires support for `-d`) -- `uuencode` (requires support for `-m` and `-r`) -- `xargs` (requires support for `-0`) +- `tr` +- `uuencode` +- `uudecode` +- `xargs` - `xmlcatalog` (provided by `libxml2`) - `xmllint` (provided by `libxml2`) - `xsltproc` (provided by `libxslt`) The following additional variables can be used to control the behaviour - of ⛩️📰 书社 :⁠— + of ⛩📰 书社 :⁠— - **`SRCDIR`:** The location of the source files (default: `sources`). @@ -140,10 +244,18 @@ The following additional variables can be used to control the behaviour Multiple include directories can be provided, so long as the same file subpath doesn’t exist in more than one of them. +- **`DATADIR`:** + If set to the location of a directory, ⛩📰 书社 will run a two‐stage build. + In the first stage, only files in `SRCDIR` which match `FINDDATARULES` (see below) will be built, with files in `DATADIR` serving as includes. + In the second stage, the remaining files in `SRCDIR` will be built, with the files built during the first stage, in addition to any files in `INCLUDEDIR`, serving as includes. + Files built during the first stage are copied into `DESTDIR` alongside those from the second stage when installing. + + This functionality is intended for sites where the bulk of the site can be built from a few data files which are expensive to create. + - **`BUILDDIR`:** The location of the (temporary) build directory (default: `build`). `make clean` will delete this, and it is recommended that it not be - used for programs aside from ⛩️📰 书社. + used for programs aside from ⛩📰 书社. - **`DESTDIR`:** The location of directory to output files to (default: `public`). @@ -159,10 +271,10 @@ The following additional variables can be used to control the behaviour ensure stale content is removed. - **`THISDIR`:** - The location of the ⛩️📰 书社 `GNUmakefile`. + The location of the ⛩📰 书社 `GNUmakefile`. This should be set automatically when calling Make and shouldn’t ever need to be set manually. - This variable is used to find the ⛩️📰 书社 `lib/` folder, which is + This variable is used to find the ⛩📰 书社 `lib/` folder, which is expected to be in the same location. - **`MAGIC`:** @@ -179,6 +291,8 @@ The following additional variables can be used to control the behaviour those which end with a cloparen, and those which contain a hash, buck, percent, asterisk, colon, semi, eroteme, bracket, backslash, or pipe. + It is important that these rules not produce any output, as anything + printed to `stdout` will be considered a result of the find. - **`EXTRAFINDRULES`:** The value of this variable is appended to `FINDRULES` by default, to @@ -193,6 +307,18 @@ The following additional variables can be used to control the behaviour default, to enable additional rules without overriding the existing ones. +- **`DATAEXT`:** + A list of file extensions which signify “data” files during a two‐stage build using `DATADIR`. + +- **`FINDDATARULES`:** + Rules to use with `find` when searching for data files. + By default, these rules are derived from `DATAEXT`. + +- **`EXTRAFINDDATARULES`:** + The value of this variable is appended to `FINDDATARULES` by + default, to enable additional rules without overriding the existing + ones. + - **`PARSERS`:** A white·space‐separated list of parsers to use (default: `$(THISDIR)/parsers/*.xslt`). @@ -201,6 +327,15 @@ The following additional variables can be used to control the behaviour The value of this variable is appended to `PARSERS` by default, to enable additional parsers without overriding the existing ones. +- **`PARSERLIBS`:** + A white·space‐separated list of parser dependencies (default: + `$(THISDIR)/lib/split.xslt`). + +- **`EXTRAPARSERLIBS`:** + The value of this variable is appended to `PARSERLIBS` by default, to + enable additional parser dependencies without overriding the + existing ones. + - **`TRANSFORMS`:** A white·space‐separated list of transforms to use (default: `$(THISDIR)/transforms/*.xslt`). @@ -209,21 +344,41 @@ The following additional variables can be used to control the behaviour The value of this variable is appended to `TRANSFORMS` by default, to enable additional transforms without overriding the existing ones. +- **`TRANSFORMLIBS`:** + A white·space‐separated list of transform dependencies (default: + `$(THISDIR)/lib/serialize.xslt`). + +- **`EXTRATRANSFORMLIBS`:** + The value of this variable is appended to `TRANSFORMLIBS` by default, + to enable additional transform dependencies without overriding the + existing ones. + - **`XMLTYPES`:** - A white·space‐separated list of media types to consider X·M·L - (default: `application/xml text/xml`). + A white·space‐separated list of media types or media type suffixes to + consider X·M·L (default: `application/xml text/xml +xml`). -- **`GENERATOR`:** - The name of the generator program (default: `⛩️📰 书社`). +- **`FINALIZE`:** + A program to run on (unspecial) X·M·L files after they are + transformed (default: `xmllint --nonet --nsclean`). + This variable can be used for postprocessing. -- **`VERSION`:** - The current version of `GENERATOR` (default: derived from the current +- **`THISREV`:** + The current version of ⛩📰 书社 (default: derived from the current git tag/branch/commit). - **`SRCREV`:** The current version of the source files (default: derived from the current git tag/branch/commit). +- **`QUIET`:** + If this variable has a value, informative messages will not be + printed (default: empty). + Informative messages print to stderr, not stdout, so disabling them + usually shouldn’t be necessary. + This does not (cannot) disable messages from Make itself, for which + the `-s`, `--silent` ∕ `--quiet` Make option is more likely to be + useful. + - **`VERBOSE`:** If this variable has a value, every recipe instruction will be printed when it runs (default: empty). @@ -256,11 +411,11 @@ Source files whose media type does not have an associated X·S·L·T contain Ascii white·space, colons (`:`), semis (`;`), pipes (`|`), bucks (`$`), percents (`%`), hashes (`#`), asterisks (`*`), brackets (`[` or `]`), erotemes (`?`), backslashes (`\`), or control - characters, must not begin with a hyphen‐minus (`-`), and must not - end with a cloparen (`)`).** + characters, must not begin with a hyphen‐minus (`-`), and must not end + with a cloparen (`)`).** The former characters have the potential to conflict with make syntax, - a leading hyphen‐minus is confusable for a command‐line argument, and - a trailing cloparen [activates a bug in G·N·U Make + a leading hyphen‐minus is confusable for a commandline argument, and a + trailing cloparen [activates a bug in G·N·U Make 3.81](https://stackoverflow.com/questions/17148468/capturing-filenames-including-parentheses-with-gnu-makes-wildcard-function#comment24825307_17148894). ## Parsers @@ -268,25 +423,24 @@ The former characters have the potential to conflict with make syntax, Parsers are used to convert plaintext files into X·M·L trees, as well as convert plaintext formats which are already included inline in existing source X·M·L documents. -⛩️📰 书社 comes with some parsers; namely :⁠— +⛩📰 书社 comes with some parsers; namely :⁠— - **`parsers/plain.xslt`:** - Wraps `text/plain` contents in a `` element. + Wraps `text/plain` contents in a `` element. - **`parsers/record-jar.xslt`:** - Converts `text/record-jar` contents into a - `` of `` elements (one for - each record). + Converts `text/record-jar` contents into a `` of + `` elements (one for each record). - **`parsers/tsv.xslt`:** - Converts `text/tab-separated-values` contents into an - `` element. + Converts `text/tab-separated-values` contents into an `` + element. -New ⛩️📰 书社 parsers which target plaintext formats should have an +New ⛩📰 书社 parsers which target plaintext formats should have an `` element with no `@name` or `@mode` and whose `@match` attribute… -- Starts with an appropriately‐namespaced qualified name for a +- Starts with an appropriately‐name·spaced qualified name for a `` element. - Follows this with the string `[@type=`. @@ -314,7 +468,7 @@ For example, the trivial `text/plain` parser is defined as follows :⁠— ``` -⛩️📰 书社 will scan the provided parsers for this pattern to determine +⛩📰 书社 will scan the provided parsers for this pattern to determine the set of allowed plaintext file types. Multiple such `` elements may be provided in a single parser, for example if the parser supports multiple media types. @@ -343,9 +497,33 @@ Parsers are applied *prior* to embedding (and can be used to generate embeds); transforms are applied *after*. It is **strongly recommended** that auxillary templates in parsers be - namespaced (by `@name` or `@mode`) whenever possible, to avoid + name·spaced (by `@name` or `@mode`) whenever possible, to avoid conflicts between parsers. +### Attributes added during parsing + +⛩📰 书社 will add a few attributes to elements which result from + parsing plaintext `` elements. +These include :⁠— + +- A `@书社:parsed-by` attribute, giving a space‐separated list of + parsers which parsed the node. + (Generally, this will be a list of one, but it is possible for the + result of a parse to be another plaintext node, which may be parsed + by a different parser.) + +- A `@书社:media-type` attribute, giving the identified media type of + the plaintext node. + +## Output Redirection + +By default, ⛩📰 书社 installs files to the same location in `DESTDIR` + as they were placed in their `SRCDIR`. +This behaviour can be customized by setting the `@书社:destination` + attribute on the root element, whose value can give a different path. +This attribute is read after parsing, but before transformation (where + it is silently dropped). + ## Embedding Documents can be embedded in other documents using a `<书社:link>` @@ -362,41 +540,53 @@ Embeds are replaced with the parsed contents of a file, unless the file is an asset, in which case an `` element is produced instead (with the contents of the asset file provided as a base64 `data:` u·r·i). +Embed replacements will be given a `@书社:identifier` attribute whose + value will match the `@xlink:href` of the embed. Embedding takes place after parsing but before transformation, so parsers are able to generate their own embeds. -⛩️📰 书社 is able to detect the transitive embed dependencies of files +⛩📰 书社 is able to detect the transitive embed dependencies of files and update them accordingly; it will signal an error if the dependencies are recursive. -## Output Redirection +### Attributes added during expansion -By default, ⛩️📰 书社 installs files to the same location in `DESTDIR` - as they were placed in their `SRCDIR`. -This behaviour can be customized by setting the `@书社:destination` - attribute on the root element, whose value can give a different path. -This attribute is read after parsing, but before transformation (where - it is silently dropped). +⛩📰 书社 will add a few attributes to toplevel result elements, both + in the main document and any embedded documents, during the expansion + phase prior to the main transformation. +These include :⁠— + +- A `@书社:cksum` attribute giving the `cksum` checksum of the + corresponding source file. + +- A `@书社:mtime` attribute giving the last modified time of the + corresponding source file. + +- A `@书社:identifier` attribute giving the ⛩📰 书社 identifier + (i·e, starting with `about:shushe?`) of the corresponding source + file. + +- For elements in the `html` namespace, an `itemscope` attribute and an + `itemtype` attribute with a value of + `urn:fdc:ladys.computer:20231231:Shu1She4:document` (for the main + document) or `urn:fdc:ladys.computer:20231231:Shu1She4:embed` (for + embedded documents). + These attributes are used to scope any nested `` elements + with `@itemprop` attributes to their containing documents. ## Transforms Transforms are used to convert X·M·L files into their final output, after all necessary parsing and embedding has taken place. -⛩️📰 书社 comes with some transforms; namely :⁠— - -- **`transforms/attributes.xslt`:** - Applies transforms to the children of any `<书社:apply-attributes>` - elements, and then applies the attributes of the - `<书社:apply-attributes>` to each result child, replacing the - element with the result. - This is useful in combination with image embeds to apply alt‐text to - the resulting ``. +⛩📰 书社 comes with some transforms; namely :⁠— - **`transforms/asset.xslt`:** Converts `` elements which correspond to recognized media types into the appropriate H·T·M·L elements, and deletes `` elements from the body of the document and moves them to the head. + This conversion happens during the finalization phase, after the main + transformation. - **`transforms/metadata.xslt`:** Provides basic `` metadata. @@ -406,11 +596,38 @@ Transforms are used to convert X·M·L files into their final output, - **`urn:fdc:ladys.computer:20231231:Shu1She4:title`:** Provides the title of the page. - ⛩️📰 书社 automatically encapsulates embeds so that their metadata - does not propogate up to the embedding document. + ⛩📰 书社 automatically encapsulates H·T·M·L embeds so that their + metadata does not propogate up to the embedding document. To undo this behaviour, remove the `@itemscope` and `@itemtype` attributes from the embed during the transformation phase. +- **`transforms/serialization.xslt`:** + Replaces `<书社:serialize-xml>` elements with the (escaped) + serialized X·M·L of their contents. + This replacement happens during the finalization phase, after most + other transformations have taken place. + + If a `@with-namespaces` attribute is provided, any name·space nodes + on the toplevel serialized elements whose U·R·I’s correspond to the + definitions of the provided prefixes, as defined for the + `<书社:serialize-xml>` element, will be declared using name·space + attributes on the serialized elements. + Otherwise, only name·space nodes which _differ_ from the definitions + on the `<书社:serialize-xml>` element will be declared. + The string `#default` may be used to represent the default + name·space. + Multiple prefixes may be provided, separated by white·space. + + When it comes to name·spaces used internally by ⛩📰 书社, the + prefix used by ⛩📰 书社 may be declared _in addition to_ the + prefix(es) used in the source document(s). + It is not possible to selectively only declare one prefix for a + name·space to the exclusion of others. + + `<书社:raw-output>` elements may be used inside of + `<书社:serialize-xml>` elements to inject raw output into the + serialized X·M·L. + The following are recommendations on effective creation of transforms :⁠— @@ -418,10 +635,10 @@ The following are recommendations on effective creation of It is likely an error if two transforms have templates which match the same element (unless the templates have different priority). -- Namespace templates (with `@name` or `@mode`) whenever possible. +- Name·space templates (with `@name` or `@mode`) whenever possible. - Set `@exclude-result-prefixes` on the root `xslt:transform` element - to reduce the number of declared namespaces in the final result. + to reduce the number of declared name·spaces in the final result. ## Global Params @@ -431,33 +648,29 @@ The following params are made available globally in parsers and - **`BUILDTIME`:** The current time. -- **`GENERATOR`:** - The value of the `GENERATOR` variable (if present). +- **`IDENTIFIER`:** + The ⛩📰 书社 identifier of the source file (a u·r·i beginning with + `about:shushe`). - **`SRCREV`:** The value of the `SRCREV` variable (if present). -- **`SRCTIME`:** - The time at which the source file was last modified. - -- **`VERSION`:** - The value of the `VERSION` variable (if present). - -The following params are only available in transforms :⁠— - -- **`CATALOG`:** - The path of the catalog file (within `BUILDDIR`). - -- **`PATH`:** - The path of the output file (within `DESTDIR`). +- **`THISREV`:** + The value of the `THISREV` variable (if present). ## Output Wrapping -⛩️📰 书社 will wrap the final output of the transforms in appropriate - `` and `` elements, so it is not necessary for - transforms to do this explicitly. -After performing the initial transform, ⛩️📰 书社 will match the root - node of the result in the following modes to fill in areas of the +Provided at least one toplevel result element belongs to the H·T·M·L + namespace, ⛩📰 书社 will wrap the final output of the transforms in + appropriate `` and `` elements, so it is not + necessary for transforms to do this explicitly. +If a toplevel result element _is_ a `` and `` + element, it will be merged with the one that ⛩📰 书社 creates. +Consequently, wrapping the result in a `` element can be + used to enable wrapping for non‐H·T·M·L content, when desired. + +As a part of this process, after performing the initial transform + ⛩📰 书社 will match in the following modes to fill in areas of the wrapper :⁠— - **`书社:header`:** @@ -472,16 +685,13 @@ After performing the initial transform, ⛩️📰 书社 will match the root The result of matching in this mode is inserted into the `` of the output. -In addition to being called with the transform result, each of these - modes will additionally be called with a `` element - corresponding to each transform. -If a transform has a `<书社:id>` top‐level element whose value is an - i·r·i, its `` element will have a corresponding - `@书社:id` attribute. -This mechanism can be used to allow transforms to insert content - without matching any elements in the result; for example, the - following transform adds a link to a stylesheet to the `` - of every page :⁠— +The document being matched will contain the full transform result + prior to wrapping as well as an `<书社:id>` element for each + transform. +The latter elements can be matched to enable transforms to provide + content _without_ matching any elements in the result; for example, + the following transform adds a link to a stylesheet to the + `` of every page :⁠— ```xml @@ -494,7 +704,7 @@ This mechanism can be used to allow transforms to insert content version="1.0" > <书社:id>example:add-stylesheet-links.xslt -