X-Git-Url: https://git.ladys.computer/Shushe/blobdiff_plain/023ca463e9644ebe5ceac3c15d73607583da00e5..d76a5f3dbbc1f19ef2033a9b9d3bf2c5c0218d34:/README.markdown?ds=inline diff --git a/README.markdown b/README.markdown index cafda3a..d459604 100644 --- a/README.markdown +++ b/README.markdown @@ -1,6 +1,10 @@ + # ⛩️📰 书社 -An X·S·L·T‐based static site generator. +A make·file for X·M·L. ⛩️📰 书社 aims to make it easy to generate websites with X·S·L·T and G·N·U Make. @@ -17,7 +21,12 @@ It makes things easier by :⁠— - Enabling easy inclusion of source files within each other. It aims to do this with zero dependencies beyond the programs already - installed on your computer. + installed on your computer†. + +† Assuming an operating system with a fairly featureful, and + Posix‐compliant, development setup (e·g, macOS). +In fact, on Linux you will probably need to install a few programs: + `libxml2-utils`, `xsltproc`, `sharutils`, and `pax`. ## Nomenclature @@ -39,6 +48,91 @@ The name 书社 was chosen to play on this pun, as In Ascii environments, ⛩️📰 书社 should be written `Shushe`, following the pinyin transliteration. +## Prerequisites + +In most cases, ⛩️📰 书社 aims to require only functionality which is + present in all Posix‐compliant operating systems. +There are a few exceptions. +Details on particular programs are given below; if a program is not + listed, it is assumed that any Posix‐compliant implementation will + work. + +### `date` + +This is a Posix utility, but ⛩️📰 书社 currently depends on + unspecified behaviour. +When the G·N·U version of `stat` is being used, then the G·N·U version + of `date` is also expected. + +### `file` + +This is a Posix utility, but ⛩️📰 书社 currently depends on + unspecified behaviour. +It requires support for the following additional options :⁠— + +- **`-C`**, when supplied with `-m`, must be useable to compile a + `.mgc` magicfile for use with future invocations of `file`. + +- **`--files-from`** must be useable to provide a file that `file` + should read file·names from, and `-` must be useable in this + context to specify the standard input. + +- **`--mime-type`** must cause `file` to print the internet media type + of the file with no charset parameter. + +- **`--separator`** must be useable to set the separator that `file` + uses to separate file names from types. + +These options are implemented by the + [Fine Free File Command](https://darwinsys.com/file/), which is used + by most operating systems. + +### `git` + +This is not a Posix utility. +Usage of `git` is optional, but recommended (and activated by default). +To disable it, set `GIT=`. + +### `make` + +This is a Posix utility, but ⛩️📰 书社 currently depends on + unspecified behaviour. +⛩️📰 书社 requires specifically the G·N·U version of `make`, and + depends on functionality present in version 3.81 or later. +It is not expected to work in previous versions, or with other + implementations of Make. + +### `pax` + +This is a Posix utility, but not included in the Linux Standard Base or + installed by default in many distributions. +Only `ustar` format support is required. + +### `stat` + +This is not a Posix utility, and nor is it particularly portable. +To get around incompatibilities, ⛩️📰 书社 attempts to recognize G·N·U + `stat` by searching for the string `GNU` when invoked with the + `--version` option, and falls back to B·S·D behaviour otherwise. + +### `uudecode` and `uuencode` + +These are Posix utilities, but not included in the Linux Standard Base + or installed by default in many distributions. +The G·N·U [Sharutils](https://www.gnu.org/software/sharutils/) package + can be installed to access them. + +### `xmlcatalog` and `xmllint` + +These are not a Posix utilities. +They is a part of `libxml2`, but may need to be installed separately + (e·g by the name `libxml2-utils`). + +### `xsltproc` + +This is not a Posix utility. +It is a part of `libxslt`, but may need to be installed separately. + ## Basic Usage Place source files in `sources/` and run `make install` to compile @@ -60,9 +154,13 @@ Compilation involves the following steps :⁠— 5. ⛩️📰 书社 uses the dependency tree to establish prerequisites for each output file. -6. ⛩️📰 书社 compiles each output file to `build/public`. +6. ⛩️📰 书社 compiles each output file to `build/result`. + +7. ⛩️📰 书社 copies most output files from `build/result` to + `build/public`, but it does some additional processing instead on + those which indicate a non‐X·M·L desired final output form. -7. ⛩️📰 书社 copies the output files to `public`. +8. ⛩️📰 书社 copies the final resulting files to `public`. You can use `make list` to list each identified source file or include alongside its computed type and dependencies. @@ -70,19 +168,23 @@ As this is a Make‐based program, steps will only be run if the corresponding buildfile or output file is older than its prerequisites. -## Namespaces +## Name·spaces -The ⛩️📰 书社 namespace is `urn:fdc:ladys.computer:20231231:Shu1She4`. +The ⛩️📰 书社 name·space is `urn:fdc:ladys.computer:20231231:Shu1She4`. -This document uses a few namespace prefixes, with the following +This document uses a few name·space prefixes, with the following meanings :⁠— -| Prefix | Expansion | -| -------: | :----------------------------------------- | -| `html:` | `http://www.w3.org/1999/xhtml` | -| `xlink:` | `http://www.w3.org/1999/xlink` | -| `xslt:` | `http://www.w3.org/1999/XSL/Transform` | -| `书社:` | `urn:fdc:ladys.computer:20231231:Shu1She4` | +| Prefix | Expansion | +| ---------: | :-------------------------------------------- | +| `catalog:` | `urn:oasis:names:tc:entity:xmlns:xml:catalog` | +| `exsl:` | `http://exslt.org/common` | +| `exslstr:` | `http://exslt.org/strings` | +| `html:` | `http://www.w3.org/1999/xhtml` | +| `svg:` | `http://www.w3.org/2000/svg` | +| `xlink:` | `http://www.w3.org/1999/xlink` | +| `xslt:` | `http://www.w3.org/1999/XSL/Transform` | +| `书社:` | `urn:fdc:ladys.computer:20231231:Shu1She4` | ## Setup and Configuration @@ -91,23 +193,32 @@ In every case, you may supply your own implementation by overriding the corresponding (allcaps) variable (e·g, set `MKDIR` to supply your own `mkdir` implementation). +- `awk` - `cat` +- `cksum` - `cp` - `date` -- `echo` - `file` - `find` -- `mkdir` (requires support for `-p`) +- `git` (optional; set `GIT=` to disable) +- `grep` +- `ln` +- `ls` +- `mkdir` - `mv` +- `od` +- `pax` (only when generating archives) - `printf` - `rm` - `sed` - `sleep` -- `stat` +- `stat` (BSD *or* GNU) - `test` - `touch` -- `tr` (requires support for `-d`) -- `uuencode` (requires support for `-m` and `-r`) +- `tr` +- `uuencode` +- `uudecode` +- `xargs` - `xmlcatalog` (provided by `libxml2`) - `xmllint` (provided by `libxml2`) - `xsltproc` (provided by `libxslt`) @@ -117,16 +228,32 @@ The following additional variables can be used to control the behaviour - **`SRCDIR`:** The location of the source files (default: `sources`). + Multiple source directories can be provided, so long as the same + file subpath doesn’t exist in more than one of them. - **`INCLUDEDIR`:** - The location of the source files (default: `sources/includes`). + The location of source includes (default: `sources/includes`). This can be inside of `SRCDIR`, but needn’t be. + Multiple include directories can be provided, so long as the same + file subpath doesn’t exist in more than one of them. - **`BUILDDIR`:** The location of the (temporary) build directory (default: `build`). + `make clean` will delete this, and it is recommended that it not be + used for programs aside from ⛩️📰 书社. - **`DESTDIR`:** The location of directory to output files to (default: `public`). + `make install` will overwrite files in this directory which + correspond to those in `SRCDIR`. + It *will not* touch other files, including those generated from files + in `SRCDIR` which have since been deleted. + + Files are first compiled to `$(BUILDDIR)/public` before they are + copied to `DESTDIR`, so this folder is relatively quick and + inexpensive to re·create. + It’s reasonable to simply delete it before every `make install` to + ensure stale content is removed. - **`THISDIR`:** The location of the ⛩️📰 书社 `GNUmakefile`. @@ -135,29 +262,62 @@ The following additional variables can be used to control the behaviour This variable is used to find the ⛩️📰 书社 `lib/` folder, which is expected to be in the same location. -- **`MAGICDIR`:** - The location of the magic files to use (default: `$(THISDIR)/magic`). +- **`MAGIC`:** + A white·space‐separated list of magic files to use (default: + `$(THISDIR)/magic/*`). -- **`FINDOPTS`:** - Options to pass to `find` when searching for source files (default: - `-LE`). +- **`EXTRAMAGIC`:** + The value of this variable is appended to `MAGIC` by default, to + enable additional magic files without overriding the existing ones. - **`FINDRULES`:** - Rules to use with `find` when searching for source files (default: - `-flags -nohidden -and -not -name '.*'`). + Rules to use with `find` when searching for source files. + The default ignores files that start with a period or hyphen‐minus, + those which end with a cloparen, and those which contain a hash, + buck, percent, asterisk, colon, semi, eroteme, bracket, backslash, + or pipe. + +- **`EXTRAFINDRULES`:** + The value of this variable is appended to `FINDRULES` by default, to + enable additional rules without overriding the existing ones. + +- **`FINDINCLUDERULES`:** + Rules to use with `find` when searching for includes (default: + `$(FINDRULES)`). + +- **`EXTRAFINDINCLUDERULES`:** + The value of this variable is appended to `FINDINCLUDERULES` by + default, to enable additional rules without overriding the existing + ones. - **`PARSERS`:** A white·space‐separated list of parsers to use (default: `$(THISDIR)/parsers/*.xslt`). +- **`EXTRAPARSERS`:** + The value of this variable is appended to `PARSERS` by default, to + enable additional parsers without overriding the existing ones. + - **`TRANSFORMS`:** A white·space‐separated list of transforms to use (default: `$(THISDIR)/transforms/*.xslt`). +- **`EXTRATRANSFORMS`:** + The value of this variable is appended to `TRANSFORMS` by default, to + enable additional transforms without overriding the existing ones. + - **`XMLTYPES`:** A white·space‐separated list of media types to consider X·M·L (default: `application/xml text/xml`). +- **`THISREV`:** + The current version of ⛩️📰 书社 (default: derived from the current + git tag/branch/commit). + +- **`SRCREV`:** + The current version of the source files (default: derived from the + current git tag/branch/commit). + - **`VERBOSE`:** If this variable has a value, every recipe instruction will be printed when it runs (default: empty). @@ -186,18 +346,18 @@ Text formats with associated X·S·L·T parsers are wrapped in a H·T·M·L Source files whose media type does not have an associated X·S·L·T parser are considered “assets” and will not be transformed. -For compatibility with this program, source filenames should conform to - the following rules :⁠— - -- They should not start with a hyphen‐minus. - This is to prevent confusion between filenames and options on the - commandline. - -- They should not contain spaces, colons, percent signs, backticks, - question marks, hashes, or backslashes. - -In general, filenames should be such that they do not require - percent‐encoding in the path component of an i·r·i. +**☡ For compatibility with this program, source file·names must not + contain Ascii white·space, colons (`:`), semis (`;`), pipes (`|`), + bucks (`$`), percents (`%`), hashes (`#`), asterisks (`*`), brackets + (`[` or `]`), erotemes (`?`), backslashes (`\`), or control + characters, must not begin with a hyphen‐minus (`-`), must not end + with a cloparen (`)`), and must not contain quoted braces (`"{` or + `}"`).** +The former characters have the potential to conflict with make syntax, + a leading hyphen‐minus is confusable for a command‐line argument, a + trailing cloparen [activates a bug in G·N·U Make + 3.81](https://stackoverflow.com/questions/17148468/capturing-filenames-including-parentheses-with-gnu-makes-wildcard-function#comment24825307_17148894), + and quoted braces are used internally by the program. ## Parsers @@ -209,14 +369,19 @@ Parsers are used to convert plaintext files into X·M·L trees, as well - **`parsers/plain.xslt`:** Wraps `text/plain` contents in a `` element. +- **`parsers/record-jar.xslt`:** + Converts `text/record-jar` contents into a `` of + `` elements (one for each record). + - **`parsers/tsv.xslt`:** Converts `text/tab-separated-values` contents into an `` element. -New ⛩️📰 书社 parsers should have a `` element with no - `@name` or `@mode` and whose `@match` attribute… +New ⛩️📰 书社 parsers which target plaintext formats should have an + `` element with no `@name` or `@mode` and whose + `@match` attribute… -- Starts with an appropriately‐namespaced qualified name for a +- Starts with an appropriately‐name·spaced qualified name for a `` element. - Follows this with the string `[@type=`. @@ -234,8 +399,10 @@ For example, the trivial `text/plain` parser is defined as follows :⁠— + <书社:id>example:text/plain @@ -250,9 +417,49 @@ Alternatively, you can set the `@书社:supported-media-types` attribute on the root element of the parser to override media type support detection. -It is **strongly recommended** that all templates in parsers other than - those described above be namespaced (by `@name` or `@mode`), to avoid - conflicts between templates in multiple parsers. +Even when `@书社:supported-media-types` is set, it is a requirement + that each parser transform any `` elements with a + `@type` which matches their registered types into something else. +Otherwise the parser will be stuck in an endless loop. +The result tree of applying the transform to the `` + element will be reparsed (in case any new `` elements + were added in its subtree), and a `@书社:parsed-by` attribute will be + added to each toplevel element in the result. +The value of this attribute will be the value of the `<书社:id>` + toplevel element in the parser. + +It is possible for parsers to support zero plaintext types. +This is useful when targeting specific dialects of X·M·L; parsers in + this sense operate on the same basic principles as transforms + (described below). +The major distinction between X·M·L parsers and transforms is where in + the process the transformation happens: +Parsers are applied *prior* to embedding (and can be used to generate + embeds); transforms are applied *after*. + +It is **strongly recommended** that auxillary templates in parsers be + name·spaced (by `@name` or `@mode`) whenever possible, to avoid + conflicts between parsers. + +### Attributes added during parsing + +⛩️📰 书社 will add a few attributes to the output of the parsing step, + namely :⁠— + +- A `@书社:cksum` attribute on toplevel result elements, giving the + `cksum` checksum of the corresponding source file. + +- For the elements which result from parsing plaintext `` + elements :⁠— + + - A `@书社:parsed-by` attribute, giving a space‐separated list of + parsers which parsed the node. + (Generally, this will be a list of one, but it is possible for the + result of a parse to be another plaintext node, which may be + parsed by a different parser.) + + - A `@书社:media-type` attribute, giving the identified media type of + the plaintext node. ## Embedding @@ -270,6 +477,8 @@ Embeds are replaced with the parsed contents of a file, unless the file is an asset, in which case an `` element is produced instead (with the contents of the asset file provided as a base64 `data:` u·r·i). +Embed replacements will be given a `@书社:identifier` attribute whose + value will match the `@xlink:href` of the embed. Embedding takes place after parsing but before transformation, so parsers are able to generate their own embeds. @@ -277,20 +486,21 @@ Embedding takes place after parsing but before transformation, so and update them accordingly; it will signal an error if the dependencies are recursive. +## Output Redirection + +By default, ⛩️📰 书社 installs files to the same location in `DESTDIR` + as they were placed in their `SRCDIR`. +This behaviour can be customized by setting the `@书社:destination` + attribute on the root element, whose value can give a different path. +This attribute is read after parsing, but before transformation (where + it is silently dropped). + ## Transforms Transforms are used to convert X·M·L files into their final output, after all necessary parsing and embedding has taken place. ⛩️📰 书社 comes with some transforms; namely :⁠— -- **`transforms/attributes.xslt`:** - Applies transforms to the children of any `<书社:apply-attributes>` - elements, and then applies the attributes of the - `<书社:apply-attributes>` to each result child, replacing the - element with the result. - This is useful in combination with image embeds to apply alt‐text to - the resulting ``. - - **`transforms/asset.xslt`:** Converts `` elements which correspond to recognized media types into the appropriate H·T·M·L elements, and deletes @@ -299,17 +509,44 @@ Transforms are used to convert X·M·L files into their final output, - **`transforms/metadata.xslt`:** Provides basic `` metadata. - This metadata is generated from `` elements with one o. + This metadata is generated from `` elements with one of the following `@itemprop` attributes :⁠— - **`urn:fdc:ladys.computer:20231231:Shu1She4:title`:** Provides the title of the page. - ⛩️📰 书社 automatically encapsulates embeds so that their metadata - does not propogate up to the embedding document. + ⛩️📰 书社 automatically encapsulates H·T·M·L embeds so that their + metadata does not propogate up to the embedding document. To undo this behaviour, remove the `@itemscope` and `@itemtype` attributes from the embed during the transformation phase. +- **`transforms/serialization.xslt`:** + Replaces `<书社:serialize-xml>` elements with the (escaped) + serialized X·M·L of their contents. + This replacement happens during the application phase, after most + other transformations have taken place. + + If a `@with-namespaces` attribute is provided, any name·space nodes + on the toplevel serialized elements whose U·R·I’s correspond to the + definitions of the provided prefixes, as defined for the + `<书社:serialize-xml>` element, will be declared using name·space + attributes on the serialized elements. + Otherwise, only name·space nodes which _differ_ from the definitions + on the `<书社:serialize-xml>` element will be declared. + The string `#default` may be used to represent the default + name·space. + Multiple prefixes may be provided, separated by white·space. + + When it comes to name·spaces used internally by ⛩️📰 书社, the + prefix used by ⛩️📰 书社 may be declared _in addition to_ the + prefix(es) used in the source document(s). + It is not possible to selectively only declare one prefix for a + name·space to the exclusion of others. + + `<书社:raw-output>` elements may be used inside of + `<书社:serialize-xml>` elements to inject raw output into the + serialized X·M·L. + The following are recommendations on effective creation of transforms :⁠— @@ -317,15 +554,42 @@ The following are recommendations on effective creation of It is likely an error if two transforms have templates which match the same element (unless the templates have different priority). -- Namespace templates (with `@name` or `@mode`) whenever possible. +- Name·space templates (with `@name` or `@mode`) whenever possible. - Set `@exclude-result-prefixes` on the root `xslt:transform` element - to reduce the number of declared namespaces in the final result. + to reduce the number of declared name·spaces in the final result. + +## Global Params + +The following params are made available globally in parsers and + transforms :⁠— + +- **`BUILDTIME`:** + The current time. + +- **`CKSUM`:** + The checksum of the source file (⅌ `cksum`). -The params `$buildtime`, `$srctime`, and `$path` are available within - transforms and are initialized to the current time, the time that the - source file was last modified, and the path of the output file within - $(DESTDIR). +- **`IDENTIFIER`:** + The ⛩️📰 书社 identifier of the source file (a u·r·i beginning with + `about:shushe`). + +- **`SRCREV`:** + The value of the `SRCREV` variable (if present). + +- **`SRCTIME`:** + The time at which the source file was last modified. + +- **`THISREV`:** + The value of the `THISREV` variable (if present). + +The following params are only available in transforms :⁠— + +- **`CATALOG`:** + The path of the catalog file (within `BUILDDIR`). + +- **`PATH`:** + The path of the output file (within `DESTDIR`). ## Output Wrapping @@ -352,7 +616,7 @@ In addition to being called with the transform result, each of these modes will additionally be called with a `` element corresponding to each transform. If a transform has a `<书社:id>` top‐level element whose value is an - i·r·i, its `` element will have a corresponding + i·r·i, its `` element will have a corresponding `@书社:id` attribute. This mechanism can be used to allow transforms to insert content without matching any elements in the result; for example, the @@ -379,11 +643,92 @@ This mechanism can be used to allow transforms to insert content Output wrapping can be entirely disabled by adding a `@书社:disable-output-wrapping` attribute to the top‐level element in the result tree. +It will not be performed on outputs whose root elements are + `<书社:archive>`, `<书社:base64-binary>`, or `<书社:raw-text>` + (described below). + +## Applying Attributes + +The `<书社:apply-attributes>` element will apply any attributes on the + element to the element(s) it wraps. +It is especially useful in combination with embeds. + +The `<书社:apply-attributes-to-root>` element will apply any attributes + on the element to the root node of the final transformation result. +It is especially useful in combination with output wrapping. + +In both cases, attributes from various sources are combined with + white·space between them. +Attribute application takes place after all ordinary transforms have + completed. + +Both elements ignore attributes in the `xml:` name·space, except for + `@xml:lang`, which ignores all but the first definition (including + any already present on the root element). +On H·T·M·L and S·V·G elements, `@lang` has the same behaviour as + `@xml:lang`. + +## Other Kinds of Output + +There are a few special elements in the `书社:` name·space which, if + they appear as the toplevel element in a transformation result, cause + ⛩️📰 书社 to produce something other than an X·M·L file. +They are :⁠— + +- **`<书社:archive>`:** + Each child element with a `@书社:archived-as` attribute will be + archived as a separate file in a resulting tarball (this attribute + gives the file name). + These elements will be processed the same as the root elements of any + other file (e·g, they will be wrapped; they can themselves specify + non X·M·L output types, ⁊·c). + Other child elements will be ignored. + + If the `<书社:archive>` element is given an `@书社:expanded` + attribute, rather than producing a tarball ⛩️📰 书社 will output + the directory which expanding the tarball would produce. + This mechanism can be used to generate multiple files from a single + source, provided all of the files are contained with·in the same + directory. + +- **`<书社:base64-binary>`:** + The text nodes in the transformation result will, after removing all + Ascii whitespace, be treated as a Base·64 string, which is then + decoded. + +- **`<书社:raw-text>`:** + A plaintext (U·T·F‐8) file will be produced from the text nodes in + the transformation result. + +## Pagination + +It is possible to have a single source file produce multiple output + files via `<书社:page>` elements, whose `@name` gives the name of the + page. +If a parsed document has a `@书社:destination` which contains `%s`, + the `%s` will be replaced with the `@name` for each `<书社:page>` (and + removed for the main output). +Otherwise, the `@name` is inserted before the first period of the + filename (or at the end of the filename for those with no period). +If `<书社:page>`s do not have a `@name`, they are numbered + sequentially. +The destination of pages must be in the same directory as their parent. + +Pagination essentially forms a limited convenience for the more + sophisticated technique of creating an archive with ⛩️📰 书社 and + then unarchiving it. +Pages are, from Make’s point of view, untracked side·effects of + installing the main output, meaning they cannot be targeted directly + and will not appear in `make list` or `make listout`. +They are intended solely for the like of indices and feeds, for which + convenience and necessity outweigh their flaws. ## License -Source files are licensed under the terms of the Mozilla Public - License, version 2.0. -For more information, see [LICENSE](./LICENSE). +This repository conforms to [REUSE][]. + +Most source files are licensed under the terms of the Mozilla + Public License, version 2.0. -[draft-phillips-record-jar-01]: \ No newline at end of file +[REUSE]: +[draft-phillips-record-jar-01]: