[2023-12-17] programming_resolutions_24

[Blog] / 2023-05-06 / touch_grass / #entry.rdf

<awol:Entry
        xml:lang="en"
        xmlns:awol="http://bblfish.net/work/atom-owl/2006-06-06/"
        xmlns:dc11="http://purl.org/dc/elements/1.1/"
        xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
        xmlns:sioc="http://rdfs.org/sioc/ns#"
>
        <dc11:title>C·I pipelines have you down? Why not ‹ touch .grass ›?!</dc11:title>
        <dc11:date>2023-05-06T23:19:48-07:00</dc11:date>
        <dc11:abstract rdf:parseType="Markdown"><![CDATA[
You’re already building your site locally to develop it and test in
your browser. You don’t need to build it again on someone else’s
machine. Just upload the dang site.
]]></dc11:abstract>
        <sioc:content rdf:parseType="Markdown"><![CDATA[
This is a blogpost targeted at those who are, or are looking to
become :—

- Developers of independent single‐administrator static websites,

- Who have access to a file server that can serve their website on the
  internet (Neocities, a personal server, a tilde club, ⁊·c),

- And are familiar with Git and plan on using Git to track changes to
  the sites they create.

If this is you, and you have an account on GitLab or GitHub, you may be
considering using the pipeline features of GitLab C·I or GitHub Actions
to build and deploy your site every time you push new changes to the
repository. (Or, if you are very adventurous, you may be considering
writing a `post-receive` Git hook to do this yourself.)

C·I pipelines are very trendy right now because GitLab and GitHub can
make money by selling you the minutes needed to run them. And to be
clear: When you are working on a team, these features are extremely
useful and convenient! Also, if you are developing a library which
other people will use, green pipelines can help assure library users
that the code they are pulling passes its own test suite (for whatever
that knowledge is worth). But if you are just building a personal
website, for yourself, on your own computer?

This blogpost will walk you through an easier way. It doesn’t require
any additional accounts or services, it runs entirely on your own
computer, and it is 100% free and uses only programs you likely already
have installed.

## Prerequisites

In order to put this method into practice, you will need the following
things installed on your computer :—

- GNU Make (check with `make --version`).

- A program for syncing your website with a remote server. This
  blogpost will use Rsync, but the [Neocities C·L·I `push`
  tool](https://neocities.org/cli) can be used if you are pushing to a
  Neocities site.

In addition, you will need basic familiarity with the command line.

## A Simple Build Script

For demonstration purposes, let’s get ourselves a build script for
generating a website. [Deno](https://deno.com/runtime) provides a fast
and convenient way to write useful scripts, so I’ll use that. The
following is a simple Deno script for converting Markdown files into
their H·T·M·L equivalents.

```js
#!/usr/bin/env -S deno run --allow-read --allow-write
// build.js
// ====================================================================
//
// Copyright © 2023 Lady [@ Lady’s Computer].
//
// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this
// file, You can obtain one at <https://mozilla.org/MPL/2.0/>.

// We’ll use the `rusty_markdown` package to parse Markdown files.
import {
  html as markdownTokensToHTML,
  tokens as markdownTokens,
} from "https://deno.land/x/rusty_markdown@v0.4.1/mod.ts";

/**
 * Processes a directory by converting all the Markdown files in it
 * (recursively) to HTML.
 */
const processDirectory = async (path) => {
  for await (const entry of Deno.readDir(path)) {
    // Iterate over each entry in this directory and handle it
    // accordingly.
    const { isDirectory, isFile, name } = entry;
    if (isFile && /\.(?:md|markdown)$/.test(name)) {
      // This entry is a file ending in `.md` or `.markdown`. Parse its
      // text contents into H·T·M·L and write the corresponding file.
      const markdown = await Deno.readTextFile(`${path}/${name}`);
      const html = markdownTokensToHTML(markdownTokens(markdown));
      await Deno.writeTextFile(
        `${path}/${name.replace(/\.(?:md|markdown)$/, ".html")}`,
        `<!DOCTYPE html>${ html }`,
      );
    } else if (isDirectory) {
      // This entry is a directory. Process it.
      await processDirectory(`${path}/${name}`);
    }
  }
};

// Process the current directory (or the one supplied to the script).
await processDirectory(Deno.args[0] || ".");
```

Obviously, this script leaves a lot to be desired: It doesn’t do any
templating, styling, or other processing of the input files, and so the
resulting H·T·M·L pages will be very boring. But it is good enough for
a demo. Feel free to substitute for this script whatever build system
you like.

## A Simple Makefile

Building our website is already pretty easy, but there’s nothing easier
than just typing `make`, so next let’s create a Makefile for it. If
you’ve never encountered a Makefile before, they simply (or
not‐so‐simply) group targets (followed by a colon and their
prerequisites) with the rules (preceded by a tab) needed to make them.
There is a whole lot more information in [the GNU Make
manual](https://www.gnu.org/software/make/manual/html_node/), but for
this post that’s the gist. A simple one might look like this :—

```make
build:
        deno run --allow-read --allow-write ./build.js
```

Make automatically builds the first target if you don’t specify one on
the command line, so now when you type `make` it should build your
site. Hooray!

While we’re at it, let’s add some rules for syncing, too.

```make
# …continued from above

# This code assumes you have defined an Rsync filter at `.rsync-filter`
# which excludes all the files you don’t want to sync. If you’re using
# the Neocities C·L·I, it should automatically ignore anything in your
# `.gitignore`.
#
# I’m not going to go into all of the details on how Rsync works right
# now, but I encourage you to read up on it if you ever need to sync a
# local and remote copy of your website.

dry-sync:
        rsync -Oclmrtvz --del --dry-run --filter=". .rsync-filter" . webserver:www/public

sync:
        rsync -Oclmrtvz --del --filter=". .rsync-filter" . webserver:www/public
```

Now `make dry-sync` will show us what will happen if we try to sync our
local repository with our webserver, and `make sync` will actually do
it. `make && make sync` is an all‐in‐one way of building and syncing
our website, which is pretty convenient.

The above Makefile is literally all you need to build and deploy your
site, but it comes with some drawbacks. If you forget to build before
you sync, you might accidentally push an older version of your site
than you intended. And what about version control? It would be nice
if we ensure that is up‐to‐date at the same time.

## Adding Version Control Support

In fact you can add all of that functionality to the same Makefile
without compromising the simplicity of running `make` and `make sync`.
Let’s start with version control: `git status --porcelain` will give us
the current status of the working tree, and be empty if all of our
changes are committed. `[ ! -z $(git status --porcelain) ]` is how you
say “`git status --porcelain` is not empty” in command line, so we can
use that to create the following check :—

```sh
if [ ! -z "$(git status --porcelain)" ]
then
  echo 'Error: There are uncommitted changes!'
  echo 'Commit changes and run `make` before syncing.'
  exit 1
fi
```

Let’s also test that the current branch is up‐to‐date. In Git, the
current branch is signified with `HEAD` and its upstream can be
signified with `@{u}`. The command `git merge-base --is-ancestor` will
error if its first argument isn’t an ancestor of its second, so we can
use it in an `if` statement directly.

```sh
if ! git merge-base --is-ancestor @{u} HEAD
then
  echo 'Error: This branch is currently out‐of‐date!'
  echo 'Pull in changes with `git pull` before syncing.'
  exit 1
fi
```

When we convert these checks into Makefile, we need to collapse them
into single lines and escape any dollar signs (with additional dollar
signs). So the Makefile rules for ensuring our files are committed and
our branch is up‐to‐date are :—

```make
# The `@` at the beginning of the if statements tells Make not to
# bother printing anything for those lines. That’s okay, since the
# lines will echo out their own content if there is an error.

ensure-branch-up-to-date:
        git fetch
        @if ! git merge-base --is-ancestor @{u} HEAD; then echo 'Error: This branch is currently out‐of‐date!'; echo 'Pull in changes with `git pull` before syncing.'; exit 1; fi

ensure-clean:
        @if [ ! -z "$$(git status --porcelain)" ]; then echo 'Error: There are uncommitted changes!'; echo 'Commit changes and run `make` before syncing.'; exit 1; fi
```

We can ensure these checks run by adding them as prerequisites to our
syncing targets. Let’s add a `git push` to our `sync` target as well,
since we now know our branch is up‐to‐date.

```make
dry-sync: ensure-clean ensure-branch-up-to-date
        rsync -Oclmrtvz --del --dry-run --filter=". .rsync-filter" . webserver:www/public

sync: ensure-clean ensure-branch-up-to-date
        rsync -Oclmrtvz --del --filter=". .rsync-filter" . webserver:www/public
        git push
```

## Checking for Builds by Touching Grass

To make sure that our latest build is current, we need to add an
additional rule to our `build` target :—

```make
build:
        deno run --allow-read --allow-write ./build.js
        touch .grass
```

`touch` is a simple command which updates the modification time on a
file, creating it if it doesn’t already exist. By touching `.grass` (an
otherwise meaningless file which we wouldn’t ever modify manually), we
can easily keep track of when our site was last built. **Add this file
to your `.gitignore` file so that Git doesn’t ever touch it for you.**

Because we already have checks in place to ensure that all of our
changes have been committed, ensuring our build is current is just a
matter of comparing times with the commit date on the latest commit.
You can get the modification date of `.grass` with
`stat -f '%m' .grass` (`%m` for “modified”), and you can get the last
commit time with `git log -1 --format='%ct'` (`%ct` for “commit time”).
So some suitable checks might be :—

```sh
# This check ensures a file named `.grass` exists.
if [ ! -f .grass ]
then
  echo 'Error: The website has not been built yet!'
  echo 'Run `make` before syncing.'
  exit 1
fi

# This check ensures the commit time isn’t greater than the modified
# time.
if [ "$(git log -1 --format='%ct')" -gt "$(stat -f '%m' .grass)" ]
then
  echo 'Error: A commit was made after the last build!'
  echo 'Run `make` before syncing.'
  exit 1
fi
```

Once again, when we move these checks into our Makefile, we need to
collapse them into single lines and escape their dollar signs :—

```make
ensure-build:
        @if [ ! -f .grass ]; then echo 'Error: The website has not been built yet!'; echo 'Run `make` before syncing.'; exit 1; fi
        @if [ "$$(git log -1 --format='%ct')" -gt "$$(stat -f '%m' .grass)" ]; then echo 'Error: A commit was made after the last build!'; echo 'Run `make` before syncing.'; exit 1; fi
```

## Putting it all together

Our final Makefile might look something like this :—

```make
# Makefile
# =====================================================================
#
# © 2023 Lady [@ Lady’s Computer].
#
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at https://mozilla.org/MPL/2.0/.

build:
        deno run --allow-read --allow-write ./build.js
        touch .grass

ensure-branch-up-to-date:
        git fetch
        @if ! git merge-base --is-ancestor @{u} HEAD; then echo 'Error: This branch is currently out‐of‐date!'; echo 'Pull in changes with `git pull` before syncing.'; exit 1; fi

ensure-clean:
        @if [ ! -z "$$(git status --porcelain)" ]; then echo 'Error: There are uncommitted changes!'; echo 'Commit changes and run `make` before syncing.'; exit 1; fi

ensure-build:
        @if [ ! -f .grass ]; then echo 'Error: The website has not been built yet!'; echo 'Run `make` before syncing.'; exit 1; fi
        @if [ "$$(git log -1 --format='%ct')" -gt "$$(stat -f '%m' .grass)" ]; then echo 'Error: A commit was made after the last build!'; echo 'Run `make` before syncing.'; exit 1; fi

dry-sync: ensure-clean ensure-branch-up-to-date ensure-build
        rsync -Oclmrtvz --del --dry-run --filter=". .rsync-filter" . webserver:www/public

sync: ensure-clean ensure-branch-up-to-date ensure-build
        rsync -Oclmrtvz --del --filter=". .rsync-filter" . webserver:www/public
        git push
```

To recap, this Makefile :—

- Builds the site with `make` (or `make build`) and pushes it to your
  webserver with `make sync`.

- Refuses to sync if your current branch isn’t up‐to‐date.

- Refuses to sync if you have uncommitted changes in your working
  directory.

- Refuses to sync if you haven’t built your site since your latest
  commit.

It still isn’t perfect: It can’t protect you against incorrect builds
which you make *after* your latest commit, for example builds based on
files you have since deleted or stashed away. But it should catch most
common mistakes. For a fuller example, see [the Makefile for this
blog](https://git.ladys.computer/Blog/blob/14b487e:/GNUmakefile), which
is a little more verbose and includes some (small) additional
functionality.

It’s rare that I see people working in website technologies
(Javascript; static site generators) talking about Makefiles, which
makes sense because they kind of have an association with old compiled
languages and application programming. But they’re dead‐simple to
write, and can really simplify automation of a lot of tasks! Mostly I
worry that people write themselves into either overly‐complicated
technical solutions or a lot of error‐prone manual work for something
which could be as easy as typing a few words on a command line.

Don’t let this be you! Do the easy thing! `make sync`!
]]></sioc:content>
        <dc11:rights rdf:parseType="Markdown"><![CDATA[
Copyright © 2023
<a href="https://www.ladys.computer/about/#lady">Lady</a>
<small>[Exquisite Grass‐Toucher]</small>.
Some rights reserved.

This blogpost is licensed under a <a rel="license"
href="http://creativecommons.org/licenses/by/4.0/"><cite>Creative
Commons Attribution 4.0 International License</cite></a>.
]]></dc11:rights>
</awol:Entry>