Self-Hosted Netlify Builds with GitLab CI

Once I had (mostly) recovered from a bungled Git LFS migration, I had no trouble switching from Netlify Large Media to eleventy-img. At least, I could do it in my local working tree. The next step was to make it work as part of my build. That was when the trouble began.

I wanted to move from having Netlify build the site to building it myself with GitLab CI on my own runners in Kubernetes. (This would provide more control over the build environment and avoid devouring the limited build minutes Netlify provides for free.) netlify deploy will deploy any files under the output directory as a draft; --alias foo will add foo to the draft URL and --prod will deploy it to the production site.

However, I wasn’t able to build the site in a Linux Docker container. I got an error referring to shared.package and dir in the esm library. esm allows CommonJS modules and ECMAScript Modules or ESM to interoperate; the original library is no longer maintained, so I use an also-no-longer-maintained fork. Eleventy doesn’t support ESM but many modern libraries are only available in that format.

Poring over the output of npm ls and even examining the esm library itself revealed no differences between my (working) Windows environment and the Linux environment other than the presence or absence of the fsevents module. However, I did find that the Windows version could only be built with that specific, existing node_modules directory; deleting and replacing the directory would stop the build from working.

I spent some time trying—as I have done twice before—to convert my blog entirely to ESM (which Node natively supports). The stumbling block continued to be Eleventy. I reverted to the esm workaround.

Through trial and error, I realized it would fail to build in Node v14 and wouldn’t run at all in v18, but the esm errors went away with v16 under both Windows and Linux. The next step was to get sharp (the image processing library powering eleventy-img) working on Linux: Windows worked with a prebuilt binary, but the version of libvips in the Linux image was too old. sharp couldn’t successfully compile a custom version, either.

I ended up creating a Docker image with Node and libvips preinstalled. That solved the problem, although I wasted a great deal of time between some silly issues caused by using a tmpfs mount and forgetting entirely that I was using LFS.

I had to add a dedicated CI machine to my Kubernetes pool because the build was chewing up 20 GB or more of RAM with the image processing… certainly not normal, considering only 10 images are being processed at a time, but I have no idea how to fix my code right now. (On my local machine, it stays under 2 GB in the host Windows but hits 20 GB under Linux in Docker.) At any rate, the first successful run took 45 minutes. After that, the cache brought it down to two.

Here are the relevant parts of my GitLab pipelines:

variables:
  LIBVIPS_IMAGE: shivjm/node-libvips:node16-libvips8.14.1

draft:
  stage: draft
  image: $LIBVIPS_IMAGE
  variables:
    GIT_DEPTH: 3
  cache:
    paths:
      - dist/assets/images/resized
      - node_modules
    key: draft
    when: on_success
  script:
    - npm install
    - NODE_ENV=production npm run build
    - npx netlify deploy --alias $CI_COMMIT_REF_SLUG
  environment:
    name: draft/$CI_COMMIT_REF_SLUG
    url: https://$CI_COMMIT_REF_SLUG--$NETLIFY_SITE_ID.netlify.app/
  only:
    - merge_requests

deploy:
  stage: deploy
  image: $LIBVIPS_IMAGE
  variables:
    GIT_DEPTH: 3
    CONTEXT: production
  cache:
    paths:
      - dist/assets/images/resized
      - node_modules
    key: deploy
    when: on_success
  script:
    - npm install
    - NODE_ENV=production npm run build
    - npx netlify deploy --prod
  environment:
    name: production
    url: https://shivjm.blog/
  only:
    - main

NETLIFY_AUTH_TOKEN and NETLIFY_SITE_ID are defined as variables elsewhere. I set GIT_DEPTH to a low number since I don’t need any history. I configure eleventy-img to produce images at dist/assets/images/resized, so that’s what I cache.

Note that GitLab ‘stops’ an environment for a merge request once it’s merged. I could add an on_stop job to delete the corresponding deployment from Netlify, but I’d rather keep them around for as long as possible (only 90 days, as it happens).

Next in series: From broken-links-inspector to muffet(#34 in Colophon: Finding A Place For My Head)