Quartz

This site is generated from a set of markdown files and directories using Quartz. I’ve always wanted to have my personal public space to talk about tech stuff and share interesting engineering experience.

Quartz is a tool that takes a directory name as an input and creates a set of publish-ready files from its contents. Its UI is amazingly minimal. Just text, a hierarchy tree and a beautiful dependency graph representing all the pages and their interconnections. I believe that a blog doesn’t need a lot of fancy stuff to do its job, it basically has to contain useful or inspirational text and be published on the web. Quartz reminds me of the old internet where it was nice to stumble upon something great buried under leaves. It was cozy and distraction-free. There are also such things as Gopher and its modern successor, Gemini, two whole mini internet spaces running on plaintext protocols with their own philosophy and web browsers.

While minimal and simple, Quartz is also very powerful with its plugin system, ability to stylize everything and change the page layout. Well, I haven’t really dug this deep so far; for now I’ve made all I needed to be able to publish my notes to the internet in a convenient way. But I’m inspired to play with styles and fonts later!

Problems

When I first tried to generate a site from .md files, I did it easily, files were actually generated. But then they needed to be published, and It became trickier. I decided not to host files on GitHub, as the authors recommended; I was going to serve it myself with nginx alongside my other services under docker.

The goal: I wanted to be able to edit my files from desktop or on-the-go via mobile and publish them to the internet automatically or with a press of a button. Just add a new file, write your text, press “publish”, and the changes are on the web.

Problem 1: Quartz can’t serve the files itself. It’s got a nice little debug web server included (--serve) that is able to serve your files on a chosen port on localhost via http, so… not a production option. So I chose nginx as a web server. Quartz docs have a documentation section on how to host the files this way.

Problem 2: Quartz doesn’t know if your source files change. Is just a tool that just generates site files from markdown files, so I’d need to rebuild it every time I change something. Again, it has a special --watch mode that’s actually said to do the thing, but I didn’t use it since I thought it wasn’t very production-ready, too. I may have been wrong; I may reconsider this in the future.

Problem 3: Quartz doesn’t give you the ability to edit the contents. You would need to edit it yourself via a text editor of some kind. That’s where Quartz’s compatibility with Obsidian comes into play.

NOTE

As I learned later, Obsidian has a paid service that hosts a full read-only version of your project. Their UI is suspiciously similar to Querz’s, so I’m thinking the latter may have been created to allow the same for free.

I was going to connect some things together in a nice hand-crafted pipeline, that took care of all those problems.

Solutions

I. Quartz can’t serve the files itself

An nginx instance can easily do this with the try_files directive.

Disclaimer: I don’t set up a full SSL here because I’m hosting a separate gateway nginx instance, that is responsible for the SSL and routing requests to different services.

So, the config:

server {
    listen 80; # proxied here from the gateway
    server_name thepunkoff.xyz; # idk if it's needed, I guess it's not, cause the gateway has the same directive
    root /public; # I've set up a local Quartz's /publish folder to be mounted at /publish in a container via a docker volume
    index index.html; # this page shows up first at the root
    error_page 404 /404.html; # an oops page, if anything
 
    location / {
        try_files $uri $uri/ $uri.html =404; # try serve as said, if not found, try a dir, if no, try with an extension, if no, oops
    }
}

Then I just ran it as a docker-compose project in a directory with my public folder where the production files dwell:

services:
  quartz:
    image: "nginx:latest" # better chose a static version later
    ports:
      - "9999:80" # ignore the exposed port here, that's for debugging
    volumes:
     - "./nginx_configs/default.conf:/etc/nginx/conf.d/default.conf"
     - "./public:/public"
    restart: "always"
    networks: ["nginx"] # this is a way to connect multiple docker compose projects into a whole network and be able to use service names as hostnames
 
networks:
  nginx:
    name: "nginx" # the gateway sees this little nginx instance and is able to proxy to it as "http://quartz:80"
    driver: "bridge"

Yay! Now I should be serving my Quartz site on thepunkoff.xyz/home!

Nope. I actually needed a couple of hours to figure out why this didn’t work.

The explorer didn’t show up, and I was seeing lots of errors in the console (btw, the errors were what helped me; I didn’t think of looking at the console for a lot of time and wasted hours editing nginx and Quartz coinfigs).

So, as you can see, the breadcrumbs show the hierarchy, which is not mirrored in the explorer on the left. It seems to be empty.

And there were errors about some security policy:

Long story short, my gateway nginx config had this sneaky thing, that disallowed downloading scripts, css-stylesheets and fonts from outer servers. Why?

Well, I am hosting a Jellyfin instance, and it’s server from the gateway nginx. When I was setting it up, I pasted the security directive from this guide:

add_header Content-Security-Policy "default-src https: data: blob: ; img-src 'self' https://* ; style-src 'self' 'unsafe-inline'; script-src 'self' 'unsafe-inline' https://www.gstatic.com https://www.youtube.com blob:; worker-src 'self' blob:; connect-src 'self'; object-src 'none'; frame-ancestors 'self'; font-src 'self'";

Because It was configured globally, it affected Quartz, too. It disallowed downloading stuff that Quartz files depends on. I moved this directive to Jellyfin-specific locations, and everything started working. Now, yay.

II. Quartz doesn’t know if your source files change

We need a tool to monitor file changes, possibly including creation, deletion and directory manipulations. There’s this inotify subsystem in linux, which you can use to monitor any change in the file system. Many tools use its API, but I just chose entr. It’s a CLI tool that you call for a target location, and it calls an arbitrary command or a script on any change. So I used it to monitor my Quartz content folder with markdown files and to call a build script when anything changed.

The build script is essentially a call to the quartz tool:

npx quartz build -d /home/thepunkoff/quartz/content -o /home/thepunkoff/quartz/public

The call to entr would look like this. You pass it the place to monitor:

#!/bin/bash
 
while true; do
  find /home/thepunkoff/quartz/content -type f | entr -d /home/thepunkoff/quartz/build.sh
  sleep 1 # maybe not needed, but just to make sure it waits a bit before restarting
done

Yeah this is a correct usage — with a loop. The thing is, entr was created to monitor files, but the -d argument makes it also monitor adding of new files to a whole directory; it exits if this happens, so I’d need to restart it again.

NOTE

I don’t know yet if it detects if a file is deleted, but I think this event is already extra. I just needed change something in any file to trigger a rebuild from scratch. The directory monitoring would not be needed, too, but it was trivial to set up, so I did it.

Cool, now we can rebuild out site on the fly when any change occurs. An entr call blocks the UI, so we need to “host” this process somewhere. The first things that come to mind are nohup and screen but those are hard to monitor, and I’m used to conveniences like statuses, restarts on errors and such stuff. I need to be able to see that my entr is running and its logs. A docker container seemed to be an overkill, because I was hosting only full scale services there. So I settled on a systemd service.

I packed my entr call as a nice little service (I just googled how it’s should be done):

[Unit]
Description=quartz site background builder
After=network.target
 
[Service]
ExecStart=/home/thepunkoff/quartz/builder.sh
Restart=always
RestartSec=5
StandardOutput=append:/home/thepunkoff/quartz/builder_log.log
StandardError=append:/home/thepunkoff/quartz/builder_log.log
 
[Install]
WantedBy=multi-user.target

It restarts if anything happens, which solves my biggest worry about background running processes. Maybe I’m paranoid and should’ve used screen but I like things organized. I’m a Virgo. It also writes all output to a log file. I’d need to figure out log rolling sometime in the future.

I packed this into a file at /etc/systemd/system/quartz-builder.service , called systemctl daemon-reexec and systemctl daemon-reload to make it detect new service. And spun it up via systemctl start quartz-builder. And it didn’t work.

It turned out entr needed to be called with a -n argument, or else it would try to interact with the TTY, which isn’t suitable for a background job.

the fix:

find /home/thepunkoff/quartz/content -type f | entr -n -d /home/thepunkoff/quartz/build.sh

Now everything worked, it detected changes and new files, and called the build script. I also needed to restart the nginx container that serves the site each time I called Quartz for it to reload new files. I don’t know why, but I didn’t want to spend time to look for a more elegant solution.

The final service code:

#!/bin/bash
 
while true; do
  find /home/thepunkoff/quartz/content -type f | entr -n -d /home/thepunkoff/quartz/build.sh
  sleep 1 # maybe not needed, but just to make sure it waits a bit before restarting
done

The final build script:

#!/bin/bash
 
cd /home/thepunkoff/repos/quartz
npx quartz build -d /home/thepunkoff/quartz/content -o /home/thepunkoff/quartz/public && \
docker compose -f /home/thepunkoff/quartz/docker-compose.yaml down && \
docker compose -f /home/thepunkoff/quartz/docker-compose.yaml up -d && \
curl -d "Quartz rebuilt 😀" ntfy.sh/thepunkoff

Yeah, there’s a little cherry on the top at the end. A call to ntfy.sh free topic. My android phone is monitoring the topic via the ntfy app, and sends me a message when quartz is rebuild. Sick!

NOTE

ntfy.sh is a paid service, but they provide free topics usage without any authorization. Well, I’m not afraid if someone DDoS’es my topic. I’d be honored, actually.

III. Quartz doesn’t give you the ability to edit the contents

Sure it doesn’t. But Obsidian does! It’s just a fantastic minimalistic tool to do its job and nothing more, just like Quartz itself. But my content folder is on my server linux machine, and even though the machine has a GUI, It’s not convenient to edit everything there, and the goal was to make everything convenient.

Obsidian has their own cloud sync service, that you can use to make shure your files sync with the cloud on multiple devices. Sounds like a perfect way to enable editing from desktop and on-the go with Obsidian app, but they charge you $4 per month for it, and that’s just another subscription to keep track of, so that wasn’t for me. I actually ended up buying a premium app (a one time purchase) for the solution, cause I needeed a quick POC, but again, sometime in the future I’ll come up with a true self-hosted scheme.

I’m hosting Nextcloud on my server and I wanted to use it for storing my .md files. Their desktop client is able to sync stuff with the cloud (for free!), and that’s how I set up the server to get the changes: I synced my cloud folder with Quartz’s content folder. It did the job on the server, but I didn’t want to install Nextcloud’s service on my workhorse PC, since I’ve been using Nextcloud from it via web and wanted to keep things that way. As for mobile, Nextcloud’s client can’t sync local folders with the cloud, unfortunately. So I bought FolderSync both for mobile (cheap) and PC desktop (not so cheap). It seamlessly integrates with multiple storage providers and keeps your data consistent. And it’s silmple to use!

Everything was ready. I could edit my site source files with Obsidian and push them to the server via FolderSync, and Nextcloud sync. There, entr called Quartz to rebuild everything, restarted nginx and sent a notification to my phone, when everything was done.

The whole pipeline looks like this: (this gnarly diagram was created with an online plantuml editor, source code)