Forgejo and Anubis
Over the past few days I’ve read about, explored, and then set up a local Forgejo instance as a self-hosted Github alternative. I’ve also deployed the Anubis scraper-blocking proxy in front of all my websites.
This is all a result of AI scraping. My decision to leave Github after
many years is because Microsoft announced that even private activity
would be used to train their Copilot bot. Meanwhile the rapacious
global scraping (and ignoring of robots.txt) by other companies'
bots led to my decision to implement Anubis.
Forgejo was surprisingly easy to get set up and running on a test machine. Getting it fully configured for production use behind Anubis wasn’t super-tricky, but did require a bit of finagling.
Anubis
I use Arch Linux, and Anubis has an Arch package, so that was easy enough:
$ sudo pacman -S anubis
Interestingly, Anubis has an @ style service definition file, which is
constructed so that you can have multiple configurations, each of
which is treated as a separate service by systemctl. Initially I
stuck with the default config file (/etc/anubis/default.cfg) to
protext my static sites, and brought up that service:
$ sudo systemctl enable anubis@default
$ sudo systemctl start anubis@default
Then I modified my nginx config, site-by-site into the split config
suggested by the Anubis docs. With that done I turned to creating a
second config for Forgejo. This was needed because Anubis is a filter
proxy, sitting in between whatever is handling TLS termination and
whatever is actually serving websites. For my static sites this was
two nginx listeners (one on port 443, one on a unix socket). But for
Forgejo, nginx would handling TLS and Forgejo itself would be
handling the HTTP requests. I copied the existing config to a new
file, forgejo.cfg which looked like this:
BIND=/run/anubis/forge.sock
BIND_NETWORK=unix
SOCKET_MODE=0666
DIFFICULTY=4
METRICS_BIND=127.0.0.1:9091
SERVE_ROBOTS_TXT=0
TARGET=http://127.0.0.1:3000
The line which sets the socket mode was critical, as Anubis and Forgejo run as different users. I brought up a second instance of Anubis and turned to Forgejo:
$ sudo systemctl enable anubis@forgejo
$ sudo systemctl start anubis@forgejo
Forgejo
The first thing to do (for me) was to enable the long-delayed setup of
rootless podman on the machine where Forgejo would be running
(consult a guide relevant to your own distro). With that done, I
created a volume which would attach to the Forgejo container’s /data
mountpoint.
podman volume create forgejo
Next, a network was needed, so that the container didn’t end up in
pasta’s default network, which would prevent any other containers
from being able to see it.
podman network create forgejo
Then I stood up a container. My first attempt was a failure, due to
not being used to how volumes behave under rootless podman. After
discovering that the bind mount needed the :U argument, I got things
working with:
$ podman run --name "forgejo" -d -p 127.0.0.1:3000:3000 -p 2222:2222 \
--network forgejo -v forgejo:/data:U \
codeberg.org/forgejo/forgejo:14-rootless
This runs Forgejo with the httpd only listening on port 3000 (its
default, and matching the Anubis config) on the localhost interface,
so that it is inaccessible except by going through Anubis. sshd, on
the other hand, attaches to port 2222 on all interfaces so that it’s
accessible anywhere/anyhow. And as mentioned, the /data volume is
mounted with rootless user-mapping enabled.
The mount mapping is important because otherwise /data becomes
unwritable from inside the container, and that’s where I wanted to put
all the datastores. In the configuration step, I set all repos, LFS,
the internal DB, and logs, to be written to locations under
/data. This, in turn, lets me snapshot the state of the Forgejo
instance using rsync (to clone the data to another local machine)
and aws sync (to then clone it to S3-compatible storage).
The only other wrinkle was that only a pretty limited amount of
configuration is possible during initial setup. To change anything
else, editing the config file is required, and can only be done within
the container – unless a copy is moved out of the container. The
/data mount makes exfiltration very easy ('podman cp' could also be
used). No matter how you decide to do it, the file is at
/var/lib/gitea/custom/conf/app.ini
So far, using Forgejo has been fast and flawless. I’ll probably start work on adding runners to my config this weekend, to bring back the build-based PR gating for my Golang projects.
Update: I’ve upgraded to Forgejo 15, and after reading the upgrade
docs I created a second volume, forgejo-conf, which holds a single
file: the configuration file. It gets mounted along with the /data dir
at runtime:
$ podman run --name "forgejo1501" -d -p 127.0.0.1:3000:3000 -p 2222:2222 \
-v forgejo:/data:U -v forgejo-conf:/var/lib/gitea/custom/conf:U \
--network forgejo codeberg.org/forgejo/forgejo:15-rootless
Also, on upgrade the server keys are regenerated. Luckily they too are just files, so they can be mounted on yet another volume, or just copied to any durable place and then restored after the upgrade. Currently, I’ve copied mine to the config mount, which makes them restorable by logging in and:
cp custom/conf/gitea.rsa* ssh/
Then restarting the container.