BP SYKORAX.EU
Infrastructure

How this is built

There was no plan. It started as a backup drive with a Forgejo instance on it, somewhere to keep my own git repos and a copy of my files. Then I started adding more, upgrading the hardware, and seeing what else the server could do. Along the way that turned into actually learning Linux administration, Docker, firewalls, and general security, mostly by taking someone else's guide and reworking it for what I actually had. The security basics started from a Hetzner tutorial written for securing Fedora, which I adapted to openSUSE instead.

That's still how it grows. The most recent addition was Grafana, after a couple of coworkers mentioned how difficult they'd found it to learn, which I took as a reason to sit down and actually give it a go. Before that it was Gentoo. I'd read through the handbook twice, but the first install (on my laptop) still took three attempts and about eight hours, tutorials and all. I was expecting to use it for a month and go back to openSUSE. That was three months ago. I've grown fond of emerge.

What follows are the five pieces of it I'd actually call engineering rather than just configuration.

Binary Package Pipeline 01

one machine compiles software once, everything else just installs the result.

Gentoo builds almost everything from source, which is powerful but slow, especially across more than one machine. Instead of every device independently spending hours compiling the same packages, one build server compiles once and the rest pull ready-to-run binaries over the network.

The interesting part isn't the building, it's the validation that happens first. A CI pipeline dry-runs every proposed change (new package, changed build flag) before anything actually gets built. It checks the change resolves cleanly, blocks anything that would touch a protected core package, and automatically reverts a change that fails validation rather than leaving the system in a broken state. Only changes that pass get queued for the next scheduled build.

Every machine on the network targets a shared baseline CPU instruction set rather than the fastest option for any single machine, which sounds like a downgrade until you realise it means a binary built once runs correctly everywhere, instead of needing a separate build per machine.

Gentoo / Portage Forgejo Actions Self-hosted CI/CD Chroot build isolation

Custom Package Overlay 02

software that isn't packaged anywhere gets packaged automatically, on a schedule.

Not everything I want to run has an existing package, and some upstream projects release new versions often enough that manually repackaging each one would be its own part-time job. A small overlay handles that instead: it watches for new upstream releases, generates the packaging automatically, and pushes the change through the exact same validation pipeline as everything else.

The point isn't the specific packages, it's that packaging stops being a manual chore and becomes something the system does to itself. If a check ever finds a release that doesn't fit the expected shape, it fails loudly instead of shipping something broken.

Scheduled automation Release tracking Automated packaging

Network & VPN 03

most of it isn't reachable from the internet at all, on purpose.

Most of what runs here has no business being reachable from the open internet, so it isn't. Internal services sit behind a WireGuard VPN and are only reachable once connected to it; a deliberately small set of services that are meant to be public sit on a separate, more exposed layer with its own protections. Traffic that doesn't need to leave the private network never does, general browsing and internal traffic are kept apart at the routing level rather than trusted to stay separate by convention.

DNS resolution for internal services only works once you're actually on the network, so an internal hostname is meaningless to anything outside it.

WireGuard Split tunnel routing Reverse proxy separation Internal DNS

Federated Services 04

running the protocol yourself, instead of just using someone else's server.

A couple of services here speak open, federated protocols rather than depending on one company's platform: a Matrix homeserver, and a Fediverse (ActivityPub) instance. Both talk to independent servers the same way any other server on those networks does. Right now that's mostly a personal sandbox for understanding how federation actually behaves in practice, identity, delivery, retries, the parts that only really show up once you're running the server side and not just a client.

It's a different kind of learning than running your own CI pipeline. Federation only works if you get the boring, specific parts of a protocol right, and the only way to find out which parts those are is to actually run it.

Matrix ActivityPub Self-hosted federation

Self-Hosted Agent 05

a language model that can propose infrastructure changes, and the guardrails that made that reasonable.

Everything above is maintained by hand, which is fine until the maintenance itself becomes the hobby. So there's now a language model running locally on the server's CPU, with access to the repositories that define how the machine is configured, and permission to open pull requests against them.

The interesting problem wasn't getting it to write changes, it was deciding what it should be able to reach. Giving a model a shell on the build host solves the problem in the laziest possible way and creates a much worse one. Instead it talks to a read-only query API I wrote for the purpose: a fixed set of verbs over the build chroot, no shell, arguments validated against a whitelist, and no endpoint that can install, build, or write anything. It can ask what version of a package is installed and why one failed to compile. It cannot act on the answer.

Everything it proposes goes through the same CI that validates my own changes, and findings from the nightly maintenance run reach it as plain files rather than a webhook, so a broken agent means nothing gets read rather than something gets triggered. It runs on six CPU cores, which means a single reply can take twenty minutes. That constraint turned out to be clarifying: when every action is expensive, you think properly about which ones are worth taking.

llama.cpp CPU inference Scoped tool access Read-only API design