I run k3s in my homelab as a single node cluster. I’m very familiar with kubernetes in general, so it’s just easier for me to reason with a control plane.

Some of the benefits I find useful:

• ArgoCD set to fire and forget will automatically update software versions as they happen. I use nix to lower the burden of maintaining my chart forks. Sometimes they break, but
• VictoriaMetrics easily collects all the metrics from everything in the cluster with very little manual tinkering, so I am notified when things break, and
• zfs-localpv provides in-cluster management for data snapshots, so when things do break I can easily roll back to a known good state.

k3s is, of course, a memory hog, I’d estimate it and cilium (my CNS of choice) eat up about 2Gb ram and a bit under one core. It’s something you can tune to some extent, though. But then, I can easily do pod routing via VPN and create services that will automatically get a public IP from my endless IPv6 pool and get that address assigned a DNS name in like 10 lines of Yaml.

Is there anything interesting at all reported in /proc/spl/kstat/zfs/dbgmsg?

I did ran out of pcie, yeah :-( the network peaks at about 26gbit/s, which is the most you can squeeze out of pcie 3.0 x4. I could move the nvmes off the pcie 4.0 x16 (I have two m2 slots on the motherboard itself), but I planned to expand the nvme storage to 4x SSDs and I’m out of the pci lanes on the other end of the fiber either way (that box has all x16 going to the gpu)

I run 3900X with a 40Gbit fiber, packed with HDDs and nvmes. The box fluctuates around 90-110W use.

when you said that Nextcloud might not meet your needs, was your concern specifically the server-side data format?

I’d prefer them as plain files. Technically it doesn’t matter much to me if it’s a database, if I have to spin up an S3-compatible API, or if I need to slice up a zvol for it, but I just prefer the files because then I can do zfs snapshots (in which I trust) and backup with restic (in which I trust)

That gives me hope, thanks. I’ll try it, then.

Lots of files. I’d offload old projects that I worked on with synology drive so they aren’t stored locally, only remotely (but are easily accessible).

Your requirements sound a lot like Chrome Remote Desktop and it’s pretty trivial to install, which might be a handy thing for family members that aren’t tech-savvy.

I don’t like helm, so I use nix to maintain my fediverse deployments in kubernetes. Typically that’d just autoupdate itself to new releases, but for lemmy specifically I upgrade by hand nowadays since one release some time ago broke my deployment and its schema change was incompatible with the automated rollback.

My setup is a combination of https://github.com/farcaller/nixdockertag (auto-updated docker imagesfor things where I fully own the deployments) and https://github.com/farcaller/nixhelm (for helm charts that I either consume verbatim PR have local patches on). Both just auto update nightly thanks to github.

PD delegates the whole prefixes, i.e. it allows the subrouters to ask for a subnet of the size they need.

I’d swap Prometheus for VoctoriaMetrics. It’s a drop-in replacement with a much better resource consumption story and a few extra goodies.

There’s way more and I already tried three implementations while trying to get a set of features I need. It’s a wild west out there and the resource usage is way higher than e.g. hosting Prosody. Seemingly it has to do with chatrooms being a full mesh, but my single user server consumes about 700mb RSS and 2.4 gb VSZ which is kinda high.

In case if you e.g. have eth0 and eth1 and neither is guaranteed to be up. It’s more of a router setup, though (Cisco routers are well-known to use the loopback interface like this).

It really depends on the specific hardware. I have Mikrotik routerOS CHR that routes between VLANs at 6Gbit/s without breaking a sweat on a $300 intel box.

At the same time, some managed switches are dirt-cheap nowadays and they generally can push the traffic around as fast as it comes in.

OpnSense is incapable of proper DHCPv6-PD, that’s when your route receives a prefix from upstream and delegates parts of it downstream. More specifically, it does the delegation, but it doesn’t add the relevant routes, effectively blackholing the allocated prefixes.

VyOS fixed this specific bug since I reported it. RouterOS and IOS never had it.

One more for mikrotik (I run the VM version on a small linux box).

I tested a ton of those (pf/opn-senses, VyOS, even Cisco), and noone of the free ones can handle IPv6 in a reasonable way in 2024, which is slightly bizzare. Mikrotik has some annoyances, but it’s rock solid as a router.

I don’t use its container features and instead run podman in a vm next to it. Works great.

I got my account closed with no reason a hair after 12 months. It was good while it lasted, and I have the backups outside of oracle’s cloud.

I wouldn’t specifically say nixOS is stable in the same sense debian is but yes, it can totally handle this use case. I mainly run k8s on it, but a few home machines run docker (or, rather, podman) containers.

A thing about nixOS is that quite often you won’t need containers at all and would be better off without them, managing your apps as part of the system state as a whole. I only do that because I can’t be bothered to properly switch to nixOS services for ELK (which is supported by nixOS).

It’s a very stable solution in general and usually ends with a configuration that either doesn’t apply at all or applies with no issues. Gitops included for pretty much free. It requires understanding nix, and it can be tricky, but not overly tricky.

All and all I haven’t had an Ubuntu in homelab for two years now and can’t be happier about that.

Try VictoriaMetrics. Basically the same feature set as Prometheus, but so much more resource friendly for homelab scale. I store some metrics for 12 months now, because it’s easy.

oh, that’s actually a fair point! You’re correct.

DHCPv6-PD is still effectively broken, though.