I use Immich. It does what you described as well.

I’m not sure why every time I look at this project, it rubs me the wrong way. Anyone found anything wrong with it?

As far as I can tell it dates back to at least 2010 - https://docs.oracle.com/cd/E19253-01/819-5461/githb/index.html. See the Solaris version. You can try it with small test files in place of disks and see if it works. I haven’t done it expansion yet but that’s my plan for growing beyond the 48T of my current pool. I use ZFS on Linux btw. Works perfectly fine.

I think data checksums allow ZFS to tell which disk has the correct data when there’s a mismatch in a mirror, eliminating the need for 3-way mirror to deal with bit flips and such. A traditional mirror like mdraid would need 3 disks to do this.

Not that I want to push ZFS or anything, mdraid/LVM/XFS is a fine setup, but for informational purposes - ZFS can absolutely expand onto larger disks. I wasn’t aware of this until recently. If all the disks of an existing pool get replaced with larger disks, the pool can expand onto the newly available space. E.g. a RAIDz1 with 4x 4T disks will have usable space of 12T. Replace all disks with 8T disks (one after another so that it can be done on the fly) and your pool will have 24T of space. Replace those with 16T and you get 48T, and so on. In addition you can expand a pool by adding another redundant topology just like you can with LVM and mdraid. E.g. 4x 4T RAIDz1 + 3x 8T RAIDz2 + 2x 16T mirror for a total of 44T. Finally, expanding existing RAIDz with additional disks has recently landed too.

And now for pushing ZFS - I was doing file based replication on a large dataset for many years. Just going over all the hundreds of thousands of dirs and files took over an hour on my setup. That’s then followed by a diff transfer. Think rsync or Syncthing. That’s how I did it on my old mdraid/LVM/Ext4 setup, and that’s how I continued doing on my newer ZFS setup. Recently I tried using ZFS send/receive which operates within the filesystem. It completely eliminated the dataset file walk and stat phase since the filesystem already knows all of the metadata. The replication was reduced to just the diff file transfer time. What used to take over an hour got reduced to seconds or minutes, depending on the size of the changed data. I can now do multiple replications per hour without significant load on the system. Previously it was only feasible overnight because the system would be robbed of IOPS for over an hour.

If you can, move to a RAID-equivalent setup with ZFS (preferred in my opinion) in order to also know about and fix silent data corruption. RAIDz1, RAIDz2 would do the equivalent to RAID5, RAID6. That should eliminate one more variable with cheap drives.

Due to risk of failure or risk of data corruption because the mirror can’t tell which drive is right when there’s a difference?

Get more drives, run higher redundancy 💪

Three-way mirror?

Buy recertified enterprise grade disks from https://serverpartdeals.com. Prices were around $160/16TB the last time I checked. Mix brands and models to reduce simultaneous failure. Use more than 1-disk redundancy. If you can’t buy from SPD, either find an alternative or buy external drives and shuck them. Use ZFS to know if your data is correct. I’ve been dealing with funny AMD USB controllers recently and the amount of silent data corruption I’d have gotten if not for ZFS is ridiculous.

I don’t think anyone makes SMR drives in the current lineups anymore.

Switching wholesale from a brand or model to another could be counterproductive. There are myriad of reasons why drives can fail that aren’t related to the brand and the model. What if you unknowingly switch to a less reliable model because of such a reason? You’d end up worse off. For example according to Backblaze’s data, Seagate is generally worse than WD.

A better way to do this is to mix brands and models so that there’s less probability to fail at the same time. I have both WD and Seagate in a single storage pool, even if the Seagate model is objectively less reliable according to Backblaze.

You already trust these people today. They run/own many large corporations today which dramatically affect our lives in multitude of ways. Except today we can’t get remove them from these positions of power under the current system.

It’s thanks to this in part that your aunt keeps indulging her imaginary pain when she thinks about your lifestyle.

All decent DP KVMs are very expensive. I got an IOGEAR which is a rebranded Aten. It was also in the same price range. Who knew high resolution needs high bandwidth and high bandwidth signaling and switching is hard…

This drama is from 2021. HA is still open source and has been great.

The Nix maintainers come off pretty entitled in this one. They ignore advice, requests and do not provide an engineering solution that could solve this without incurring the cost HA would bear. You don’t get to dictate or create work for an open source project that wasn’t designed to play well in your environment. If you want to get it to play nice in there, come up with a proposal that the upstream accepts and implement it. Better yet, come up with a design of your system that accommodates the upstream project. If you go ahead and create that work anyway, upstream can use the tools at their disposal to prevent that.

Anything but paying for the labor of a person to draw such a picture.

It’s too close to expiration for a transfer. 😔 givemoney.gif

Plot twist:

Private equity buys your registrar and jacks up the renewal price 5x.

Purely on the product side, if I decide to buy it, I wouldn’t buy it for myself. I’d buy it for friends and family who are not that tech literate. Either to make my life easier to give them self-hosted services, or ideally for themselves to be able to do so. I want this product to be a non-shitty, open source “Synology,” from a firm I can trist to support it for a very long time. Doesn’t have to have that form factor. And I’m totally fine with an ongoing subscription. I’d like to be able to say - hey friend, buy this from ACME Co-op and sign up for their support plan. Follow the wizard and you’ll have Immich, Nextcloud, etc. A support plan might include external cloud HTTP proxy with authentication and SSL that makes access trivial. Similar to how Home Assistant’s subscription (Nabu Casa) works. It could also include a cloud backup. Perhaps at a different subscription rate.

I don’t know enough to say what the structure should be but this should not be possible:

But it doesn’t protect you against more insidious forces like the founders selling to private capital

It implies that the founders have more voting power and ownership than the rest of the people in the org. In my mind, everyone should have an equal vote, which should prevent a sale on the whim of the founders or another minority group. If a sale is in the cards, a majority of the people in the org should have to approve for it to proceed. And this shouldn’t be advisory but a legal barrier to pass.

If I were to start a firm today, I’d be looking into this because not only this is the kind of firm I’d like to work in, but I think so would quite a few people in software. And those aren’t the dumb kids.

I can also say that as a customer, the few worker co-ops I’ve able to buy things from give me a much more trustworthy impression than the baseline. They just behave differently. Noticeably more ethically.