Hello there,
I am in search of inspiration for solving several issues I am having.
That’s my situation:
- ageing Synology NAS (4x4tb drives) with multiple docker containers running on it.
- old gaming rig (i7-6700, Fedora Server) which I use sparingly for game server hosting
What I want to achieve:
- Upgrade/replacement of my NAS
- Offsite backup
- ~20tb of usable storage (~2-3tb of actual important stuff)
Primary use cases:
- SMB share
- Docker hosting (Pihole, unbound, Jellyfin, *arr etc.)
Newly added requirements:
- Cloud storage for mobile devices of close family members on and off site
Complications:
- Electricity cost is a scam where I live (~0.40$ per Kwh)
I have an ageing NAS whose capacity is getting to its limits storage and horsepower wise. And since I have to do work on my setup anyways, I was wondering whether you might give me some inputs on what I could/should use to achieve my goals.
-
My NAS is getting to its teenage years and I am getting worried about its eol. Buying some old server HW is out of the question because of power usage and availability. What are my best options? Build something myself with current hardware? Buy a new NAS? What is a good way to migrate data to a new system? From a power consumption standpoint are SSD’s better than HDD’s?
-
I have an off site which i visit regularly where I could either place backup drives or put a system in a rack. What would be a good option for an offsite backup solution?
-
I have gotten my aunt (77) a tablet during covid so she could video call us. In recent months a smartphone has entered the ring because daily life is getting impacted when you don’t have one. Now she is all into taking pictures and videos and the storage on her phone is not enough. What are my options? I’ve experimented with Nextcloud but I am uncertain whether it is the right solution, especially from a usability perspective. (I want to avoid third party services for storage)
I will very much appreciate your input since I’m not working in the field and am getting to the edge of my own knowledge at this point.
Thank you in advance for your input.
Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I’ve seen in this thread:
Fewer Letters More Letters LXC Linux Containers NAS Network-Attached Storage NVMe Non-Volatile Memory Express interface for mass storage Plex Brand of media server package RAID Redundant Array of Independent Disks for mass storage SATA Serial AT Attachment interface for mass storage SSD Solid State Drive mass storage SSH Secure Shell for remote terminal access VPN Virtual Private Network
9 acronyms in this thread; the most compressed thread commented on today has 19 acronyms.
[Thread #18 for this sub, first seen 10th Aug 2023, 21:35] [FAQ] [Full list] [Contact] [Source code]
Good bot!
What are my best options? Build something myself with current hardware?
Build something based on a MicroATX board with a CPU > i5-6500, this will probably cost around 80€ second hand with RAM. Server hardware is overkill, wastes power and older CPUs are always worse in all possible ways. Even if the board only has 4 SATA ports, a PCI to 5 SATA port card and it will be enough. Use BTRFS as your filesystem and its RAID if needed.
You may be thinking something like “I want a faster CPU in order to have fast SMB”, don’t. Your gigabit network will saturate before your CPU or even mechanical drives and when this happen you’ll get something like 10-20% CPU usage. Just don’t waste your money.
From a power consumption standpoint are SSD’s better than HDD’s? It depends on the hardware, either way is isn’t feasible to have around 20TB of SSD storage. Even considering you can now buy a 4TB SSD for 195€ that would be a lot of money.
Nextcloud is garbage, yes very bad usability, more reasons and issues listed here: https://lemmy.world/comment/1571886 and https://lemmy.world/comment/346174
Syncthing is a very good piece of software, to sincronize devices with a NAS - except if you use iOS. Currently I’m running Syncthing on my NAS and all my devices sync to it (no cross-device sync to avoid issues). Then I’ve an SMB share to allow access to the files on iOS devices and FileBrowser for a cloud-like web browser access experience. Works flawlessly uses very little RAM and its solid, private, secure and manageable open-source - not something like Nextcloud that calls home, breaks everything on upgrades, wastes ram and runs slowly to only deliver an inferior experience in all possible ways.
Running stuff in your NAS: if you’re really into a low power solution you might want to stay away from Docker. Simply install a clean Debian system and manually get what you need instead of tons of containers and dependencies. This will allow you to run more stuff on less hardware. Linux isn’t that hard, you can do it.
Thank you for your answer.
I agree with you that Server Hardware is overkill. That’s why I am asking for options here. For your suggested architecture: It is quite ancient and eol per last year and I have one of these already in use as a “more than my NAS thingy”. What I am not comfortable with is that that generation is already end of life and I don’t want to invest time and money into hardware that I would have to replace a year or two from now. I’m looking for a solution (self built or not) that will drag me through the next decade. Does BTRFS include Raid support? I don’t have much experience with it. The most I did once was recover a snapshot.
CPU is the least of my concerns. I am currently looking at a low end current gen Intel CPU for my purpose. And yes, samba is slow and I will never saturate a somewhat recent CPU with it, but I have from time to time other things running on that machine.
For storage: You are overestimating a bit with the prices. For 200€ I can get 4TB SSDs. So 20TB + one for raid 5 ~1200€. That would be quite doable but on the expensive side I agree. The question was whether SSDs were considerable cause of power usage, not from a price standpoint.
As fro Syncthing… I’ll have a look at that.
Docker is just nice and simple. I remember times when deploying software on a single server was hell on earth. Conflicting libraries etc. And yes Linux isn’t hard (been using it for like 2 decades).
What I am not comfortable with is that that generation is already end of life
The thing is that those are very cheap as people want faster things for Windows desktops and as we both know Samba won’t ever saturate that CPU on a 1GbE to 2.5GbE link. If you can get a last-gen low power solution like an i3 in the same price ranges go for it and ignore my previous advice. But you know an i5-6500 or i5-7400 + motherboard + RAM for 70-80€ is a good deal and enough for the use case.
For 200€ I can get 4TB SSDs.
Sorry, my mistake. I meant 4TB.
Does BTRFS include Raid support?
Yes many possible configurations and snapshots. BTRFS also tends to be way more reliable than Ext4 and others when the hardware fails, you experience sudden power losses etc. More: https://linuxhint.com/set-up-btrfs-raid/
As fro Syncthing… I’ll have a look at that.
Syncthing + FileBrowser is a “killer” setup for a personal cloud. I’m even amazed you’ve never heard about / used Syncthing as it is very popular in self-hosting. You can also use it to sync your main NAS with the remote backup. Very reliable and easy to setup.
Docker is just nice and simple. I remember times when deploying software on a single server was hell on earth.
Never had that experience… if the software is properly done and you aren’t using a weird distro things should work out well. Eventually you can use LXC/LXD or even systemd containers to isolate problematic applications without having to deal with all the Docker overhead and mess.
Yes many possible configurations and snapshots.
Except RAID 5 and 6! Those are still broken on BTRFS and not recommended for use by the devs. It’s unfortunate because I just setup a DIY NAS and I had to go with ZFS because of this.
As if anyone should be using RAID 5 or RAID 6: https://www.zdnet.com/article/why-raid-6-stops-working-in-2019/ and https://www.linkedin.com/pulse/avoid-raid5-raid6-live-happy-life-tech-tuesdays-6-jv-roig/?articleId=6568798728638038016
Those setups are highly discouraged.
Eh, RAID 5 and 6 are still viable for home deployments. Not a lot of people want to be running massive drive arrays or expensive disks at home just to get decent storage. I ran a 4x 4TB RAID 5 for close to a decade and it’s survived 4 drive rebuilds. The Intel chip on the QNAP machine I was using to maintain that array died before the array itself did. Now I have an NVMe SSD-based array, so drive rebuilds are even less of a concern.
The other reason why I brought it up is that the article you linked doesn’t even mention BTRFS RAID 5 and 6 issues until all the way down at the bottom of the article in a small paragraph, when really it should be in bright red letters at the beginning.
Does BTRFS include Raid support? I don’t have much experience with it. The most I did once was recover a snapshot.
It does have RAID support but its RAID 5 and 6 are BROKEN! The devs themselves do not recommend using these. If you need RAID 5 and 6 and you absolutely want to use BTRFS, you’ll have to go with mdraid and then put BTRFS on top, but then you lose a lot of the BTRFS self-healing capabilities. Personally for RAID 5 and 6, I still recommend ZFS’s RAIDz. It’s quite easy to setup. I have a DIY NAS with an OS drive running BTRFS and a storage pool consisting of 4x 4TB SSDs running in RAIDz1.
Does FileBrowser support creating public links for sharing? I use Nextcloud as a way to deliver large amounts of photos and videos to my clients.
My issue with Syncthing is that doing partial sync is sort of a pain in the ass. My Nextcloud currently has 290GB of data that I’d rather not completely sync to all of my devices and AFAICT with Syncthing, you still need to fiddle around with config files to do that, and even then its clunky and doesn’t work sometimes.
Yeah I get that Nextcloud is a bit slow but it’s definitely more capable as a drop-in cloud storage replacement than other software I’ve seen.
Does FileBrowser support creating public links for sharing?
https://noted.lol/content/images/2023/01/filebrowser-self-hosted-public-shares-noted.gif
My issue with Syncthing is that doing partial sync is sort of a pain in the ass.
That’s like the ONLY feature everything is missing from Syncthing. Frankly I don’t get it, I just don’t understand why they don’t implement it. They’ve their ignore patterns that can essentially be leveraged to implement partial sync but they won’t just do it. https://forum.syncthing.net/t/selective-sync-feature-most-important/16394
Nice that share feature looks pretty slick. I might check this out.
Yeah I frankly don’t get why syncthing doesn’t implement it either. It’s like the only feature that really holds me back from using it, otherwise it’s pretty damn slick and has much faster sync than Nextcloud.
I just got through helping a friend set up a NAS. Even today I recommend people stick with Synology because you get so much with it. Security updates and software upgrades are easy, you get good software packages for free, and the Synology platform is just easier to manage unless you want to be a real power user. Honestly, I would replace your current Synology device with an updated one. The DS423+ I set my friend up with had a reasonable processor that could even do hardware transcoding for Plex. Not a lot, mind you, but plenty for his 1080p and DVD library.
I use my Synology NAS for computer backups, photo storage and display, and occasionally I use Synology Drive (Synology’s NextCloud clone – or possibly a fork of NextCloud) to host files for people to access from my network. I wouldn’t say that any self-hosted solution would be extremely easy to use, but Synology Drive was really excellent for moderately techy people.
Offsite backups are hard. I just use Synology’s HyperBackup to create an archive of the files I can’t afford to lose and physically carry those drives to an offsite location. I’ve had to restore from it from time to time, and it has been a nice experience. I especially like that I can restore only specific files and that it handles versioning. It gets hard when you need an immense amount of space for your backup. But these days you can get drives that are positively huge.
Security updates and features until they decide to change their terms of service and remove some useful piece of software just because they want. https://www.youtube.com/watch?v=XvEVEP75DYk Louis Rossmann - Synology stops hosting old reinstall files, claims “licensing” issues
Offsite backups are hard
If you build a NAS instead of using Synology stuff it will be as easy as setup SSH between the machines and rsync.
Here is an alternative Piped link(s): https://piped.video/watch?v=XvEVEP75DYk
Piped is a privacy-respecting open-source alternative frontend to YouTube.
I’m open-source, check me out at GitHub.
FYI, he posted an apology video not long after that saying he jumped the gun, and that Synology had to remove the software not because they wanted to, but because some parts of their software were licensed and expired. Video link
Here is an alternative Piped link(s): https://piped.video/YWICHNEHHeU?si=d3WNWs8yM8V-qDIw
Piped is a privacy-respecting open-source alternative frontend to YouTube.
I’m open-source, check me out at GitHub.
I’m aware of that but what practical difference does it make? You buy one of those things and suddenly you got functionality because they weren’t able to properly negotiate lifetime licensing for their devices that are sold, not rented, not leased…
Well if you’re aware of it, why continue misinforming people with a video he retracted? You’re still using the device as is. They aren’t retroactively deleting functionality off of your device after support ends. The license issue just means that they cannot legally distribute legacy software from their website anymore. AFAIK, Synology emailed their customers informing them ahead of time to download the image before they had to take it down, I think they did what they could have done already.
The original licensing problem in question was about the AAC codec, which as you probably know, is practically in every device we use and are in use daily. So until another open codec becomes the norm, or until all the patents expire in 2031, there probably will still be cases like this popping up from time to time.
Because they shouldn’t sell hardware “for life” with an underlaying license that is only temporary. That doesn’t make sense. They’ve to make better agreements or pick other solutions.
Okay.
First off, afaik, they never advertised “lifetime” warranty nor software support on their website.
Secondly, as I told you in the previous comment already, you can still use the product as is! This is no different from phone manufacturers dropping software support after 2 or 5+ years. Not to mention this probably only affects hardware that are older than 2009. That’s over 10 years of software support already. Granted they could have reuploaded a version without the AAC codec, but still.
Third, looking at Via’s AAC FAQ, license fees are due on “per unit” sale, and the term of the license is five years, with additional five years renewal. There never was a lifetime license for it. The same goes for every single hardware/software sold that has AAC encoder/decoder built in. So if your actual issue is with that, take it up on Via and the patent holders, and start using foss audio codecs like OGG Vorbis or FLAC.
Third, looking at Via’s AAC FAQ, license fees are due on “per unit” sale, and the term of the license is five years, with additional five years renewal. There never was a lifetime license for it.
Yes, that’s a scam. One thing is when you rent, lease or have hardware provided on a subscription basis, another entire different thing is when you buy hardware. Synology is just propagating this kind of BS by including AAC.
Up until now I’ve dealt with a few devices that included AAC and none of them suddenly became obsolete by the lack of license after 5 years.
Offsite backups are hard
If you build a NAS instead of using Synology stuff it will be as easy as setup SSH between the machines and rsync.
To be fair, you can do this with Synology as well. Rsync is built-in and even integrated into DSM. The advantage to using Hyperbackup is that you get block-level incremental backups.
Rsync does incremental.
TIL! Are there good GUI front-ends for Rsync for when you want to browse the file versions?
Another option, easier than rsync is simply use Syncthing to sync the main machine with the backup machine. No tunnels required and it has built in file versioning with a GUI.
I get the ease of use aspect, but having just come out of a lengthy recovery process with a broken QNAP TS-453 Pro, I can safely say I am done with turn-key NAS solutions. Having to be beholden to a specific company for hardware upgrades and replacements, all while paying an exorbitant markup for subpar hardware, no longer gives me peace of mind about the security of my data. I just personally think the ability to do what you want with both the hardware and the software storing your data is key to long-term data storage. You just never know when these companies will go out of business, start engaging in shitty business practices, or, like in the case of QNAP, swap to a custom LVM layout that makes it impossible to open up in standard Linux distros, complicating data recovery.
Security updates and software upgrades are easy, you get good software packages for free
That may very well be true, but I doubt it’d come close to the amount of software you can get with basically any standard Linux distro. There’s also TrueNAS and Unraid if you’re looking for something that’s more NAS like.
Thank you for your insights.
My primary issue with Synology NASes or prebuild NASes in common is the lack of upgradeability. I am quite happy with the one I’ve got but it is underpowered and underspeced for what I sometimes want to do. Hence I converted my old gaming rig into a server for those purposes.
I’ve tried the Synology Apps but I don’t like to link myself into one ecosystem for what should be simple tasks. So I am looking for alternatives.
I somehow have to find a middle ground in usability for me and my companions.
I’m still a fan of synology because it does a lot of what you want out of the box without you needing to constantly manage and setup all these services from scratch. I’ve upgraded through several synology units over the years, currently using a 6TBx8 unit for much of what you mention. Since drives are so big these days, you could get a newer 4bay with more horsepower and just drop a couple 20TB drives in it as a mirrored pair then in the future add more drives as needed. Dropping to 2 drives cuts your power consumption a bit, and staying with a 4 bay instead of something bigger will also keep the power down.
You can absolutely build your own, but synology comes with all the “home cloud” apps preconfigured and your time and effort is worth something too. I build enterprise cloud environments for a living and I don’t want to have to do that at home on my free time- synology is so plug’n’play.
Just a quick follow-up on how we set up self-hosted cloud storage for my friend:
Synology has an OpenVPN server built-in. We configured that to grant his offsite family members access to his network, and then set up DSM to have a custom URL specifically for Synology Drive. (It’s in the Remote Access section of the control panel.) This way users could just visit /drive and get access to a google drive-like interface that was easy for them to use. Setting up the OpenVPN client on their computer was a pain in the butt (as per usual for OpenVPN), but after that was properly configured, they just have a little toggle switch that enables them to access his NAS, which is easy for them to use.
When you share files with someone on Synology Drive, it even sends them an email telling them that you made a file available. Very convenient! They just have to remember how they access the NAS.
If you build a NAS instead of using Synology stuff it will be as easy as setup SSH between the machines and rsync. Way easier than a VPN.
My friend’s requirements were that the transfers be encrypted (which ssh does) and that his family have a server that was easy for them to use to upload and download files. The file server also had to be private – meaning not stored in the cloud. They aren’t technically savvy, so we needed an option where they could literally drag and drop a file from their desktop onto a web browser window. It worked well for them. My only regret is that the VPN was so complicated to set up. But on the bright side, Synology unifies the username and password between the VPN server and DSM, which makes it a little easier for my friend (and his family) to maintain.
they could literally drag and drop a file from their desktop onto a web browser window
Read my comment here: https://lemmy.world/comment/2284151 I’ve a solution for that. FileBrowser is a good UI for everyone, look at the video here https://github.com/filebrowser/filebrowser
