Selfhosted

I would documented everything as I go.

I am a hobbyist running a proxmox server with a docker host for media server, a plex host, a nas host, and home assistant host.

I feel if It were to break It would take me a long time to rebuild.

aa

I would have taken a deep dive into docker and containerised pretty much everything.

I should have learned Ansible earlier.

Docker compose helped me get started with containers but I kept having to push out new config files and manually cycle services. Now I have Ansible roles that can configure and deploy apps from scratch without me even needing to back up config files at all.

Most of my documentation has gone away entirely, I don't need to remember things when they are defined in code.

Go with used & refurb business PCs right out of the gate instead of fucking around with SBCs like the Pi.

Go with "1-liter" aka Ultra Small Form Factor right away instead of starting with SFF. (I don't have a permanent residence at the moment so this makes sense for me)

I already did a few months ago. My setup was a mess, everything tacked on the host OS, some stuff installed directly, others as docker, firewall was just a bunch of hand-written iptables rules...

I got a newer motherboard and CPU to replace my ageing i5-2500K, so I decided to start from scratch.

First order of business: Something to manage VMs and containers. Second: a decent firewall. Third: One app, one container.

I ended up with:

• Proxmox as VM and container manager
• OPNSense as firewall. Server has 3 network cards (1 built-in, 2 on PCIe slots), the 2 add-ons are passed through to OPNSense, the built in is for managing Proxmox and for the containers .
• A whole bunch of LXC containers running all sorts of stuff.

Things look a lot more professional and clean, and it's all much easier to manage.

Actually plan things and research. Too many of my decisions come back to bite me because I don't plan out stuff like networking, resources, hard drive layouts..

also documentation for sure

Setup for high availability. I have a hard time taking things down now since other people rely on my setup being on.

I'd have stuck with ZFS.

I always redo it lol, which is kind of a waste but I enjoy it.

Maybe a related question is what I wish I could do if I had the time (which I will do eventually. Some I plan to do very soon):

• self host wireguard instead of using tailscale
• self host a ACME-like setup for self signed certificates for TLS and HTTPS
• self host encrypted git server for private stuff
• setup a file watcher on clients to sync my notes on-save automatically using rsync (yes I know I can use syncthing. Don't wanna!)

Make sure my proxmox desktop build can do GPU passthrough.

I'm generally pretty happy with it, though I'd have used podman rather than docker if I were starting now.

I wouldn't change anything, I like fixing things as I go. Doing things right the first time is only nice when I know exactly what I'm doing!

That being said, in my current enviroment, I made a mistake when I discovered docker compose. I saw how wonderfully simply it made deployment and helped with version control and decided to dump every single service into one singular docker-compose.yaml. I would separate services next time into at least their relevant categories for ease of making changes later.

Better yet I would automate deployment with Ansible... But that's my next step in learning and I can fix both mistakes while I go next time!

My current homelab is running on a single Dell R720xd with 12x6TB SAS HDDs. I have ESXi as the hypervisor with a pfsense gateway and a trueNAS core vm. It's compact, has lots of redundancy, can run everything I want and more, has IPMI, and ECC RAM. Great, right?

Well, it sucks back about 300w at idle, sounds like a jet engine all the time, and having everything on one machine is fragile as hell.

Not to mention the Aruba Networks switch and Eaton UPS that are also loud.

I had to beg my dad to let it live at his house because no matter what I did: custom fan curves, better c-state management, a custom enclosure with sound isolation and ducting, I could not dump heat fast enough to make it quiet and it was driving me mad.

I'm in the process of doing it better. I'm going to build a small NAS using consumer hardware and big, quiet fans, I have a fanless N6005 box as a gateway, and I'm going to convert my old gaming machine to a hypervisor using proxmox, with each VM managed with either docker-compose, Ansible, or nixOS.

...and I'm now documenting everything.

Not accidentally buy a server that takes 2.5 inch hard drives. Currently I'm using some of the ones it came with and 2 WD Red drives that I just have sitting on top of the server with SATA extension cables going down to the server.

More ram 🐏

Get a more powerful but quieter device. My 10th gen NUC is loud and sluggish when a mobile client connects.

I'd make my own nas.

Instead of a 4-bay NAS, I would have gone with a 6-bay.

You only realize just how expensive it is to expand on your space when you have to REPLACE HDDs rather than simply adding more.

I would spend more time planning and understanding docker. My setup works, but it's kinda messy

I'd put my storage in a proper nas machine rather than having 25tb strewn across 4 boxes

For me:

• Document things (configs, ports, etc) as I go
• Uniform folder layout for everything (my first couple of servers were a bit wild-westy)
• Choosing and utilizing some reasonable method of assigning ports to things. I do not even want to explain what I need to do when I forget what port something in this setup is using.

I would have gone with an Intel CPU to make use of iGPU for transcoding and probably larger hard drives.

I also would have written down my MariaDB admin password... Whoops

Buy an actual NAS instead of a rats nest of USB hub and drives. But now it works so I'm too lazy and cheap to migrate it off.

That's a pretty good question: Since I am new-ish to the self-hosting realm, I don't think I would have replaced my consumer router with the Dell OptiPlex 7050 that I decided on. Of course this does make things very secure considering my router is powered by OpenBSD. Originally, I was just participating in DN42 which is one giant VPN semi-mesh network. Out of that hatched the idea to yank stuff out of the cloud. Instead, I would have put the money towards building a dedicated server instead of using my desktop as a server. At the time I didn't realize how cheap older Xeon processors are. I could have cobbled together a powerhouse multi-core, multi-threaded Proxmox or xcp-ng server for maybe around 500-600 bucks. Oh well, lesson learned.

To be honest, nothing. Running my home server on a nuc with proxmox and a 8 bay synology Nas (though I'm glad that I went with 8 bay back then!).
As a router I have opnsense running on a low powered mini pc.

All in all I couldn't wish for more (low power, high performance, easy to maintain) for my use case, but I'll soon need some storage and ram upgrade on the proxmox server.

I have things scattered around different machines (a hangover from my previous network configuration that was running off two separate routers) so I’d probably look to have everything on one machine.

Also I kind of rushed setting up my Dell server and I never really paid any attention to how it was set up for RAID. I also currently have everything running on separate VMs rather than in containers.

I may at some point copy the important stuff off my server and set it up from scratch.

I may also move from using a load balancer to manage incoming connections to doing it via Cloudflare Tunnels.

The thing is there’s always something to tinker with and I’ve learnt a lot building my little home lab. There’s always something new to play around with and learn.

Is my setup optimal? Hell no. Does it work? Yep. 🙂

I recently did this for the second time. Started on FreeNAS, switched to TrueNAS Scale when it released and just switched to Debian. Scale was too reliant on TrueCharts which would break and require a fresh install every couple of months. I should've just started with Debian in the first place.

I would’ve gone with a less powerful nas and got a separate unit for compute. I got a synology nas with a decent amount of compute so I could run all my stuff on the nas, and the proprietary locked down OS drives me a bit nuts. Causes all sorts of issues. If I had a separate compute box I could just be running some flavor of Linux, probably Ubuntu and have things behave much more nicely