Self-Hosted Main

The things of "how to set up" are usually covered by software's documentation, further supplemented by googling certain specific questions and going on various forums. Never just use the first page you find but crossreference information as well.

​

Having gone into self-hosting as a hobby with the intention of supplementing knowledge in my career path (currently still studying), it can become quite daunting quite fast, especially if you try looking at a big picture, rather than taking it step-by-step. The best thing to do would be to follow a tutorial within the documentation of something you're trying to host, then expanding per step as far as dependencies goes, and then go from there. Compartmentalizing is a very effective way to not get lost.

On the one hand, I feel you.
On the other, man, I am so happy you are suffering! lol If you, a Software Engineering student are struggling, then it does not make me feel quite as clueless anymore as a normie :)

I think this is going to be a super unpopular opinion here, but you don't necessarily have to go with what "everybody else" is using just because they're the big shiny pennies that everybody else is using. Years ago, "everyone" was using Windows and Internet Explorer. Does that mean that nothing else was worth running?

As to VPN, Docker, Cloudflare, Traefik, Crowdsec, Authentik... this definitely isn't a "don't use those things" post. If you want to learn Cloudflare, and want to use Cloudflare, then by all means, do so. This isn't an anti-Docker, anti-Cloudflare, or anti-anything post.

But that being said, I started self-hosting with a Dell laptop running XAMPP with a single, tiny, intranet web portal on port 80. It was a couple of clicks to get a functioning LAMP stack, despite me being a complete noob. It was incredibly insecure, and not the best, newest, fanciest choice to host my little website. But it allowed me to learn incrementally, and quickly, rather than feeling like I was thrown into the deep end way over my head with -- as an example -- Docker and virtualization.

• I got sick of XAMPP pretty fast, and learned how to host my own database and Apache instances on bare metal outside of XAMPP
• then I learned how to buy and set up an actual domain name
• then I learned how to install SSL certificates for my domain
• then I learned about hardening Apache, and things like honeypots and fail2ban
• later, I acquired several more small servers with their own services, all behind the same router, and I learned about subdomains, virtual hosts, and proxies
• even later, with all of that foundation, I learned about moving individual services into Docker. I only had to learn Docker. I already knew the rest of what I needed.
• etc.

I wanted to self-host my own intranet-only DNS server, but I found BIND to be overwhelmingly daunting at the time, with poor information for how to actually set it all up and get it working. So I found dnsmasq and installed that instead. The configuration was incredibly simple, and it worked almost out-of-the-box. Am I going to have pretentious snobs telling me how much dnsmasq sucks and how I'm stuck in the 90s if I still use it? Probably. Does it serve my own, personal, use case for the features I needed? Yes, it's been great and simple to maintain for years.

My "dynamic DNS service" is a bash script running from a cron job, accessing a webhook that my domain registrar offers, to update my DNS records automatically. Surely, there is some great piece of software with a catchy name and a GUI dashboard that I could be using for this purpose. But I don't feel that I have to. My laughably basic bash script has worked without touching it for about 10 years now.

For my VPN, for a long time, I had a PPTP daemon running on my server. That, too, had been very easy to set up for a young sysadmin who knew nothing about anything. It wasn't the most secure, I used it long past when PPTP fell out of favor, but it worked at the time, and there was ample documentation that was easy to follow to set it up.

I've moved on from most of the things I started out with when I was learning the basics, but all of that to say: Sometimes, it's okay for something to be "good enough". It's okay if your server isn't PCI-DSS-level secure and protected from absolutely everything. It's okay to do the best you can, get your setup working... and then don't stop there. Continue learning how to make it better, more secure, more reliable, more scaleable. But just because the popular kids are going to insist that you should never consider anything less than Nginx in a Docker container, managed by Portainer, behind Wireguard, behind Cloudflare... that setup can be your end goal, but you don't have to start there and learn everything all at once.

I'm confused why you say you can't find information on how to set up home lab projects. There are hours and hours of content on YouTube that also do sometimes point back to blogs and GitHub repos.

I will say that the problem I have is once they show you how to setup say a reverse proxy there's generally not follow through in either deep diving into all the additional settings or more advanced setups. If there is the other problem you'll run into is that it can become outdated quickly so it's a double edged sword. But setup and getting things running in a home lab there shouldn't be any shortage of info to consune.

networks are black magic and printers are voodoo.

TBH, I don't think Software Engineering, especially just 2 years of experience, is going to really help you understand network architectures/distributed systems. You can probably build some small apps, but designing larger systems is a skill that requires intentional practice.

The best thing you can do is figure out how to containerize the stuff you want to run and store the configurations in source control. Figure out how to isolate your data and back it up, and then experiment with configuration changes to see how they change system behavior.

A few specific things to learn/practice.

• Learn how to break down a larger problem into distinct components with specific responsibilities.
• Learn about docker, what problem it solves.
• Learn how to experiment with the tools to find out what they can do, and how you can configure their behavior (docker is very helpful here, because you can spin up a temporary sandbox to figure things out without risk of breaking a "live" system).

Use a decent VPS instead of trying to host off a residential internet connection behind 47 layers of NAT and you don't have to worry about 90% of that stuff

i.e. use the right tool for the job

Depends on what you're trying to do.. Plex/Jellyfin server on a Pi with Zerotier for remote access? Super easy and you can get running in 20 minutes with zero previous knowledge.

Self hosted email and reverse proxy with auto SSL and best security practices? Well that's a rabbit hole...

Start small

Set up a nextcloud or something then use NGINX on another VM to connect to it as a reverse proxy

That'll help you understand cf a little more

You can't just drive in and be an expert, it'll take years to get good

I strongly suggest to not only read articles on the internet but get decent books and read them carefully. It will fill in plenty of gaps you have now and in a blink of an eye self-hosting will become a smooth and fun experience.

I have been self hosting for like 20 years and I must say it has never been easier to bring up a service and make it secure as well.

Good network understanding is key, forget about IPv4 it's dead. If you engineer new stuff, concentrate on IPv6. Also a good book or two about Linux, it's the platform to go for the next decade. If you're still hungry dive into containers and container orchestration.

Most importantly, keep experimenting. I'd say 80%+ of my empirical data comes from my own experience.

I think the reason is that there's a preferred method for doing things for each person.

Like take reverse proxies for instance.

You can use nginx proxy manager if you're comfortable with docker but uncomfortable with CLI or text file configs.

Or you can just use straight nginx.

Or if you want you can use Caddy, which is what I personally do, because I dig how well Caddy does automatic management of letsencrypt certs. But that means being comfortable grokking how to write a Caddyfile (the config), and how to launch a daemon on some sort of Linux.

So if you want to take my route you've got even more choices, you can run it on a Linux VM, a Linux container, or let docker manage your Linux container. Or you can use someone's Caddy docker container and config that for yourself.

Personally I hand configured a Linux LXC container on proxmox and am using that for all the reverse proxying on my proxmox NUC server.

It's just so many variables now.

I remember back in the day you just installed a LAMP server and ran everything with PHP.

I'm not sure which is better.

But I've got to say, the sheer amount of abstractions and ways to orchestrate a system are mind numbing.

Like you can do proxmox or truenas or truenas scale or Synology or rancher or bare metal. There are so many things to decide on now.

And I've been running Linux for the past 18 years or so.

The biggest thing for me is that you should decide on the outset if you want a simple setup, or if you want to play with bleeding edge enterprise stuff.

One is relatively plug and play and somewhat tends to be less secure.

The other can be fun for some or job training for others. I find the people who find it fun to be a bit weird, like they are basically LARPing being a system administrator. But it might be just dressing up for the job they want instead of the job they've got, or they just like it and that's fine for them.

It's not really for me. I look for simplicity, but also security. Luckily I'm a fan of minimal installs and am happy with operating on a CLI. OpenBSD is a pretty solid base to build on. It makes you think about everything you add and how it's a possible security hole.

Anyway, enjoy reading manuals. That's most of what we do here. Join a forum or two (a real one). Find a supportive community there, ask good questions, contribute good answers. It's computer karma and by helping others you're broadening and solidifying your own comfort levels.

Don't give opinions as fact. They are opinions.

This comment is clearly riddled with opinions. YMMV. Good luck with your goals.

This is true for any thing in life. Especially technical stuff. It helps that you have some understanding to start with, but you should learn how to start small and organically grow. You can't setup all and everything in one go.

I would do in this order to make self hosting bite size,

1. Get docker working, with basic docker stuff.
2. Host one service and use it over local lan.
3. More services, but still strictly internal lan only. Get comfortable.
4. Get VPN and try accessing services through VPN
5. Now start research on how to expose, tools to use etc etc.

Maybe my path can help you. I don’t have any educational / professional background in tech, but it is my hobby. I started simple, wanting a NAS for my home. I downloaded TrueNAS and reading the docs and using a lot of YouTube videos (from multiple creators) I managed to get it up and running. Then I tried sharing but using permissions. It took me weeks to understand ACL and NFS shares. It took me a week to figure out that I need to share both parent and child dataset to access a child via NFS. I had to goggle a lot, but it was such a stupid mistake that there was no information about, so I resort to try and error. After I had it setup I thought it would cool to install pinhole. Then I had to learn VM on trueNas, Linux server, package managers and so on. I had zero experience in Linux. I managed to install docker and portainer via a tutorial but then I learned docker compose files and it made everything easier. I destroyed and created that vm multiple times. Tried Debian, Ubuntu server, fedora server … The pattern goes on, home lab will be about reading a lot and not afraid to try and break stuff. It is better to try when you don’t have a lot of services. Always backup your vm, if something breaks, nuke it and start from the backup. Keep it a piece each time. Don’t try to setup all at once. Have an idea about the end state but solve a small problem at time. They say the best engineers are the ones that are able to break complex problems in really small ones. Doing that I have a vm on true nas running more than 20 containers, a redundant pinhole on a separated server, everything accessible via VPN.

Now I’m learning about podman and kubernetes, probably I will spin a second vm, try to migrate what I have a nuke docker. It never ends while you are curious.

so many more things get thrown around so much in this sub and other resources, yet I can barely find info on HOW to set up this things.

no offense but maybe you just aren't using the right search terms.

People are overcomplicating stuff because they fear for security.

Move fast and break stuff, and just start somewhere.

You will not die if you run your setup without reverse proxy for a week.

Just go step for step till everything is nice and secure.

Also the thing about understanding everything is a habit that you get in school. But in bigger software projects it's not feasible at least in my experience.

Yeah I agree it can be quite daunting, I recommend starting out small and trying to understand and getting one piece working at a time.

That is the approach I've taken while migrating my workloads from manually deployed services on old PC hardware into a fully declarative Kubernetes setup on a more energy efficient mini PC. Sounds scary but it's been a lot of fun actually!

I tried writing down some help on getting started, as well as short explanations on most of the infrastructure here in my homelab git repo: https://github.com/FruitieX/homelab/blob/main/docs/configuration.md

Hopefully that's helpful if you decide to try setting up something similar! 😎

Well I mean there's a reason why these skills are in such high demand.

just chip away at it, I'm slowly learning SWAG/reverse proxy. Its complicated for me but its rewarding and eventually becomes easy. Such is the way with linux from my experience

For me and I'm not sure if it's been mentioned here already or not, but I don't really have any programming skills or super geek out on this stuff. But I like the tools, right? So teaching myself some of this is hard, especially when there is an attitude with most of this stuff that you should know everything about docker containers and weird networking rules, etc. Sometimes people are helpful, more often than not, it's a super short answer you have to interpret and decipher to figure out.

I self host for the same reason I play Factorio, it's a constant stream of solving problems and adding new ways to do things. There is a lot to do, but you don't need to do everything right away. Like a lot of people said, just break it up. I use Vikunja to create a list of things I want to and then break those things down into hour or weekend projects.

There is a challenge in the "bootstrap" process. But once you get something going then its easier to keep adding. Stuff like "Reverse proxy, VPN, Cloudfare bullshit" is not super necessary at first. Start with just getting an app going at an ip address and port that's only accessible on you LAN, then add the other stuff.

If you don't want to deal with these things and it's just you or maybe a few others, here's the easy way:

• Define the services for each host (i.e. physical + virtual machine) in a single Docker Compose file. You can even put all of these docker-compose files in a single Git repo

• Install Tailscale on all your machines - this is really easy and will let you securely access everything without having to forward ports or worry about the attack surface of the public web

• Set up something like Heimdall or https://github.com/tailscale/golink to have easy to use shortcuts for your services

This gets you out of SSO signin, certificates, reverse proxy setups, all the things that are (understandably!) complex and annoying to set up

Networking is extremely dense and extremely hard.

Its also a space filled with professionals who do it for a job. Many on this sub are current or ex networking or security professionals who understand this shit like the back of their hand.

Im no idiot with IT - I’ve had pc’s my whole life and work in close proxiimity to IT. But networking, security and the clusterfuck of everything associated with it is honestly about as hard to get into as actuarial science.

It requires such a heavy base of fundamental concepts that if you dont start at the bottom and work up, you have no clue wtf is going on.

I can’t get anything actually running!

Most of my time in my sysadmin work is sitting there staring at the screen reading error logs about why some new product isn't working.

make sure you ACTUALLY have a solid grasp of networking. dig into the osi model or the 3 layer dod model. Go further than "yup. encapsulation. got it". Look at the actual headers in layer 2 and layer 3. Figure out how packets get routed and how frames get forwarded. Look at how NAT actually mangles packets.

Understanding hos the traffic flows will make troubleshooting much easier.

If you've never gone and built your own system, or started a network from scratch without a container based system, and you've relied on stacking your own environment for a long time, you might necognize it as you would any skill.

I'm sure I could build a shed, for E.g. I know to get the wood, the panels, the cement, and all the things I'll have all documented in front of me.

Then I will start, realize I forgot I also needed a permit, which I then found out I might also need to run power to it, so now shed has become something that doesn't feel impossible, it feels daunting

Why? Well, I don't build sheds, I don't keep building things that would allow me to have all the framework, project of getting my shed built. So, as with a homelab, take it one step at a time, and if rhe shed isn't the house, you're allowed to skip a few things and stick with what works for you.

I use homelab and self hosting as a passion, but I also have 25 years of networking and systems administration experience so docker and containers just made my normal, tedious job of building up complex network infrastructure, feel like a job, and my homelab and self host happens to be a hobby

Treat your homelab as a passion or simple hobby, or use it as a skill building tool for work

Overall it's just time, experience and expectations that either drive you forward or set you back

Decide what you want to self-host. Keep it simple at first then work your way up as need be. You could first self-host a simple web site. You could use Tailscale free vpn with 100.x.x.x ip's to test your setup from another computer anywhere.

Back in the day you had to find a book if you didn't know people

Because security isn't easy. Know you know why system admins have a problem with programmers that just expect things to work.

Most blogs and articles I find are about what they are which I already know.

Exactly.. I neither want "what z is" nor all the history behind z, the Z inventors life, his grand parents lives etc. I just want a solution to the problem im having..

Because it's a complicated topic, you're just feeling the actual responsibilities of doing it right that many are ignoring now.

This repo is worth checking out.

my love here :D

That's not bad

The greatest advice my teachers in university told me was to tackle everything in life with the foundation/philosophy of K.I.S.S. = Keep It Simple Stupid.

It's more of a code to live by; to which you should ask yourself the question first "why do I need this thing?" What problem is it solving, and is there a simpler method?

Usually, most labbers don't even need 10% of what we think we need. We introduce complexity for the sake of complexity.

Just be comfortable with an ecosystem or make your own that aligns with your values/principles.

Hone your senses by questioning the conceptual integrity of others (everyone has an opinion, don't take their opinion unless they're a trusted person that you respect). If you want to emulate someone based on their own system that matches yours, then do exactly to the letter what they have done and mimic their approach. Most times, you find that once you've concluded your journey you often feel it wasn't worth the journey in the end and you take those hard lessons and sometimes failures and it bridges your understanding further to the truth of a matter.