tl;dr: Proxmox or bare metal? Containers yes/no? What is your use case?
I used a Dell R710 when I first started self-hosting, it ran ESXi with one VM for each service I wanted. Restarting the server required me to first shutdown each VM in order and then the whole host. When setting up a new service to host I had to create a new VM (allocate RAM, disk etc), install the OS (I ran Debian) and then follow instructions for how to setup the service. This mostly was not a problem but one software I never managed to get working was Apache Guacamole.
Nowadays I have a Dell Optiplex I salvaged for parts, got a new case and all my HDDs from my R710. Because it has much less RAM and an old Intel i5 (6th or 7th generation), I decided to get into Docker. With Docker containers you write your compose file and it will just work. No more need to dig through documentation for which version of a dependency to use, how to handle if two services on the same host need different versions (this was part of the reason for one VM/service). With Docker, I can try out a software in seconds and have it configured to my liking in minutes.
Today I have NixOS (bare metal) and it comes with Podman which uses systemd. Hence restarting my OS (albeit not that often, almost never unless I mess up my config) is a no-brainer because Podman via systemd will manage everything. Adding, stopping or removing containers in general is easy. I have a script running as a service which will stop a container, create a BTRFS subvolume snapshot, start the service again and start borg backup to backup from the snapshot.
For me using Proxmox would just an extra layer of complexity I don’t need. I only have one server and I am the only user.
Questions:
- Do you use Proxmox instead of a bare metal installation?
- Do you use containers or do you install manually?
- What is you use case that requires your setup the way it is?
You must log in or # to comment.
It’s not an either-or scenario. Running services in Docker/Podman is great and makes a lot of sense, as you’ve found. But there’s no reason the OS running those Docker containers can’t be a VM on a hypervisor like Proxmox. Then you get the simplicity of Docker, in addition to the isolation and segmentation (network and process) provided by VMs, and snapshot-based incremental backups from PBS. It’s the best of both worlds. You wouldn’t have a VM per service like you ran before, instead you’d have a VM per group of related services with common networking and security requirements. For example, all of your publicly exposed services can run in Docker in their own isolated VM that’s walled off from the rest of your network, while your internal-only services also run in Docker, but on a separate VM on your internal network.
Proxmox with LXC’s and a few VM’s.
Often with Portainer as well.It’s just nice to use and easy to automate backups.
I used to run barebone on Debian, but I accepted that I’m not good enough in the terminal and to used to graphical solutions to go back.
Proxmox running LXC’s and VM’s, all of them also have docker installed
I don’t care much for proxmox but that’s because I live in the terminal anyway and have a lot of experience administering Linux systems. So for me it’s just an extra layer of complexity, the benefits don’t really apply when my setup has IaC, most of my containers are in Podman rather than LXCs, my backups are automated with my own scripts and cronjobs, and I only run a couple of VMs. Also it makes encrypting the base system more difficult than it is with a standard distro
I’ll die on this hill.
Containers run on “bare metal” in the same way that other processes on your system run.
Who was saying it was otherwise? Containers are not virtualization and never have been.
Maybe I misunderstood - The “NixOS (bare metal)” seemed to imply to me that the containers were not bare-metal by omission. If not then my complaint is retracted.
I moved away from proxmox about 3 years ago to incus, primarily because Proxmox has a lot going on that I don’t need, but also incus is much more amenable to configuration with ansible. Also, and maybe this has changed recently, but passing through PCI devices to incus is much easier.
I use Proxmox as a VDI server. I understand that this isn’t homelab territory, but selfhosted VDIs have proven exponentially more reliable and easier to manage than cloud based ones (after the initial PitA setup). Flawless copy/paste, screen resize, and most importantly, file transfers. When you connect to 12 different clients, each with a different set of security requirements, system hardening, monitoring, and VPN access, being able to console amd fully interact with a sandboxed desktop becomes priceless.
because sometimes i need a vm (home assisstant has more features thsn home assisstant container), sometimes a container (i mostly use lxc).
its not really an extra lauer because proxmox is an os.you use nixos i use proxmox. everything is out of the box. proxmox has comminntiu scripts to run everything and set them up. there is an i stall script for about everything https://community-scripts.org/
sure podman has similar setup but i also need vms and they are all mamaged in one place
EDIT: you can read the script, its text
