I have a server with a bunch of services just as Docker containers. I see that Proxmox is popular among the self-hosting community. I was wondering why?
I understand that running things in a VM provides better security than running them in a container. But is the difference so important given the relatively low risk that an exploit happens inside a container that leads to doing damage to the host machine?
There’s also obviously the additional overhead of using Proxmox. It wouldn’t be an issue for me as I should have enough resources to, say replace all my Docker containers with VMs. I’m more wondering if the security difference is so massive, or if there is another reason I’m missing why people use Proxmox.
Or am I misunderstanding how people use Proxmox? I was assuming people would use it like how you use Docker, i.e. different services get their own VM/container. If you have a different kind of setup I’d be interested in hearing it.
Edit: I would appreciate if people stop being pedantic and actually read the post. Obviously I am aware that you can run containers in VMs, or containers on bare metal alongside VMs. That’s not what the question is and you know it.
VMs are managed by you. You’re responsible for dealing with prerequisites, updates, security.
Docker is a dev stating “works on my machine” and giving you a copy of their machine.
You can run docker within proxmox, and doing so gives you the ability to run containers in addition to VMs.
There are advantages and disadvantages to both.
My vote is Podman with an immutable distro, like OpenSUSE MicroOS or Fedora Silverblue. Here are my reasons:
- rolling base, with very minimal footprint, so you don’t need to worry about upgrades
- podman runs proper rootless containers, so you get better security vs docker, which tends to run as root (breaking out does less damage if you manage permissions properly)
- deploying a new service (or moving a service) just means copying configs and running, no concerns about what the host has
- there’s nothing special about the host, so if MicroOS or Silverblue are abandoned, just copy the configs and data to a new host
It’s a little more work to set up, but once things are running, it’s drama free. And I think that’s the best thing to optimize for, keeping things boring is a good thing.
“I run an immutable distro, BTW”
Proxmox or even just lazy old KVM GUI for anything that needs to be deployed manually in a VM (Home Assistant, WIndows VM, etc.). Otherwise you can even just spin up whatever manual service you want to run on an LXC container or bare metal host with the correct security settings with systemd and selinux if you want to be extra careful.
Docker/Podman (the superior one lol) is just an automated deployment system in container form (like Ansible). It great for automated deployment without having to manually configure the installation process and worry about upgrades, changes, etc. You can even easily create your own images on the fly just for the purpose of having it run a single service inside a container.
Proxmox equivalent would be like using Terraform/OpenTofu to deploy VMs to do the same thing. Its possible, but just not that common because of the reduced overhead with containers, and well supported deployment images with docker/podman specifically.
Generally speaking, I’ve seen proxmox used more in lab environments were you want to emulate something like a complete network of machines whereas docker/podman has become the defacto server deployment platform.
You’re just much more likely to find software with a published docker container and default docker compose script than the same thing in Terraform or even K8s/K3s.
Proxmox or Docker?
It’s not mutually exclusive? I have a 3-node proxmox config on which I have 3 VMs running as kubenetes nodes to which I deploy containers. I also have some VMs setup for things which either don’t work well as containers or which I simply don’t want as containers (e.g. a couple Windows VMs for doing Windows things). Also home assistant runs in a VM since it was just easier to do USB passthrough this way.
I understand that running things in a VM provides better security than running them in a container.
Not sure what you mean by this - containers are typically easier to secure as they’re minimalist. But I doubt anyone is using VMs because they think they’re more secure.
Run a proxmox VM with docker services. ZFS snapshots and backups via PBS.
I keep landing back to Proxmox, My primary use is to run the Home Assistant OS VM which is quite fantastic there. And also, I have NFS sharing setup on the Proxmox server so I can share it between my machines and my home Linux boxes. I’m on Proxmox 8 though and not 9. Debian 13 with Proxmox 9 it turns out at least when I tried it, is really locked down now for running Docker via the host. (Proxmox machine) With Proxmox 8, I can still install Docker and run my containers there, then use Portainer to manage them sometimes, but rarely now days. You can also probably do it the “Correct way” as some may believe by setting up a VM or LXC in Promox to host docker containers. I do that with one subset of containers but not all.
Another option you may want to consider is XCP-NG, which is another hypervisor and IMHO ran Home Assistant a tad bit faster for me, but it will not allow you to mount existing drives without erasing them (I can’t do that with my disks). Additionally, it seems to be on an out of date CentOS build which is no longer updated. (My notes from this are from a year ago when I tried it and I think some of it has changed, but for storage: https://docs.xcp-ng.org/storage/) You can see what’s going on there.
Most people will say to host Truenas or something like that in a VM via Proxmox but honestly, it isn’t too difficult to set up with a tool like Cockpit to manage the shares. I’ve played with most of the setups recently and recently tried going with a Debian 12 install on bare metal with the Home Assistant VM running which I could, but I had more crashes with the server and it never started the VM in spite of being told to do so. I honestly didn’t stick around though, so YMMV if you go that route.
There are a few reasons why someone might use Proxmox. It doesn’t have to be just security, it can also be different network architectures that don’t work as well in Docker and it can also be just greater control over the services which is less comfortable to do in Docker as it’s meant to have built images that are running and are ephemeral. There are also certain services that either don’t have a pre-built docker and someone might not want to bother with making their own docker infrastructure around it or have technologies that are not well supported or are not well executed in docker.
There is also the fact that Proxmox is meant to be used in production, which means that it’s more stable (than some casual docker rubning on whatever distro they have) and it does have a very low overhead, even if you do use dockers you can use them within Proxmox and it gives you a lot of capabilities that add to stability and manageability.
Generally speaking if your threat model is very small, you’re running this within your private network, and it’s not exposed to the internet or anything large like that, then it doesn’t really make a big difference and you should probably just use whatever is comfortable for you.
I personally moved to Proxmox for three reasons which are security, customizability and stability. I felt that within Docker containers it was a lot more annoying to have to pull the images and make my own Docker files and update them and build them every time. I find it easier to have my own server with its dedicated service and that I know how to update and how to modify more properly and that I built from scratch. There is also the advantage that I can use whatever OS I want for different situations. Of course I personally use exclusively Linux but even within that I can use different distros and I can have all kinds of different services running without interfering with one another in any way, and in extreme cases I can have a windows vm.
And another major factor for me was that I just wanted to learn how to do it. I think it’s cool and it was interesting and I have already experienced Docker to a level that I felt comfortable with it and it was time to move on and expand my horizons.
I use proxmox because I am a tinkerer and VMs help me tinker without worrying about making major mistakes that might brick my server. If I want to try something new, just spin up a test VM and try it out, the rest of my stacks are safe and if I muck up the test VM I’m tinkering with, just delete it and start again.
I started with KVM-QEMU, which proxmox is based on, with virt-manager front end. Can do all the same things, but can be installed on most distros. Will let you get your feet wet with VMs without having to format and install proxmox.
Podman and Proxmox is your answer. Both are great for everything you will ever need virtually. No reaon to choose one or the other, just how you are going to configure your setup.
if you don’t need proxmox’s admin tools
try running podman in NixOS on ZFS
Why ZFS?
Snapshots
