I’ve rebuilt the Proxmox servers – the migration was pretty uneventful and incredibly easy. I installed PVE 7.2 onto a standby server, got that all configured to the way I kinda want it (LAGs, VLANs, storage). Backed up all LXC’s and VM’s, and used SCP to copy them over to the standby server. Switching them over was so uneventful you’d expect they hadn’t even gone offline – which I was pretty pleased with.
I reinstalled Proxmox onto the main two servers and created a cluster, and did the same thing. Backed up the VM’s, SCP over, switch over!
I’ve since re-installed Proxmox onto the standby server and have decided it’ll join the cluster, so that I can properly have 3 servers in the pool. Having only two servers caused some issues, especially during a shutdown/boot procedure. I’m hoping 3 servers will help reduce issues with that.
VLANs and LAGs
I’ve also implemented VLAN’s to my network, to finally separate the each security level, unsecured, trusted, DMZ, secret. Each VLAN contains at least 3 subnets, which will be managed by the DHCP server. I still need to reconfigure the DHCP server so that IP addresses are allocated to unknown clients to the least secure level (IoT). Currently, clients need to have their MAC addresses registered in the DHCP server to get an IP address outside of the unsecured-IoT network.
I’ve also kept Link Aggregation Groups in the Proxmox server – though I found that only LACP appears to work between Proxmox and the Dell Powerconnect switch. If I understand this correctly, it means that a new switch will need to be preconfigured with the LACP LAG first before I can switch a PVE server across to the new switch.
I’ve also added these VLANs into the UniFi AP’s, so that cameras and workstations can still connect to the trusted and secret networks.
Ubuntu to Debian
I’ve since begun to regret going with Ubuntu Server for LXC containers, and have started switching over to Debian. The Ubuntu servers have too much overhead and other little systems screwing with what should be fairly simple processes. Most particularly frustrated with LDAP and SystemD ResolveD. Also found that for LXC’s I can’t do do-release-upgrade, after trying a couple times it totally borks the whole system. I have a couple LXC’s stuck on Ubuntu 18.04 and 19.04. For each old Ubuntu server I need to build a completely new Debian one, which will require some work. Oh well.
I plan on posting a nice write-up on how I’ve configured my network. It’s probably not so ideal, but I see it as a poor-mans enterprise network, it’ll make do.