Proxmox VE is a complete virtualization management solution for servers. By design it support two very popular virtualization techniques: OpenVZ and KVM. OpenVZ is container-based virtualization for Linux. However, only supported by pretty ancient kernel: 2.6.32. Lately I gave another try to check whether we can use and replace OpenVZ with a new functionality present in latest kernels (3.14) the cgroups.
And guest what? Not perfect, but more or less it's working!
If you are familiar with OpenVZ you should know about vzctl. Basically vzctl is the primary tool for Container management. Since version 4.0 (now we have 4.7) some work has been made to support upstream kernels with cgroups (for more info visti here). A few days ago version 4.7 was released. Support for cgroup is pretty decent. After a few tests I managed to run CT's on Linux 3.14 with mixed success. Some of things were not working (memory limits, venet0), some of them were working just fine (cpu, userns).
I managed to fix most of the issues and integrated support for upstream vzctl in Proxmox VE.
You can inspect all changes on my GitHub repos:
- https://github.com/ayufan/pve-vzctl-for-upstream - vzctl with fixes for venet0, netns and memory limits
- https://github.com/ayufan/pve-cgroupfs-mount - backported cgroup mounter for debian (newer version requires systemd)
- https://github.com/ayufan/pve-manager-for-upstream - pve-manager with fixes to support reading statistics from vzctl cgroups
- https://github.com/ayufan/linux-kernel-with-vga-passthrough/tree/cgroup - linux kernel 3.14.1 with example config for cgroup - kernel supports cgroups with cpu, memory, network and userns limits and apparmor profiles.
- CPU limits
- CPU shares
- Memory, Kernel memory, swap limits
- TCP limits
- CPU, disk and network usage in Proxmox
- Dynamic limit switching
- User namespace!
- vzctl exec and vzctl enter
- veth (IP address) and vnet (Network device)
What doesn't work?
- Disk quotas
- VNC and vzctl console
- CT reboot
- First you have to install Proxmox VE. Proxmox requires Debian Wheezy - you can use their ISO or install it by hand.
For more info visit: http://pve.proxmox.com/wiki/InstallProxmoxVEonDebian_Wheezy
After installation reboot and try to login to Proxmox's webpanel: https://PROXMOX_IP:8006/
Enable AppArmor and memory cgroup with swap
Download modified pve-manager, vzctl, cgroupfs-mount and linux kernel
Install AppArmor and all downloaded packages
That's all. Reboot system and you should be ready! Just verify that you are running correct kernel.
You can now login to Proxmox's webpanel: https://PROXMOX_IP:8006/
To fire up CT from command line do:
Running containers in User namespace
By default vzctl uses User namespaces. If you want to disable it just comment out LOCALUID and LOCALGID from /etc/vz/vz.conf. However, it greatly reduce the security of containers. I adwise you to leave it as it is. All new containers will be build in User namespace context.
This is how
ps auxf looks from host:
This is how
ls -al /var/lib/private/100 looks from host:
So it's strongly advised to use user namespaces. However it requires a few additional steps to make it run well.
Remap containers to User NS
Use this simple tool. It will remap uid and gid to 100000 to all files and directories in /var/lib/vz/private.
If you are willing to go back remap all uids and gids back to 0.
Fix Debian-based containers
Running containers in User namespace requires two additional entries in /etc/fstab:
Fix Ubuntu-based containers
The same thing apply for Ubuntu containers:
Also fix for mountall is required in /etc/init/mountall.conf:
The work here cannot be considered production ready. Use it with caution!
- v1: first public release (2014-04-27)