Proxmox VE is a complete virtualization management solution for servers. By design it support two very popular virtualization techniques: OpenVZ and KVM. OpenVZ is container-based virtualization for Linux. However, only supported by pretty ancient kernel: 2.6.32. Lately I gave another try to check whether we can use and replace OpenVZ with a new functionality present in latest kernels (3.14) the cgroups.

And guest what? Not perfect, but more or less it's working!


If you are familiar with OpenVZ you should know about vzctl. Basically vzctl is the primary tool for Container management. Since version 4.0 (now we have 4.7) some work has been made to support upstream kernels with cgroups (for more info visti here). A few days ago version 4.7 was released. Support for cgroup is pretty decent. After a few tests I managed to run CT's on Linux 3.14 with mixed success. Some of things were not working (memory limits, venet0), some of them were working just fine (cpu, userns).

I managed to fix most of the issues and integrated support for upstream vzctl in Proxmox VE.


You can inspect all changes on my GitHub repos:

  • - vzctl with fixes for venet0, netns and memory limits
  • - backported cgroup mounter for debian (newer version requires systemd)
  • - pve-manager with fixes to support reading statistics from vzctl cgroups
  • - linux kernel 3.14.1 with example config for cgroup - kernel supports cgroups with cpu, memory, network and userns limits and apparmor profiles.

What works?

  • CPU limits
  • CPU shares
  • Memory, Kernel memory, swap limits
  • TCP limits
  • CPU, disk and network usage in Proxmox
  • Dynamic limit switching
  • User namespace!
  • vzctl exec and vzctl enter
  • veth (IP address) and vnet (Network device)

What doesn't work?

  • Disk quotas
  • Ploop
  • VNC and vzctl console
  • CT reboot


  1. First you have to install Proxmox VE. Proxmox requires Debian Wheezy - you can use their ISO or install it by hand.
cat <<EOF >> /etc/apt/sources.list
deb wheezy pve-no-subscription
wget -O- "" | apt-key add -
apt-get update && apt-get dist-upgrade
apt-get install pve-manager

For more info visit:

After installation reboot and try to login to Proxmox's webpanel: https://PROXMOX_IP:8006/

Enable AppArmor and memory cgroup with swap

cat <<EOF >> /etc/default/grub
GRUB_CMDLINE_LINUX="\$GRUB_CMDLINE_LINUX security=apparmor apparmor=1 cgroup_enable=memory swapaccount=1"

Download modified pve-manager, vzctl, cgroupfs-mount and linux kernel


Install AppArmor and all downloaded packages

apt-get install apparmor apparmor-profiles
dpkg -i pve-manager_3.2-2_amd64.deb
dpkg -i vzctl_4.7-1pve1_amd64.deb
dpkg -i cgroupfs-mount_1.1_all.deb
dpkg -i linux-image-3.14.1-cgroup-00007-ge19f531_3.14.1-cgroup-00007-ge19f531-6_amd64.deb

That's all. Reboot system and you should be ready! Just verify that you are running correct kernel.

$ uname -a
Linux upstream.home 3.14.1-cgroup-00007-ge19f531 #6 SMP PREEMPT Sun Apr 27 12:18:10 CEST 2014 x86_64
Debian GNU/Linux 7.4


You can now login to Proxmox's webpanel: https://PROXMOX_IP:8006/

To fire up CT from command line do:

vzctl start VMID

Running containers in User namespace

By default vzctl uses User namespaces. If you want to disable it just comment out LOCALUID and LOCALGID from /etc/vz/vz.conf. However, it greatly reduce the security of containers. I adwise you to leave it as it is. All new containers will be build in User namespace context.

This is how ps auxf looks from host:

100000   10858  0.0  0.0   2196   300 ?        Ss   Apr27   0:00 init [2]      
100000   12787  0.0  0.0   2292   236 ?        Ss   Apr27   0:00  \_ /sbin/rpcbind -w
100000   12908  0.0  0.0  19568   540 ?        Sl   Apr27   0:00  \_ /usr/sbin/rsyslogd -c5
100001   12933  0.0  0.0   2068   180 ?        Ss   Apr27   0:00  \_ /usr/sbin/atd
100000   13009  0.0  0.0   2224   408 ?        Ss   Apr27   0:00  \_ /usr/sbin/cron

This is how ls -al /var/lib/private/100 looks from host:

root@upstream:/var/lib/vz/private/100# ls -la
total 84
drwxr-xr-x 21 100000 100000 4096 Apr 27 21:46 .
drwxr-xr-x 10 100000 100000 4096 Apr 27 17:43 ..
drwxr-xr-x  2 100000 100000 4096 Sep 13  2013 bin
drwxr-xr-x  2 100000 100000 4096 Jun  4  2013 boot
drwxr-xr-x  3 100000 100000 4096 Apr 27 14:21 dev
drwxr-xr-x 66 100000 100000 4096 Apr 27 21:46 etc
drwxr-xr-x  5 100000 100000 4096 Sep 21  2013 home

So it's strongly advised to use user namespaces. However it requires a few additional steps to make it run well.

Remap containers to User NS

Use this simple tool. It will remap uid and gid to 100000 to all files and directories in /var/lib/vz/private.

if [ ! -f /usr/bin/uidmapshift ]; then
    gcc -o uidmapshift uidmapshift.c
    sudo mv uidmapshift /usr/bin/uidmapshift
    rm -f uidmapshift.c

uidmapshift -b /var/lib/vz/private/ 0 100000 65536

If you are willing to go back remap all uids and gids back to 0.

uidmapshift -u /var/lib/vz/private/ 100000 0 65536

Fix Debian-based containers

Running containers in User namespace requires two additional entries in /etc/fstab:

none /dev/pts devpts newinstance,rw,gid=5,mode=620 0 0
/dev/pts/ptmx /dev/ptmx none bind 0 0

Fix Ubuntu-based containers

The same thing apply for Ubuntu containers:

tmpfs /sys/kernel/debug tmpfs optional 0 0
tmpfs /sys/kernel/security tmpfs optional 0 0
tmpfs /sys/fs/fuse/connections tmpfs optional 0 0
tmpfs /proc/sys/fs/binfmt_misc tmpfs optional 0 0

Also fix for mountall is required in /etc/init/mountall.conf:

mount -t devpts devpts /dev/pts -o rw,newinstance,gid=5,mode=620
mount --bind /dev/pts/ptmx /dev/ptmx

exec mountall --daemon $force_fsck $fsck_fix


The work here cannot be considered production ready. Use it with caution!


  • v1: first public release (2014-04-27)