Russell Coker <russell@coker.com.au> writes:
My fallback would be NFS shared ZFS in the domU - much cheaper because you only have one ARC, set of checksums, etc, to manage, but with the added bonus of NFS between the domU and dom0. Fun times.
Are you sure that LXC or OpenVZ wouldn't better fit your needs than Xen? You trade off marginally less isolation between containers for the simplicity of having a single kernel image - so native ZFS performance.
That's a possibility. What is the support for them like in Debian/Wheezy?
I am currently running LXC on 2.6.32, to separate services (e.g. apache, postfix/dovecot, nsd3), on consolidated hardware. IME LXC as at 2.6.32 was inadequate for this task -- in particular, per-container resource allocation and encapsulation (e.g. of /sys) was not ready. Wheezy will be 3.2. I have not done any significant work on LXC on a post-2.6.32 system, but AFAICT it has vastly improved, especially in the areas that annoyed me. IMO LXC on 3.2 is definitely worth at least considering. Important things to note: - the "lxc" and "libvirtd" packages provide COMPETING implementations of the LXC userland tools; you only need one. The former had much more functionality last time I looked; the latter might be useful if you need to give partial privileges to other users (e.g. to manage their own containers). - by default LXC containers are not very secure; if you care about security it is definitely worth spending time dropping as many pcaps as you can (esp. CAP_SYS_ADMIN and mount privileges), and limiting what resources you meaningfully can. This *will* break containers that are stock debootstrap, and you will need to fiddle with init scripts increasingly as you lock it down. http://cyber.com.au/~twb/snarf/lxc-create may be helpful there. - obviously since the kernel is shared, and you (probably) drop modprobe privs within the containers, stuff like iptables-restore can't implicitly load new modules, they have to be modprobed in advance in the host OS. Likewise exposing hotplug equipment to the container is extremely nontrivial. - OTOH, you can choose which parts to containerize, e.g. you can have a separate filesystem but the same network interfaces, or vice-versa. I'm not sure how useful this is in practice. Regarding OpenVZ, AFAIK it was dropped by both Debian and Ubuntu years ago, so the only reason I can think of to run it if you're already invested and LXC doesn't yet do what you need.