On Sat, 25 May 2013, Craig Sanders <cas@taz.net.au> wrote:
On Thu, May 23, 2013 at 11:01:47AM +0000, James Harper wrote:
You say "One is to use a single BTRFS filesystem with RAID-1 for all the storage and then have each VM use a file on that big BTRFS filesystem for all it's storage"
note that if you don't have any other particular reasons to use btrfs rather than zfs, then zfs is a better choice for this job.
I noted in the first paragraph that 4G of RAM for ZFS alone seems inadequate. I had ongoing problems with a server that was running nothing but Samba and NFS file serving with ZFS storage and 4G of RAM that gave repeated kernel errors about memory allocation. The recommended way of solving that problem didn't work, so I upgraded it to 12G of RAM (8G of RAM was a lot cheaper for the client than paying me to figure out the ZFS problem).
zfs allows you to create disk volumes (called "zvol") as well as filesystems from the pool, which are similar to a disk partition or an LVM logical volume but with better performance and all the other benefits of being part of a zpool.
I've got a Xen server that uses ZVols for the DomU block devices. I've been wondering if it really gives a benefit. If I had used a regular ZFS filesystem with large files to contain all the virtual block devices in question then I could have made a snapshot backup of all of them with one command. I've also been moving some of the larger storage (such as mailing list archives) from filesystems on block devices in the VM to NFS mounts of ZFS filesystems, this makes snapshotting a little more useful.
http://pthree.org/2012/12/21/zfs-administration-part-xiv-zvols/
a zvol can also be exported via iscsi, so a VM on a compute node could use a zvol exported from a zfs file-server. could even use 2 or more zvols from different servers and raid them with mdadm (i haven't tried this myself but there's no reason why it shouldn't work - synchronised snapshotting may be problematic, you'd probably want to pause the VM briefly so you can snapshot the zvols on the file servers).
Why would pausing the VM be necessary? -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/