I'm looking at converting some Xen servers to ZFS. This includes a couple of servers for a reasonable size mail store (8,000,000 files and 600G of Maildir storage). For much of the Xen on ZFS stuff I'll just use zvols for block devices and then use regular Linux filesystems such as Ext3 inside them. This isn't particularly efficient but for most DomUs it doesn't matter at all. Most of the DomUs have little disk access as they don't do much writing and have enough cache to cover most reads. For the mail spool a zvol would be a bad idea, fsck on a 400G Ext3/4 filesystem is a bad thing and having the double filesystem overhead of Ext3/4 on top of a zvol is going to suck for the most disk intensive filesystem. So it seems that the correct solution is to do one of the following: 1) Run the mail store in the Dom0 which will be good for performance at the cost of management. A server which has direct user access in any form (including POP and IMAP) needs to be running all the latest security patches while a Dom0 can have patches delayed if they don't seem relevant to network issues or virtualisation. 2) Use NFS to mount a ZFS filesystem from the Dom0. This will be good for management but there's the problem of caching. I don't think that NFS caches that aggressively so I'd need to give more RAM to the Dom0 for ZFS caching and I'd probably still lose some read performance. 3) Run ZFS in the DomU separately from the Dom0. This will work well for the DomU as long as there is enough RAM. But having a ZFS filesystem in the Dom0 as well as a separate one in the DomU (which would use different partitions of the same disks) would be difficult (ZFS in the Dom0 will probably want to grab all zpools). Also write performance will take a hit if there are two separate zpools on the same disks as there will be seeks between writes - this will be particularly bad for mail delivery where the message and the log entry will be written to different parts of the disk. 4) Run ZFS for mail storage in the DomU and use something other than ZFS for the Dom0. This has the same performance problems as 3) but without the issue of different ZFS instances fighting about it. Also I would lose the support for hashes on the zvol data, I could use files on BTRFS for similar data integrity (I'm using RAID-1 so RAID-Z isn't an option and therefore the benefits of ZFS over BTRFS are fewer) but that would still give performance issues. Any suggestions? -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/