James Harper <james.harper@bendigoit.com.au> wrote:
CEPH does sound exciting. Is anyone here doing it with Xen?
It was still under heavy development when last I read about it. The kernel module entered the mainline and, I assume, has undergone further work since then.
File systems take a long time to mature. Testing and bug finding efforts always help, however.
Full of enthusiasm, I've just created 3 xen VM's and created a 3 node CEPH cluster inside. It's amazingly easy to set up, but from what I can see some of the init tools automate a lot of the steps that I assume need to be done manually for ongoing maintenance, so I may go back and set it up the long way around to get a feel for the steps involved, or maybe just add a few more nodes. For a production CEPH installation, I think I would still use RAID1 underneath and then deploy CEPH on top, as it lowers the risk of a node-down situation, at the expense of requiring additional disks. Debian appears to have some OCF RA's which I haven't investigated yet, as I'd still want to use pacemaker for managing the actual VM's. For a 5 node cluster with 2 copies of all xen virtual disks I assume it becomes important to run VM's on a node that has a local replica of the data... or is the data striped across all nodes transparently in a "you don't need to know" way? I'd better do some more reading I guess :) James