On Wed, Aug 10, 2016 at 03:39:40PM +1000, Russell Coker wrote:
On Monday, 8 August 2016 2:05:47 PM AEST Robin Humble via luv-main wrote:
I wanted to test ZFS for other reasons though - Lustre ZFS OSDs. Why can't Lustre run on BTRFS?
Lustre doesn't use the ZFS POSIX layer. infographics tell me that it hooks into the ZFS ZAP and DMU directly - for performance reasons I presume. I think I vaguely knew what those were once, but no longer. the entirity of Lustre server code is in-kernel using in-kernel APIs. so adding btrfs to the object storage options (currently ~ext4 and ZFS) wouldn't be trivial. I think there was some talk of btrfs a few years ago though, and the re-architecting to allow ZFS has no doubt made it easier to add a third choice. it might happen one day, especially if btrfs is more stable now. hmmm, maybe Lustre not using the POSIX layer also means that there is no ZFS vs. Linux VM cache conflicts because it's _all_ ZFS cache. that would be nice. I'll have to run some tests and check that... so is it possible to extremely heavily prioritise metadata over data in the ZFS read caches? never dropping metadata would be my preference. that's how we get excellent IOPS to our (ext4 OSD) Lustre currently. sadly it's all too easy for GB/s of use-once data to flush caches of metadata that then takes ages to re-read from disk... cheers, robin