On Mon, 27 Jul 2015 09:46:41 AM Craig Sanders wrote:
btrfs probably handles this slightly better because it's easier to add a single disk or two to increase capacity...and, of course, you can rebalance to redistribute your data evenly over all disks in the pool.
The most significant thing about how BTRFS handles this better is that BTRFS allows removing disks and changing RAID configuration on the fly. Admittedly while RAID-5/6 hasn't been tested enough there is a limit to what can be done here, but in the future a live migration from RAID-1 to RAID-6 after adding a few more disks will be an option. ZFS only allows replacing disks and adding pools and vdevs. AFAIK it's impossible to create a ZFS array with 4*1TB disks and migrate it to 2*4TB without send/recv and downtime.
the free space issue on zfs is not a "rule of thumb" but a well known fact about it - getting over 80% full is really bad for performance, and just about every tuning or setup guide will mention it. i don't know btrfs as well but it wouldn't surprise me if the <10% free issue that was mentioned is also a hard rule (as in "don't do it")
In BTRFS it used to be that the filesystem could deadlock if you ran out of space. But I think that adding GlobalReserve has solved that. I don't recall seeing performance problems with BTRFS filesystems that were near full, but the history of BTRFS has led me to take more effort to avoid them getting full than I did with Ext* filesystems.
If I wanted good performance on a BTRFS array I would make the filesystem as a RAID-1 array of SSDs. Then I would create a huge number of small files to allocate many gigs of metadata space. A 4TB array can have 120G of metadata so I might use 150G of metadata space. Then I'd add 2 big disks to the array which would get used for data chunks and delete all the small files. Then as long as I never did a balance all the metadata chunks would stay on the SSD and the big disks would get used for data. I expect that performance would be great for such an array.
with zfs, you'd just do:
zfs set secondarycache=metadata <fs>
that tells it to use the L2ARC (e.g. ssd or other fast block device cache) to only cache metadata. this can be set on a per-filesystem ("subvolume" in btrfs terms) basis.
you can also set primarycache (i.e. ARC in RAM) to the same values - all, none, or metadata with the default being all (actually, inherit from parent with the ultimate parent's default being all) .
zfs doesn't do rebalancing, or rseizing, (unfortunately - they're the key features that btrfs has that zfs doesn't) but if it did you wouldn't have to avoid using them so that a kludge like that keeps working. clever tricks can be cool but designed reliable features are better.
Yes. BTRFS is lacking in that regard, but it's not surprising given that they haven't even written code to optimise reading for both disks in a RAID-1 last time I checked. There hasn't been much work done to give BTRFS good performance. Unfortunately the BTRFS developers seem to have little interest in features like the copies= feature of ZFS. I hope they will change their minds about that when BTRFS has the more common reliability issues solved. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/