On Fri, 24 Jul 2015 10:12:04 PM James Harper wrote:
I have had serious metadata performance issues with BTRFS on my 4TB RAID- 1 array, such as a "ls -l" taking many seconds to complete. For that array I can just wait for those cases, for that system all the data which needs good performance is stored on a SSD.
If I wanted good performance on a BTRFS array I would make the filesystem as a RAID-1 array of SSDs. Then I would create a huge number of small files to allocate many gigs of metadata space. A 4TB array can have 120G of metadata so I might use 150G of metadata space. Then I'd add 2 big disks to the array which would get used for data chunks and delete all the small files. Then as long as I never did a balance all the metadata chunks would stay on the SSD and the big disks would get used for data. I expect that performance would be great for such an array.
That sounds unreasonably fragile. Especially if you are unable to ever do a balance.
You should only ever need to do a balance if you have too much space allocated to one of data/metadata and need to free some for the other or when you are doing things like changing RAID levels. In normal use you shouldn't need to do it. The fact that it is sometimes needed in normal use is due to deficiencies in BTRFS that might have been fixed now.
My first testing of btrfs was on top of bcache, and performance was awesome. I went back to entirely rotating media for production though as I only had a single SSD at my disposal, didn't really need the extreme performance, and had other things to spend money on. Also at the time there were reports of incompatibilities between btrfs and bcache. I expect bcache would out-perform the hot-relocation project, for most workloads. For my server which does lots of streaming writes (mythtv) and lots of random io (other stuff), it would balance things nicely.
The concept of Bcache sounds good, but the bug reports are concerning.
This guy claims success with bcache + btrfs http://www.spinics.net/lists/linux-btrfs/msg42125.html and raises some interesting points (interesting to me, at least).
Very impressive.
Btw, when you say 5TB RAID1, what exactly do you mean? Is the 5TB referring to the raw disks or the usable redundant space? I'm never quite sure.
5TB disks are quite affordable nowadays. 6TB is still a little expensive. So a RAID-1 array of 5TB disks is a good option. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/