On Tue, 17 Apr 2012, Toby Corkindale <toby.corkindale@strategicdata.com.au> wrote:
On 17/04/12 11:25, Trent W. Buck wrote:
Russell Coker wrote:
One of my clients needs to reliably store terabytes of data which is mostly comprised of data files in the 10MB - 15MB size range. The data files will almost never be re-written [...]
Will they ever be re-read? ;-)
Yes. There will be batch jobs that process them.
For that matter, do you really need all the files stored on a single machine? No matter what level of RAID you use, you still have a single point of failure in the server hardware.
Why not pick up several smaller servers and use a cluster filesystem instead?
A cluster filesystem requires significantly more work to administer. It needs to have at least 3 servers to avoid split-brain problems and will either involve performance loss (in the case of a cluster where the blocks are shared via Ethernet) or significant extra expense (in the case of shared block devices). http://www.dell.com/au/business/p/poweredge-t110-2/pd http://www.dell.com/au/business/p/poweredge-t610/pd A cluster filesystem also doesn't solve any real problems. A Dell server which can handle 8 disks (all that is needed for the next few years) is not particularly expensive. A PowerEdge T610 costs $2100 plus about $200 per disk (or a lot more per disk if you buy disks from Dell). A PowerEdge T110 costs $700 and can handle 4 disks. If you had three of the T110 servers in a cluster they would cost the same as a single T610 and provide the same effective capacity, however you would purchase 12 disks instead of 8 and thus the hardware cost would involve an additional 4 disks ($800 or more if you buy disks from Dell). Then there's the cost of sysadmin work. Now one thing that a cluster can solve is a server failing at some unexpected time (*). But in this case the use is going to be 9-5 operation. Batch jobs will be run overnight, but getting a cluster fail-over event to not interrupt the batch jobs would be more effort than it's worth. Getting a system that can handle 8*SATA disks isn't THAT difficult. In the unlikely event that a Dell server entirely broke and Dell couldn't fix it fast enough it would be OK to install a white-box system as a temporary replacement. So the failure recovery case will not be preserving 24*7 operation, but not wasting too much 9-5 time after the problem has been discovered. http://etbe.coker.com.au/2010/08/04/clusters-dont-work/ (*) In theory, in practice I haven't observed that happening. See the above URL. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/