I've got a Dell Poweredge tower server running RAID-Z1 on 4 internal disks. I'd like to extend it with another maybe 8 disks in a RAID-Z2 with the option of easily adding another 8 or 16 later. The system has an eSATA connector so an enclosure that provides 24*3.5" disks with a SATA interface (not SCSI because I need SMART to work) at 6Gb/s (600MB/s) will do. Does anyone know of such an enclosure that's cheap? Ideally it wouldn't be excessively noisy (although 24 disks isn't going to be that quiet) because it's not a rack-mount system. Before someone comments on the speed, this is mostly for archival and the data will be accessed over a single Gig-E connection. So I need 100MB/s data transfer out of the system (maybe 140MB/s if a USB device is being used at the same time) and the 600MB/s will only be for zpool scrub. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
On Fri, 26 Jul 2013 10:29:12 +1000 trentbuck@gmail.com (Trent W. Buck) wrote:
Russell Coker <russell@coker.com.au> writes:
(not SCSI because I need SMART to work)
Stupid question: are you saying SMART doesn't work over SAS? Or are you talking about parallel SCSI? I thought parallel SCSI was *long* gone.
http://en.wikipedia.org/wiki/Esata#eSATA # Some low-level drive features, such as S.M.A.R.T., may not operate # through some USB [26] or FireWire or USB+FireWire bridges; eSATA does # not suffer from these issues provided that the controller manufacturer # (and its drivers) presents eSATA drives as ATA devices, rather than as # "SCSI" devices The above section from the SATA Wikipedia page is what concerns me. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
Russell Coker wrote:
On Fri, 26 Jul 2013 10:29:12 +1000 trentbuck@gmail.com (Trent W. Buck) wrote:
Russell Coker <russell@coker.com.au> writes:
(not SCSI because I need SMART to work)
Stupid question: are you saying SMART doesn't work over SAS? Or are you talking about parallel SCSI? I thought parallel SCSI was *long* gone.
http://en.wikipedia.org/wiki/Esata#eSATA
# Some low-level drive features, such as S.M.A.R.T., may not operate # through some USB [26] or FireWire or USB+FireWire bridges; eSATA does # not suffer from these issues provided that the controller manufacturer # (and its drivers) presents eSATA drives as ATA devices, rather than as # "SCSI" devices
The above section from the SATA Wikipedia page is what concerns me.
That... sounds misleading. Certainly USB mass storage enclosures are (IME) not SMARTable, but I ascribe that to the translation to USB, not because there's SCSI in there somewhere. Having said that, I'm not 100% sure I've ever SMARTed a SAS drive.
On Fri, 26 Jul 2013 02:17:02 PM Trent W. Buck wrote:
Having said that, I'm not 100% sure I've ever SMARTed a SAS drive.
Just had a look but all our SAS drives in Linux boxes are RAID'ed from what I see, and of course those aren't SMARTable. cheers, Chris -- Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC This email may come with a PGP signature as a file. Do not panic. For more info see: http://en.wikipedia.org/wiki/OpenPGP
Chris Samuel <chris@csamuel.org> wrote:
Just had a look but all our SAS drives in Linux boxes are RAID'ed from what I see, and of course those aren't SMARTable.
My SAS drives will respond to SMART if you load the sg module, which makes it possible to address the drives directly (as /dev/sg1 and /dev/sg2) rather than through the LSI SAS1064E RAID controller card. It isn't proper hardware RAID, though.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Chris In my experience hardware RAID blocks access to the smart data via the standard Linux tools. Normally you need to use tools which come with the RAID hardware. Where as software RAID allows direct access to the drives, meaning that the smarttools work. Cheers Mike On 27/07/13 1:58 PM, Chris Samuel wrote:
On Fri, 26 Jul 2013 02:17:02 PM Trent W. Buck wrote:
Having said that, I'm not 100% sure I've ever SMARTed a SAS drive.
Just had a look but all our SAS drives in Linux boxes are RAID'ed from what I see, and of course those aren't SMARTable.
cheers, Chris
_______________________________________________ luv-main mailing list luv-main@luv.asn.au http://lists.luv.asn.au/listinfo/luv-main
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (Darwin) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlHzawQACgkQfgQf7OYYFPU3owCfeZ0PzMfCdvgQmjQpGfKBha06 DCgAn3dVzHLltp4Y1U9O5TunXBDsd8nK =BLBn -----END PGP SIGNATURE-----
On 26/07/13 12:47 PM, Russell Coker wrote:
On Fri, 26 Jul 2013 10:29:12 +1000 trentbuck@gmail.com (Trent W. Buck) wrote:
Russell Coker <russell@coker.com.au> writes:
(not SCSI because I need SMART to work) Stupid question: are you saying SMART doesn't work over SAS? Or are you talking about parallel SCSI? I thought parallel SCSI was *long* gone. http://en.wikipedia.org/wiki/Esata#eSATA
# Some low-level drive features, such as S.M.A.R.T., may not operate # through some USB [26] or FireWire or USB+FireWire bridges; eSATA does # not suffer from these issues provided that the controller manufacturer # (and its drivers) presents eSATA drives as ATA devices, rather than as # "SCSI" devices
The above section from the SATA Wikipedia page is what concerns me.
Hi Russel, Trent I'm using an external ESATA, system have been for a number of years. When I first started out the Linux kernel did not support SMART pass-though for SATA, but it did shortly after I started using the system. smartctl -d ata -a /dev/sd[abcdefghijklm] just works :) Cheers Mike
Hi Russel, Trent
I'm using an external ESATA, system have been for a number of years. When I first started out the Linux kernel did not support SMART pass-though for SATA, but it did shortly after I started using the system.
smartctl -d ata -a /dev/sd[abcdefghijklm]
just works :)
FWIW, I haven't found any devices in the last few years that don't work with smartctl, even if they require a bit of tinkering with the -d option to make it work. James
On Thu, Jul 25, 2013 at 05:58:32PM +1000, Russell Coker wrote:
I've got a Dell Poweredge tower server running RAID-Z1 on 4 internal disks. I'd like to extend it with another maybe 8 disks in a RAID-Z2 with the option of easily adding another 8 or 16 later. The system has an eSATA connector so an enclosure that provides 24*3.5" disks with a SATA interface (not SCSI because I need SMART to work) at 6Gb/s (600MB/s) will do.
don't use esata, especially with port multipliers. sata port multipliers suck - they have all sorts of weird controller and drive compatibility issues and are unreliable garbage. http://www.zdnet.com/are-sata-port-multipliers-safe-7000011897/ https://www.usenix.org/system/files/fastpw13-paper7_0.pdf http://forums.storagereview.com/index.php/topic/28641-sata-port-multiplier-r... if you have spare PCIe slots get a cheap SAS HBA (i.e. non-raid) with external ports. 1 or two 8 port SAS cards should be able to handle 24 disks with an SAS expander in the external drive box....but since SAS expanders cost about $500 and 8-port SAS cards can be had for about $100, it's much cheaper to get three 8-port cards, or a 16-port and an 8-port. either way, the dell will be connected to the external drive box by SAS SFF-8088 to SFF-8088 cables (1 cable per 4 SAS ports) but see below about cable costs - cables are expensive, so a SAS expander (or a m/b) will probably work out cheaper.
Does anyone know of such an enclosure that's cheap? Ideally it wouldn't be excessively noisy (although 24 disks isn't going to be that quiet) because it's not a rack-mount system.
Norco RPC-4224. 24 hot swap bays. $479 from techbuy in sydney. road freight to melbourne would be about $22. http://www.techbuy.com.au/p/149123/CASINGS_SERVER_-_4U/Norco/RPC-4224.asp the supplied fans sound like the usual server fans (i.e. like a jet engine warming up) so you'll want to replace the case fans with low-noise fans, and order the replacement internal fan bracket for $12 (it allows you have two quiet 120mm fans rather than three noisy 80mm fans). figure about $20-$40 each for the fans. you'll also need a power supply and extra SAS cabling. the cables will be the most annoyingly expensive part of the project. don't buy them from techbuy or anywhere else in australia, look on ebay for cheap chinese suppliers...they're a third or less of the price of any australian importer. even so, the cables are still going to be expensive. you may find it's cheaper and better to just install a motherboard, CPU, RAM and SAS cards directly into the RCP4224 case and export the zpool to the Dell server as iscsi with one or more gigabit NICs. this will give much better performance on your regular zpool scrubs too. optionally upgrade to infiniband or 10Gbit ethernet (last I looked about a year ago, you can get cards for around $500) if you need faster connection. craig PS: i bought one of these cases at work about 18 months ago. very nice. i'd have one at home except I already had an antec 1200 tower case, and only needed to replace the antec's non-hotswap drive bays with some Lian Li EX-H34B hot-swap bays (4x3.5" drives in 3x5.25" drive slots). The EX-H34Bs cost about $80 each. -- craig sanders <cas@taz.net.au>
participants (8)
-
Chris Samuel -
Craig Sanders -
James Harper -
Jason White -
Mike O'Connor -
Russell Coker -
Trent W. Buck -
trentbuck＠gmail.com