We have a few Netgear ReadyNAS 104's, and they all crash under high iscsi load. They are being used to hold backups from a windows server, and the one I've been working on rarely gets through a single overnight backup. I have ssh'd in and it seems that the iscsi backing store is a file sitting on a btrfs mount. The kernel is 3.0.101, which strikes me as a little old for running btrfs although I guess being an appliance they could well have backported everything from a newer more stable release. Or not, going on how much crashing it's doing. I was planning on using netconsole to see if I could capture its dying gasp, but it's not compiled in and not built as a module. A warranty claim is the obvious answer, but that's always a pain so I'm wondering if it's possible to work around the problem by tweaking some things. Netgear are aware of the problem, and have asked things like how full is it getting so I guess they know about the btrfs limitations relating to running out of space. So far the only solution they have offered is to try the latest beta kernel and do yet another factory reset, which we've done. Some info (abridged): # mount /dev/md127 on /data type btrfs (rw,noatime,nodiratime,nodatasum,space_cache) /dev/md127 on /apps type btrfs (rw,noatime,nodiratime,nodatasum,space_cache) /dev/md127 on /home type btrfs (rw,noatime,nodiratime,nodatasum,space_cache) # uname -a Linux RPCV-NAS1 3.0.101.RN_ARM.1 #1 Thu Mar 6 09:31:05 PST 2014 armv7l GNU/Linux Any hints appreciated! Thanks James
On 31/03/2014 12:06 pm, James Harper wrote:
We have a few Netgear ReadyNAS 104's, and they all crash under high iscsi load. They are being used to hold backups from a windows server, and the one I've been working on rarely gets through a single overnight backup.
I have ssh'd in and it seems that the iscsi backing store is a file sitting on a btrfs mount. The kernel is 3.0.101, which strikes me as a little old for running btrfs although I guess being an appliance they could well have backported everything from a newer more stable release. Or not, going on how much crashing it's doing.
I was planning on using netconsole to see if I could capture its dying gasp, but it's not compiled in and not built as a module.
A warranty claim is the obvious answer, but that's always a pain so I'm wondering if it's possible to work around the problem by tweaking some things.
Netgear are aware of the problem, and have asked things like how full is it getting so I guess they know about the btrfs limitations relating to running out of space. So far the only solution they have offered is to try the latest beta kernel and do yet another factory reset, which we've done.
Hi James I assume you have tried the version 6.1.6 ? (http://kb.netgear.com/app/answers/detail/a_id/24560/~/readynas-os-version-6....) they talk about a crash issue with high load while using iscsi. Mike
Hi James
I assume you have tried the version 6.1.6 ? (http://kb.netgear.com/app/answers/detail/a_id/24560/~/readynas-os- version-6.1.6) they talk about a crash issue with high load while using iscsi.
Yep. We're trying 6.1.7rc-something now. After I posted this I got to wondering if there was an iscsi freeze and a btrfs-too-full freeze, which are going to have identical symptoms in that the box is frozen the next morning. Some anecdotal forum messages seem to suggest problems once the disk gets over 80% full. We were running it at 60% full and still getting crashes, but that was earlier and maybe they were iscsi crashes... more testing required! Thanks James
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mon, 31 Mar 2014 01:36:26 AM James Harper wrote:
I have ssh'd in and it seems that the iscsi backing store is a file sitting on a btrfs mount. The kernel is 3.0.101, which strikes me as a little old for running btrfs although I guess being an appliance they could well have backported everything from a newer more stable release. Or not, going on how much crashing it's doing.
A Netgear person on the btrfs list said: # Our OS is a Debian derivative, but we have pulled in lots of changes # from Oracle UEK and the SUSE SLE11-SP3 kernel tree. And we do # indeed run atop MD. So at least it's not just vanilla. All the best! Chris - -- Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQEVAwUBU15PxY1yjaOTJg85AQIsyAf+JRpgB1TkoCHF1r9GSEdS89dSzVmY1VSb RU13TtzTZ9wthbWT795ol7CgtLDfE1UwHJJ/EE5HyStkBGyRQa5KBIauJW8kjp0N S2bBFaEM+JCAUddoiof8fyQMVMKdfI1nxWd04sLcYCetOAA0QOjakT/tclu4NlVD T2Tosv0DR7BeODdtKieNF6wTDZOxyoNQi4mTovfehNKo2LEZb18Jb05sGRSFoguU l7WyAcoqr+nHW9mh4qR4OjPoBYbkKKP5RiiCM6+AI8nXFZvydzMLoxUWWynxJup6 z/VkDRNCKoRpwrEeY/eu5kaXSmH4TrQ3V8LqPoHvgXjUVVjHadVCBQ== =+N6I -----END PGP SIGNATURE-----
participants (3)
-
Chris Samuel -
James Harper -
Mike O'Connor