On Sunday 06 May 2012 14:41:24 Craig Sanders wrote:
IB and 10Gb ethernet cards seem to go for around $600-$800 new. so a pair would get you a 10-40Gbps connection for around $1200-$1600. not cheap.
Remember that for IPoIB the bandwidth you can get is determined by the subnet manager as well as the devices that are connected together, so a common issue with "managed" switches is that the built in subnet manager will only allow 10Gbps unless you find some way to tweak that. You can use "ipdiagnet" to spot that: -I--------------------------------------------------- -I- IPoIB Subnets Check -I--------------------------------------------------- -I- Subnet: IPv4 PKey:0x7fff QKey:0x00000b1b MTU:2048Byte rate:10Gbps SL:0x00 -W- Suboptimal rate for group. Lowest member rate:20Gbps > group-rate:10Gbps The MTU there is for IB in datagram mode, you can go way over that if you swap to connected mode.. cheers, Chris -- Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC This email may come with a PGP signature as a file. Do not panic. For more info see: http://en.wikipedia.org/wiki/OpenPGP