Dell Server, ESX 6.5 and iSCSI
Hi all, we have new Dell Servers with Dell's VMware ESX 6.5 installed. They have multiple 10GB interfaces. With NFS we get expected speed, basically to flood the disks up to their limits. However, with iSCSI the network becomes the bottleneck, we only reach 20% of the possible performance. We installed Ubuntu 16.04 on one of the servers, for testing. With Ubuntu we are able to run iSCSI properly. We tried iSCSI with ESX against a Powerfault MD3860i and against a Synology NAS, both setups show the same slowness. So there must be something wrong with the ESX setup in this environment, I believe. So far we looked at "Best practise guides" for VMWare and tried to play with Jumbo frames, made sure they are enabled (size 9000 as recommended in these guides) on all components and disabled flowcontrol. It did not help. My colleague exhausted all options he can think of, I cannot think of anything either, and we asked VMware but did not have a good answer yet.. Of course it is urgent;-) So I wonder whether I can pick your brains. Do you have any ideas what could cause the issue? Thank you for all ideas Peter
On Thu, 11 May 2017 02:47:08 PM Peter Ross via luv-main wrote:
we have new Dell Servers with Dell's VMware ESX 6.5 installed. They have multiple 10GB interfaces.
With NFS we get expected speed, basically to flood the disks up to their limits.
However, with iSCSI the network becomes the bottleneck, we only reach 20% of the possible performance.
How many clients are using it? Do you have problems when all 10GB interfaces are in use or do they also occur when only 1 interface is in use?
We installed Ubuntu 16.04 on one of the servers, for testing. With Ubuntu we are able to run iSCSI properly.
Why not just run Ubuntu then? What does ESX give you that Ubuntu + Xen or KVM doesn't? Do you have any advice on affordable 10GB gear? One of my clients has been holding off on 10GB for financial reasons. Give me some good advice on affordable 10GB and I might be able to help you with 10GB performance tuning a month later. ;) -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
Hi Russell, There are a few things I cannot change, e.g. using VMWare. The problem was solved using "software iSCSI", that means not using the network card's abilities to support iSCSI. Now the speed is as expected. We are using Dell R730 with 4 interfaces, 2x10GB/sec and 2x1GBsec (Broadcom QLogic 57800) and a dual-port QLogic 57810 (both 10GB/sec). The switches are Dell N4032. I did not shop around for quotes (that was done before I started to work here) but I am certain pricing was an important factor when choosing it. My colleague tested throughput with iSCSI, and is happy with it, besides the mentioned issue. E.g. he is practically able to "flood" the disks (some of them SSD and 15k spindles) until they reach the disk related limits. Unfortunately I cannot give you benchmark details because I did not benchmark them myself and I just have my colleague's summary. Regards Peter On Sun, May 14, 2017 at 12:37 PM, Russell Coker <russell@coker.com.au> wrote:
On Thu, 11 May 2017 02:47:08 PM Peter Ross via luv-main wrote:
we have new Dell Servers with Dell's VMware ESX 6.5 installed. They have multiple 10GB interfaces.
With NFS we get expected speed, basically to flood the disks up to their limits.
However, with iSCSI the network becomes the bottleneck, we only reach 20% of the possible performance.
How many clients are using it? Do you have problems when all 10GB interfaces are in use or do they also occur when only 1 interface is in use?
We installed Ubuntu 16.04 on one of the servers, for testing. With Ubuntu we are able to run iSCSI properly.
Why not just run Ubuntu then? What does ESX give you that Ubuntu + Xen or KVM doesn't?
Do you have any advice on affordable 10GB gear? One of my clients has been holding off on 10GB for financial reasons. Give me some good advice on affordable 10GB and I might be able to help you with 10GB performance tuning a month later. ;)
-- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
participants (2)
-
Peter Ross -
Russell Coker