The best way to see if disk IO is the problem is to run iostat. I run "iostat -x 10" as I find that if a disk is at high load for 10 seconds that usually means that there is a serious performance issue. In the top output you can see that multiple users have IMAP processes blocked on disk IO which is an indication that disk IO speed is probably the issue.
sda = a usb backup drive dm-0 = / (MySQL) RAID dm-1 = / (MySQL) RAID dm-2-5 = /home (also where VM's Live) [1 x hot spare] Does this help at all? Thanks for your input. Cheers Piers iostat -x 10 Linux 2.6.32-573.7.1.el6.x86_64 19/01/16 _x86_64_ (12 CPU) avg-cpu: %user %nice %system %iowait %steal %idle 5.27 0.00 1.74 3.98 0.00 89.01 Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util sda 144.40 398.34 92.54 95.71 5790.73 4493.15 54.63 0.32 1.71 1.38 25.92 dm-0 0.00 0.00 154.36 81.59 1234.90 652.68 8.00 0.04 0.16 0.44 10.43 dm-1 0.00 0.00 154.36 81.59 1234.90 652.68 8.00 0.04 0.17 0.44 10.44 dm-2 0.00 0.00 0.00 0.00 0.00 0.00 8.00 0.00 3.36 3.07 0.00 dm-3 0.00 0.00 0.00 0.00 0.00 0.00 8.00 0.00 0.00 0.00 0.00 dm-4 0.00 0.00 9.63 8.24 77.01 65.93 8.00 0.22 12.41 1.52 2.72 dm-5 0.00 0.00 73.20 404.46 4478.78 3774.52 17.28 0.24 0.27 0.46 21.75 avg-cpu: %user %nice %system %iowait %steal %idle 7.60 0.00 1.51 7.34 0.00 83.55 Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util sda 574.70 116.50 191.70 62.50 8408.00 1175.20 37.70 2.62 9.83 1.67 42.39 dm-0 0.00 0.00 654.50 168.90 5236.00 1351.20 8.00 17.86 9.75 0.36 29.93 dm-1 0.00 0.00 654.50 168.90 5236.00 1351.20 8.00 17.86 9.76 0.36 29.93 dm-2 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 dm-3 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 dm-4 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 dm-5 0.00 0.00 111.40 10.50 3216.00 233.60 28.30 1.46 11.89 2.98 36.29 avg-cpu: %user %nice %system %iowait %steal %idle 6.20 0.00 1.83 16.75 0.00 75.22 Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util sda 1108.30 102.70 499.10 31.70 18019.20 1560.00 36.89 4.53 8.76 1.88 99.87 dm-0 0.00 0.00 1346.00 126.90 10768.00 1015.20 8.00 28.12 25.76 0.50 74.09 dm-1 0.00 0.00 1346.00 126.90 10768.00 1015.20 8.00 28.12 25.76 0.50 74.08 dm-2 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 dm-3 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 dm-4 0.00 0.00 3.90 0.00 31.20 0.00 8.00 0.04 10.62 4.38 1.71 dm-5 0.00 0.00 257.80 7.10 7170.40 135.20 27.58 2.67 10.09 3.76 99.68 avg-cpu: %user %nice %system %iowait %steal %idle 7.48 0.00 2.10 11.25 0.00 79.17 Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util sda 70.20 54.90 421.30 33.90 12352.00 855.20 29.01 2.13 4.69 2.08 94.63 dm-0 0.00 0.00 86.80 77.70 694.40 621.60 8.00 1.43 8.66 1.29 21.17 dm-1 0.00 0.00 86.80 77.70 694.40 621.60 8.00 1.43 8.66 1.29 21.18 dm-2 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 dm-3 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 dm-4 0.00 0.00 12.40 0.00 99.20 0.00 8.00 0.19 15.31 3.06 3.79 dm-5 0.00 0.00 392.50 11.10 11556.00 233.60 29.21 1.65 4.10 2.34 94.48 avg-cpu: %user %nice %system %iowait %steal %idle 7.65 0.00 2.05 13.18 0.00 77.12 Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util sda 672.70 144.50 447.50 78.20 16368.00 1842.40 34.64 4.91 9.32 1.81 95.16 dm-0 0.00 0.00 820.20 208.60 6561.60 1668.80 8.00 24.26 23.57 0.68 70.00 dm-1 0.00 0.00 820.20 208.60 6561.60 1668.80 8.00 24.26 23.57 0.68 70.00 dm-2 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 dm-3 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 dm-4 0.00 0.00 4.30 0.00 34.40 0.00 8.00 0.13 29.98 4.95 2.13 dm-5 0.00 0.00 295.80 14.20 9770.40 174.40 32.08 2.75 8.87 3.03 93.81 avg-cpu: %user %nice %system %iowait %steal %idle 10.51 0.00 2.53 14.68 0.00 72.28 Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util sda 45.00 88.70 128.50 94.70 3609.60 1676.80 23.68 35.44 131.92 4.00 89.20 dm-0 0.00 0.00 83.40 182.10 667.20 1456.80 8.00 44.11 135.81 2.08 55.29 dm-1 0.00 0.00 83.40 182.10 667.20 1456.80 8.00 44.11 135.81 2.08 55.29 dm-2 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 dm-3 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 dm-4 0.00 0.00 3.30 0.00 26.40 0.00 8.00 0.09 28.73 6.48 2.14 dm-5 0.00 0.00 86.90 15.60 2922.40 388.80 32.30 1.93 18.82 8.25 84.59 avg-cpu: %user %nice %system %iowait %steal %idle 9.36 0.00 1.62 6.39 0.00 82.63 Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util sda 39.10 49.00 67.60 180.80 1193.60 3053.60 17.10 2.95 36.02 1.67 41.40 dm-0 0.00 0.00 64.60 75.90 516.80 607.20 8.00 3.23 80.40 2.02 28.38 dm-1 0.00 0.00 64.60 75.90 516.80 607.20 8.00 3.24 80.40 2.02 28.39 dm-2 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 dm-3 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 dm-4 0.00 0.00 15.80 0.00 126.40 0.00 8.00 0.25 15.58 4.72 7.45 dm-5 0.00 0.00 26.20 139.60 544.00 2277.60 17.02 0.87 5.20 2.20 36.51 avg-cpu: %user %nice %system %iowait %steal %idle 12.29 0.00 1.80 12.47 0.00 73.44 Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util sda 43.40 123.30 194.70 323.90 2689.60 4763.20 14.37 4.36 8.42 1.39 72.07 dm-0 0.00 0.00 77.90 233.60 623.20 1868.80 8.00 2.75 8.83 1.26 39.38 dm-1 0.00 0.00 77.90 233.60 623.20 1868.80 8.00 2.75 8.83 1.26 39.39 dm-2 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 dm-3 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 dm-4 0.00 0.00 28.90 0.00 231.20 0.00 8.00 0.29 9.65 1.99 5.76 dm-5 0.00 0.00 131.40 213.50 1838.40 2893.60 13.72 2.91 8.45 1.93 66.50 avg-cpu: %user %nice %system %iowait %steal %idle 8.93 0.00 1.91 8.96 0.00 80.20 Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util sda 45.30 51.80 144.70 142.80 1862.40 2254.40 14.32 4.33 15.05 2.30 66.21 dm-0 0.00 0.00 59.30 85.50 474.40 684.00 8.00 4.01 27.67 2.42 35.08 dm-1 0.00 0.00 59.30 85.50 474.40 684.00 8.00 4.01 27.67 2.42 35.11 dm-2 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 dm-3 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 dm-4 0.00 0.00 36.00 0.00 288.00 0.00 8.00 0.43 11.91 2.08 7.50 dm-5 0.00 0.00 94.60 109.10 1096.80 1570.40 13.09 1.28 6.27 2.59 52.83