NVMe benchmark results (Was: Re: [MLUG] Next meeting - 26th March)
On Tue, Mar 27, 2018 at 02:02:33PM +1100, Michael Pope wrote: [...]
Duncan, Thanks for sharing your NVMe experiences at the meeting last night. Would you be able to run some benchmarks using 'bonnie++' please and post the results? from Michael
Hi Michael, I ran a pair. For sda2 (ext3) I made the mistake of testing random creates / stat / delete which ran all night and then some, just soaking up CPU. So I didn't run that for nvme0n1p2 (ext4).
18:07:37$ bonnie++ -d /sda2/bonnie -s 16g -n 1024 Writing with putc()...done Writing intelligently...done Rewriting...done Reading with getc()...done Reading intelligently...done start 'em...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.03e ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP dimstar 16G 27653 53 27340 16 9023 3 18208 33 20106 3 136.6 0 ------Sequential Create------ --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 1024 57 99 932164 99 51223 56 58 99 175 99 105 66 dimstar,16G,27653,53,27340,16,9023,3,18208,33,20106,3,136.6,0,1024,57,99,932164,99,51223,56,58,99,175,99,105,66 10:20:25$ bonnie++ -d /nvme0n1p2/bonnie -s 16g -n 0 Writing with putc()...done Writing intelligently...done Rewriting...done Reading with getc()...done Reading intelligently...done start 'em...done...done...done... Version 1.03e ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP dimstar 16G 63069 71 332703 41 142806 19 71545 75 375326 25 +++++ +++ dimstar,16G,63069,71,332703,41,142806,19,71545,75,375326,25,+++++,+++,,,,,,,,,,,,, 10:33:31$
Per Chr Sequential Output is 2.26 times faster Block Sequential Output is 12.17 times faster Rewrite Sequential Output is 15.83 times faster Per Chr Sequential Input is 3.93 times faster Block Sequential Input is 18.67 times faster NVMe Random Seeks completed too quickly for bonnie++ to report a result (<0.5s) Bear in mind this is probably with only a PCIe version 2 bus, Cheers ... Duncan.
On 28/03/18 11:43, Duncan Roe wrote:
On Tue, Mar 27, 2018 at 02:02:33PM +1100, Michael Pope wrote: [...]
Duncan, Thanks for sharing your NVMe experiences at the meeting last night. Would you be able to run some benchmarks using 'bonnie++' please and post the results? from Michael
Hi Michael,
I ran a pair. For sda2 (ext3) I made the mistake of testing random creates / stat / delete which ran all night and then some, just soaking up CPU. So I didn't run that for nvme0n1p2 (ext4).
18:07:37$ bonnie++ -d /sda2/bonnie -s 16g -n 1024 Writing with putc()...done Writing intelligently...done Rewriting...done Reading with getc()...done Reading intelligently...done start 'em...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.03e ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP dimstar 16G 27653 53 27340 16 9023 3 18208 33 20106 3 136.6 0 ------Sequential Create------ --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 1024 57 99 932164 99 51223 56 58 99 175 99 105 66 dimstar,16G,27653,53,27340,16,9023,3,18208,33,20106,3,136.6,0,1024,57,99,932164,99,51223,56,58,99,175,99,105,66 10:20:25$ bonnie++ -d /nvme0n1p2/bonnie -s 16g -n 0 Writing with putc()...done Writing intelligently...done Rewriting...done Reading with getc()...done Reading intelligently...done start 'em...done...done...done... Version 1.03e ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP dimstar 16G 63069 71 332703 41 142806 19 71545 75 375326 25 +++++ +++ dimstar,16G,63069,71,332703,41,142806,19,71545,75,375326,25,+++++,+++,,,,,,,,,,,,, 10:33:31$
Per Chr Sequential Output is 2.26 times faster Block Sequential Output is 12.17 times faster Rewrite Sequential Output is 15.83 times faster Per Chr Sequential Input is 3.93 times faster Block Sequential Input is 18.67 times faster NVMe Random Seeks completed too quickly for bonnie++ to report a result (<0.5s)
Bear in mind this is probably with only a PCIe version 2 bus,
Cheers ... Duncan.
Thanks Duncan looks like a great speed improvement.
participants (2)
-
Duncan Roe -
Michael Pope