One of the issues we encountered designing a small virtualization farm was that as you add more DIMMs to the system, the clock speed of the RAM had to go down - addressing more addresses over a limited bandwidth meant ou traded of speed for capacity... we chose capacity - and lower speed, lower cost RAM - which worked out for our use case (a few large high capacity VM's). If we had been planning one of the bioinformatics grid compute modes, it would have been a different story... On 20 May 2018 at 01:17, Russell Coker via luv-main <luv-main@luv.asn.au> wrote:
https://en.wikipedia.org/wiki/DDR4_SDRAM
At the LUV meeting today someone asked me how servers can have so much RAM. Firstly the above page is worth reading, DDR3 (which is in most desktop PCs in use now) can have a maximum of 16G per DIMM, some PCs only have 2 slots, 4 is very common, and 6 is reasonably common for high end systems (I own a couple of those). 6*16G=96G as the theoretical maximum in a high-end DDR3 desktop system and 6*64G=384G as the theoretical maximum of a DDR4 desktop, of course the BIOS or chipset might not support so much.
https://en.wikipedia.org/wiki/List_of_Intel_chipsets
The above page lists Intel chipsets and gives the maximum RAM supported. A lot of the Core 2 family were limited to 4G of address space, NB that's NOT 4G of RAM, that's 4G including address space for video cards etc - so 3.25G was a common amount of usable RAM on such systems. The Wikipedia page doesn't give information on the RAM limits of the i3/i5/i7 systems.
The DDR4 Wikipedia page says that one of the benefits of DDR4 is a maximum module size of 64G. A quick check of my favorite PC parts store showed that they don't sell DDR3 larger than 8G modules or DDR4 larger than 16G (preumably I could get larger via mail order). If I got a motherboard that took 6*DDR4 DIMMs then I could have 96G of RAM, but that would cost me 6*$295=$1770 so I'm not about to do it.
http://www.dell.com/en-au/work/shop/povw/poweredge-t640
The Dell page for the PowerEdge T640 says that it has 24*DDR4 slots for up to 3TB of RAM with a caveat that the 128G modules aren't available yet (as an aside it seems like the DDR4 Wikipedia page needs an update in this regard).
https://en.wikipedia.org/wiki/Fully_Buffered_DIMM
Threre are significant engineering issues related to supporting large numbers of DIMM sockets. The above Wikipedia page is a good place to start if you want to learn about how server RAM is different from (and incompatible to) desktop RAM because of such issues.
https://en.wikipedia.org/wiki/ECC_memory https://en.wikipedia.org/wiki/Hamming_code
Another difference with server RAM is the use of Error Correcting Codes (see the above Wikipedia pages). Server RAM has extra bits in each word to store codes that allow single bit errors to be detected and corrected and double bit errors to be detected. It's a really good feature to have if you don't want your data to be corrupted. It's also supported in high-end workstation systems and some systems have support for both ECC and non-ECC RAM. ECC RAM is usually "buffered" and won't work in systems that take "unbuffered" RAM (IE all the desktop systems that don't use ECC RAM). There is unbuffered ECC RAM for low-end server systems. Apart from buffered vs unbuffered there's apparently no reason why ECC RAM shouldn't work in non-ECC systems, although when I tried this back in the P4 days (before buffered RAM was invented) it didn't work.
One of the more dedicated members of this list got a free server system from LUV and uses it as his personal workstation. It has something like 96G of RAM but makes more noise than most people want in the same building they are in. There's no reason why you couldn't design a system with 24 DIMM sockets that doesn't sound like an aircraft taking off, but most people who want so much RAM have a soundproofed server room.
-- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
_______________________________________________ luv-main mailing list luv-main@luv.asn.au https://lists.luv.asn.au/cgi-bin/mailman/listinfo/luv-main
-- Dr Paul van den Bergen