Since the start of virtualisation as we know it, two CPU sockets configurations were the obvious choice since it allowed to increase density and run more VMs on a single server, saving in the meantime money on the other components such as RAID controllers, network cads/HBAs, chassis, disks, you name it.
This is all great until you choke on the price of licenses… Like many other software companies, VMware licenses most of its products on a per-socket basis. Not cores, so there’s a trick!
A servers with 2 CPUs of 8 cores would require 2 vSphere licenses right? So if you are using vRops you will need 2 vRops licenses, if you use VSAN you will need 2 VSAN and so on (NSX, Network Insight, …). As you can imagine the number at the bottom of the bill can quickly skyrocket, but there is a way to get more for your money.
And how do you save money then? Easy:
- Instead of two 8 cores CPUs, why not using a single socket 16 cores CPUs?
With the evolution of processors, every year the manufacturers such as Intel and AMD increase the density of cores per CPU. In the 4th generation of E5-2600 (v4), the biggest CPU includes 22 cores!
Let’s have a look at the evolution of the core count on the biggest CPUs in the E5-2600 family over the years:
| Year | CPU Model | Core count |
|---|---|---|
| 2012 | E5-2690 | 8 |
| 2013 | E5-2697 v2 | 12 |
| 2014 | E5-2699 v3 | 18 |
| 2016 | E5-2699 v4 | 22 |
source: https://ark.intel.com/
Of course M. Picky will say:
But the more cores in a CPU, the lower the clock speed!
True, historically this is the case, the more cores the less GHz.
Why does it not bother me?
Most virtualised environments need more cores over clock speed to improve the concurrency. Especially in environments with lot’s of small-size VMs.
So let’s include the Money factor and have a look at it to prove that it could save you good money to switch to single socket servers!
Parameters of the study
We will see here 2 scenarios:
- Scenario A : Small infrastructure, 3 hosts running vSphere Standard only.
- Scenario B : Bigger Infrastructure, 8 Hosts running vSphere with Operations Manager Enterprise Plus and VSAN advanced.
In each scenario we will detail 2 configurations:
- Single socket servers equipped with E5-2698 v4 (20 cores each)
- Dual socket servers equipped with E5-2640 v4 (10 cores each so 20 cores per server)
I picked these models deliberatly to have the same number of cores in each cluster.
The price of the processors come from the Intel ARK website and the price of the VMware licenses from the Dell configurator website as of January 2017. Whether the prices are perfectly accurate doesn’t really matter as the point is to show a difference in relation to each scenario. vCenter is not included because the price will always be the same (vCenter Standard pricing).
For these configs I set a fixed base price for the servers (no license, no CPU) of 3000$ for the small infra and 5000$ for the bigger one. Again this is not relevant as it will be a constant, so let’s dig in.
Scenario A
An SMB company running a 3 hosts cluster on vSphere Standard.
- Single socket, E5-2698 v4 - Cluster price: $22,788
- Dual socket, E5-2640 v4 - Cluster price: $22,854
Here we can see that the dual socket servers are almost at the same price as the single socket one.
To get a more accurate idea, let’s check the price per GHz.
Config A1
- Single socket, E5-2698 v4
1 Socket x 20 cores x 2.20GHz x 3 hosts = 132 GHz
$22,788 / 132 GHz = $172.63/GHz
Config A2
- Dual socket, E5-2640 v4
2 Sockets x 10 cores x 2.4 GHz x 3 hosts = 144 GHz
$22,854 / 144 GHz = $158.70/GHz
Eventhough the price of purchase is greater for the dual socket server, it proves to actually be cheaper for the resources you get. For a $66 difference I’d rather have an extra 12 GHz.
On a dual socket server you also have the possibility to use all the memory and PCI slots, which is not true on a single socket server. Fair enough, one point for dual socket.
Outcome scenario small environment: DUAL SOCKET WINS (as long as other socket based licensed software isn’t added)
Scenario B
A middle sized company running an 8 hosts cluster on vSphere Entperprise Plus, monitor it with vRops and stretches the storage with VSAN advanced edition.
- Single socket, E5-2698 v4 - Cluster price: $156,208
- Dual socket, E5-2640 v4 - Cluster price: $235,824
In this scenario for the same number of cores (not GHz) the cluster is $79,616 more expensive with 2 socket servers than single socket ones. Ouch!
I think these figures are self explanatory, but let’s follow the same reasoning we did with scenario A and get a price per GHz.
Config B1
- Single socket, E5-2698 v4
1 Socket x 20 cores x 2.20GHz x 8 hosts = 352 GHz
$156,208 / 352 GHz = $443.77/GHz
Config B2
- Dual socket, E5-2640 v4
2 Sockets x 10 cores x 2.4 GHz x 8 hosts = 384 GHz
$235,824 / 384 GHz = $614.12/GHz
Again, $170.35 difference for every single GHz in the cluster (The same applies to memory GB, local storage …).
I bet the same M. Picky will say:
But there are 32 GHz more on the dual socket servers.
True, however this time let’s make no compromise and show M. Picky how it’s done!
What if I added a host to the (single CPU) cluster to give M. Picky his 32 GHz?
- 9 x Single socket, E5-2698 v4 - Cluster price: $175,734
1 Socket x 20 cores x 2.20GHz x 9 hosts = 396 GHz
So what is to note here :
- Still $60,090 cheaper than the 8 hosts single socket cluster ! (Nice pay raise for the team * wink * to my boss)
- 12 more GHz (to reduce the impact of the hypervisor overhead for example)
- 20 more physical cores to decrease the vCPU to physical core ratio
- 1 more host to improve redundancy and reduce failure domains
- Room for growth.
The single socket servers has more resources and are cheaper than the dual socket ones.
Should we stop here? we could, the point is made.
Will we? Boy no!
Let’s try to match the $235,824 of the dual socket cluster with single socket servers to see what resources we would get for the same price.
And here it is:
- 12 x Single socket, E5-2698 v4 - Cluster price: $234,312
To summarize, what do you get for $235,824?
(Let’s say each server have 192 GB of memory and 3TB of raw local for VSAN for the sake of this example)
| Dual Socket | Single Socket |
|---|---|
| 160 cores | 240 cores |
| 384 GHz | 528 GHz |
| 1,536GB RAM | 2304GB RAM |
| 28TB RAW | 36TB raw |
| 8 hosts | 12 hosts |
| $235,824 | $234,312 |
Outcome scenario bigger environment: SINGLE SOCKET WIN BY FAR. With a higher core density, single socket servers are much more cost efficient as you license more cores at once with the VMware (socket base licensed) products.
Conclusion
If like M. Picky you have always worked with dual socket servers and thought that single sockets are for peasants, consider looking into it from now on. With the constant evolution of processors it makes more and more sense to go for a single processor with lot’s of cores instead of two smaller ones, especially in environments running a lot of small/medium size VMs.
You might realise that you’ll save a lot of money or maybe it won’t change anything. Again it will depend on your environment. If you have planned to start small, typically because of budget restrictions, and expand later on with solutions like VSAN, VROPS or NSX, I highly recommend to take the full picture into account when doing your sizing. Because when the time comes to write the second check you will be happy to have single socket servers!
If you are thinking “Why not have half as many servers but each with two processors of 22 cores and full of RAM?”. Well, technically you can but I wouldn’t recommend it.
Yes you will maybe save a few $ grands on the price of chassis, raid controllers, motherboards… But what if you lose a host? Where HA would need to restart 30 VMs on a 22 cores host, it will have to restart 60 VMs on a 44 cores one, twice as much sweat during the failover process! In doing so you are reducing your failure domain and that’s something you (as a sane admin) want to avoid. I won’t get into the Scale-out versus Scale-up debate as this is not the point of this post but it’s definitely something to take into account when sizing and avoid getting carried away on crazy configurations.