Is the Emperor Under Dressed?

Today, Dell became the third x86 server vendor to deliver a rack mount servers specifically optimized for server virtualization workloads. To understand the difference between a standard server lets look compare the new PowerEdge 805 with the PowerEdge 2950 III, both support two processors with up to four cores each and occupy the same amount of space in a rack. What separates them is the memory and I/O capacity:

	PowerEdge 805	PowerEdge 2950 III
Memory slots	16	8
Maximum memory	128 GB	32 GB (Dell doesn't offer 8GB DIMMs for the 2950 III)
PCIe 8x	3 slots	2 slots
PCIe 4x	1 slot	1 slots
Onboard Ethernet I/F	4 x 1Gbit/s	2 x 1Gbit/s

So that's four times the memory, 40% more I/O bandwidth, and double the number of networking interfaces in the same rack footprint. What I find particularly interesting about these "optimized" servers is what they say about blade servers as a virtualization platform, consider the PE 805 compared to Dell's two processor blade:

	PowerEdge 805	PowerEdge M605
Memory slots	16	8
Maximum memory	128 GB	32 GB (Dell doesn't offer 8GB DIMMs for the 2950 III)
PCIe 8x	3 slots	2 slots
PCIe 4x	1 slot
Onboard Ethernet I/F	4 x 1Gbit/s	2 x 1Gbit/s

The PE 805 still has the memory advantage, but the I/O throughput advantage is even more marked at a whopping 75%. So much for blades as a server virtualization platform!

If you listen to some of the x86 server vendors you could be forgiven for thinking that anybody that doesn't implement blades for their entire x86 server infrastructure, is not only doing them self a disservice, but also endangering the entire planet through their profligate use of energy. If you talk to customers though, you get a very different story; many of them have serious concerns including:

• The density myth: achieving anything close the theoretical density of blade servers is nigh on impossible given limits of power distribution and cooling in most data centers. Hands up all the data center managers who are delivering 20-30 Kwatts of power to each rack? I had one customer using a co-lo that was talking about energy densities of 1000w/ square foot. Given the amount of space around a rack that translates to about 9Kw of power available for each rack. 9Kw is about enough power for one or two (if you're really lucky) fully populated blade chassis, in a rack that can physically hold four or five blade chassis.
• The consolidation myth: I've talked to several customers about data center consolidation, and blades always come up as a topic of conversation. But consider this; if the plan is to move workloads from conventional servers to an equivalent number of blades, then very little consolidation is actually achieved. Yes, the servers may have been geographically consolidated into a single data center, but any gains in power and cooling efficiency will be negligible. If the goal of consolidation is to centralize IT and to reduce the overall energy consumption, then server virtualization represents a much more attractive path. Consider the following example where twenty conventional servers are to be consolidated. Using just blades we go from twenty poorly utilized servers in remote locations to twenty poorly utilized blades in a central location, big deal! Throw server virtualization into the mix, where running ten virtual machines on a single server is very doable, then we end up with three efficiently utilized servers (assuming an N+1 failover cluster) in the central location. Clearly the latter approach will be much more efficient from an energy perspective, will consume less space in the data center, and will reduce the number of physical hosts being managed from twenty to three.

But surely blades must be good for something, other wise why have them? Let's consider three typical use cases:

• Server virtualization: Blades are fundamentally unsuited to large scale server virtualization projects because of limits in terms of memory and I/O scalability which place artificial constraints on how many VMs can be consolidated onto a single host. Even the vendors who are most forthright in their adoration for blades have tacitly admitted this by launching conventional servers optimized for server virtualization applications. The optimizations they make are telling. First, these new servers support very large memory footprints, especially compared to blades. Second, these optimized servers sport considerably more I/O bandwidth than blades or less well endowed rack mount servers. Examples of such server virtualization optimized platforms include the HP ProLiant DL785 and Dell’s recently announced PowerEdge R805 and PowerEdge R905 servers.
• High performance compute: At first glance, HPC seems like the "killer app" for blades. But if you dig in a little more it's not so clear. For example Dell was proud to announce a new HPC lab at Purdue University, but it was built from standard rack mount servers not blades . Similarly the Texas Advanced Computing Center has two Dell HPC configurations built from PowerEdge 1955 servers. Granted, the largest system at TACC is built using Sun's 4-core Opteron blade, but that barely qualifies as a blade system with respect to density since the exact same density could be achieved without blades.
• A future in the clouds: I hear a lot of talk about how blades are natural fit for cloud computing because of their density. Hmm, if so, then why isn't Google using blades in its data centers? Equally to the point, why did IBM just announce the iDataPlex (what genius came up with that name?) system specifically for this market. iDataPlex is built with half depth 1U rack mount servers with a nominal density of 80 servers in a rack along with water cooling and a small nuclear reactor (both of which are optional).

The truth is, blades are not the right choice for most environments, and customers should be very wary of any vendor or reseller that tells them anything different. If you don't mind a proprietary product that costs more to acquire and locks you into a single vendor while simultaneously limiting scalability and I/O throughput, then blades are just what you need, otherwise think long and hard before deploying them.

 

Posted by: Nik Simpson

Comments

[blizxt]

Not all blades are limited in the IO category. IBM blades support up to 6 Ethernet NICs and 2 Fiber Channel HBA's for at total of 8 IO ports. The blades are still limited in the memory bandwidth category with the maximum memory on an IBM HS21-XM blade being 32GB today. There are 16GB (2x8GB) memory options available for the x3850M2 and x3950M2 rack mount servers which could eventualy trickle down to IBM's blade servers increasing this to 64GB, but that is pure speculation. The point I am trying to make is that not all blades are as IO limited. Also, do not discount the ability to run 14 quad core cpu's in 9U of rack space with consolidated power, fiber, and ethernet connectivity. While I agree blades are not a fit for everything they do have value for virtualization as well as consolidation.

[Nik Simpson]

Thanks for the comment, but I stand by my contention. The IBM blade you describe is little better (or worse) than any similar blade system. It is however substantially worse than what can be achieved with conventional rack mount servers.

I also don't discount the value of density, however if you can't fully utilize that density because of insufficient I/O and memory expandability, then density becomes a red herring.