HP has a BIG baby

As part of HP's Monday announcements, they introduced a new server (ProLiant DL785 G5) which marks HP's reentrance into the 8-way x86-server market after pulling out in 2005. The specs for this new machine are pretty impressive...

• 8 x AMD Opteron "Barcelona" 4-core processors
• 64 (yes, you did read that correctly) DIMM sockets for memory, that's 256 GB of memory with 4 GB DIMMs, and 512 GB at some point in the not too distant future.
• 11 PCIe expansion slots (includes two 16x slots that can also be used for HTX accelerator cards in number crunching applications). Many of the slots are also PCIe rev 2.0 compliant which doubles the bandwidth.
• Up to 16 x 2.5" SAS disk drives (8-bays are standard, with 8 more as an option).

Impressive as these hardware specs are, the more interesting question is "why is HP getting back into a market that it previously abandoned?" That question itself has a number facets:

• Why leave in the first place? - 8-way x86 server design became a reality in the late 90s with Corollary's ProFusion chipset (originally designed for Pentium Pro, but ultimately shipped with XEON). There were other contenders at the time, Hyundai was one that I recall, and COMPAQ designed their own.  But there were two problems, first the designs didn't scale well because the shared memory architecture which meant that the 8-way boxes never really performed as advertised. The second was a market problem, there simply weren't many applications on Intel-based hardware that needed that sort of scalability which limited the size of the market and prevented commoditization. Put simply, it was a solution looking for a problem.
• What's changed? - First, AMD's Opteron design doesn't have a shared-memory bus, each processor has dedicated memory and I/O channels which should help a lot with scalability. Second, and perhaps more important from HP's perspective is that the market has changed. Now, with server virtualization, you can easily run multiple workloads on the box, so the issue of finding a single application that can do something useful an 8-way has changed dramatically. It's not one workload it's tens of virtual machines sharing the box.
• But aren't blades the future? - HP has spent a good deal of time telling us how wonderful blade-based servers are, so why come out with a system that is the very antithesis of blade style computing? The truth is, blades work well for some things, but if you are looking for really high consolidation rates as you virtualize workloads, you really can't beat a "honking big box" .

So HP rejoins a market that currently features Sun and IBM. Sun's 8-way is rather similar in basic architecture (i.e. Opteron-based) while IBM continues to push it Intel-based NUMA cluster (multiple 4-way servers with a high speed interconnect). It will be interesting to see if HP regains the market leadership it had before exiting the 8-way market?  I expect things to only get more competitive over time, particularly when Intel's Nehalem processor is available to support 8-way designs which I would expect by 2010 at the latest. My guess is that by 2011-2012 timeframe we have 8-way/8-core designs with hyperthreading for 128 processing threads which will be quite a beast! What a box of this caliber means for HP's Itanium systems, not too mention Sun's uSPARC and IBM's Power processors, is an interesting question that only time will answer.

 

Posted by: Nik Simpson

Comments

[Ed]

With all those PCIe slots, I would bet HP has way over committed bandwidth.

The HT to PCIe bridge chips offer 8GB/sec, I believe there are two, for a 16GB/sec total. I count HP to have 46GB
of theoretical PCIe BW, but could only feed about 16GB/sec.

HP is 7RU, Sun's x4600 is 4RU. IBM's is a 3RU/4 socket brick, that grows via an external expansion cable to another 3RU brick.

[Nik Simpson]

I just got see one of these systems up close and personal. The bandwidth for the PCIe slots is not oversubcribed. The system has four I/O hubs (i.e. twice the number you'd see in a 4-way system).

That said, HP's marketing somewhat oversold the system when they originally described it, the only PXIe2.0 element is the voltage supplied to the 16x slots. All the slots are pCIe 1.0 from a bandwidth perspective. So for the record, the DL785 has 3 16x slots, 3 8x slots and 5 4x slots.