Data Center Strategies

Cisco continues Unified Fabric Push

===Updated 3Feb2009===

Many of you have seen the press announcement this week from Cisco about their next generation Nexus switches.  These products continue Cisco’s push towards their “Data Center 3.0” vision that includes a unified fabric – network and storage traffic over the same physical medium.

Burton Group shares a similar vision about the future data center; we call it the Dynamic Data Center.  A dynamic data center requires un-tethered mobility of resources.  Server virtualization is but one key building block offering agility, with storage and network virtualization being others.  Cisco introduced the unified fabric concept over a year ago, and with this recent announcement, is focusing on supporting the growth of server virtualization within the data center.

Now onto the subject of unified fabric.  A year ago when the unified fabric concept was first pushed on the industry in earnest, I heard from a number of IT server, network, and storage admins that they had no intention of ever running storage and network traffic through a common switch, let alone on a common fabric.  This was their visual way to keep the separation:  “you do your job, I’ll do mine.”   We are now a year later, and the next installments of unified fabric support are now announced with an increased push towards fanning the virtualization wild fire.  So has the attitude changed a year later?

I believe the answer is yes and no.  There are two opposing forces moving IT organizations to rethink the way they do things.  The biggest force being the downturn economy.  Tightening budgets mean postponing upgrades, slowing of data center growth, and holding onto what you have.  The opposing force to this is that often business slow downs are an opportunity to circle the wagons and seek out improved efficiencies - re-architect that which was hastily deployed just to keep ahead of the rapid growth during good times.  Re-thinking IT infrastructure goes beyond technological improvements to include policy and organizational changes.  I had a discussion last year with the IT ops manager of a company that was feeling the stress from the downturn.  They had a project they had attempted twice to accomplish over a nine month period of time.  It failed each time.  Their economic situation forced layoffs and an IT organizational restructure.  They attempted the project again, with fewer staff and succeeded in less than four weeks.  The ops manager indicated that hind-sight was 20x20 in this case.  The reason the project had failed the previous two times was because of their larger pre-reduction silo organization.  The layoff forced a restructure and broke down the silos and they were now successful.

The tug of war between these two forces: keep what you have to weather the storm versus opportunity to re-architect for a more agile future tend to play out differently organization by organization.  Fierce competition that accompanies a downturn economy tends to swing the strong towards preparing for future growth.  Then there are the few that had the foresight to implement change for agility during the good times - they are well positioned now.

I bring up the opposing forces of a downturn economy because I’ve seen them weathering a change on IT’s attitude towards many concepts including unified fabrics.  The arguments against a unified fabric have historically been grounded in the comfort of status quo and supported by organizational silos.  The need for increased IT agility that comes from a dynamic data center coupled with organizational changes has broken down the barriers of status quo.  The industry is now more prepared for the move to agile computing.

I’ve noticed a few organizations just starting to warm up to the ideas of a unified fabric.  Have they moved there yet?  No. The technologies are not mature enough to complete an entire unified fabric infrastructure.  Cisco knows this and is positioning the Nexus switch capabilities as a future proofing move. I found it interesting that of the three service provider case studies that Cisco used to promote their new products only one mentioned unified fabric as a future possibility.  Rather all focused interest in the product’s increased port density, total bandwidth, and non-stop maintenance.

We’d like to hear your take on unified fabrics.  Is it something you are considering for the future? If so, what intrigues you and if not, what are the problems you see?

===Update=== 

Cisco is openly wanting to break down the IT silos.  Burton Group analyst, Eric Siegel updated his blog with a discussion on this topic and link to Cisco's CTO Padmasree Warrior's blog on this subject.  Go check it out.

[Posted by: Richard Jones]

SSDs based subsystems are getting interesting...proceed carefully

Two interesting things happened in the computer storage biz on Monday: Sun announced their new "Amber road" storage products and Violin Memory, a small startup, introduced a high performance pizza box storage appliance. Both these products are leveraging NAND flash SSD technology. Both in interesting ways.

BAM, new era...sorry couldn't resist.

It would be easy to go all gaga over these products but caution is advised. Both are innovative in their own right, both challenge the status quo of the data center class storage system business. These are innovative products. No matter what my comments to follow are, keep this in mind: these products are integrating new technologies in new ways and therefore, regardless of the technical prowess of the suppliers, caution should be exercised before inserting these products into a data center on a large scale.

Try'em to verify'em.

In my past blogs about SSDs, I've mentioned that flash SSDs are about transactional performance not capacity.  I stated that capacity will have to wait until flash memory density increases and pricing decreases dramatically - we will be waiting for awhile. So, I went on to suggest that a storage system based on SSDs, to be well rounded, must offer high performance but solve any capacity shortfall by including high capacity SATA disks. To make that workable, some magic is needed to move data around appropriately to match the data's dynamics.

That's what the new Sun storage product claims to do.

Sun, by leveraging their open source ZFS file system, with some crucial tweaks, marries SSD technology for performance and SATA hard disk technology for capacity. Compared to an all HDD system, Sun claims up to 3x read transactional performance, 5x less power, at about the same cost.

Not bad.

Turns out Sun uses SSDs tailored both for write performance to handle an internal logging function and then read optimized SSD to act essentially as a really large cache. All wrapped around their open-source ZFS file system hidden within the storage subsystem. Sun is demonstrating technology leadership by tightly integrating SSD technology into a complete storage subsystem - yes others have done a pluggable replacement for a HDD, but that's fairly obvious and less then optimal. And there have also been some demonstrations such as IBM's Quicksilver science experiment based on Fusion-io, but no significant shipping product.

There are certainly other nice features in the Sun "Amber Road" products such as an improved management interface, analytics and such but blah, blah, blah. The industry expects those things, they are the price of entry and are the nuts and bolts of any system. Of course you need good pricing, support, quality and the like - that's all good, but its expected. If a vendor does not live up to the expectation - they get voted off the island - fast.

Sun has a window of opportunity here to make some market share gains, before other major vendors show up to the party. Its on you Sun. Price the product to move, forget premium pricing. Get support right, and accounts will be won. Don't forget: others will arrive at the SSD party soon as well. Grab market share while you can. Show customers how Sun is setting a new price-performance bar only reachable with SSD technology. Game on. Watch your back, I expect strong competition in the SSD space over time, the vendors who don't react are no worry since they will not survive in the long run.

Back to Violin. Violin introduced a 2u (that's about 2 pizza boxes) flash based storage appliance. While this is not an holistic performance and capacity solution, it does offer an excellent example of what happens when you take the "D" out of SSD. No longer constrained by the HDD form factor, the Violin 1010 uses memory card like flash modules that plug into a motherboard. As part of the design, there is a RAID like redundancy for the flash modules with hot plug support in case one fails - but I gotta admit, I wouldn't touch the thing while its running. Find an intern to do it and if things go wrong make the intern the scape goat.

For those running transactional applications, like a database, this product could be a godsend. But I fear the pricing may be too high and you do need to accept working with a newbie in the space. The Violin 1010 attaches directly to PCIe (it also does FC and Ethernet) . PCIe is needed to achieve high iops - fibre channel and scsi in general add too much latency but that's the subject of a future blog.

The Violin product is interesting on its own but consider: Combine the Violin 1010 with a bunch of 1TB SATA hard disks and Sun's open storage software with ZFS. Add in a few tweaks for auto-tiering, do some soul searching about pricing and you could have an awesome product. 

So where is all this SSD technology going? Into data centers, sooner than anyone has imagined.

Fall SNW 2008 Trip Report

get ready for some rambling - lots to cover....

For those unfamiliar, Storage Networking World is a Computerworld magazine and SNIA sponsored event. Held twice a year in the US. The largest storage-only event. This is a vendor love fest, designed to bring end-users and vendors together to talk things over, get educated and generally develop a sense of the storage market for businesses. Definitely not for consumers.

As an analyst, at SNW I spend my time meeting with vendors, to hear their latest, discuss hot topics and encourage them to address customer needs. It's also a great opportunity to talk with end-users to discuss what's of interest to them.

Prior to attending SNW, I recently posted blogs on SNIA's SMI-S and SSDs and was interested to hear views on FCOE as was my Research Director Drue Reeves.

Frankly I was unprepared for the intense reaction I got regarding my SMI-S blog - I could hardly walk around without someone bending my ear about it - but more on that later.

Here's a quick drive by of what I learned:

FCOE:

This turned out to be an intense topic - I was not expecting the passion on this subject. In a meeting with Cisco, it was clear that they are all over it and pushing it hard.

We at Burton Group have been somewhat tepid on the FCOE subject. Yes, FCOE helps bridge the gap between an FC and Ethernet topology, but iSCSI offers a better price point for new installs. There is a place and value for both technologies.

Intel (they've got a 10Gig adapter) expressed an agnostic view on the subject, supporting both FCOE and iSCSI. Intel is definitely in the "let's see what develops" camp providing product support either way. Of course Emulex is all over the FC/FCOE market with their adapters. Netapp, bravely leading the market from the FCOE target side and taking a "let's see what happens view", will be delivering an FCOE target - nothing new here just reiteration of past activity.

Mostly I heard - "let's see how it goes". For the risk adverse, its still early for FCOE. Expect to see credible full implementations the second half 2009.

So we'll see how FCOE adoption goes - the market will decide. To hear more of the Burton Group view, jump on a plane to Prague and attend the Catalyst event starting this week.  Good luck with the airplane ticket prices!

SMI-s:

Based on my SMI-S blog, at times I thought I might need a security detail to escort me around the convention ;-} I had barely left the registration desk before being hit up for a discussion. My comments are meant to be constructive. There are some inconvenient  truths to be dealt with here. After meeting with the leadership of the SNIA board, it's clear that it's time for SNIA folks to huddle and develop a SMI-s game plan. A restatement of the goals for SMI-s would be a good first step. SNIA is teeming with smart people. Best wishes.

Storage Products:

Storage is getting so complicated - you'd think that it would be getting simpler. So many vendors, so many product variants. The product overlap is overwhelming.  There are lots of great storage products out there, it must make customers' heads explode...hmmm bad image...

Microsoft told about where they're taking DPM. They said that ...opps can't talk about it but take a look at this blog and look for significant enhancements. And Sun let us know about their plans for ...opps can't talk about it but look for leveraging of the Sun open storage initiative and the application of SSDs...IBM ran through their SAN volume controller for mid-range businesses - a full featured yet less performing version of their enterprise product. HDS has a new mid-range truly active-active 3Gb/s SAS disk array. Xiotech with their over-the-top marketing sung themselves praises. Hats off to Xiotech for having an actual customer in the briefing. Riverbed is making noise about getting into the storage biz with an appliance that does inline dedup, consolidation and WAN acceleration - I'm not sure how to categorize it. Bluearc has a nice story for performance NAS using TMS's SSDs as a high performance tier. And finally F5-Acopia  describe their existing network based virtualization appliance to me.

All good stuff, too much to cover in this blog.

Drum roll please..*************BAM

SSD's

My favorite subject. You may recall in my SSD related blog I said, "we need storage subsystem providers to start shipping product with SSDs in them!" Products are starting to arrive. Yes, they are somewhat brutish and much optimization lies ahead, but the game is on.

First, Intel announced availability of their enterprise SSD. Awesome performance. Small capacity. But a great start. Intel continues to legitimize the SSD enterprise offerings. Yes, kudo's to the other suppliers out there. Request to Intel: please help to standardize the spec'ing of SSDs. Performance, power, wear, bit error rates and failure rates need to be addressed. Intel, you know what to do.

The vendors offering SSD's in their products increased dramatically. Joining the existing SSD gang, EMC, Fusion-IO, TMS are vendors that either announced product or intent: Compellent, Verari, Sun, Wasabi and Rackable. More are coming - I just can't spill the beans just yet. If I missed someone, please comment on the blog - the list grows every week. Lots of rumors out there. IBM's showing a SVC-based million IOP beast using Fusion-IO under the covers, expect this to be productized as well.

Of course, HP now ships blade servers, not to mention laptops, with SSDs. How long will it be before SSD chip sets end up on a server motherboard?

For Burton Group clients, watch for my upcoming in depth research document on SSD's.

I declare the SSD games open!

But I would call these Novelty products - designed for extreme IOPs, not so much a blended IOP-capacity capability and, unfortunately, with premium pricing. Close but not yet where the market needs to go.

Here's what the market will really love: a blended system with SSDs for performance and terabyte SATA disks for capacity. To make this work, auto-tiering will be needed under the covers, transparent to users. Ideally, this product would allow policy-based data movement leveraging usage patterns and storage costs. Compellent is uniquely positioned to do this, but I fear their infrastructure is not yet optimized for SSDs. SUN's got some interesting ideas using ZFS. And, Wasabi, a small but innovative vendor, is combining SSDs with an object-based file system, which theoretically can easily identify and move data objects across performance tiers.

And to drive down SSD subsystem costs, the "D" must be removed from SSD and replaced with "PM" for persistent memory. Repackaging SSPM's within a storage subsystem will greatly reduce costs and allow for performance optimization, not unlike what Fusion-IO and Violin already conceive.

Exciting times. But be patient - this may take years to unfold...development cycles can be long...and, of course, whatever we end up with will likely be complex beyond comprehension.

Perhaps the geniuses who dreamed up credit default swaps can help out ;-}

Let me know what you thought of SNW by commenting on this blog. Thanks!

Posted by Gene Ruth

Has SNIA’s SMI-S failed?

“We’re not enamored of truth. It is too often painful, discouraging and it tends to undermine our self-image. We prefer comfort. Reassurance. Well being. Good Cheer. Naked optimism. Nobody wants to hear the facts when they clash with a happily imagined reality. It is after all, a terrible thing to be the only person in town who can see what’s really happening" [from “Odyssey” by Jack mcDevitt].

To IT data center storage vendors: Are you fooling yourselves about SMI-S? Is SMI-S relevant anymore? Is anybody really implementing it? And where is that mystical standards-based storage management console? All difficult questions and, I fear, with unpleasant answers. It’s been over 5 years since SMI-S was first kicked off. The truth must be faced. I’m not seeing truly complete standards-based storage management implementations. I mean implementations that completely manage a storage system or device within the confines of SMI-S, even assuming a few vendor unique extensions – which, btw, are allowed by the spec.

At this point, you may be scratching your head and wondering what SMI-S stands for. If you don’t know, stop reading this blog unless you relish the anticipation of a possible train wreck. For those who do, you may be wondering about my sanity or nodding your head in agreement.

Let us digress. For some education, go to this link and take a look at the SMI-S document. I'll wait. It might be awhile - the spec is over 1000 pages.

SMI-S is very complicated with a lot of loose ends but it CAN BE MADE to work.

The promise of SMI-S is simple: By getting storage vendors to instrument their gear in a standard way, theoretically, a single management infrastructure with a single console could manage multi-vendor storage environments in a common way. Sounds great doesn’t it? We all dream about this interoperability.

Trouble is, no-one-bothers in a serious fashion.

Sure, some of the top tier players - two, three, and four letter names – do implement some of SMI-S in their products. But it’s the “some” part that’s the rub. I frequently tele-conference with enterprise class storage subsystem vendors– large and small – to hear their latest, and I usually ask this question: Do you implement or plan to implement SMI-S? If so, how much and when? …Far and away, there’s a pause, I hear some shuffling and throats clearing, and get one of two answers: “we are watching that space and will provide an implementation when our customers ask for it”. In other words: forget about it.

Or I get told: “Yes, we do!”

And I ask: “Completely? Does that mean I don’t need your management application to configure and monitor your gear?”

And they respond: “Well, ah, no…”, again paper shuffling and whispering….

Back in June, Burton Group held its annual Catalyst Conference. During the conference, Drue Reeves (VP & Research Director) challenged the industry with his “count-down to interoperability”. He asked, “when will truly interoperable, standards-based storage management applications and instrumented systems be available?” He gave a one-year time frame – I fear he will be disappointed.

So has SMI-S failed? Will vendors seriously implement it? Vendors please let me know. Tell me you are nearly there. After all, as a matter of confession, I and others at Burton Group contributed to the spec and know many very smart people who have worked long and hard on it.

I want to see it succeed.

But what do I tell my clients asking for a unified storage management environment?

If SMI-S has failed then what went wrong? Did vendors realize or anticipate that cooperating to make their equipment interoperable - at a management level - hurt revenues? Did storage systems get so complex and feature rich, that the modeling approach taken is too detailed? Or, is it simply that SMI-S is too complicated and vendors don’t see the payback for investing precious development dollars on it?

I suspect it’s all of the above.

Perhaps a new approach is needed.

Enterprise class storage subsystems have become feature rich over the past few years – what top tier vendor doesn’t do thin provisioning, tiering, replication, etc. All these features are uniquely implemented and a cause for vendor differentiation. The real value of these new features is the delivery of a Quality of Service (QOS) such as data protection, scalability in the face of data growth, performance, and the like. Unfortunately, SMI-S focuses on the down and dirty modeling of the internal specifics of these features – the place where vendors innovate and add their secret sauces.

I’d suggest a new approach: model storage management based on QOS or, more simply, service delivery. For example, rather than modeling the intricacies of thin provisioning or RAID sets, instead, through standard api’s, identify that the equipment supports specific features characterized by key standardized parameters associated with each feature. Why should users specify what disks go into a RAID set? Instead, let them define the level of redundancy and expected performance, letting the equipment figure it out from there. Some vendors already do this in a proprietary way – it's not a new idea. This can be conveniently modeled with XML.

So, back to the main question: Has SMI-S failed? What do you think? Is there hope or should the storage industry rethink how to achieve interoperability in IT data centers or, more to the point, do vendors want to do it to begin with? Where does the SNIA board stand on this issue? After all, the members represent the vendors whom this blog is addressed to.

I know one truth for sure: IT data center customers want storage management interoperability and they want it now.

Posted by Gene Ruth

The Good, the Bad, and the Ugly - Part 3

So, I've described the good and the bad parts of my friend's experience, what about the ugly?

Let's go back to morning after the license timeout fiasco that kept my friend up all night rebuilding systems. Once everything was back up and running around 5am in the morning, he composed a short and very professional email to the integrator expressing his understandable frustration. I've seen the email, and I can only say I would have not shown the same restraint in the circumstances. After sending the email he went to bed to catch a few hours of badly needed sleep before going back to work. At 7am, the integrator took time out at the start of his busy day, to call my friend on the phone and proceeded to swear at him, claiming that the problems were all my friends fault and while questioning his ancestry, mental health, mating habits, and intelligence amongst other things!

So what lessons should we learn from this, first the IT lessons...

• When working with small software or hardware companies who insist on using integrators for the install, don't take their integrator recommendations at face value, insist on references. 
• If at all possible, don't put yourself in the position of putting new solutions directly into production.

For software vendors...

• The best product can be really hurt by a clueless integrator. 
• Integrators who show this level of incompetence should be shown the door immediately. 
• When an installation starts to go wrong in this way, prompt action by both support and sales organizations are critical. To be fair, my friend only had good words for the software company's support organization which was responsive and knowledgeable throughout. Sadly the same cannot be said of the salesperson who managed the account.

But the real lesson that everybody needs to take to heart is that for many of the more innovative solutions on the market, it's still the wild west out there for integrators, and it's not the Indians we need to worry about, it's the cowboys!

Posted by: Nik Simpson

The Good, the Bad, and the Ugly - Part 2

It's now time to talk about the bad part of my friend's experience building a software iSCSI SAN...

Many software vendors in this space are wary of customers doing their own installs, and insist on bringing in an integrator with experience of the product to perform the installation. That's a fine plan in theory, but is totally reliant on the abilities of the integrator, which is where the plan frequently falls apart. Small software vendors tend to get stuck with small integrators who are basically "two men and a dog" outfits, and frequently the dog is the brains of the organization!

Anyway, enough preamble, on to the gory details of the implementation. The consultant (recommended as one of the best by the vendor) came in with a plan to use dual GigE NICs in an adapter teaming configuration to provide load balancing and fail-over. Unfortunately adapter teaming requires switches that support the Link Aggregation Control Protocol (LACP) which meant buying a new GigE switch at around $5K on top of the original price, in addition to upgrading firmware on the servers. Implementing adapter teaming also required the entire VLAN architecture of the data center to be redesigned with addition of two new two VLANs (one for iSCSI traffic and one for everything else) which is where things started to go wrong. The Broadcom based GigE interfaces used in Dell servers assign the same MAC address to each VLAN created on a particular NIC. Use of a single MAC address causes problems with iSCSI because iSCSI requires the MAC addresses to be unique otherwise it can't work out which VLAN to bind to. The integrator insisted that they could work around this problem by creating the iSCSI VLAN, configuring iSCSI, and then creating the second VLAN. They insisted that this approach had worked for them before. Unfortunately it relies on the system creating the VLANs in the same order every time it boots, and at least for Windows Server 2003 this is not the case. After some days spent trouble shooting the problems this caused, a support call to the software vendor revealed that this was an unsupported configuration for the network.  In the end it was decided to use the NICs in a simple fail-over pair with no load balancing or link aggregation which worked correctly.

Strike #1 for the integrator - they'd recommended a network architecture that would never work, and cost my friend $5K extra for a switch, plus several days redesigning VLANs and troubleshooting iSCSI problems, none of which would have been necessary had the integrator known what they were doing.

The next hurdle related to the software licensing, the system was initially installed with 30-day evaluation licenses. Four days before the licenses were set to expire (a Monday) my friend sent emails to the vendor and integrator asking whether the licenses would expire at 12pm Thursday or 12pm Friday. He got no response, so he emailed them again on the Tuesday, and on the Wednesday, and on the Thursday! He finally got a response from the integrator on the Thursday who assured him that the license would expire on Friday night, and that they would install the full licenses during business hours on the Friday. I'm sure you can guess what happened next, around 12pm on the Thursday the mail system went down because the licenses for the iSCSI SAN expired and the iSCSI volumes disappeared! My friend spent the Thursday night rebuilding everything, while entertaining himself with thoughts of boiling oil, integrators, and excruciating agony.

Strike #2 for the integrator - they didn't check the correct expiration date for the licenses, causing significant downtime for mail servers.

The final job for the integrator was to move the mail store volume over to use thin-provisioning. The plan was to break the original mirror, associate one half of that original mirror with a thin-provisioned volume, synchronize the new mirror, then repeat the process for the other half of the mirror. Not rocket science you'd think, but the integrator fouled it up with consummate skill. Instead of breaking the mirror as my friend suggested, he unmapped the entire mirrored pair from the application server, with entirely predictable results and more downtime for the mail system.

Strike #3 for the integrator - they clearly didn't understand how the software worked, despite being strongly recommended by the software vendor!

 

Posted by: Nik Simpson