Standards, We Don't Need no Stinking Standards!

Yes folks, it's the wild west out there and the backup vendors are holding your data hostage.  Let's face it, backup is the only area of IT technology that has proven completely immune to standards, largely because the vendors concerned have not been just  indifferent to the idea, they've been actively hostile, determined to strangle any effort in the cradle. The result is that we have:

• No standards for the backup metadata catalog   
• No APIs for getting information from the backup catalog   
• No standard APIs or CLIs for controlling a backup application   
• No standard format for on-tape or on-disk data formats   
• No standard and an ever changing selection of tape media   
• No standard for scheduling

So "We don't need no stinking standards" sums up the attitude of the backup vendors. With no meaningful standards, you are faced with some unpleasant truths:

• Once you select a backup vendor, changing that selection is both difficult and expensive   
• Integrating IT systems from an acquisition is a nightmare if they chose a different backup vendor   
• Maintaining long-term readability of a tape archives means regularly migrating the data to new media and the latest on-tape format or media that vendor has decided to foist on you   
• There is very little 3rd-party software to help manage backups and report on the success/failure of backup operations, you're stuck with whatever the backup vendor has chosen to produce

The big question is why have the backup vendors been so successful in avoiding standards that would make life easier for their customers? The answer is surprisingly simple, the vendors have decided that standards aren't good for them and in effect operate a Cartel to control the standards process. They know that however distasteful it maybe, you have no choice but to continue buying their products, it's not like you can stop making backups in protest! As a customer you have almost no leverage since the backup companies are all equally guilty of avoiding standards to ensure your continued "loyalty." Of course, the backup vendors say that there is no demand for a standard format and attempts in the past by Novell with System Independent Data Format (SIDF) and Microsoft with Microsoft Tape Format (MTF) have not been popular. But was the lack of popularity the result backup vendors' indifference and FUD, or do consumers really not care about backup standards?

In my opinion, the only way backup standards moving is for 2nd-tier backup vendors to decide they have nothing to gain by preserving the status quo, and band together to create some de-facto standards for their products. Such a move could revolutionize the backup market, companies would have to compete on the performance, scalability and ease of use of their products, rather than relying on proprietary data formats. If it happened, customers would finally have an alternative to the proprietary systems that might just begin to put pressure on the rest of the backup industry. Maybe a first step would be for one the vendors to put their format in the public domain, let people (i.e. the open source community) develop applications that can read/write the format without any royalty or IP issues.

So, I'm interested to hear from you, the consumer, would you be willing to look at an alternative to your current backup application if there was a consortium of backup vendors, ISVs and open source projects creating an interoperable suite of products, or are you happy living on the wild frontier?

Posted by: Nik Simpson

Comments

[Drew]

got it in one. weve largly reverted to rsync/robocopy straight to sata disk for backups. We were spending hundreds of thousands of dollars each year on tape, agents and mainataince. understandably we were just small fry to vendors so we had to look at other solutions. 22 months later we're still spending the same $$$ on backups but far less headaches (can we say archival data!) and far more operational value for money.

[xfer_rdy]

Just saw the article. and your are right about the standards. However, lets clarify a few items. Novell was not the driving force behind SIDF.

I formed the committee that produced the System Independent Data Format (SIDF) later becoming ECMA-208 & ISO/IEC 14863 while at Tallgrass and then later Exabyte. We (I) made the decision to start off with Novell's format because, 1) they were considered an OS vendor not a backup products competitor and 2) all 14 other backup companies did not want to give any competitor an unfair market advantage by using it's media format as a starting point for the standard.

Shortly after the approval of the ISO version of the standard, the market compressed. In many companies, all non-essencial development (SIDF) was put on hold. Additionally, many backup product vendors, wanted to guarantee loyalty from their customers channels and only supported SIDF as an interchange format.

At the same time, the IEEE's mass storage group was coming out with a set of APIs for secondary and tertiary storage. However, the APIs and the IEEE's architecture were riddled with patented technologies which became a unsolvable obstacle for adoption.

Additionally at the time, most data centers didn't care about data or media interchange, interoperability or standards. Or should I say, they didn't want to pay for it. Once the backup industry started consolidating, all we heard from data center managers was the crying over lost data. But, when interchange and conversion companies sprung up to quiet the cries, almost none of the data centers spent the money to convert the media formats. I guess they figured out the data would have obsoleted by the time they needed to use the old backup media. Data centers only converted media formats when needed. You can't blame them.

That's how we got here and why we don't have deployed standards in this segment of the IT product space.