« ASHRAE says turn it up | Main | Ticket to Ride (REDUX) »

February 02, 2009


Han Solo

See: LVM Snapshots...and have been around since forever. VSS is just a snapshot manager, and linux already has that facility with having a real volume manager built in.

And Linux had this ability about 4 years before VSS in Win2003.

Not sure what else your waiting for. BTRFS maybe which is now in Linux?

Richard Jones

As I mention at the bottom end of the blog, Linux has had the dm-snapshot capabilities and ability to manage snapshots and restoring from snapshots. However, it lacks any framework for application integration to signal apps to "freeze" their data for a snapshot and then to "thaw" after the snapshot has occurred.

A discussion with RedHat engineers on this subject prompted this blog. We are not aware of any such frameworks for Linux in the open source. Veritas has their closed source Application Quiescence product that offers such a framework, but that's it.

Robert Wipfel

Hey Richard, you remember we wrote up a list of requirements for Data Center Linux (DCL), and published them by working with the Open Source Development Lab (OSDL). We included the need for a Freeze/Thaw framework (like NetWare added due to SAN integration experience). The Linux kernel community, in keeping with the philosophy of mechanism not policy, has now added a freeze/thaw API in 2.6.29 [1]. With a frozen file system, one can then snapshot the backing storage; as the previous poster noted either using Linux host based dm suspend/resume and snapshot etc [2], or the external SAN's snapshot feature. This all has to be coordinated of course, with open APIs that allow integration with 3rd party components in the whole stack. That is the policy part of an overall solution, that builds on the mechanisms present in the Linux kernel. I guess your point is that Linux should provide that part of the solution too. With respect to API support, you might be interested to know that Novell's SLES11 provides some of the SMI host profiles that are necessary to Orchestrate an end to end solution: e.g. including freezing the higher layer applications running in the same address space as the host OS, or in their own address space, is a distributed management problem that requires coordination of multiple layers including the Linux kernel (again, either in the hosting or hosted OS when running apps in VMs). So one might argue that the Linux kernel APIs are present, what's actually missing is policy based Orchestration of the entire storage stack, driven by knowledge of the distributed components in that stack. Some vendors are working on this [3] ;-)


[1] http://www.kernel.org/pub/linux/kernel/v2.6/testing/ChangeLog-2.6.29-rc1
[2] http://linux.die.net/man/8/dmsetup
[3] http://debaer.org/blog/wp-content/uploads/2008/02/osm.pdf

Richard Jones

Thanks Robert! Looks like progress is being made. A framework to orchestrate the entire stack is what I'm referring to. Sounds like any distribution built on 2.6.29 or later will have the foundation to build the framework for the rest of the stack. In my post, I was outlining that this is what Microsoft did with VSS in Win 2003. VSS includes the foundation plus the framework to for whole stack integration, and Microsoft (of course) integrated their own applications and within about 3 years, third party vendors began to integrate their apps. I remember talking to a number of customers that were looking for similar support in Linux back in 2006 and it just wasn't there - they opted down the Windows path just because the third party solution integration was lacking in Linux at the time to enable an orchestrated whole stack snapshot.

Craig Ringer

(I know this is an old entry, but):

This is something where D-BUS can be very useful, as it provides another piece of the puzzle in the form of inter-application and applicationsystem signaling. The trouble is that, like with most things in the Linux-related world, some kind of agreement needs to be reached by interested parties.

All it really takes for app consistency is two d-bus events:

1) "Prepare for snapshot - pause activity, fsync, etc."
2) "Snapshot taken/failed/cancelled, safe to resume"

... though there needs to be some aliveness-checking and time-limiting in place so that a broken application can't cause indefinite outages in other apps by not responding to a snapshot message.

Off the top of my head, products that'd benefit from this include Cyrus IMAPd, PostgreSQL, MySQL, any other database you care to name...

I've recently fallen "in envy of" VSS. Other than the unfortunate naming clash with Visual Source Safe, it's a perfect piece of sysadmin heaven. By contrast on Linux:

- Snapshots require you to use LVM. LVM is badly broken with respect to write barriers, and can trash your data if your app/file system expects write barriers to be honoured.

- Snapshots are block-level not file-system-level, so the file system isn't aware of the snapshot being taken.

- Because snapshots are block-level and not filesystem-aware, the snapshot must track even low-level file system activity like file defragmentation, erasing free space, etc. This means they grow fast and have a higher impact on system I/O load.

- Accessing a snapshot requires mounting it manually to another location and faffing around with the changed paths. There's no way for an app to simply request that it sees a consistent point-in-time view of the FS instead of the current state, as in VSS.

- The file system being mounted has to be able to cope with being mounted read-only in crashed state - it can't do journal replay/recovery etc. LVM doesn't even give the *filesystem* a chance to make its self consistent before the snapshot.

- Snapshots don't age out particularly gracefully. You have to guess how much space you'll need to store changes, as LVM isn't smart enough to just use free space in the VG until it runs out. The snapshot's change tracking space is reserved and can't be used for anything else (even another snapshot) until the snapshot is freed. If you over-estimate you can have fewer snapshots and need to keep more free space around. If you under-estimate your snapshot may die before you finish using it. Argh!

So: even with the ability to pause app activity, there's unfortunately a lot more to be done at the LVM level before anything even half as good as VSS is possible. LVM is more than a bit half-baked.

Richard Jones

Thanks Craig! Excellent comment!

The comments to this entry are closed.

  • Burton Group Free Resources Stay Connected Stay Connected Stay Connected Stay Connected

Blog powered by Typepad