Its been nearly six years since Microsoft released Windows Server 2003 and its accompanying Volume Shadow Copy Service (VSS). Most people think of VSS as Microsoft’s storage volume snapshot. Yes it includes snapshot, but it’s much more than just snapshot, it is a snapshot framework that includes a rich set of APIs to allow third-party software integration. This enables storage array vendors such as EMC or Network Appliance the ability to integrate their array based snapshot solutions with VSS. It also enables application vendors the ability to construct their applications to participate in the snapshot process so that the volume snapshot of that application’s data is a consistent image. Here’s an architectural diagram of VSS (courtesy Microsoft):
The framework of VSS is the most important aspect of this feature. It forms the basis for enabling data protection using server-less backup. In the architectural diagram, applications are the “Writers” and server-less backup software or snapshot management consoles are the “Requestor.” Microsoft includes providers for its storage subsystem and storage vendors can plug-in their own providers into the framework.
The nice thing about VSS is that applications, especially databases, are given a signal to make their data in storage consistent. They can flush buffers and transactions as if they were cleanly shutting down and then signal back that they are ready. The snapshot then gets a good looking image of the application’s data, one that the application knows it can easily access without requiring lengthy repairs or playing re-do logs.
So now let’s go back to the beginning of my blog. It’s been six years since Microsoft released this feature. Storage demands have continued growing and don’t look to stop anytime soon. The number of server-less backup products available for Windows Server continues to grow and enterprises have been deploying these products over the past couple of years to shorten the dreaded “backup window.”
But where’s Linux?
Linux distributions don’t have any equivalent feature. Yes, modern Linux distributions include the dm-snapshot module, but that just does the snapshot, there is no framework for application developers, storage vendors, or backup vendors to integrate with. The result has been a few one-off solutions focused on a particular application stack, but for the most part applications just suffer from inconsistent snapshots on Linux. These are what we call crash consistent, meaning that the application is responsible for figuring out the state of its data within the snapshot and getting it to a position it can begin running. Most databases can take upwards of twenty minutes to mount a crash consistent snapshot.
So this is a call to the Linux community: Please band together and architect a snapshot framework for all Linux application developers, storage developers, and users to benefit from.
[Posted by: Richard Jones]