Saving DataSets Locally With Compression
So you have an ADO.NET application that saves its data locally. Maybe you're caching, maybe you never upload the data to a server and it's a client-side-only application. I fall into the latter category. How do you do it? Well, the easiest approach is to save the DataSet object to disk using its WriteXml method:
ds.ReadXml(fileName); // do some stuff ds.WriteXml(fileName);
That's just dandy except for the bloat you get with XML - what should be a fairly small file turns out to be pretty big once you add all those plain-text tags in there.
So my solution was to compress the XML before I saved it to disk. Check it out:
public void ReadFile(string fileName) { using (FileStream fs = new FileStream(fileName, FileMode.Open)) { Stream s; if (Path.GetExtension(fileName) == ".cmx") { s = new GZipStream(fs, CompressionMode.Decompress); } else if (Path.GetExtension(fileName) == ".cmz") { s = new DeflateStream(fs, CompressionMode.Decompress); } else { s = fs; } ReadXml(s); s.Close(); } } public void WriteFile(string fileName) { using (FileStream fs = new FileStream(fileName, FileMode.Create)) { Stream s; if (Path.GetExtension(fileName) == ".cmx") { s = new GZipStream(fs, CompressionMode.Compress); } else if (Path.GetExtension(fileName) == ".cmz") { s = new DeflateStream(fs, CompressionMode.Compress); } else { s = fs; } WriteXml(s); s.Close(); } }
So the code above compresses the file in two different ways, depending on the extension passed in the filename. It was a good way for me to test which worked better (GZip in my tests, by the way).
These methods are actually methods of my own strongly-typed DataSet, which is why I'm able to call ReadXml and WriteXml directly. There's no reason you can't adapt them so that they accept a DataSet as a parameter, though!
The new version of Microsoft Office uses a technique similar to this - it saves its files out as a folder structure containing XML files, then compresses them in ZIP format. That might work for you, too.
Comments
# Fregate
5/06/2006 3:02 PM
Mate,
Did you try to pre-load an XML schema to speed up XML Serialization with
Type[] myTypes = new Type[] {Type.GetType("Customer") };
XMLSerializer.FromTypes(myTypes);
on OnApplicationStart in ASP.NET or in Start method for Web Forms?
# mabster
5/06/2006 5:37 PM
I haven't tried that, Fregate. Although ... working with business objects like "Customer" is a different case to simple DataSet serialization like I'm doing in this article, isn't it?