Please ship your next Windows Phone app with GZip: speed requests 50-80%

Posted January 22nd 2012 to C#,Windows Phone

Please ship your next Windows Phone app or app update using this awesome GZipWebClient library by Morten Nielsen (NuGet, blog post, source) – it’s been out there since August and we should all be using this now for HTTP-heavy apps!

The next update of my app is in the works and performance is significantly improved by this library. I used both generation 1 and gen 2 devices to gather a ton of data and am loving the results. Can’t wait to submit the update to the marketplace. I’ve been seeing requests speed up 50%-80%, it’s amazing – but I should note, these are unofficial numbers, I’m just grabbing these from my app that is JSON-heavy.

GZipChart

Some sample data: a large set of averaged requests of a very large JSON request to the Foursquare web services used in my app, 4th & Mayor for Windows Phone 7.5.

These numbers aren’t official or anything like that, just what I was observing using my 3G data connection here in Seattle or home WiFi. Also interesting are the fastest times for the content – I often found that once a data connection was open, I would get as low as 100ms for some GZip requests. Just amazing.

One of the awesome modern web server features that nearly every browser happily supports today is Gzip compression of its output. Text and JSON beautifully compress, often by more than half, resulting in quick content downloads. It’s a simple and awesome way to improve the performance and responsiveness of your applications.

This isn’t a new technology, but unfortunately it isn’t just a default, built-in option for you to use in your Windows Phone apps, and you can expect that competing apps on other platforms (and even on Windows Phone) will be using Gzip to improve the performance. Hopefully we fix this in the future.

Frankly it isn’t a default in most mobile platforms, either, and you need to opt-in. However, some, like iOS, seem to automatically deflate incoming streams as long as you opt-in by setting the content encoding headers; in an iOS app, you just addValue “gzip” forHTTPHeaderField before your request and you’re good from there. Android is similar but you do need to check for gzip in responses and manually new up a deflating stream with GZipInputStream.

HTTP Gzip and Windows Phone apps

So Morten Nielsen created a wrapping WebClient library over 6 months ago and through a few iterations has made it great.

So now it’s super easy to get this perf win. To make this work:

  • Add the library to your project via NuGet [SharpGIS.GZipWebClient]
  • Replace your WebClient code to use GZipWebClient instead – yes, just rename! (GZipWebClient is in Morten’s `SharpGIS` namespace)
  • Test and ship your app!

Now I’m not considering the server-side, but if you’re rolling your own web services, you might need to do some work to make sure that you’ll serve GZip content when the capability is ready on the client. Check with the documentation for your server-side app platform, cloud provider or web server technology.

I can totally see myself adding this to my core libraries for all future projects. Hope this helps. Thanks Morten for providing this to the community! You can follow Morten on Twitter (@dotMorten) or check out his blog (http://www.sharpgis.net/).

(And on the performance side: the smaller payload means networking rocks and is fast; it is worth noting that GZip adds a small amount of CPU load – hopefully in your background thread – but I haven’t detected anything negative from this yet)

23 Comments

  • Niall

    This library sounds great – will check it out.. I have been using a modified Silverlight 2/3 BZip library with one of my apps since oct 2011 (where xml payloads of around 2mb are compressed down to around 50k) – but the decompression time has always been an issue (compressing it server side is no issue of course).

    I really think Microsoft should really bake native API support for gzip into the o/s for wp7 developers – like what we get already with .net on windows (so we can have a blazing fast native/c++ implementation). Seeing as large % of apps need to get data from somewhere online – it doesn’t make sense not to have it.

  • http://www.sharpgis.net/ Morten

    Niall: .net for Windows doesn’t automatically compress your requests either. It does come with the logic but you have to opt in for it there too. And this it not even an obvious property (its not directly off WebClient) so my guess is that 99% of .net apps are inefficient here as well.

  • Niall

    yes – i understand this is the case (when using standard HTTP GZip compression) – however in my case my server is not actually running IIS (it’s a windows service acting as a Web Server) – so I actually am gzip’ing the stream and then encrypting it on server as it’s sensitive data (and then doing the reverse when it hits the client).

    What I want built into WP7 is the entire System.IO.Compression namespace (found in .NET 3.5) – ie. the GZipStream and DeflateStream classes in particular. (the encryption ones would be similarly desirable to have fast native implementations).

  • http://www.facebook.com/cristovao.morgado Cristóvão Morgado

    Even nicer would be this with RestSharp :)

  • Peter Kuhn (Mister Goodcat)

    For me it’s still a pity that this is not built into the framework out of the box. It’s not just the improved performance for response times; at least here in Europe you still often have to pay for mobile connections based on traffic, and even if you have unlimited plans the connection speed is throttled after a certain amount is transferred/month. Which means that network traffic still is an extremely precious and/or costly resource on the phone, and the fact that you often can achieve 80%+ compression rates with gzipped content makes it hard to believe this hasn’t been considered as core feature in Silverlight for the phone even in the Mango iteration.

    I’ve been using similar setups like Morten’s library since the beginning (but based on the low-level request/response classes instead of WebClient), and you should too. If you put the processing on a background thread it doesn’t really hurt, and your users will thank you for both the improved performance and lower traffic amounts.

  • Anonymous

    Definately something everyone doing text based HTTP requests (XML/JSON) should be doing. On of the main reasons for starting to write my WM6 http://peshirtweets.peshir.nl/ Twitter client way back, was to get it to do gzip compression on HTTP traffic. This took a huge chuck out of performance and network traffic.

  • http://www.withinwindows.com Rafael R.

    Hmm. I’m curious — back in the day, modems handled compression for us. Do 3G chips do this as well? If so, what kind of compression is it? And does it null (or worse, negate) the savings from this lib?

  • Niall

    i’d imagine it would be doing some compression where possible – but only at a much smaller packet level. Hence – compression on data in large xml/csv streams would not benefit much from this (as the repeating element/attribute names and values would not be identifiable in smaller chunks without analyzing entire set of data).

  • http://www.withinwindows.com Rafael R.

    Makes sense, but given the state of tech today I’d imagine chipsets are doing more than merely sending packets one at a time. They could be tuned, by the mobile operator, to — perhaps — store and forward when some arbitrary buffer fills, say 4096 bytes in size. (No idea.)

    Don’t get me wrong, this gzip advice is probably sound in most cases but I’m worried about the call to action without the raw data in hand.

  • http://www.pedrolamas.com Pedro Lamas

    I just added a request for this over on RestSharp forum:
    https://groups.google.com/group/restsharp/browse_thread/thread/56cc3ea707926fe2#

  • Peter Kuhn (Mister Goodcat)

    > They could be tuned, by the mobile operator, to — perhaps –
    > store and forward when some arbitrary buffer fills,
    > say 4096 bytes in size.

    In certain situations, things like these are already in place on the transport protocol level. For example in TCP we have the Nagle algorithm that tries to avoid sending out too small chunks of data by doing such a buffering (it’s active by default).

    I personally very much dislike the idea of mobile operators optimizing this, because then I as a developer do not have full control over it anymore. Sometimes it’s highly undesirable to buffer data for optimization, for example in real-time communication or streaming applications. Then I want to be able to turn this off or enforce the behavior I’m looking for.

    I agree that it depends on the data whether gzip actually will result in better results, so you as a developer should decide when it makes sense, or not, and test thoroughly for your scenario. For everything text-based it’s usually a tremendous improvement though, independent of the underlying networking technology.

  • http://www.sharpgis.net/ Morten

    Grab v1.1 on NuGet and add this line of code and it should work with RestSharp too:
    WebRequest.RegisterPrefix(“http://”, SharpGIS.WebRequestCreator.GZip);
    WebRequest.RegisterPrefix(“https://”, SharpGIS.WebRequestCreator.GZip);

    This means you don’t have to touch WebClient at all! (it will automatically start using the SharpGIS.WebRequest/Response classes under the covers). Just call this line of code once, and you’re good to go for the rest of the app’s lifetime.

  • DrAlani

    Some carriers do server-side compression before the data is sent to the phone. But htis is valid for video and images only. The raw data is still being sent as raw data with no actual “data-compression”.

  • http://twitter.com/AlexSorokoletov Alex Sorokoletov

    Jeff, isn’t that strange? You suggest not to use WebClient as it works on UI and and at the same time you post update to WebClient with gzip support.

  • Anonymous

    WebClient for 7.0 and WebClient for Mango are different: in 7.0 its on the UI always, in Mango it is on the calling thread via sync. context.

  • http://openid.richardszalay.com/ Richard Szalay

    I don’t see why this is necessary. GZipped responses are supported natively in Mango and are automatically unzipped for you. The only thing you have to do is make sure you add an Accept-Encoding header to the request.

  • http://openid.richardszalay.com/ Richard Szalay

    It’s worth pointing out that RegisterPrefix only works for WebRequest.Create(), calling WebRequest.CreateHttp() will always use the default IWebRequestCreate

  • Anonymous

    Maybe that’s true for the WebClient (is it?), but if you use HttpWebRequest/Response the response stream you get is not automatically unzipped.

  • http://twitter.com/AlexSorokoletov Alex Sorokoletov

    Thank you for clarifying that, most people including me still think that WebClient is always tied to UI thread.

  • Josh Schlesinger

    That is not true for WebClient. You can set the accept-encoding but you will get an exception because it can’t process the response. In the full framework there is a property on the HttpWebRequest called AutomaticDecompression but it isn’t available on the phone…hence the need for this library.

  • http://www.pedrolamas.com Pedro Lamas

    Well, yesterday I had a 2.5 hours trip on train so I managed to cook up RestSharp with GZip support… I’ve forked the original RestSharp over at GitHub and added a pull request, I’m now waiting to see what happens!
    In the meantime, you can get it here: https://github.com/PedroLamas/RestSharp

  • http://www.openmymind.net/ Karl Seguin

    Maybe I’m missing something, but I find the numbers meaningless without knowing the initial size of the payload???

  • Anonymous

    Yes, sorry the numbers were not meant to be a true indicator of performance. You’ll want to do some testing on your end – but for JSON and other textual data types the increase can be significant.

    I saw some simple but large requests (150k or so) improve by 80%, while others (like a 20k JSON payload) increased about 50%.

    But a lot of things can affect it, sorry!