31 March 2011
In my Windows Phone 7 app I’ve enabled anonymous feedback via an opt-in mechanism: when setting up their account, they can opt-in to the use of the location service, plus opt-in to sharing anonymous use and crash reports.
In return I’m using the data to better understand:
A side benefit is knowing that thousands of times the app has been used, it’s in real hands. My app has used for over 37,360 check-ins since being released (as of a few nights ago), and I can keep track of things ahead of waiting for the marketplace’s data that comes out many days behind.
As interesting as data is, it takes work to do this all right. App developers should consider using analytics at some point if it makes sense – at a minimum, during your beta development period, to learn what parts of the app are used the most.
By understanding better what key parts of an app my users are spending time on, I can work to figure out where to invest time in making improvements in future updates, and if you’re smart with the data (don’t be greedy, don’t be inappropriate with it, etc.), everyone should be OK with that value.
With such great power comes responsibility, too, so let’s tread carefully out of respect of our users… there was a short exchange of emails about this between some passionate Windows Phone developers today and I just wanted to at least share my experience with this post this evening.
Analytics can range from the basic (what size screen is being used? what “page” in an app has been navigated to?) to crazy (where was the last touch on-screen? a unique representation of the phone? the phone number of the user?). Thankfully the Windows Phone is very secure thanks to its nice sandbox for apps to play in, and so app developers cannot get ahold of data like other platforms (phone numbers, etc., are off-limits).
That’s a good thing and a bad thing: as a developer, I’d love to be able to give my user’s a big “find friends I may already know by examining my address book” option… but the idea that “Joe’s random duck noise app” might be able to get my contact list without me knowing is scary. So I’m glad that restriction is in place today.
The Wall Street Journal has a nice interactive analysis of many of the top iPhone and Android phone apps online here – they did some interceptive research into the data stream that apps had, what was being sent over the wire by apps, etc. Very good investigative journalism! It is clear that there’s a lot of interesting things happening, and the data can be useful for many purposes.
At this time there is not a marketplace ingestion requirement that your users must opt-in to all the data types sent from your app, so as developers we have a responsibility to tread with care – it’d be hard to enforce, but letting users know is important in the trust relationship between developer and customer.
I personally don’t like the idea of apps that use analytics without asking for opt-in first, but they’re out there, and clearly controls like the ad controls get tons of data for those advertisers. These things fund apps, so who knows...
To re-emphasize, Windows Phone is secure if you compare it with many other platforms: today a Windows Phone app cannot troll the list of contacts on the device; if it asks for the unique phone identifier, or other information like the make/model of the phone, that capability is detected at ingestion time, so a user about to download your app will receive a clear indication that you app makes use of that data, leaving the consumer to make a trust decision before even agreeing to install & download the application. Slick!
One interesting thing is that the concept of the “unique phone identifier” is combined with other key properties, all exposed through an interface called the ‘extended phone properties’ – info like the amount of memory available on the device, the peak working set of the process, etc. It would be nice if this was more clear: advertising controls probably want the unique ID, but many developers may just want the simple memory counter, etc., for debugging and error reporting purposes.
I have a few mitigations to worries that people have about analytics and data, and I hope as app developers you will consider both…
Treat your users with respect, offer an opt-in to the analytics. Please!
This is easier for apps that already must prompt users, such as the location sharing/location service apps, so understandably apps that are “ready to go” from the start might need to come up with a better process. Considering many settings screens are hidden away, I would rather prefer an Opt-in prompt than one day finding an Opt-out toggle switch hidden away…
In the code path for enabling and updating analytics, check for the user’s explicit opt-in permission (in isolated storage, etc.). If they haven’t given it, don’t even invoke that part of your app to startup analytics.
If you include these things, users that are very concerned about their privacy and how you might use the data will have something to fall back on. Not storing personally identifiable data is a good thing all-around, but gathering statistical data (like pages hit) seems acceptable for most folks.
Now that my app has been in the Windows Phone Marketplace for more than a week, I have some data, though it’s difficult to interpret at this point.
These stats are available to all app developers, right in the App Hub, but they lag by many days behind (credit cards need to clear, etc.), so you’ll need to be patient. Clearly I’m getting a few downloads, but who knows how the last 7 days have gone, given this report ends in last week.
So without analytics, at least I know “a few thousand people have downloaded my app – sweet”.
Now, if I open up my analytics’ provider’s site, I can drill into the data that the analytics feed has been providing. I don’t have any way of knowing what percent of my users have left the “anonymous use and feedback” checkbox checked, but I hope a lot of them have. I don’t have a way of knowing if someone decides not to opt-in.
Just a quick look, right around 3/21, I can see the same jump as the marketplace’s cumulative total: obviously the app went out and people downloaded it.
From then on, although the marketplace still hasn’t given me information, I can see that my users must be happy: they’re continuing to use the app, day in and out. You can also roughly see a pattern emerging where around the weekends the app seems to be used just a little bit more – fluctuating maybe 20% by day.
Let’s dig a little deeper into what “pages” were viewed. These pages are effectively the XAML pages. I’ve also highlighted in Photoshop two of the “pages” for specific reason. In my analytics implementation, I decided to only have an analytics event when a full page is navigated to – so if you navigate from the “friends” to the “places” pivot item, I don’t record that as an event – it seems simpler, and is less chatty from a data standpoint.
Most users of the app will probably have a total of 5-7 analytics events during that use of the app, and the amount of data sent is super small.
So looking at this set of data, here’s what pops out:
And now, on to the map…
This data tells me that I have a lot of US users, and users across the world, but right now nowhere is highlighting as a place that I might want to consider localization a must-have.
Good and fun to know!
My initial, unexperienced take on this data…
At this time I’m not sharing my analytics implementation (I’m using Google Analytics b.t.w., free, easy, and I already use it with my web sites and blog, so I’m familiar with it), but there are a few things – from the Silverlight analytics framework to other providers people are using. I’ve also heard that the Preemptive product is good. I need to cleanup my code in the meantime!
In my app, I simply hook analytics up to the navigation service events, so every time a new primary page (such as “About” or “Explore a place”) are used, I fire off an analytics event that is very small (a few hundred bytes sent).
Hope this is interesting.
Jeff Wilcox is a Principal Software Engineer at Microsoft in the Open Source Programs Office, helping Microsoft scale to 15,000+ engineers using, contributing to and releasing open source.