Why Analytics?

Merav Davidson

January 23rd, 20150 0

Analytics gives us the ability to understand our users and what they are experiencing, so we can build the right applications for them. It alerts us quickly to any issues and helps us diagnose them rapidly so as to provide users with the best possible service level.

Users have high expectations and little tolerance for hiccups, whether your app is an online service aimed at attracting consumers, a phone app intended to entertain people, or internal to your enterprise. Analytics enables us to identify problems early on, diagnose and solve them faster, and continuously improve our applications so they can become an integral part of our users’ day-to-day routine.

Detect and solve problems quickly

You need to solve problems before they actually impact customers and without analytics, you’re flying blind. Users may be suffering crashes, poor performance, or simply finding your app tough to navigate, while you have no clue until it’s much too late. You must be alerted early on, and have the ability to investigate what is wrong and diagnose the problem. Once you have a fix, you should be able to validate that the issue is indeed solved. In many cases, diagnosis requires analyzing data coming from different sources to understand dependencies across sub-systems, or nailing down the exact sequence that leads to a crash on a specific mobile device.

The time it takes to detect and repair any issues (MTTD, MTTR) has a direct impact on the service level users experience, and as a result, how soon they’d want to come back. It’s crucial to be able to analyze all the relevant information in real time. A problem might be in the application itself, or some other system it’s dependent on, the device it’s running on, or some combination of those. You need to be able to look at performance metrics, investigate exceptions and app crashes, search through logs, and correlate multiple sources of information.

I was once running a web service. One time when we deployed our latest & greatest version to a new datacenter it showed slower performance than the existing one, before experiencing any real user traffic. Naturally the fingers were pointed at the feature crew who had done the most recent changes, and they started investigating. But analyzing our data, we ended up identifying a routing issue within the new datacenter that no-one was aware of – one that was also affecting other applications running there. We got it resolved before any of our customers was affected.

More often than we’d like, engineering teams have to deal with multiple issues in parallel, and must understand the magnitude of each problem and its actual impact on users, in order to prioritize the efforts. Having a clear view of what customers are really doing with the app, how critical a certain functionality is vs. another, and in some cases even who the users currently affected actually are, allows optimizing the efforts for the right business outcome.

Know your users to delight them better

Deep understanding of our users and their behavior is also key in driving product direction: what does the customer need? How does she use the app to address this need? How well do we satisfy it and what are we missing? How can we get her to do more with us? Today’s market is very competitive, and customers have options. If we don’t delight them, eventually someone else would. Getting a clear view on who our users are and segmenting them into groups, analyzing what each group is after, and how well we address that need, enables continuously improving the experience and growing the value of the application. Agility is key in responding to users and seizing market opportunities. With short cycles of Build-Measure-Learn, we can deliver incremental improvements, measure their impact, and choose the next step based on it, iteratively.

My team was once split over one of our key features. It was complex and tough to maintain and some were arguing that it had to be significantly limited. Others believed it was very useful and an important part of our value prop, and therefore had to be expanded. Both sides could bring a data-point from a recent conversation with a customer to back up their point of view. An initial look at our usage data seemed to support Camp A: a very small percentage of the overall users were actually utilizing this feature. We performed further analysis segmenting the users by how advanced they were and looking deeper at the different usage patterns. The conclusion was that while being a small percentage, the users of this feature were actually our most engaged and committed ones, with which we had the most opportunity to grow our business. Camp B won after all. In this case too, having the relevant information at hand and the ability to thoroughly analyze it, enabled a data-driven decision, the right one for our customers and business.

Beyond drawing more people to use the applications we build, we want to see them increasing their engagement. We’d like to see them trying our app out and loving it, and give them good reasons to want to spend more time in it and come back. We aspire for our application to become an integral part of our customers’ day-to-day routine, whether work or life. Being intimately familiar with your users, analyzing new vs. returning, and looking at how Active Users trend over time is key to being successful.

Application Insights

With [Application Insights][3] from Microsoft you get a 360° view of your application’s availability, performance and usage for device and server applications, in the cloud and on-premises.

You go to one place to get an overview of how your application is doing, from which you can drill into different aspects, investigate performance bottlenecks, search for specific events, slice and dice the data to understand your usage growth with different audiences, and can understand the relationships between all of these. Application Insights was designed to be an integral part of the development process and it is easy to add to both new and existing applications. It allows the collection of vital application data with zero effort for developers.

Application Insights provides great support for both aspects we’ve discussed:

Detect, triage and diagnose issues so that you can fix them before they have a significant impact on your customers.
Analyze usage patterns and find out how your customers use your app, so you can prioritize working on the scenarios they find important.

You’ll find more in our other blog posts, and in our doc pages. If you haven’t tried it yet, here’s how to get started.

[3]: http://blogs.msdn.com/controlpanel/blogs/posteditor.aspx/Application Insights “http://azure.microsoft.com/en-us/services/application-insights/”