{"id":3401,"date":"2012-04-11T07:19:42","date_gmt":"2012-04-11T07:19:42","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/bharry\/2012\/04\/11\/preemptive-analytics-in-visual-studio-and-tfs-11\/"},"modified":"2024-05-13T12:49:12","modified_gmt":"2024-05-13T19:49:12","slug":"preemptive-analytics-in-visual-studio-and-tfs-11","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/bharry\/preemptive-analytics-in-visual-studio-and-tfs-11\/","title":{"rendered":"Preemptive Analytics in Visual Studio and TFS 11"},"content":{"rendered":"

Over the past couple of years there\u2019s been a growing connection between development and operations.\u00a0 The \u201cold\u201d world where development teams throw applications over the wall at ops is disappearing the same way the world where developers threw applications over the wall at test did.\u00a0 Whether you\u2019re talking about continuous deployment, DevOps or \u201cBuild, Measure, Learn\u201d, these are all catchy phrases that are various aspects of getting the development team connected more closely with the customer and with the application in production.<\/p>\n

I\u2019ve had the privilege of living this life for the past couple of years as we\u2019ve been bringing the Team Foundation Service to life.\u00a0 It\u2019s really challenged me to think about the world in a different way.\u00a0 Over the next couple of months, I\u2019m going to try to write a series of posts that shares some of the things I\u2019ve been learning along the way.\u00a0 One of those things, though, is that you have to have very good visibility into production.\u00a0 Without it you will have a low quality, expensive, undesirable service.\u00a0 With it you can solve problems before your customers even realize they exist, prioritize work based on what people actually use, test changes to see how it affects user behavior, drive your costs (both hard and soft) to the lowest level possible and much, much more.<\/p>\n

In the last year or so, we\u2019ve started to make some significant investments to help with this.\u00a0 The first was the Team Foundation Server and System Center integration<\/a> we announced about a year ago that enables production tickets to be \u201cescalated\u201d to TFS along with all of the diagnostic information.\u00a0 Another coming in the VS 11 release is Intellitrace in production<\/a> that allows you to get detailed diagnostic data from your production environment.\u00a0 We\u2019ve also announced some follow on improvements coming to Intellitrace in production<\/a> that will be available this fall.<\/p>\n

Another area we\u2019ve been working is in production telemetry.\u00a0 We\u2019ve partnered with Preemptive (the makers of Dotfuscator) on a basic telemetry included with Team Foundation Server\/Visual Studio 11 called PreEmptive Analytics for TFS Community Edition.\u00a0 This built in capability allows you to instrument your application and receive reports from your users on any crashes they experience.\u00a0 The reports are analyzed, correlated with other reports and distilled to a set of \u201cproduction incidents\u201d that appear to be the same underlying cause.\u00a0 These show up as work items in your Team Foundation Server database.\u00a0 You can also purchase the Pro edition and get additional capabilities like the ability to analyze what features of your application get the most use, etc.<\/p>\n

This kind of telemetry has been an important part of our process in Visual Studio for many years.\u00a0 We view it as a critical part of understanding the experience real customers are having, addressing the issues they are having and measuring our progress.\u00a0 We create a number of reports and set goals for every release.\u00a0 For instance, here\u2019s a list of the top 10 crashing bugs in the VS 11 Beta reported by real user telemetry and the current status of the bug.<\/p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Bug Info:<\/b><\/td>\nWatson Info:<\/b><\/td>\n<\/tr>\n
Rank<\/b><\/td>\nTFS ID<\/b><\/td>\nResolution<\/b><\/td>\nPU<\/b><\/td>\nHits<\/b><\/td>\nCABs<\/b><\/td>\nDetails<\/b><\/td>\n<\/tr>\n
#1<\/td>\n358582<\/a><\/u><\/td>\nFixed<\/td>\nVS: Pro<\/td>\n1010<\/td>\n6.0%<\/td>\n45<\/td>\nMSENV.DLL!== [Crash32_Normal: 0xC0000005]<\/b><\/td>\n<\/tr>\n
#2<\/td>\n276670<\/a><\/u><\/td>\n(Active)<\/td>\nVS: Ultimate<\/td>\n454<\/td>\n2.7%<\/td>\n9<\/td>\nVSDEBUG.DLL!CAddressPosition::UpdateMarker [Crash32_Normal: 0xC0000005]<\/b><\/td>\n<\/tr>\n
#3<\/td>\n378926<\/a><\/u><\/td>\n(Active)<\/td>\nNDP: WPF<\/td>\n326<\/td>\n1.9%<\/td>\n42<\/td>\nNVD3DUM.DLL!unknown [Crash32_Normal: 0xC0000005]<\/b><\/td>\n<\/tr>\n
#4<\/td>\n372955<\/a><\/u><\/td>\n(Active)<\/td>\nVS: Pro<\/td>\n313<\/td>\n1.9%<\/td>\n21<\/td>\nUNKNOWN.DLL!Microsoft.VisualStudio.Editor.Implementation.Find.FindTarget.CalculateWrapping [Clr20r3: system.nullreferenceexception]<\/b><\/td>\n<\/tr>\n
#5<\/td>\n374532<\/a><\/u><\/td>\nFixed<\/td>\nVS: Ultimate<\/td>\n226<\/td>\n1.3%<\/td>\n8<\/td>\nVSDEBUGENG.SCRIPT.DLL!ScriptDM::CProviderEventCallback::AttachToProgramImpl [Crash32_Normal: 0xC0000005]<\/b><\/td>\n<\/tr>\n
#6<\/td>\n362918<\/a><\/u><\/td>\nFixed\n(excluded)<\/td>\nVS: Pro<\/td>\n181<\/td>\n1.1%<\/td>\n18<\/td>\nMSENV.DLL!EnableBrowserSecurityFeatures [Crash32_Normal: 0x\nC0000420]\n[Exclude reason: internal only]<\/b><\/td>\n<\/tr>\n
#7<\/td>\n373304<\/a><\/u><\/td>\nExternal\n(excluded)<\/td>\nVS: Ultimate<\/td>\n161<\/td>\n1.0%<\/td>\n9<\/td>\nMICROSOFT.VISUALSTUDIO.ARCHITECTURETOOLS.PROGRESSIVEREVEALPROV!Microsoft.VisualStudio.ArchitectureTools.ProgressiveReveal.ProgressiveRevealProvider.Finalize [Clr20r3: system.missingfieldexception]\n[Exclude reason: unsupported install]<\/b><\/td>\n<\/tr>\n
#8<\/td>\n378927<\/a><\/u><\/td>\n(Active)<\/td>\nVS: Pro<\/td>\n123<\/td>\n0.7%<\/td>\n12<\/td>\nMSENV.DLL!CDelayProjectLoadManager::LoadProject [Crash32_Normal: 0xC0000005]<\/b><\/td>\n<\/tr>\n
#9<\/td>\n376355<\/a><\/u><\/td>\nFixed<\/td>\nExpression: VS Integ<\/td>\n116<\/td>\n0.7%<\/td>\n28<\/td>\nSYSTEM.RUNTIME.REMOTING.NI.DLL!System.Runtime.Remoting.Channels.Ipc.IpcPort.Read [AppHangB1: 0]<\/b><\/td>\n<\/tr>\n
#10<\/td>\n354401<\/a><\/u><\/td>\n(Active)<\/td>\nVS: WinC++<\/td>\n113<\/td>\n0.7%<\/td>\n30<\/td>\nMFC110U.DLL!CView::~CView [Crash32_Normal: 0xC0000374]<\/b><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

We also create visualizations to show how we are doing overall.\u00a0 Here\u2019s a bar chart \u2013 1 bar for each bug, height indicates number of reports and color is current status of the bug.<\/p>\n

\"Image<\/a><\/p>\n

So, with VS 11 & TFS 11, you get get the same kinds of telemetry on your applications.\u00a0 Both the server side and client side components for the Community Edition come in the box.\u00a0 The installation for the server side pieces have been integrated to the TFS administration console and the client side pieces are available on the VS tools menu.<\/p>\n

In short, you install the server side pieces.\u00a0 Then you instrument your app (including with an url to an exposed server side collector).\u00a0 When customers run your app and experience a crash, it will automatically send data, including things like a stack trace to your server and the aggregation service will file or update bugs on your TFS service.<\/p>\n

Once you\u2019ve got it all configured and have an app reporting failures, you\u2019ll start seeing production incidents in TFS that look like this:<\/p>\n

\"Image<\/a><\/p>\n

Then you can start looking at the built in reports like these.\u00a0 Here\u2019s an overview of incidents by application, including the status of the incidents.<\/p>\n

\"Image<\/a><\/p>\n

Or trends of incidents over time.<\/p>\n

\"Image<\/a><\/p>\n

Conclusion<\/h3>\n

Telemetry and Analytics are increasingly important aspects of the software development process.\u00a0 They enable you to \u201cclose the loop\u201d and ensure you are delivering a great experience for your customers.\u00a0 They are important regardless of whether you are building a mission critical server or a client running on desktops in your organization or PCs and phones around the world.\u00a0 For the past several years, we\u2019ve been investing in making Visual Studio a great tool for developers to get good insight into application behavior and you can expect a great deal more in the coming years.<\/p>\n

Everything you need is in the VS 11 and TFS 11 beta releases we published last month.\u00a0 We\u2019re still working to streamline and improve some of the experiences but I think it\u2019s starting to get to where we want it to be.\u00a0 I encourage you to check it out.<\/p>\n

You can visit these sites to learn more:<\/p>\n