{"id":773,"date":"2013-08-15T15:08:00","date_gmt":"2013-08-15T15:08:00","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/dotnet\/2013\/08\/15\/announcing-traceevent-monitoring-and-diagnostics-for-the-cloud\/"},"modified":"2021-10-04T12:26:22","modified_gmt":"2021-10-04T19:26:22","slug":"announcing-traceevent-monitoring-and-diagnostics-for-the-cloud","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/dotnet\/announcing-traceevent-monitoring-and-diagnostics-for-the-cloud\/","title":{"rendered":"Announcing TraceEvent \u2013 Monitoring and Diagnostics for the Cloud"},"content":{"rendered":"

In this post, Vance Morrison<\/strong>, software developer on the .NET Runtime team, will talk about the TraceEvent NuGet library we just shipped. — Immo<\/em><\/p>\n

I am happy to report that we have just released the TraceEvent<\/a> library on (prelease) NuGet. This library is an important part of the .NET, end-to-end, cloud scale diagnostics and monitoring story. It allows you to easily control and process any Event Tracing for windows (ETW)<\/a> logging events, and in particular the logging events generated by the .NET System.Diagnostics.Tracing.EventSource<\/a> class. Together EventSource and TraceEvent form a complete logging system that is light weight, efficient, interoperable with operating system logging, and which can easily be automated for cloud-scale installations.<\/p>\n

The TraceEvent library has been available for some time via the BCL codeplex site, however repackaging this library as a NuGet package makes it significantly easier for you to add to your app and also to get all of our updates. In addition we have created a \u2018TraceEvent Samples\u2019 NuGet package that makes it even easier to get started.<\/p>\n

Here\u2019s what it does<\/h2>\n

Here is a quick demo of what you can do. In the following demo we have two processes. The process on the left represents an application being monitored. This application EvenSource to log a \u2018MyFirstEvent\u2019 when it start a user\u2019s request, and a \u2018MySecondEvent\u2019 when it is done. For demo purposes, this process also logs messages to the console when these events are logged. The process on the left is using ETW to monitor \u2018interesting\u2019 events machine wide, including the \u2018MyFirstEvent\u2019 and \u2018MySecondEvents\u2019 as well as .NET runtime garbage collection (GC) events. It uses this information to perform its analysis: for this demo, it is how long requests take and how long.NET GCs take. In this particular screenshot, you can see we highlighted a GC that happened on the third request and the resulting logging that happened in the monitor. This scenario is typical, how you can combine both your own events along with a wealth of built in ones to perform better analysis (in this case determining if GC has impact on response times). Building something like this is straightforward with the TraceEvent library, but you can also do so much more. The PerfView<\/a> tool was built entirely on top of the TraceEvent library.<\/p>\n

\"\"<\/p>\n

How to get started<\/h2>\n

It is really easy to get started with our sample in Visual Studio:<\/p>\n

    \n
  1. Create an empty console application,<\/li>\n
  2. Right click on the References<\/strong> node in Project Explorer and select Manage NuGet Packages<\/strong><\/li>\n
  3. Set the filtering to include pre-release packages and search for \u2018TraceEvent Samples\u2019 and select that package a part of that project.<\/li>\n
  4. Add the line TraceEventSamples.AllSamples.Run();<\/code> to your main program.<\/li>\n
  5. Browse the sample code in the \u2018TraceEventSamples\u2019 directory the NuGet package inserted into your project.<\/li>\n
  6. Hit \u201cF5\u201d when you want to see it run.<\/li>\n<\/ol>\n

    The samples package shows you several of the most common scenarios for using TraceEvent. It demonstrates how to enable both your own EventSource or .NET\/OS events and then send the data either to a file or process them in real time.<\/p>\n

    The TraceEvent NuGet package also comes with a \u2018TraceEventProgrammersGuide.docx\u2019 file (I have included a copy at the end of the blog but as you will see it is not finished. The most up to date one will always be in the package). Once you have decided that you want to use this library, reading this doc is the fastest way to get truly ramped up.<\/p>\n

    Enough Overview. Show me the code!<\/h2>\n

    The EventSource class in the .NET Framework 4.5 makes logging as easy as defining a class that has a method for each different kind of event. You can see what that looks like in the example definition below, and learn more in my EventSource tutorial<\/a>.<\/p>\n

    class MyEvents : EventSource <\/span>
    {<\/span>
     public void MyFirstEvent(string MyName, int MyId) { WriteEvent(1, MyName, MyId); }<\/span>
     public static MyEvents Log = new MyEvents();<\/span>
    }<\/span><\/pre>\n
    The MyFirstEvent method would then be called from the appropriate places within an app.<\/p>\n
    MyEvents.Log.MyFirstEvent("hello", 35); <\/code><\/pre>\n
    Events then \u2018go to subscribers\u2019. Those subscribers could be EventListeners in the same process as the EventSource. EventSource also supports out-of-process listeners, using its built in support to send events to the Event Tracing for Windows (ETW) infrastructure. There are tools like PerfView<\/a>, which let you enable EventSource providers \u2018manually\u2019 (see my tutorial<\/a> for details).<\/p>\n
    You can also listen to events from other processes from code, which is valuable for automation. This is where the TraceEvent library comes in, which make it easy to collect ETW events. Here is the code to turn on monitoring of the \u2018MyEvents\u2019 EventSource (from another process) for 10 seconds and send the data to MyEventData.etl. It is about as easy as you can get.<\/p>\n
    using (var session = new TraceEventSession("MySession", "MyEventData.etl"))<\/span>
    {<\/span>
     session.EnableProvider("MyEvents");<\/span>
     \/\/ Collect for 10 seconds then stop. <\/span>
     System.Threading.Tasks.Task.Delay(TimeSpan.FromSeconds(10)).Wait();<\/span>
    }<\/span><\/pre>\n
    Once you have data, TraceEvent makes it easy to process the events that you\u2019ve collected. Here is the code that plucks out the events you want, using lambda syntax.<\/p>\n
    using (var source = new ETWTraceEventSource("MyEtlFile.etl"))<\/span>
    {<\/span>
     source.Dynamic.AddCallbackForProviderEvent("MyFirstEvent", "MyEvents", (TraceEvent data) =><\/span>
     {<\/span>
     Console.WriteLine("GOT {0}({1}, {2})", data.EventName,<\/span>
     data.PayloadByName("MyName"), data.PayloadByName("MyId"));<\/span>
     });<\/span>
     \/\/ Typically you have several of these callback registration calls. <\/span>
     source.Process();<\/span>
    }<\/span>  <\/pre>\n
    You might wonder why the API signature includes an Actionparameter, which you see above as the lambda, instead of a more traditional IEnumerable pattern. TraceEvent also supports \u2018real time\u2019 processing of events, so exposes an API that works well for that. In that situation, an implementation needs to be called as events come in. In the scenario above, which processes an ETL file, that approach isn\u2019t needed, however, that\u2019s the reason for the API shape. Because the code is written in this way, it is easy to modify this sample to process the data in real time rather than from a file.<\/p>\n
    The code above registers callbacks for the set of events that it is interested in. In this example, that\u2019s only \u2018MyFirstEvent\u2019 from the \u2018MyEvents\u2019 provider). This implementation is simple: it finds the \u2018MyName\u2019 and \u2018MyId\u2019 values and print them. A typical implementation would register several callbacks, one for each kind of event needed for analysis. Once all the callbacks are registered, \u2018Process\u2019 is called to invoke the callbacks, as the events are read from the source (in this case the ETL file).<\/p>\n
    Not bad. In a dozen lines of code or so, I have set up all the code needed to log an event, pass different types of argument to it, enable it from outside the process, and receive that event data to parse those arguments to do whatever monitoring logic I want. That\u2019s not a lot of effort to create the beginnings of a real cloud monitoring solution.<\/p>\n
    And that is only the beginning. For those of you who like the Reactive Extensions<\/a> framework, TraceEvent supports that too, and the NuGet TraceEvent Samples package has examples of this. You also have the wealth of .NET and OS events that let you know about the GC, the Just in time compiler, CPU, Disk I\/O, Network activity, Blocked time and more. Events can also have stack traces associated with them, which are SUPER useful for doing \u2018deep dive\u2019 diagnosis. We have only scratched the surface.<\/p>\n
    If you want to learn more:<\/p>\n
      \n
    1. Read the Programmers guide at the end of this blog entry.<\/li>\n
    2. Read Vance Morrison\u2019s Blog on TraceEvent<\/a> and EventSource<\/a>.<\/li>\n
    3. Read Grigori\u2019s blog on Semantic (strongly typed) logging<\/a> and the Semantic Logging Application Block<\/a> code on codeplex that lets you send EventSource events to other places besides ETW (e.g. a local file, or a database, or Azure storage \u2026)<\/li>\n<\/ol>\n
      Next steps!<\/h2>\n
      You should follow my suggestion above about creating an empty console project in Visual Studio and referencing the \u2018TraceEvent Library Samples\u2019 NuGet Package in your empty project (currently is a pre-release package so make sure your filters don\u2019t hide it). If you want to skip the samples, you can simply reference the \u2018TraceEvent Library\u2019 directly and start writing your code that uses it. In either case, reading the TraceEventProgrammersGuide.docx (either at the end of this blog or in the TraceEvent package) is highly recommended.<\/p>\n
      How do I report bugs or make suggestions?<\/h2>\n
      We are definitely interested in your feedback. For now please simply respond to this blog entry with any bug reports or suggestions.<\/p>\n
      Happy Eventing!<\/strong><\/p>\n
      TraceEventProgrammersGuide.docx<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"
      In this post, Vance Morrison, software developer on the .NET Runtime team, will talk about the TraceEvent NuGet library we just shipped. — Immo I am happy to report that we have just released the TraceEvent library on (prelease) NuGet. This library is an important part of the .NET, end-to-end, cloud scale diagnostics and monitoring […]<\/p>\n","protected":false},"author":335,"featured_media":58792,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[685],"tags":[11,30,31,59,77,104,117],"class_list":["post-773","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-dotnet","tag-net-framework","tag-announcement","tag-asp-net","tag-diagnostics","tag-fundamentals","tag-nuget","tag-releases"],"acf":[],"blog_post_summary":"
      In this post, Vance Morrison, software developer on the .NET Runtime team, will talk about the TraceEvent NuGet library we just shipped. — Immo I am happy to report that we have just released the TraceEvent library on (prelease) NuGet. This library is an important part of the .NET, end-to-end, cloud scale diagnostics and monitoring […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts\/773","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/users\/335"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/comments?post=773"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts\/773\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/media\/58792"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/media?parent=773"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/categories?post=773"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/tags?post=773"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}