{"id":24709,"date":"2021-03-31T13:19:45","date_gmt":"2021-03-31T20:19:45","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/aspnet\/?p=24709"},"modified":"2022-10-10T21:33:02","modified_gmt":"2022-10-11T04:33:02","slug":"monitoring-and-observability-in-cloud-native-asp-net-core-apps","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/dotnet\/monitoring-and-observability-in-cloud-native-asp-net-core-apps\/","title":{"rendered":"Monitoring and Observability in Cloud-Native ASP.NET Core apps"},"content":{"rendered":"<p>Distributed applications are complex and bring in their own set of challenges for developers to debug and fix production issues. Though the microservices architecture helps maintain a smaller team that works autonomously and focuses on separate business domains, it introduces newer challenges due to its distributed nature. For example, in case of an issue during a business transaction, the request needs to be traced end-to-end, which may span across multiple services and infrastructure. Some of the challenges include:<\/p>\n<ul>\n<li>Managing known and unknown failures<\/li>\n<li>Failures are also distributed<\/li>\n<li>Legacy monitoring systems will not work<\/li>\n<\/ul>\n<p>This is where monitoring and observability come into the picture. While monitoring records the overall health of an application while observability helps you dig deeper with contextual data. During the entire month of March, we focused on bringing you newer content around Microservices. On the .NET show, Cecil and I talked in-depth about the Observability and Monitoring in a Cloud-Native app.<\/p>\n<p><iframe width=\"800\" height=\"450\" src=\"https:\/\/www.youtube-nocookie.com\/embed\/PDdHa0ushJ0\" allowfullscreen><\/iframe><\/p>\n<p>We looked at Observability and Monitoring&#8217;s key pillars like <strong>Logging<\/strong>, <strong>Metrics<\/strong>, and <strong>Tracing<\/strong> and deep-dived into the <strong>Health checks<\/strong> in the above video.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/aspnet\/wp-content\/uploads\/sites\/16\/2021\/03\/Monitoring-300x167.png\" alt=\"Image Monitoring and Observability\" width=\"300\" height=\"167\" class=\"alignnone size-medium wp-image-24710\" srcset=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2021\/03\/Monitoring-300x167.png 300w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2021\/03\/Monitoring-1024x569.png 1024w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2021\/03\/Monitoring-768x427.png 768w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2021\/03\/Monitoring-1536x854.png 1536w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2021\/03\/Monitoring.png 1994w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/p>\n<p>Here are some of the fundamental concepts discussed in the video:<\/p>\n<h2>Health checks<\/h2>\n<p>Health checks are implemented in microservices, preferably using HTTP endpoints that various real-time monitoring systems can query. At a very minimum, the health check endpoints should respond to:<\/p>\n<ul>\n<li>Is the system running?<\/li>\n<li>Can it perform tasks?<\/li>\n<\/ul>\n<p>In the Kubernetes world, these directly transform into the <strong>liveness<\/strong> and <strong>readiness<\/strong> probes, respectively. They are defined in the deployment configuration of the Kubernetes YAML file.<\/p>\n<p><img decoding=\"async\" width=\"941\" height=\"863\" src=\"https:\/\/devblogs.microsoft.com\/aspnet\/wp-content\/uploads\/sites\/16\/2021\/03\/Kubernetes-Yaml-File.png\" alt=\"Image Kubernetes Yaml File\" class=\"alignnone size-medium wp-image-24711\" srcset=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2021\/03\/Kubernetes-Yaml-File.png 941w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2021\/03\/Kubernetes-Yaml-File-300x275.png 300w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2021\/03\/Kubernetes-Yaml-File-768x704.png 768w\" sizes=\"(max-width: 941px) 100vw, 941px\" \/><\/p>\n<ul>\n<li>A <em>liveness<\/em> path is an endpoint that Kubernetes queries periodically to check for failures. Kubernetes provides liveness probes to detect failing applications and restarts them when they don&#8217;t return success codes.<\/li>\n<li>A <em>readiness<\/em> path is an endpoint that Kubernetes queries to know when a service is ready to start accepting traffic. It returns the HTTP status code 200 when all registered checks are successful. <\/li>\n<\/ul>\n<p>ASP.NET Core offers Health checks, middleware, and libraries for reporting health to a monitoring system. To get started, check out <a href=\"https:\/\/docs.microsoft.com\/aspnet\/core\/host-and-deploy\/health-checks?WT.mc_id=friends-0000-NANIL\" rel=\"noopener\" target=\"_blank\">Health checks in ASP.NET Core<\/a>.<\/p>\n<h2>Logging<\/h2>\n<p>No matter what tools you utilize to investigate issues in production, in the end, it is always the logs that give you the root cause of the problem. In the distributed world, you need to ensure that the log records have in-depth information to debug. They&#8217;re queryable from a central place. Each log record needs to have a correlation id, so they&#8217;re tracked in a single transaction to get the big picture.<\/p>\n<h3>Structured Logging<\/h3>\n<p>Using structured logging, you can add serialized objects to the logs that are efficiently queryable by log monitoring systems. For e.g., you can query the entire transaction log based on a <em>customerID<\/em> or a <em>transactionID<\/em>. In ASP.NET Core apps, you can use Serilog, which provides structured logging. Checkout <a href=\"https:\/\/docs.microsoft.com\/\/aspnet\/core\/fundamentals\/logging\/?WT.mc_id=friends-0000-NANIL\" rel=\"noopener\" target=\"_blank\">Logging in .NET Core and ASP.NET Core <\/a>for getting started and <a href=\"https:\/\/serilog.net\/\" rel=\"noopener\" target=\"_blank\">Serilog <\/a>to understand structured logging.<\/p>\n<h3>Centralized Logging &amp; correlation ID<\/h3>\n<p>In traditional applications, log files are stored on the local machine. Logging to a flat-file on a single machine is not helpful in a distributed environment. Applications producing logs may not have access to the local disk, or the local disk may be highly transient as containers are moved around virtual machines. Because of the challenges associated with using file-based logs in cloud-native apps, centralized logs are preferred. Logs are collected by the applications and shipped to a central logging application which indexes and stores the logs. This class of systems can ingest tens of gigabytes of logs every day. Serilog provides <a href=\"https:\/\/github.com\/serilog\/serilog\/wiki\/Provided-Sinks\" rel=\"noopener\" target=\"_blank\">sinks <\/a>for writing log events to centralized systems like Azure Application Insights, a feature of Azure Monitor. It&#8217;s also helpful to follow some standard practices when building logging that spans many services. For instance, generating a <a href=\"https:\/\/blog.rapid7.com\/2016\/12\/23\/the-value-of-correlation-ids\/\" rel=\"noopener\" target=\"_blank\">correlation ID<\/a> at the start of a transaction and then logging it in each message related to that transaction makes it easier to search for all related messages from the centralized logging systems.<\/p>\n<h2>Distributed Tracing<\/h2>\n<p><img decoding=\"async\" width=\"1509\" height=\"1117\" src=\"https:\/\/devblogs.microsoft.com\/aspnet\/wp-content\/uploads\/sites\/16\/2021\/03\/End-to-End-tracing.png\" alt=\"Image End to End tracing\" class=\"alignnone size-medium wp-image-24712\" srcset=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2021\/03\/End-to-End-tracing.png 1509w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2021\/03\/End-to-End-tracing-300x222.png 300w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2021\/03\/End-to-End-tracing-1024x758.png 1024w, https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2021\/03\/End-to-End-tracing-768x568.png 768w\" sizes=\"(max-width: 1509px) 100vw, 1509px\" \/><\/p>\n<p>Distributed tracing is the equivalent of call stacks for modern cloud and microservices architectures, with the addition of a performance profiler tossed in. Distributed tracing or distributed request tracing helps look at the request end-to-end and enables you to identify a problem holistically. Trace can give you detailed answers to questions like What time did the event happen? How long did it take? Why did it take so long? What microservice handled it? Etc. Open-source distributed tracing systems like the <a href=\"https:\/\/github.com\/openzipkin\/zipkin\/?WT.mc_id=friends-0000-NANIL\" rel=\"noopener\" target=\"_blank\">openzipkin\/zipkin<\/a> are prevalent in this space.<\/p>\n<p>Enabling distributed tracing in your app could be as simple as adding a corresponding distributed trace provider&#8217;s SDK into every microservice. E.g., With the <a href=\"https:\/\/docs.microsoft.com\/azure\/azure-monitor\/app\/distributed-tracing#how-to-enable-distributed-tracing?WT.mc_id=friends-0000-NANIL\" rel=\"noopener\" target=\"_blank\">Application Insights SDK<\/a> installed and configured in your app, tracing information is automatically collected for popular frameworks, libraries, and technologies by SDK dependency auto-collectors.<\/p>\n<p>There is a need for standardization with several different systems and tooling in place for observability. <a href=\"https:\/\/opentelemetry.io\/\" rel=\"noopener\" target=\"_blank\">OpenTelemetry<\/a> standardizes how different applications and frameworks collect and emit observability telemetry. OpenTelemetry provides a vendor-neutral specification, a set of APIs, SDKs and tooling and integration for observability telemetry (distributed tracing, metrics, etc.). Check out the blog post <a href=\"https:\/\/devblogs.microsoft.com\/dotnet\/opentelemetry-net-reaches-v1-0\/?WT.mc_id=friends-0000-NANIL\" rel=\"noopener\" target=\"_blank\">OpenTelemetry .NET reaches v1.0<\/a> for detailed insights.<\/p>\n<h2>Hands-on Modules<\/h2>\n<p>We have built a series of modules to help you learn to build .NET microservices and cloud-native technologies. Check out the below modules that will help you understand monitoring and observability.<\/p>\n<ul>\n<li>For health-checks check out <a href=\"https:\/\/docs.microsoft.com\/learn\/modules\/microservices-aspnet-core\/?WT.mc_id=friends-0000-NANIL&amp;ns-enrollment-type=Collection&amp;ns-enrollment-id=8mq4i2mzgjwn10\" rel=\"noopener\" target=\"_blank\">Create and deploy a cloud-native ASP.NET Core microservice<\/a><\/li>\n<li>For monitoring and Observability, check out <a href=\"https:\/\/docs.microsoft.com\/learn\/modules\/microservices-logging-aspnet-core\/?WT.mc_id=friends-0000-NANIL&amp;ns-enrollment-type=Collection&amp;ns-enrollment-id=8mq4i2mzgjwn10\" rel=\"noopener\" target=\"_blank\">Instrument a cloud-native ASP.NET Core microservice<\/a><\/li>\n<li>For other topics checkout <a href=\"https:\/\/aka.ms\/aspnet-microservices?WT.mc_id=friends-0000-NANIL\" rel=\"noopener\" target=\"_blank\">https:\/\/aka.ms\/aspnet-microservices <\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Distributed applications are complex and bring in their own set of challenges for developers to debug and fix production issues. ASP.NET Core offers a number of solutions to add monitoring and observability to your cloud-native applications.<\/p>\n","protected":false},"author":593,"featured_media":24712,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[197,7237],"tags":[7539,7572,7530],"class_list":["post-24709","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-aspnet","category-containers","tag-cloud","tag-healthchecks","tag-logging"],"acf":[],"blog_post_summary":"<p>Distributed applications are complex and bring in their own set of challenges for developers to debug and fix production issues. ASP.NET Core offers a number of solutions to add monitoring and observability to your cloud-native applications.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts\/24709","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/users\/593"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/comments?post=24709"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts\/24709\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/media\/24712"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/media?parent=24709"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/categories?post=24709"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/tags?post=24709"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}