First, let’s briefly explain what each tool does.<\/p>\n
.NET Object Allocation Tracking tool<\/h2>\n
This tool helps you track how many instances of each type are allocated to the heap and their aggregate size and the methods they are allocated from. It helps you answer the questions: Where was this type allocated from? How many instances of this type are<\/u> allocated? Which method accounts for the most allocations? etc. It also collects information about each garbage collection that occurs, such as which types were freed, and which ones survived.<\/p>\n
This can be useful when you want to understand allocation patterns in your .NET code. It can also help you optimize your app\u2019s memory usage by tracking down the methods that are most expensive in terms of memory allocations.<\/p>\n
The tool shows us where things are allocated from. But it does not tell us why an object is still in memory.<\/p>\n
Memory Usage tool<\/h2>\n
This tool allows you to take heap snapshots while your app is running. Each snapshot contains information about objects that were alive at that time and which objects hold references to them. It also displays diffs between different snapshots. This allows you to see what has changed between two points in the app\u2019s execution: which types of objects were garbage collected, which new objects were allocated, etc.<\/p>\n
This tool is helpful when investigating memory leaks. It helps you answer questions such as: Which objects were alive on the heap at a certain point in the app\u2019s execution? What are the references keeping this instance alive? Did this object survive a garbage collection run?<\/p>\n
However, the tool does not tell you exactly where objects were allocated from.<\/p>\n
To illustrate the usage of these tools, we\u2019ll go through some practical examples using sample applications and use cases.<\/p>\n
Note on managed memory<\/h2>\n
The tools in this article focus on the managed heap. This is memory that is automatically managed by .NET the garbage collector. When you create a new instance of a class, it gets stored on the managed heap. This contrasts with the unmanaged heap which is not tracked by the garbage collector, it\u2019s mostly used by native code or code that interacts directly with the operating system. Whenever this article mentions the \u201cheap\u201d, it refers to the managed heap. For more information about memory management in .NET visit this page<\/a>.<\/p>\n In this first example, we have a demo application that stores a list of orders in memory and exposes an API endpoint that returns the total and average sales from orders from a particular region and category.<\/p>\n You can find the full source code in this GitHub repository<\/a>.<\/p>\n You can clone the repository locally and switch to the The code defines a Let’s run the server to test it. Once the server is running, we make the following requests from a web browser or HTTP client like Postman:<\/p>\n <\/p>\n But we have received reports that the server\u2019s memory usage grows over time until the server runs out of memory and crashes. We have been asked to investigate this issue.<\/p>\n First, let\u2019s try to reproduce this issue locally to understand what\u2019s happening. Sending a single request to the server might not be enough to observe any strange behavior. There are tools that we can use to send many requests to a server for load testing. For this example, I created a simple Console application for demonstration purposes and included it in the demo repository.<\/p>\n To run the application: while the server is still running, open a terminal window, then navigate to the root of the repository that you cloned, then navigate to the subfolder: This sends 10,000 requests to the server.<\/p>\n Now let\u2019s stop the server and proceed to the profiling.<\/p>\n We would like to see how memory usage evolves over time as the application is running, whether it increases indefinitely as reported. This makes the Memory Usage profiler a suitable tool to use.<\/p>\n Before we start profiling, it is recommended to set our project configuration to Release<\/strong> mode instead of Debug<\/strong> mode. When an app is built in Release<\/strong> mode, the compiler performs optimizations that might affect the results reported by profiling tools. Profiling our app in Release<\/strong> mode therefore leads results which more accurately reflect how our app behaves in production.<\/p>\n To start the Memory Usage<\/strong> profiler, go to the top menu then click Debug<\/strong> -> Performance Profiler<\/strong><\/p>\n This will open a tab that allows us to select the profiling tools we want to run the target application we want to profile. The Visual Studio profiler allows you to profile a running application or launch an executable, but we\u2019re currently just interested in profiling our open project. Make sure the Analysis Target<\/strong> is set to Startup Project<\/strong> ( Click the Start<\/strong> button. This will launch our application with the profiler attached to it. Visual Studio will open a diagnostics session tab that displays real-time information about the memory usage of your app.<\/p>\n Now let\u2019s stop the collection by clicking the Stop Collection<\/strong>\u00a0button at the top of the diagnostics pane. This will also stop the server.\u00a0 Visual Studio will process the results then present a report like the one below:<\/p>\n At the top, we have a timeline graph that shows how much memory the application used over time. Note that this is the overall memory and not just the managed heap. The blue markers indicate at which points we took snapshots of the heap. If garbage collection occurred, it would be indicated by yellow marks. Nevertheless, taking a snapshot also induces a full garbage collection. Consequently, in our case we had at least 3 garbage collections.<\/p>\n The bottom part displays a table that contains a summary of each snapshot taken. Each row represents a snapshot and contains the following fields:<\/p>\n Each snapshot allows us to inspect the types of objects on the managed heap at that point. Let\u2019s inspect the objects in the second snapshot. Clicking on the number of objects in the first snapshot will open a new tab that looks like the screenshot below:<\/p>\n The main part of the window is a table that contains information about each type of object on the heap. Each row has the following columns:<\/p>\n For more information about the controls provided by this tool, visit the documentation<\/a>.<\/p>\n We see that the top type is a class called Now, click the Paths To Root<\/strong> is a tree view that displays the chain of objects that have a reference to the type we clicked. If you expand the nodes on the tree, it will eventually lead to a \u201cGC root\u201d. Roots include static fields, local variables on the stack, CPU registers, GC handles and the finalize queue (visit this page<\/a> to learn more). The garbage collector will not free an object so long as there\u2019s a path of references from a root to that object. If an object has no chain of references linking it to a GC root, then it\u2019s considered unreachable, and its memory will be freed by the garbage collector. Therefore, this tree view allows us to see what references are keeping objects alive and is a valuable resource for investigating memory leaks.<\/p>\n In the case of the The Referenced Types<\/strong> tab shows a tree view that displays all the types that the selected references. It\u2019s basically the inverse of the Paths To Root<\/strong>. The Registering the cache as a singleton means the instance will remain alive in memory for the entire lifetime of the process. This also means any object it references (directly or indirectly) will also remain in memory. This is expected. And since our application does not implement any eviction logic, it\u2019s also expected that cached items will remain indefinitely in memory. However, that does not explain why we have 10,000 instances each of The cache stores statistics by region and category. The orders in our data store only span 3 unique region-category groups. So, we should have at most 3 entries in the cache: 3 instances of After making 10,000 requests, we had 10,000 entries in the cache. It seems like each request inserts a new entry. Let\u2019s look at the final snapshot. If this trend is consistent, then the 10,000 entries should still be in memory, and we should have 10,000 additional entries from the second batch of requests.<\/p>\n As expected, we have 20,000 total instances. We can compare this snapshot with the previous one by clicking the Compare with Baseline<\/strong> dropdown and selecting the target snapshot ID (Snapshot #2<\/strong> in this case). This will display a diff report:<\/p>\n We can see most of the change is in the types in the cache. They have all increased by 10,000 instances. There\u2019s not been much change in the remaining types, some even have reduced instance count from garbage collection.<\/p>\n We now have a decent idea where the issue comes from. Now we need to dig deeper to figure out the root cause and how to fix it. If we make multiple requests to the same endpoint, e.g., http:\/\/localhost:5018\/stats\/Nairobi\/Clothing<\/a>, we always get the same response. This is the expected behavior. Let\u2019s look at the code that handles that request<\/a>:<\/p>\n It tries to fetch data from the cache first, and if it doesn\u2019t find it there, it computes the data manually then stores it in the cache. This looks okay. Since our cache is growing, we must be hitting the cache at some point. We might be storing duplicate data. Let\u2019s look at our cache implementation in Do you see anything wrong with the code above? In the Well, we have not defined any equality logic for the Before we go ahead and fix the code. Let\u2019s use .NET Object Allocation Tracking<\/strong> tool to compare the type of reports and insights that we would get from both tools in this scenario. We go back to the Performance Profiler<\/strong> page, but this time select .NET Object Allocation Tracking<\/strong> and click Start<\/strong>.<\/p>\n This will launch the app and a profiling session. Like before, we use the client app to send requests to the server. When we\u2019re done collecting data for the scenario, click the Stop collection to view .NET Object Allocation data.<\/strong><\/p>\n This will process the data then display a report like the following:<\/p>\n The top pane has 3 graphs. The Live Objects<\/strong> chart shows the number of live objects in the heap over time. The second chart shows the percentage increase in the number of objects at different points in time. We can see we have noticeable increases when we make a request then it flattens. The red bar indicates that a garbage collection occurred at that point, between the first and second wave of requests. Notice that this coincides with a drastic drop in the number of live objects. The Private Bytes (MB)<\/strong> shows how much memory the app uses over time, like what we had in the Memory Usage<\/strong> report.<\/p>\n This tool does not allow us to take snapshots of the heap. It allows us to see which types were allocated within a specific time range. It can also tell us which types were cleaned up by garbage collection and which ones survived. But it does not tell us which types were on the heap at a specific point in time.<\/p>\n The bottom pane contains tabs with the different tabular report. The Allocations<\/strong> tab shows how many instances of each type were allocated on the heap.<\/p>\n We see that there were 40,000 instances of This information proves to be valuable when we\u2019re trying to track down allocations to reduce memory usage or GC pressure. We will explore more of what this tool offers in the next part of this series. For now, let\u2019s go back to fixing our memory issue.<\/p>\n Let\u2019s go with the simplest fix: turn our class into a Record. We simply change the definition of the Now let\u2019s re-run the Memory Usage<\/strong> profiler and re-run the same experiment: collecting snapshots after each wave of requests. I got the following report:<\/p>\n At first glance, we can see that the number of objects in the last 2 snapshots has significantly reduced from the first report we got earlier. When we open the second snapshot, we see that our We can use the filter search box to find the field: We only have 3 instances of When we look at the diff report of the last snapshot, we see that the number of instances for If we collected .NET Object Allocation data, we would not see a difference in the number of allocations for the If we were to define the struct manually, we could do it as follows:<\/p>\n Implementing Now when we re-run our Memory Usage<\/strong> experiment and look at a snapshot that was taken after making requests, we find that there\u2019s no instance of For comparison, let\u2019s collect a report using the .NET Object Allocation Tracking <\/strong>tool. We find that the Sample investigation<\/h2>\n
CachedOrderStatsSample<\/code> directory to follow along with this section. Inside the repository, navigate to the CachedOrderStatsSample<\/code> directory and open the CachedOrderStatsSample.sln<\/code> in Visual Studio.<\/p>\nStatsService<\/code> class which implements a GetStatsByRegionAndCategory()<\/code> method that returns statistics for a specific region and category. The statistics are computed from an in-memory collection of orders that are returned from the DataStore<\/code> class. After computing the statistics for a region-category group the first time, the values are stored in an in-memory cache so that they won\u2019t have to be computed the next time they\u2019re requested. The cache is implemented by the OrderStatsCache<\/code> class, and it uses an object that comprises of the region and category values as key.<\/p>\n\n
![\"Sample](\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2022\/08\/Sample-request-of-orders-1-163x300.png\")
<\/a><\/p>\n\n
![\"Sample](\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2022\/08\/Sample-request-returning-stats-of-orders-in-Clothing-category-in-Nairobi-300x158.png\")
<\/a><\/p>\nCachedOrderStatsSample\/TestClient<\/code> and then run the following command:<\/p>\ndotnet run<\/pre>\n
![\"Image](\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2022\/08\/Terminal-screenshot-of-command-executing-10000-requests-300x19.png\")
<\/a><\/p>\n![\"Visual](\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2022\/08\/Visual-Studio-Switch-to-Release-Mode-300x100.png\")
<\/a><\/p>\n![\"Visual](\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2022\/08\/Performance-Profiler-Menu-300x172.png\")
<\/a><\/p>\nOrderCachedStatsSample<\/code>). Then we select Memory Usage<\/strong>\u00a0from the list of available tools.<\/p>\n![\"Image](\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2022\/08\/Performance-profiler-tools-selector-in-Visual-Studio-300x114.png\")
<\/a><\/p>\n\n
![\"Image](\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2022\/08\/Visual-Studio-Memory-Usage-graph-with-snapshots-300x85.png\")
<\/a><\/p>\n\n
![\"Image](\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2022\/08\/Visual-Studio-Managed-Memory-Snapshot-300x128.png\")
<\/a><\/p>\n\n
Node<CacheKey, OrderStats><\/code> type holds a reference to a CacheKey<\/code> and OrderStats<\/code> objects. And this accounts for all CacheKey<\/code> and OrderStats<\/code> instances in memory. This means that the total inclusive size of Node<CacheKey, OrderStats><\/code> should be equal to its own (exclusive) size plus the total size of the CacheKey<\/code> and OrderStats<\/code> instances it references. And it adds up: 2,186,600 = 480,000 + 1,066,600 + 640,000<\/li>\n<\/ul>\nCacheKey<\/code>. This is a class defined in our application that\u2019s used as key to store computed statistics data in a ConcurrentDictionary<\/code> that backs our cache. The CacheKey<\/code> is a simple class that contains the properties Region<\/code> and Category<\/code>. OrderStats<\/code> is the class that stores the computed statistics for each region-category group.<\/p>\nCacheKey<\/code> row. This will reveal two tabs on the bottom pane: Paths To Root<\/strong> and Referenced Types<\/strong>.<\/p>\n![\"Image](\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2022\/08\/Paths-to-root-tree-view-300x146.png\")
<\/a><\/p>\nCacheKey<\/code>, all instances are referenced by Node<\/code> objects inside of a ConcurrentDictionary<\/code>. The dictionary is part of our custom OrderStatsCache<\/code>, which is used by the StatsService<\/code> to store statistics. The StartService<\/code> is registered in the dependency injection service container as a singleton<\/a>:<\/p>\nbuilder.Services.AddSingleton<IDataStore, DataStore>();\r\nbuilder.Services.AddSingleton<IOrderStatsCache, OrderStatsCache>();\r\nbuilder.Services.AddSingleton<IStatsService, StatsService>();\r\n<\/pre>\n
CacheKey<\/code> has only 2 string properties which account for all its outgoing references:<\/p>\n![\"Image](\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2022\/08\/Memory-Usage-Object-Referenced-Types-300x31.png\")
<\/a><\/p>\nCacheKey<\/code> and OrderStats<\/code>.<\/p>\nCacheKey<\/code> and 3 instances of OrderStats<\/code> regardless of how many requests we make.<\/p>\n![\"Image](\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2022\/08\/Memory-Usage-Order-Stats-Sample-Snapshot-2-300x46.png\")
<\/a><\/p>\n![\"Image](\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2022\/08\/Memory-Usage-Snapshots-Diff-Report-300x114.png\")
<\/a><\/p>\npublic OrderStats GetOrderStatsByRegionAndCategory(string region, string category)\r\n{\r\n OrderStats? stats = null;\r\n if (!cache.TryGetStats(region, category, out stats))\r\n {\r\n stats = ComputeStats(region, category);\r\n if (stats.Count > 0)\r\n {\r\n cache.SetStats(region, category, stats);\r\n }\r\n }\r\n\r\n return stats!;\r\n}<\/pre>\nOrderStatsCache<\/code><\/a>:<\/p>\npublic class OrderStatsCache : IOrderStatsCache\r\n{\r\n private readonly ConcurrentDictionary<CacheKey, OrderStats> cache = new();\r\n public void SetStats(string region, string category, OrderStats stats)\r\n {\r\n cache.AddOrUpdate(new CacheKey(region, category), stats, (key, existingStats) => stats);\r\n }\r\n\r\n public bool TryGetStats(string region, string category, out OrderStats? stats)\r\n {\r\n return cache.TryGetValue(new CacheKey(region, category), out stats);\r\n }\r\n\r\n private class CacheKey\r\n\r\n {\r\n public CacheKey(string region, string category)\r\n {\r\n Region = region;\r\n Category = category;\r\n }\r\n public string Region { get; }\r\n public string Category { get; }\r\n\r\n public override int GetHashCode()\r\n {\r\n return HashCode.Combine(Region, Category);\r\n }\r\n }\r\n}<\/pre>\nSetStats<\/code> method, we create a new instance of CacheKey<\/code> from the region and category to update the cache entry. In TryGetStats<\/code> we also create a new CacheKey<\/code> to look up the cache. Two keys with the same region and category should be considered equal and should be matched in the cache. But that\u2019s not what\u2019s happening. Have you seen why?<\/p>\nCacheKey<\/code> class, so the default object equality semantics are used. By default, objects are considered equal only if they refer to the same object in memory. So different instances of CacheKey<\/code> will result in different entries in the cache even if they contain the same data. This is an issue that\u2019s easy to fix. Here are a couple of approaches we can take:<\/p>\n\n
Equals<\/code> method that overrides the default Object.Equals<\/code><\/a> method<\/li>\nrecord<\/a><\/code> instead (C# 9 and later). Records implement value-equality semantics out of the box. The compiler generates custom equality comparison methods that compare each field of the records.<\/li>\nCacheKey<\/code> as a struct<\/code>. Structs also use value-equality semantics by default. They have the added advantage that they\u2019re allocated on the stack when instantiated, instead of the heap. This may be convenient for small objects that are allocated frequently. Using structs in such scenarios can help reduce the pressure on the garbage collector.<\/li>\nCacheKey<\/code> as a record struct<\/a> (C# 10 and later)<\/li>\n<\/ul>\n![\"Image](\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2022\/08\/Performance-Profiler-.NET-Object-Allocation-Tracking-Selected-300x114.png\"){kind=tracking}
<\/a><\/p>\n![\"Image](\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2022\/08\/NET-Object-Allocation-Tracking-session-300x142.png\"){kind=tracking}
<\/a><\/p>\n![\"Image](\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2022\/08\/NET-Object-Allocation-Tracker-report-300x148.png\"){kind=tracking}
<\/a><\/p>\nCacheKey<\/code> allocated. We allocate a new CacheKey<\/code> in both the SetStats()<\/code> and TryGetStats()<\/code> methods of StatsCache<\/code>. But the instances we create in TryGet()<\/code> get garbage collected. That\u2019s why the Memory Usage<\/strong> tool only reported 20,000 instances. When you double-click the CacheKey<\/code> entry, it opens a tree view on the right pane that back-traces the methods that allocate that type.<\/p>\n![\"Image](\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2022\/08\/NET-Object-Allocation-Tracker-Allocations-Tab-300x69.png\"){kind=tracking}
<\/a><\/p>\nCacheKey<\/code> class to the following:<\/p>\nprivate record CacheKey(string Region, string Category);<\/pre>\n
![\"Image](\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2022\/08\/Memory-Usage-after-fix-300x61.png\")
<\/a><\/p>\nCacheKey<\/code> or dictionary Node<\/code> are no longer among the top types:<\/p>\n![\"Image](\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2022\/08\/Memory-Usage-with-fewer-cache-keys-300x49.png\")
<\/a><\/p>\nCacheKey<\/code> and only 3 nodes in the cache. That\u2019s because we only have 3 unique region-category groups:<\/p>\n![\"Image](\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2022\/08\/Manage-Heap-filtered-types-300x38.png\")
<\/a><\/p>\nCacheKey<\/code> has not changed:<\/p>\n![\"Image](\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2022\/08\/Managed-heap-last-snapshot-has-new-cache-keys-300x34.png\")
<\/a><\/p>\nCacheKey<\/code>. We\u2019re still allocating just as many instances on the heap when we call new CacheKey(\u2026)<\/code>. But most of them get garbage collected. To do away with the heap allocations altogether, we can define the CacheKey<\/code> as a struct<\/code>. The easiest way to do this would be to declare CacheKey<\/code> as a record struct if you’re usig C# 10 or later:<\/p>\nprivate record struct CacheKey(string Region, string Category);<\/pre>\n
private struct CacheKey : IEquatable<CacheKey>\r\n{\r\n public CacheKey(string region, string category)\r\n {\r\n Region = region;\r\n Category = category;\r\n }\r\n\r\n public string Region { get; }\r\n public string Category { get; }\r\n\r\n public bool Equals(CacheKey other)\r\n {\r\n return Region == other.Region && Category == other.Category;\r\n }\r\n\r\n public override bool Equals([NotNullWhen(true)] object? other)\r\n {\r\n if (other is CacheKey otherKey)\r\n {\r\n return Equals(otherKey);\r\n }\r\n\r\n return false;\r\n }\r\n\r\n public override int GetHashCode()\r\n {\r\n return HashCode.Combine(Region, Category);\r\n }\r\n}<\/pre>\nIEquatable<CacheKey><\/code> and custom equality logic improves performance and avoids boxing<\/a>, which would cause the struct to be allocated on the heap when treated as Object<\/code>.<\/p>\nCacheKey<\/code> on the heap. We still have dictionary Node<\/code> instances on the heap, and the CacheKey<\/code> values are now \u201cembedded\u201d inside the Node instances instead of being stored as separate objects on the heap.<\/p>\n![\"Image](\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2022\/08\/Memory-Usage-no-cache-key-on-heap-300x38.png\")
<\/a><\/p>\nCacheKey<\/code> is no longer listed because it has not been allocated to the heap. Calling new CacheKey(\u2026)<\/code> now allocates the instance on the stack, and it’s automatically freed when the method goes out of scope. If we had not implemented IEquatable<CacheKey><\/code> we would see many CacheKey<\/code> instances allocated on the heap.<\/p>\n
![\"Image](\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2022\/08\/NET-Object-Allocation-Tracker-no-CacheKey-allocations-300x104.png\"){kind=tracking}
‘<\/a><\/p>\n