{"id":406,"date":"2017-01-05T13:27:46","date_gmt":"2017-01-05T05:27:46","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/seteplia\/?p=406"},"modified":"2017-01-05T13:27:46","modified_gmt":"2017-01-05T05:27:46","slug":"understanding-different-gc-modes-with-concurrency-visualizer","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/premier-developer\/understanding-different-gc-modes-with-concurrency-visualizer\/","title":{"rendered":"Understanding different GC modes with Concurrency Visualizer"},"content":{"rendered":"

In this post I\u2019m going to visualize what exactly happens during Garbage Collection (GC) and how different GC modes can significantly affect application performance. <\/p>\n

I assume that the reader is familiar with garbage collection basics. If this isn\u2019t the case I encourage you to spend 15 minutes to fill this gap, for instance from the following article \u2013 \u201cFundamentals of Garbage Collection\u201d<\/a> or from a chapter in your favorite book on C#\/.NET (*). <\/p>\n

The Garbage Collector in the CLR is a very complicated, configurable and self-tuning creature that may change behavior based on application needs. To satisfy different memory usage requirements the GC has some options to configure how it operates. There are two main modes: Workstation mode (designed to minimize delays) and Server mode (designed for maximum application throughput). The GC also supports one of two \u201csub-modes\u201d – concurrent or non-concurrent (**). <\/p>\n

Workstation GC vs. Server GC<\/h4>\n

Workstation GC is designed for desktop applications to minimize the time spent in GC. In this case GC will happen more frequently but with shorter pauses in application threads. Server GC is optimized for application throughput in favor of longer GC pauses. Memory consumption will be higher, but application can process greater volume of data without triggering garbage collection. <\/p>\n

All managed objects are stored in segments. There is one segment for young generations (called the ephemeral segment) and many segments for generation 2 and large object heap. When the ephemeral segment is full, CLR will allocate a new one. But before that, GC will happen. The size of the segment varies depending on whether a system is 32- or 64-bit, and on the type of the garbage collector. Workstation GC uses smaller segments and Server GC uses bigger segment, although the size depends on the number of CPU cores. Smaller the segments are more frequently GC will occur. Workstation GC is used by default in all managed apps and is best suited for UI applications. Server GC could be turned on by the CLR host or configured in <gcServer><\/a> element in the application configuration file and intended for server applications. <\/p>\n

GC flavors like \u2018concurrent\u2019 or \u2018non-concurrent\u2019 may help fine tune the garbage collection to gain maximum performance and\/or responsiveness for your application. Concurrent mode reduces the overall time spent in GC because the mark phase for 2nd<\/sup> generation happens in dedicated thread in parallel with application threads. In this mode, GC suspends user threads for shorter amount of time but will use slightly more memory. <\/p>\n

Concurrent Workstation GC is best suited for UI applications and non-concurrent Workstation GC should be used for lightweight server processes or for server apps on single-core machines. <\/p>\n

To visualize the GC, I\u2019ll be using a tool called Concurrency Visualizer, observing a simple console application. Concurrency Visualizer is a Visual Studio extension<\/a> that shows various threading aspects of the application, like lock contention, thread synchronization, input-output operations, GC pauses and other. The app is simply allocates byte arrays. Some arrays are kept in the internal lists and some of them are eligible for garbage collection immediately. <\/p>\n

Now, let\u2019s take a look at each mode in more details using Concurrency Visualizer<\/a>. <\/p>\n

Workstation GC: non-concurrent mode<\/h4>\n

There are a few reasons for GC to happen: Generation 0 is full or Gen0 budget is reached, GC.Collect<\/b> was called, or the system memory is low. We are only interested in the first option. <\/p>\n

Here is a very rough algorithm for workstation non-concurrent GC: <\/p>\n

    \n
  1. \n
    Application thread allocates an object and GC can\u2019t fulfill the request. GC is started.<\/div>\n<\/li>\n
  2. \n
    CLR suspends all managed threads.<\/div>\n<\/li>\n
  3. \n
    CLR collects the garbage in the thread that triggered the GC.<\/div>\n<\/li>\n
  4. \n
    CLR resumes all application threads once GC is done.<\/div>\n<\/li>\n<\/ol>\n

    For testing purposes, I\u2019m using a laptop with a Core i7 processor. The sample application is using 8 threads to do its job, but I will show fewer threads for the sake of simplicity. <\/p>\n

    Steps #1 and 2: CLR suspends all managed threads: <\/p>\n

    \"clip_image002[8]\"<\/a><\/p>\n

    Above, we see GC was triggered by thread 2948<\/b> and it waits for the CLR to suspend all managed threads. After that, the thread will collect the garbage and (as we will see in a moment) compact the heap. Note, heap compaction isn\u2019t happening for every GC. The CLR tries to maximize GC performance and compacts the heap only when garbage\/survivor ratio is high and compaction is useful. <\/p>\n

    Step #3: garbage collection: <\/p>\n

    \"clip_image004[8]\"<\/a><\/p>\n

    While GC is in progress, all managed threads are suspended waiting for GC: <\/p>\n

    \"clip_image006[8]\"<\/a><\/p>\n

    This example shows GC for Gen0, but in non-concurrent workstation GC, the process is the same for older generations as well. It just takes more time. <\/p>\n

    Now let\u2019s look at more sophisticated mode: concurrent Workstation GC. <\/p>\n

    Workstation GC: concurrent mode<\/h4>\n

    In concurrent (or background) mode, the CLR creates a dedicated high-priority thread for Gen2 collection<\/b>. In this case, the first phase of the garbage collection, mark phase, is happening in parallel with application threads. During this phase the application is still running, so user threads can allocate new objects and even trigger GC for young generations. <\/p>\n

    This is the main difference between old Concurrent GC available in pre .NET 4.0 era and the new Background GC. Concurrent GC also had a dedicated worker thread for Gen2 collection, but unlike Background GC if a user thread triggered GC, the thread was blocked while current GC is in progress. Background GC allows ephemeral collection in the middle of the background one. Background GC supersedes Concurrent GC and the same key is used to turn it on. In .NET 4.0+ there is no way to use Concurrent GC any more. <\/p>\n

    Here is how GC looks like for background Workstation GC: <\/p>\n

      \n
    1. \n
      Application thread allocates an object and GC can\u2019t fulfill the request. GC is started.<\/div>\n<\/li>\n
    2. \n
      CLR suspends all managed threads.<\/div>\n<\/li>\n
    3. \n
      CLR collects Gen0 and Gen1.<\/div>\n<\/li>\n
    4. \n
      CLR starts background collection and resumes all managed threads.<\/div>\n<\/li>\n
    5. \n
      Background thread marks all reachable objects in memory and suspends application threads for sweep or compact phase.<\/div>\n<\/li>\n
    6. \n
      CLR resumes all application threads once GC is done.<\/div>\n<\/li>\n<\/ol>\n

      This is a very basic description and set of steps could differ based on some heuristics, like the degree of heap fragmentation or if there are any GC requests during background collection. <\/p>\n

      In the following case GC was triggered by thread 12600<\/b>, and the thread waits till all the threads are suspended: <\/p>\n

      \"clip_image008[6]\"<\/a><\/p>\n

      Then the thread 12600<\/b> collects Gen0 and Gen1: <\/p>\n

      \"clip_image010[6]\"<\/a><\/p>\n

      Then GC starts background collection for Gen2: <\/p>\n

      \"clip_image012[6]\"<\/a><\/p>\n

      And thread 15972<\/b> starts background collection: <\/p>\n

      \"clip_image014[6]\"<\/a><\/p>\n

      After the mark phase, the background thread suspends the worker threads until GC is done: <\/p>\n

      \"clip_image016[6]\"<\/a><\/p>\n

      The background thread sweeps the heap: <\/p>\n

      \"clip_image018[6]\"<\/a><\/p>\n

      And releases free segments while application threads are running: <\/p>\n

      \"clip_image020[6]\"<\/a><\/p>\n

      Server GC<\/h4>\n

      Server GC has a few very important aspects that affect garbage collection: <\/p>\n

      1. Server GC is using bigger segments (few times bigger than for workstation GC). <\/p>\n

      2. The CLR creates 1 managed heap per core. This means that for an 8 core machine, the CLR will allocate 8 distinct<\/b> managed heaps. <\/p>\n

      3. GC happens in dedicated threads: one thread per managed heap. <\/p>\n

      Server GC trades memory in favor of throughput. Larger heaps mean that memory saturation happens less frequently, but once it happens, the CLR needs to do more work to traverse the heap. As a result, the application consumes more memory and GC will happen less frequently, but every GC will take longer period of time even for collecting Gen0 and Gen1. <\/p>\n

      To speed up the GC CLR is uses a dedicated high priority thread even for ephemeral collection. In the case of background GC the CLR will create yet another set of threads (one per core) for background analysis. Managed applications with background server GC will use 16 additional threads for an 8 core machine! <\/p>\n

      Now let\u2019s take a look at Server GC with Background mode. (Note:<\/b> Background Server GC is available only from .NET Framework 4.5). Because the number of threads is so high I\u2019ll show only a part of them. <\/p>\n

      The basic workflow for Background Server GC is as following: <\/p>\n

        \n
      1. \n
        Application thread allocates an object and GC can\u2019t fulfill the request. GC is started.<\/div>\n<\/li>\n
      2. \n
        CLR suspends all managed threads.<\/div>\n<\/li>\n
      3. \n
        CLR collects Gen0 and Gen1 in dedicated GC worker threads.<\/div>\n<\/li>\n
      4. \n
        CLR suspends GC worker threads and starts background collection. All the managed threads are resumed.<\/div>\n<\/li>\n
      5. \n
        Background threads mark all reachable objects in memory and suspend application threads for sweep or compact phase.<\/div>\n<\/li>\n
      6. \n
        CLR resumes GC worker threads to sweep the heap.<\/div>\n<\/li>\n
      7. \n
        Application threads wait for GC to finish.<\/div>\n<\/li>\n<\/ol>\n

        The following screenshots shows the 3 groups of threads: <\/p>\n

        \u00b7 First 4 threads are foreground GC threads responsible for collecting its own heap. <\/p>\n

        \u00b7 Second 4 threads are application worker threads. <\/p>\n

        \u00b7 Last 4 threads are dedicated for background GC. <\/p>\n

        The screenshot below shows that application threads are suspended waiting for foreground GC to finish: <\/p>\n

        \"clip_image022[6]\"<\/a><\/p>\n

        GC Worker threads are doing Gen0\/Gen1 collection: <\/p>\n

        \"clip_image024[6]\"<\/a><\/p>\n

        GC triggers a background collection: <\/p>\n

        \"clip_image026[6]\"<\/a><\/p>\n

        Then the CLR resumes GC worker threads to compact the heap:
        \"clip_image028[6]\"<\/a><\/p>\n

        Meanwhile application threads are blocked waiting for GC to finish: <\/p>\n

        \"clip_image030[6]\"<\/a><\/p>\n

        As you can see, background Server GC is more complicated that the workstation GC. It requires more resources and more complicated cross-thread collaboration. <\/p>\n

        Server GC vs. Workstation GC in a real application<\/h4>\n

        GC has significant effect on any performance critical managed application. Allocations are very cheap, but garbage collection is not. Different GC modes are more suitable for different kinds of apps; and a basic understanding of how GC works can help you pick the right mode. Just switching from one GC mode to another could increase end-to-end application performance significantly. <\/p>\n

        In my spare time I work on a Roslyn analyzer project called ErrorProne.NET<\/a>. The tool helps find some common errors in C# programs like invalid format strings or suspicious\/invalid exception handling. Like every analyzer, ErrorProne.NET could be integrated in Visual Studio but in some cases console mode (CLI, Command Line Interface) is more preferable. <\/p>\n

        To validate newly created rules and to check performance, I\u2019m constantly running ErrorProne.NET on different open-source projects, like StylecopAnalyzers<\/a> or the Roslyn<\/a> codebase itself. To do that I\u2019m using a console application that opens the solution, runs all the analyzers and prints a report in a human readable form. <\/p>\n

        By default every console application uses Background Workstation GC and recently I\u2019ve decided to check what will happen if I\u2019ll switch to Server GC. Here is what I\u2019ve got by running my app with different GC modes and \u2018Prefer 32bit\u2019 flag enabled. I\u2019ve used PerfView<\/a> to collect this information:<\/p>\n\n\n\n\n\n\n\n
        \n

        GC Mode<\/b><\/p>\n<\/td>\n

        		\n

        E2E time (ms)<\/b><\/p>\n<\/td>\n

        		\n

        Total GC Pause (ms)<\/b><\/p>\n<\/td>\n

        		\n

        % Time paused for GC<\/b><\/p>\n<\/td>\n

        		\n

        Gen0<\/b> Count<\/b><\/p>\n<\/td>\n

        		\n

        Gen1 Count<\/b><\/p>\n<\/td>\n

        		\n

        Gen2 Count<\/b><\/p>\n<\/td>\n

        		\n

        Total Allocations (Mb)<\/b><\/p>\n<\/td>\n

        		\n

        Max GC Heap Size (Mb)<\/b><\/p>\n<\/td>\n<\/tr>\n

        \n

        Workstation GC<\/p>\n<\/td>\n

        		\n

        132 765<\/p>\n<\/td>\n

        		\n

        46 118<\/p>\n<\/td>\n

        		\n

        35.1%<\/p>\n<\/td>\n

        		\n

        1674<\/p>\n<\/td>\n

        		\n

        1439<\/p>\n<\/td>\n

        		\n

        35<\/p>\n<\/td>\n

        		\n

        174 691<\/p>\n<\/td>\n

        		\n

        1 561<\/p>\n<\/td>\n<\/tr>\n

        \n

        Background Workstation GC<\/p>\n<\/td>\n

        		\n

        132 008<\/p>\n<\/td>\n

        		\n

        39 798<\/p>\n<\/td>\n

        		\n

        30.4%<\/p>\n<\/td>\n

        		\n

        2109<\/p>\n<\/td>\n

        		\n

        1451<\/p>\n<\/td>\n

        		\n

        65<\/p>\n<\/td>\n

        		\n

        225 554<\/p>\n<\/td>\n

        		\n

        1 676<\/p>\n<\/td>\n<\/tr>\n

        \n

        Server GC<\/p>\n<\/td>\n

        		\n

        102 553<\/p>\n<\/td>\n

        		\n

        9 026<\/p>\n<\/td>\n

        		\n

        9.1%<\/p>\n<\/td>\n

        		\n

        28<\/p>\n<\/td>\n

        		\n

        130<\/p>\n<\/td>\n

        		\n

        8<\/p>\n<\/td>\n

        		\n

        17 959<\/p>\n<\/td>\n

        		\n

        1 667<\/p>\n<\/td>\n<\/tr>\n

        \n

        Background Server GC<\/p>\n<\/td>\n

        		\n

        99 867<\/b><\/p>\n<\/td>\n

        		\n

        8 040<\/b><\/p>\n<\/td>\n

        		\n

        8.5%<\/b><\/p>\n<\/td>\n

        		\n

        23<\/p>\n<\/td>\n

        		\n

        148<\/p>\n<\/td>\n

        		\n

        9<\/p>\n<\/td>\n

        		\n

        16 610<\/p>\n<\/td>\n

        		\n

        1 724<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

          <\/p>\n

        This table clearly shows the huge difference between Server GC and Workstation GC for this application: just by switching from default Workstation GC to Server GC End-to-End time dropped by 30%! There are two reasons for this: number of managed heaps and segment size. Bigger segments allow allocating more objects in ephemeral segment and drastically reduces number of Gen0 and Gen1 collections: from 3500 to 200. Lower number of garbage collections significantly reduced the total allocations size (from 170Gb to 17Gb). <\/p>\n

        Another interesting data point is the number of GC pauses that took longer than 200ms and mean GC duration for different workstation GC flavors:<\/p>\n\n\n\n\n\n
        \n

        GC Mode<\/b><\/p>\n<\/td>\n

        		\n

        Number of GC pause > 200ms<\/b><\/p>\n<\/td>\n

        		\n

        Gen0 (ms)<\/b><\/p>\n<\/td>\n

        		\n

        Gen1 (ms)<\/b><\/p>\n<\/td>\n

        		\n

        Gen2 (ms)<\/b><\/p>\n<\/td>\n

        		\n

        All (ms)<\/b><\/p>\n<\/td>\n<\/tr>\n

        \n

        Workstation GC<\/p>\n<\/td>\n

        		\n

        11<\/p>\n<\/td>\n

        		\n

        6.8<\/p>\n<\/td>\n

        		\n

        16.2<\/p>\n<\/td>\n

        		\n

        322.6<\/p>\n<\/td>\n

        		\n

        14.7<\/p>\n<\/td>\n<\/tr>\n

        \n

        Background Workstation GC<\/p>\n<\/td>\n

        		\n

        2<\/p>\n<\/td>\n

        		\n

        7.9<\/p>\n<\/td>\n

        		\n

        15.4<\/p>\n<\/td>\n

        		\n

        12.7<\/p>\n<\/td>\n

        		\n

        11.0<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

         <\/p>\n

        The table shows that Background mode reasonably reduces amount of long GC pauses by reducing Gen2 collection time. <\/p>\n

        Conclusion<\/h4>\n

        Not everyone is working on high performance managed applications. In many cases GC can efficiently do its job without any human intervention. Application performance is unaffected and the main concern of the developer lies elsewhere. <\/p>\n

        But this isn\u2019t always the case. <\/p>\n

        Many of us are working on system-level or high performance software written in C# like games, database servers or web-servers with huge load. In this case, good understanding of what GC does is crucial. It is very important to understand the behavior for different GC modes, what a memory segment is, and why a GC pause in one case could be way higher than in another. <\/p>\n

        I hope this post helped you to build a mental model in your head for the various GC modes and gave enough information to take GC seriously. <\/p>\n

        Additional resources<\/h4>\n