A few years back, I started an internal Microsoft project called R9. Our\ngoal was to create a robust SDK to accelerate the development of\nreliable high-scale services on .NET. Hundreds of microservices within\nMicrosoft now leverage this SDK and happily many of the technologies we\ncreated have been folded into .NET 8 for the benefit of the entire .NET\necosystem.<\/p>\n
We knew starting the R9 project that asking service teams to adopt\nour technology would be challenging. Teams have a very high quality bar\nand if we didn’t meet that bar, it might\nbe a long time before they gave us another look, if ever. We therefore\nput a strong emphasis on quality, establishing a target of 100% code\ncoverage, embracing mutation testing<\/a>, and\nadopting a zero-bug policy.<\/p>\n Since the R9 components generate a fair bit of telemetry, we decided it\nwas critical for us to test the quality of our telemetry. Service teams\ndepend on having the right signals coming from their applications to be\nable to successfully deploy, monitor, and repair their services. We\nneeded to make sure our telemetry was part of the solution, not the\nproblem.<\/p>\n At first, we were timidly using generated mocks\nto capture telemetry from our components in ad hoc ways. Although this\ntechnically worked, it was ugly. Using this approach usually requires tests\nto assume the internal structure of the components being tested, which\nmakes tests brittle as they become susceptible to break when the internal\nstructure of a component is refactored. Since it was clumsy to write, we\nended up with few tests validating telemetry throughout our code base.<\/p>\n After a while, it became clear that to achieve our quality objectives,\nwe would need to improve how telemetry is tested, which led me to\ncreate the logging and metric fakes. The model was successful as R9\ndevelopers quickly embraced it and the quality of our telemetry tests\nimproved dramatically. The fakes model caught on within the team, we\ncreated several other fakes to help improve both our tests and our\ncustomer\u2019s tests.<\/p>\n Anybody that\u2019s been part of a large software engineering team knows the\ninherent problems caused by flaky tests. Flaky tests are those that\ngenerally work, but occasionally trigger a false negative. If you have\nthousands of test cases, a handful of flaky ones can make it difficult\nto have a fully successful test run, which in turn makes it difficult to\nknow if changes you\u2019re making to the code base are causing test\nfailures, or it\u2019s merely a flaky test doing mischief.<\/p>\n Most flaky tests are caused by some sort of race condition, and often\nthese races exist purely in the tests and not in the software under\ntest. This happens because the environment in which software is tested\nis often hacked together, being different than how the software is\nnormally used.<\/p>\n To help tame the race conditions, I decided to create an At some point, we searched through a bunch of internal Microsoft code\nbases and found literally hundreds of variations of .NET 8 introduces several fakes to simplify test authoring. In this\narticle, I\u2019ll describe the logging fake, the metering fake, and the time\nprovider fake. We don\u2019t have the space to cover those here, but be on\nthe lookout for other useful fakes such as the fake host, fake\nredactor, and fake data taxonomy.<\/p>\n The idea behind the logging fake is to use a custom implementation of\n You’ll find the logging fake in the Microsoft.Extensions.Diagnostics.Testing<\/a> package.<\/p>\n Getting started is easy, you can just create a Sometimes, code wants an Once you\u2019ve got a logger provider or logger wired-in, anything logged\nwill be captured in a Once you have the Check the Call Use the The most common pattern is to check that The metric fake uses a similar model of capturing all the reported\nmetric updates and making it easy for you to inspect the resulting\nstate. Since metrics integrate very differently into .NET than logging\ndoes, the fake model is also different and simpler.<\/p>\n You’ll find the metering fake in the Microsoft.Extensions.Diagnostics.Testing<\/a> package.<\/p>\n The There are several different constructors for the Here\u2019s an example showing how to use the The The really useful aspect of If you provide a fake time provider in all those places, you can\ncompletely control how time behaves in those APIs. So instead of having\nrace conditions all over your test suite, you can have fully\ndeterministic tests that execute much faster than before (since you can\navoid the nasty delays that typically get added to test suites to work\naround the race conditions).<\/p>\n The fake time provider is quite different than the logging and metric\nfakes we just discussed. Instead of recording events, the fake time\nprovider is there as a manually controlled clock. Whereas the system\nprovider, which you get from You’ll find the time provider fake in the Microsoft.Extensions.TimeProvider.Testing<\/a> package.<\/p>\nWhy we Started Faking It<\/h2>\n
Time Is a Problem<\/h2>\n
IClock<\/code>\nabstraction. We\u2019d use this throughout our SDK to do anything time\nrelated. IClock<\/code> let us use a fake clock in our tests to completely\ncontrol the passage of time. Once deployed and used, this dramatically\nreduced the number of flaky tests in our code base.<\/p>\nIClock<\/code> and\nassociated types to use in testing. Clearly, this was a common problem\nthat deserved some attention. This work ultimately led to the\nintroduction of the TimeProvider<\/code><\/a>\ntype in .NET 8.<\/p>\nFakes in .NET 8<\/h2>\n
Logging Fake<\/h2>\n
ILogger<\/code><\/a>\nwhose job is to capture and accumulate all state being logged in\nan in-memory buffer such that it can be inspected from a test suite.<\/p>\nFakeLogger<\/code><\/a>\ninstance and pass it to any code that is asking for an ILogger<\/code> or ILogger<T><\/code>:<\/p>\n[Fact]\npublic static void TestLogging()\n{\n \/\/ a plain logger\n var logger1 = new FakeLogger();\n var c1 = new ClassToTest1(logger1);\n\n \/\/ a logger with a specific logging category\n var logger2 = new FakeLogger<ClassToTest2>();\n var c2 = new ClassToTest2(logger2);\n}<\/code><\/pre>\nILoggerProvider<\/code> out of which it creates its own\nlogger. For those cases, you can use the FakeLoggerProvider<\/code><\/a>\ntype either directly or through dependency injection.<\/p>\nFakeLogCollector<\/code> object. If you manually created\nthe logger, you can get the collector via the FakeLogger.Collector<\/code>\nproperty. If you used FakeLoggerProvider<\/code> instead and you don\u2019t have\nimmediate access to the logger object in your code, you can then use the\nGetFakeLogCollector<\/code> method to extract the FakeLogCollector<\/code><\/a>\nused by the logger instances created by dependency injection.<\/p>\nFakeLogCollector<\/code> object, you can do one of these\nthings:<\/p>\n\n
Count<\/code> property to make sure the expected number of log\nrecords were produced.<\/p>\n<\/li>\nGetSnapshot<\/code> to get a copy of all the log records collected in the\ntest case.<\/p>\n<\/li>\nLatestRecord<\/code> property to get access to the last log record\nthat was collected.<\/p>\n<\/li>\n<\/ul>\nCount<\/code> equals 1, and then\nuse LatestRecord<\/code> to ensure the logged data is correct. The FakeLogRecord<\/code>\nclass contains properties to let you access all the state captured with\nevery log record including the log level (debug, warning, error, etc),\nthe category, the message, and the full set of name\/value tags.<\/p>\n[Fact]\npublic static void TestLogging()\n{\n \/\/ create a fake logger for this test\n var logger = new FakeLogger();\n\n \/\/ pass the fake logger in to the code being tested\n var c = new ClassToTest1(logger);\n\n \/\/ trigger the code to do some work\n c.DoSomethingThatFails();\n\n \/\/ verify that the DoSomethingThatFails method has logged what it should have\n\n \/\/ only one log record\n Assert.Equal(1, logger.Collector.Count); \n\n var r = logger.LatestRecord;\n\n \/\/ should be logged as an error\n Assert.Equal(LogLevel.Error, r.Level);\n\n \/\/ the message should include the text \u201cCould not do something\u201d\n Assert.Contains(\"Could not do something\", r.Message);\n\n \/\/ there should be a \u201ccause\u201d tag with the value \u201cno_support\u201d\n Assert.Contains(r.StructuredState, tag => tag.Key == \"cause\" && tag.Value == \"no_support\"); \n}<\/code><\/pre>\nMetric Fake<\/h2>\n
MetricCollector<T><\/code><\/a> object lets you record all updates to a metric\ninstrument or observable instrument. Once you\u2019ve created the collector,\nyou can use the LastMeasurement<\/code> property to query the last update to the\ninstrument or call GetMeasurementSnapshot<\/code> to get a list of all the\nmeasurements captured for the instrument.<\/p>\nMetricCollector<\/code> class,\neach letting you specify the specific instrument to collect from in a\ndifferent way, which tailors to most use cases.<\/p>\nMetricCollector<\/code> type:<\/p>\npublic sealed class ClassToTest : IDisposable\n{\n private const string CounterName = \"MyCounter\";\n private readonly Meter _meter;\n\n public ClassToTest()\n {\n _meter = new Meter(Guid.NewGuid().ToString());\n Counter = _meter.CreateCounter<long>(CounterName);\n }\n\n public void Dispose() => _meter.Dispose();\n\n public void DoWork(string workType, int unitsOfWork)\n {\n Counter.Add(unitsOfWork,\n new KeyValuePair<string, object?>[]\n {\n new(\"workType\", workType),\n });\n }\n\n internal Counter<long> Counter { get; }\n}\n\n[Fact]\npublic static void TestMetering()\n{\n const string WorkType = \"Bootstrap\";\n const int WorkUnits = 3;\n\n using var classToTest = new ClassToTest();\n using var collector = new MetricCollector<long>(classToTest.Counter);\n\n \/\/ show the collector has no state\n Assert.Empty(collector.GetMeasurementSnapshot());\n Assert.Null(collector.LastMeasurement);\n\n \/\/ now do some work that should update the counter\n classToTest.DoWork(WorkType, WorkUnits);\n\n \/\/ now verify the right telemetry was updated\n var snap = collector.GetMeasurementSnapshot();\n Assert.Equal(1, snap.Count);\n\n \/\/ get the sole measurement taken\n var m = snap[0];\n\n \/\/ make sure the value is what it should be\n Assert.Equal(WorkUnits, m.Value);\n\n \/\/ make sure the tags are exactly what they should be\n Assert.True(m.MatchesTags(new KeyValuePair<string, object?>(\"workType\", WorkType)));\n\n \/\/ make sure the exact set of tags are present, regardless of the specific values they carry\n Assert.True(m.MatchesTags(\"workType\"));\n\n \/\/ make sure at least the given tags are present\n Assert.True(m.MatchesTags(new KeyValuePair<string, object?>(\"workType\", WorkType)));\n\n \/\/ make sure at least the given tags are present, regardless of the specific values they carry\n Assert.True(m.ContainsTags(\"workType\"));\n}<\/code><\/pre>\nTimeProvider Fake<\/h2>\n
TimeProvider<\/code><\/a> type is new to .NET 8 and it provides a simple set of\nmethods to get the current time, get the current time zone, and create\ntimers. This is like functionality already present in .NET, except for\nthe fact the methods are not static. Instead, they are instance methods\non a type that\u2019s designed to have both real and fake implementations.<\/p>\nTimeProvider<\/code> is that it\u2019s been integrated\ninto core functionality of the framework. For example, there are now\nconstructors for CancellationTokenSource<\/code> and PeriodicTimer<\/code> which take a\nprovider, Task.WaitAsync<\/code> and Task.Delay<\/code> now have overloads that take a\nprovider.<\/p>\nTimeProvider.System<\/code>, tracks real-world\ntime, the apparent passage of time is instead entirely controlled by the\nfake time provider.<\/p>\n