{"id":10221,"date":"2015-08-14T07:02:00","date_gmt":"2015-08-14T14:02:00","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/visualstudioalm\/2015\/08\/14\/intellitest-hands-on\/"},"modified":"2022-08-02T04:09:48","modified_gmt":"2022-08-02T12:09:48","slug":"intellitest-hands-on","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/devops\/intellitest-hands-on\/","title":{"rendered":"IntelliTest – hands on"},"content":{"rendered":"

In practical terms, white box unit test development includes an iterative workflow informed by code coverage – write a unit test, see what parts of the code are not covered by the test, write more tests to cover those parts, repeat until all of the code is covered \u2013 a workflow not different from what we would use while working with IntelliTest, as we will show in this demonstration with an application (https:\/\/github.com\/dylan-smith\/pokerleaguemanager<\/a>) that ALM MVP Dylan Smith<\/a> kindly permitted to use for the purpose.<\/p>\n

Step: install the application and build PokerLeagueManager.sln.<\/strong><\/p>\n

The application tracks stats for a weekly poker league.\u00a0There is a table that has the stats on each player (Games Played, Total Winnings, Total Profit, etc.).\u00a0The method we want to test is in: PokerLeagueManager.Queries.Core EventHandlers GetPlayerStatisticsHandler.cs Handle(GameDeletedEvent), as shown below:<\/p>\n

\"\"<\/p>\n

When a Game is deleted this method is responsible for updating the stats of the affected players. \u00a0It does this in steps:<\/p>\n

    \n
  1. Retrieve the list of Players that were in the deleted Game<\/li>\n
  2. For each player get their current stats<\/li>\n
  3. Update the stats object to reflect the necessary changes<\/li>\n
  4. Save the data<\/li>\n<\/ol>\n

    As with most real-world code, this code interacts with other objects and layers. Our goal with this demonstration is to enable IntelliTest reach 100% code coverage on the Handle method.<\/p>\n

    Understanding the warnings<\/span><\/strong><\/p>\n

    Step: \u201cRun IntelliTest\u201d on the Handle method.<\/strong><\/p>\n

    We see only a couple of tests generated and low coverage (6\/42 blocks), but also 5 warnings.<\/p>\n

    \"\"<\/p>\n

    Step: Click on the warnings button.<\/strong><\/p>\n

    \"\"<\/p>\n

    The first warning \u201cRuntime Warning\u201d says that IntelliTest has discovered, and will use, \u201cPokerLeagueManager.Queries.Core.QueryDataStore\u201d [PQCQ]<\/strong> as IQueryDataStore. Browsing through the code we see that IQueryDataStore is the type returned by the *getter *from the QueryDataStore property on the base class BaseHandler. In order to unit test this method, a concrete instantiation of this type is required \u2013 PQCQ is the instance IntelliTest has gone and discovered.<\/p>\n

    But is that the type you want to use?<\/em><\/p>\n

    Further, it has also discovered publicly accessible APIs though which to instantiate PQCQ (in this case that happens to be the public constructor). The APIs need to be publicly accessible because IntelliTest needs to actually call them to instantiate the type. So, the first of the Object Creation Warnings alerts us about the APIs that it discovered. If we\u00a0prefer, those calls can be persisted as a Factory method.<\/p>\n

    \"\"<\/p>\n

    When IntelliTest actually called them to instantiate PQCQ, it could not create an instance it could reason about. That is the second warning.<\/p>\n

    \"\"<\/p>\n

    As it turns out the constructor had ended up calling into some, as yet, uninstrumented code.<\/p>\n

    \"\"<\/p>\n

    On inspecting the stack trace\u00a0we can see a call from the constructor into DbContext code, where execution transitioned into uninstrumented code. IntelliTest works by instrumenting code and monitoring execution<\/a>. However, it does not instrument the entire universe of code for two reasons (1)<\/strong> it cannot know *a priori *what comprises that universe of code (2)\u00a0<\/strong>that would make the system very slow. So, that is why we see that \u201cuninstrumented method\u201d warning.<\/p>\n

    And by then, the number of branches in the code path that IntelliTest is exploring is so large, that it trips an internal bound that has been setup for fast interactive performance. Hence, it raises a warning and stops the exploration.<\/p>\n

    \"\"<\/p>\n

    Providing mock implementations<\/span><\/strong><\/p>\n

    To proceed further, we need to answer that first question: is that the type you want to use?<\/em> To unit test the method, we need to provide a *mock *implementation of IQueryDataStore. Browsing through the solution, we see a FakeQueryDataStore. Let\u2019s tell IntelliTest to use that (instead of the PQCQ that it discovered).<\/p>\n

    To start assisting IntelliTest like this we need the Parameterized Unit Test (PUT aka IntelliTest)<\/a>.<\/p>\n

    Step: Click on the Warnings button to toggle back to the tests. Select, and Save, all the tests, and then delete the .g.cs file (these did not cover all of the code, remember?).<\/strong><\/p>\n

    What we have in the GetPlayerStatisticsHandleTest.cs file is the PUT<\/a>.<\/p>\n

    Step: Add a reference to PokerLeagueManager.Common.Tests in the generated test project, and add the lines in red as shown to the PUT.<\/strong><\/p>\n

    \"\"<\/p>\n

    Step: Do a \u201cRun\u201d from the exploration Results window OR \u201cRun IntelliTest\u201d on the PUT (or on the Handle method).<\/strong><\/p>\n

    This time that bounds exceeded warnings is gone. We see 2 tests, but only 4 warnings this time.<\/p>\n

    \"\"<\/p>\n

    Focusing on \u2018just my code\u2019<\/span><\/strong><\/p>\n

    Step: Click on the warnings button.<\/strong><\/p>\n

    IntelliTest says it has discovered how to instantiate PokerLeagueManager.Common.Tests.FakeQueryDataStore [PCTF]<\/strong>, and alerts us about the APIs it can use to instantiate it (if we prefer, those calls can be persisted as a Factory method). That is the first warning.<\/p>\n

    But when it went ahead and called those APIs to instantiate PCTF, it ended up calling into uninstrumented code again. On inspecting the stack trace associated with these warnings,\u00a0we can see the calls where execution transitions into uninstrumented code – it is at the constructor and at the GetData methods.<\/p>\n

    Since we are not testing the mock, lets us suppress these warnings.<\/p>\n

    Step: Select the Object Creation warning and click on Suppress to suppress the warning.<\/strong><\/p>\n

    Note that all of the warnings related to uninstrumented method calls are coming from the same type [PCTF].<\/p>\n

    Step: Select any one of them click on Suppress to suppress all such warning coming from that type.<\/strong><\/p>\n

    We\u00a0should see this lines added in PexAssemblyInfo.cs<\/p>\n

    \"\"<\/p>\n

    **Step: Do a \u201cRun\u201d from the exploration Results window OR \u201cRun IntelliTest\u201d on the PUT (or on Handle method.). **<\/p>\n

    We should see those same 2 tests generated. This time there are no more warnings.<\/p>\n

    But we should remember to setup PCTF to be able to return data from calls to the GetData method! If not we will not be able to exercise the code-under-test further (note that we have covered only 15\/42 blocks).<\/p>\n

    Crafting the PUT<\/span><\/strong><\/p>\n

    In the PUT, \u2018target\u2019 is the object that contains data that will be returned by calls to GetData – in particular, T is either a LookupGamePlayersDto[] or a GetPlayerStatisticsDto[]. We need fill up PCTF with concrete instances of these. Let\u2019s ask IntelliTest to generate concrete instances of these.<\/p>\n

    Since IntelliTest can synthesize data values<\/a>, we will add this to the PUT\u2019s signature. We want 2 instances of LookupGamePlayersDto, and 1 instance of GetPlayerStatisticsDto. Further, we will associate the statistics for the first player.<\/p>\n

    Step: Add a reference to PokerLeagueManager.Common.DTO in the generated test project, and add the lines in red to as shown to the PUT (NOTE we are changing the signature of the PUT).<\/strong><\/p>\n

    \"\"<\/p>\n

    Next, we will prime the target with these:<\/p>\n

    \"\"<\/p>\n

    and, then we will exercise the code under test:<\/p>\n

    \"\"<\/p>\n

    at this point, we will simply query for the statistics and assert the observed value of its fields.<\/p>\n

    Add a using System.Linq<\/span> and then assert:<\/p>\n

    \"\"<\/p>\n

    Step: Since we changed the signature of the PUT, delete the g.cs file once again.<\/strong><\/p>\n

    Full code coverage<\/span><\/strong><\/p>\n

    Step: Do a \u201cRun\u201d from the exploration Results window OR \u201cRun IntelliTest\u201d on the IntelliTest (or on Handle method.).<\/strong><\/p>\n

    Now we should see all of the code covered (52\/52 blocks), and 3 passing tests, and 4 failing tests. And 7 warnings.<\/p>\n

    \"\"<\/p>\n

    Step: Click on the warnings button.<\/strong><\/p>\n

    Note that none of the warnings is related to the code under tests, and may therefore be suppressed.<\/p>\n

    \"\"<\/p>\n

    Step: Select each of them click on suppress.<\/strong><\/p>\n

    We should see this lines added in PexAssemblyInfo.cs:<\/p>\n

    \"\"<\/p>\n

    Step: Do a \u201cRun\u201d from the exploration Results window OR \u201cRun IntelliTest\u201d on the IntelliTest (or on Handle method.).<\/strong><\/p>\n

    Now we should see all of the code covered (52\/52 blocks), and 3 passing tests, and 4 failing tests. And 0 warnings.<\/p>\n

    \"\"<\/p>\n

    2 tests fails because they uncover a NulReferenceException; when \u2018e\u2019 is null (in that case e.AggregateId will raise the exception).<\/p>\n

    One of the test uncovers a DivideByZeroException. This will happen if stats.GamesPlayed has value of 1. In that case, the statement stats.GamesPlayed– will make it 0, and subsequently stats.Profit \/ stats.GamesPlayed will raise the exception.<\/p>\n

    One of the tests uncovers an OverflowException. For the following data values: stats.Profit = -2147483648 and stats.GamesPlayed = -1 the statement stats.ProfitPerGame = stats.Profit == 0 ? 0 : stats.Profit \/ stats.GamesPlayed; will raise the exception. We can see this from the details pane (the exact values seen may differ from those shown here).<\/p>\n

    IntelliTest has generated tests that uncover errors in the code. If we plug in more assertions about the expected behaviour then it will generate tests for validating that as well!<\/p>\n

    Q.E.D.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"

    In practical terms, white box unit test development includes an iterative workflow informed by code coverage – write a unit test, see what parts of the code are not covered by the test, write more tests to cover those parts, repeat until all of the code is covered \u2013 a workflow not different from what […]<\/p>\n","protected":false},"author":765,"featured_media":45953,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[226,1,252],"tags":[],"class_list":["post-10221","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ci","category-devops","category-testing"],"acf":[],"blog_post_summary":"

    In practical terms, white box unit test development includes an iterative workflow informed by code coverage – write a unit test, see what parts of the code are not covered by the test, write more tests to cover those parts, repeat until all of the code is covered \u2013 a workflow not different from what […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/posts\/10221","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/users\/765"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/comments?post=10221"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/posts\/10221\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/media\/45953"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/media?parent=10221"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/categories?post=10221"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/tags?post=10221"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}