{"id":27516,"date":"2018-09-27T05:54:51","date_gmt":"2018-09-27T12:54:51","guid":{"rendered":"http:\/\/devblogs.microsoft.com\/premier-developer\/?p=27516"},"modified":"2019-02-14T20:17:50","modified_gmt":"2019-02-15T03:17:50","slug":"cosmos-db-emulators-can-also-run-in-the-cloud","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/premier-developer\/cosmos-db-emulators-can-also-run-in-the-cloud\/","title":{"rendered":"Cosmos DB Emulators can also run in the Cloud"},"content":{"rendered":"<p>Dev Consultant <a href=\"https:\/\/www.linkedin.com\/in\/julienoudot\/\">Julien Oudot<\/a> demonstrates how to automate the testing of a data layer in ASP.NET Core WEB API with Cosmos DB.<\/p>\n<hr \/>\n<p>The intent of this article is to describe how to automate the testing of a data layer in an ASP.NET Core WEB API relying on Cosmos DB.<\/p>\n<p>Azure Cosmos DB is Microsoft&#8217;s globally distributed, multi-model database. Sometimes referred to as a server-less database, the promise is the ability to be able to transparently and indefinitely scale your data with high throughput, low latency and high reliability.<\/p>\n<p>To achieve this goal, an efficient design is needed along with a strong monitoring and testing strategy to give users the confidence that their systems perform as expected.<\/p>\n<p>To efficiently test a web API controller relying on Cosmos DB, three approaches are usually observed:<\/p>\n<ul>\n<li>Mock classes responsible for the interactions with the Cosmos DB back-end. This option is efficient to unit test the controller logic but requires implementing mocks. Furthermore, it does not test the interactions with the back-end.<\/li>\n<li>Spin up a test container in the Azure Cloud (we will only consider collection in this article). With this option, users need to provision their Cosmos DB environment and collection on the flight in their Azure subscription. It is closer to reality, since users run their test cases against a full fledge version of Cosmos DB.<\/li>\n<li>Rely on the Cosmos DB emulator hosted in the Build agent. This approach avoids spinning up a true Cosmos DB collection and therefore, having to pay for its provisioning. Furthermore, it does not require much effort aside from setting up the emulator. This is on this strategy that the present article focuses on.<\/li>\n<\/ul>\n<p>Lately, the Cosmos DB team published a new <a href=\"https:\/\/marketplace.visualstudio.com\/items?itemName=azure-cosmosdb.emulator-public-preview\">VSTS task<\/a> that can automatically create an instance of the Cosmos DB emulator, hosted in a Windows Container and spun up on the Build machine. Then, users can parametrize their test projects to use this emulator instance instead of an actual Cosmos DB environment. This task can be included both in the Build or Release processes.<\/p>\n<p><b><u>Setting up the code repository and Build process: <\/u><\/b><\/p>\n<p>First, an ASP.NET Core Web API application hosted in VSTS is needed. If you want to reuse an existing application, you can clone the following repository and then push it to a VSTS repository as explained here: <a href=\"https:\/\/github.com\/joudot\/CosmosDB-EmulatorTesting-VSTS\">https:\/\/github.com\/joudot\/CosmosDB-EmulatorTesting-VSTS<\/a><\/p>\n<p>Then, an existing VSTS environment is also required in order to push the code to the repository.<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/31\/2019\/04\/cosmos1.png\"><img decoding=\"async\" title=\"cosmos1\" src=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/31\/2019\/04\/cosmos1_thumb.png\" alt=\"cosmos1\" width=\"1028\" height=\"348\" border=\"0\" \/><\/a><\/p>\n<p>Finally, create a new Build process, click on <b>Build<\/b> <b>\u2013 New<\/b>, and then choose ASP.NET Core<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/31\/2019\/04\/cosmos2.png\"><img decoding=\"async\" title=\"cosmos2\" src=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/31\/2019\/04\/cosmos2_thumb.png\" alt=\"cosmos2\" width=\"1028\" height=\"333\" border=\"0\" \/><\/a><\/p>\n<p>It will come with the following steps to build, test and publish an ASP.NET Core application (the Restore step can be removed from the following list):<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/31\/2019\/04\/cosmos3.png\"><img decoding=\"async\" title=\"cosmos3\" src=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/31\/2019\/04\/cosmos3_thumb.png\" alt=\"cosmos3\" width=\"1028\" height=\"428\" border=\"0\" \/><\/a><\/p>\n<p>When we add the task to run the Cosmos DB emulator in VSTS, it will start the emulator in a Windows Container. That is the reason why we need to choose Hosted VS2017 which is a Windows agent.<\/p>\n<p><b><u>Create the test project and configure it to point to the emulator URL <\/u><\/b><\/p>\n<p>To be able to test this controller, we first need a test project. <a href=\"https:\/\/github.com\/joudot\/CosmosDB-EmulatorTesting-VSTS\/tree\/master\/CosmosDbClient\/CosmosDbTests\">This<\/a> Xunit project instantiates a Controller and run some tests.<\/p>\n<p>It also contains a <a href=\"https:\/\/github.com\/joudot\/CosmosDB-EmulatorTesting-VSTS\/blob\/master\/CosmosDbClient\/CosmosDbTests\/appsettings.json\">configuration file<\/a> with the Cosmos DB configuration used to connect and run our test against the Cosmos DB data store.<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/31\/2019\/04\/cosmos4.png\"><img decoding=\"async\" title=\"cosmos4\" src=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/31\/2019\/04\/cosmos4_thumb.png\" alt=\"cosmos4\" width=\"1028\" height=\"626\" border=\"0\" \/><\/a><\/p>\n<p>Note that the authorization key is the default one used by the <a href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/cosmos-db\/local-emulator#authenticating-requests\">emulator<\/a>. The endpoint URL will be overridden on the fly by the VSTS Build pipeline. For local testing, these fields could be overridden to point to a local emulator or to an existing Cosmos DB environment.<\/p>\n<p><b><u>Create the emulator instance in the Build process <\/u><\/b><\/p>\n<p>Now, we need to go back to the VSTS Build definition and add the new task: \u201cAzure Cosmos DB Emulator\u201d. Request the task and add it before the Test step.<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/31\/2019\/04\/cosmos5.png\"><img decoding=\"async\" title=\"cosmos5\" src=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/31\/2019\/04\/cosmos5_thumb.png\" alt=\"cosmos5\" width=\"1028\" height=\"298\" border=\"0\" \/><\/a><\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/31\/2019\/04\/cosmos6.png\"><img decoding=\"async\" title=\"cosmos6\" src=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/31\/2019\/04\/cosmos6_thumb.png\" alt=\"cosmos6\" width=\"995\" height=\"772\" border=\"0\" \/><\/a><\/p>\n<p>This is a task that will spin up a container, hosting the emulator instance. In the next step, we will override the configuration, so that the test project connects to the Cosmos DB collection hosted in the emulator.<\/p>\n<p><b><u>Update the configuration to connect to the Cosmos DB Emulator<\/u><\/b><\/p>\n<p>When running the test cases from developer machines, the string <b>CosmosEndpointUrl<\/b> as well as the authorization key contained in the <a href=\"https:\/\/github.com\/joudot\/CosmosDB-EmulatorTesting-VSTS\/blob\/master\/CosmosDbClient\/CosmosDbTests\/appsettings.json\">application settings file<\/a> can be replaced by actual connection credentials. These credentials can be found in the <b>Settings<\/b> \u2013 <b>Key<\/b> section of the Cosmos DB environment in Azure. However, in the context of the Build process, we need to automatically override the endpoint URL value, since it will depend on the container id started by the VSTS task under the cover. This id will be different for each run.<\/p>\n<p>Add a new PowerShell Task and put it between the Azure Cosmos DB Emulator and the Test steps. Select <b>Inline<\/b> as a type. Then, you need to replace the <b>CosmosEndpointUrl<\/b> string by the endpoint value that you can find in the output variable <b>CosmosDbEmulator.Endpoint<\/b>:<\/p>\n<p>Script:<\/p>\n<p>(Get-Content .\\CosmosDbClient\\CosmosDbTests\\appsettings.json) -Replace &#8216;CosmosEndpointUrl&#8217;, &#8216;$(CosmosDbEmulator.Endpoint)&#8217; | Set-Content .\\CosmosDbClient\\CosmosDbTests\\appsettings.json<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/31\/2019\/04\/cosmos7.png\"><img decoding=\"async\" title=\"cosmos7\" src=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/31\/2019\/04\/cosmos7_thumb.png\" alt=\"cosmos7\" width=\"1028\" height=\"196\" border=\"0\" \/><\/a><\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/31\/2019\/04\/cosmos8.png\"><img decoding=\"async\" title=\"cosmos8\" src=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/31\/2019\/04\/cosmos8_thumb.png\" alt=\"cosmos8\" width=\"1028\" height=\"458\" border=\"0\" \/><\/a><\/p>\n<p><b><u>Initialize the Cosmos DB Collection from the test project<\/u><\/b><\/p>\n<p>Because the emulated Cosmos DB environment created by the task won\u2019t contain any collection upfront, the collection initialization steps need to be implemented as part of the test project (or could also be isolated in an VSTS task using the <a href=\"https:\/\/docs.microsoft.com\/en-us\/cli\/azure\/cosmosdb\/collection?view=azure-cli-latest\">Azure Command Line Interface<\/a>). To do this only once for all tests, we can rely on the concept of fixture available in XUnit: <a href=\"https:\/\/github.com\/joudot\/CosmosDB-EmulatorTesting-VSTS\/blob\/master\/CosmosDbClient\/CosmosDbTests\/CosmosDBFixture.cs\">https:\/\/github.com\/joudot\/CosmosDB-EmulatorTesting-VSTS\/blob\/master\/CosmosDbClient\/CosmosDbTests\/CosmosDBFixture.cs<\/a><\/p>\n<p>The method CreateReviewCollectionAndInitializeIfNotExistsAsync creates the collection and uploads the documents expected to be in the collection by the test scenarios. The advantage is that there is no need to think about cleaning up the data once the tests are executed because a brand-new collection will be created by the following Build iterations.<\/p>\n<p>The last step is to go to the <b>Publish<\/b> step and uncheck <b>Publish Web Projects<\/b>. This will publish all projects contained in the code repository. By default, this step is going to look for a web project at the root of the repository which is not our case (the project is in a folder).<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/31\/2019\/04\/cosmos9.png\"><img decoding=\"async\" title=\"cosmos9\" src=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/31\/2019\/04\/cosmos9_thumb.png\" alt=\"cosmos9\" width=\"1028\" height=\"471\" border=\"0\" \/><\/a><\/p>\n<p>Now we can queue a new build and check that all steps are successfully executed.<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/31\/2019\/04\/cosmos10.png\"><img decoding=\"async\" title=\"cosmos10\" src=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/31\/2019\/04\/cosmos10_thumb.png\" alt=\"cosmos10\" width=\"1028\" height=\"598\" border=\"0\" \/><\/a><\/p>\n<p><b><u>Final thoughts<\/u><\/b><\/p>\n<p>It takes some time to pull the container image and start it. That is why, it might be more appropriate to use this testing strategy in the context of a Release Definition instead of doing this at Build time. Using private agents would mitigate this, since there would be <a href=\"https:\/\/docs.docker.com\/develop\/develop-images\/dockerfile_best-practices\/#leverage-build-cache\">some caching on the machine<\/a> to ensure that we don\u2019t always pull the entire image.<\/p>\n<p>Overall, thanks to this VSTS task, we will be more confident that there is no regression affecting the interactions with the Cosmos DB data layer, since it will always be tested against the containerized version of the Cosmos DB emulator.<\/p>\n<p>And the beauty of this does not require to create (and pay for) a new Cosmos DB collection every time we want to run the tests.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Dev Consultant Julien Oudot demonstrates how to automate the testing of a data layer in ASP.NET Core WEB API with Cosmos DB.<\/p>\n","protected":false},"author":582,"featured_media":27517,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[25,8,22],"tags":[83,186,3],"class_list":["post-27516","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-azure","category-data","category-devops","tag-net-core","tag-cosmosdb","tag-team"],"acf":[],"blog_post_summary":"<p>Dev Consultant Julien Oudot demonstrates how to automate the testing of a data layer in ASP.NET Core WEB API with Cosmos DB.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/posts\/27516","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/users\/582"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/comments?post=27516"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/posts\/27516\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/media\/27517"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/media?parent=27516"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/categories?post=27516"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/tags?post=27516"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}