{"id":3188,"date":"2021-07-29T08:58:26","date_gmt":"2021-07-29T15:58:26","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/cosmosdb\/?p=3188"},"modified":"2022-03-30T22:31:04","modified_gmt":"2022-03-31T05:31:04","slug":"always-encrypted","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/cosmosdb\/always-encrypted\/","title":{"rendered":"A tour of Always Encrypted for Azure Cosmos DB"},"content":{"rendered":"<p>Always encrypted for Azure Cosmos DB provides encryption support on the client-side to protect sensitive data by encrypting data such as credit card numbers or national identification numbers, before it gets transferred over the wire to be stored in Azure Cosmos DB. Always encrypted allows clients to protect JSON property values which are deemed sensitive, by means of client encryption policies which can be configured on the containers just like any other policy such as indexing policy etc.<\/p>\n<h2>How to start using always encrypted<\/h2>\n<p>Let us try to run the sample code. Encrypting data using always encrypted requires an initial setup as always encrypted relies on Azure Key Vault (when the <em>KeyEncryptionKeyResolver<\/em>\u00a0selected is Azure Key Vault &#8211; I will get to that in a bit) to store wrapped data encryption keys which are encrypted using a customer-provided key encryption key created (and stored) in Azure Key Vault. In this case, Azure Key Vault provides all the required services to wrap (Encrypt) and unwrap (Decrypt) the data encryption key in use.<\/p>\n<h4>Step 1: Enable client application access<\/h4>\n<p>When you have an application that connects to your Azure Cosmos DB instance, you can create an identity for the app. This identity is known as a service principal. Access to resources is restricted by the roles assigned to the service principal, giving you control over which resources can be accessed and at which level.<\/p>\n<p>Let us look at how to use the portal to create the service principal in the Azure portal. Let us focus on a single-tenant application where the application is intended to run within only one organization.<\/p>\n<ol>\n<li>Sign into your Azure Account through the\u00a0<a href=\"https:\/\/portal.azure.com\/\" target=\"_blank\" rel=\"noopener\">Azure portal<\/a>.<\/li>\n<li>Select Azure Active Directory\/App registrations.<\/li>\n<li>Select New registration.<\/li>\n<li>Name the application. Select a supported account type, which determines who can use the application.<\/li>\n<\/ol>\n<p>There you go, now you have your app registered.<\/p>\n<p><img decoding=\"async\" width=\"834\" height=\"752\" class=\"wp-image-3200\" src=\"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-text-application-descr-3.png\" alt=\"Graphical user interface, text, application Description automatically generated\" srcset=\"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-text-application-descr-3.png 834w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-text-application-descr-3-300x271.png 300w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-text-application-descr-3-768x692.png 768w\" sizes=\"(max-width: 834px) 100vw, 834px\" \/><\/p>\n<h4>Step 2: Get tenant and app ID values for signing in<\/h4>\n<p>You would need these two IDs to create an instance of the <em>ClientCertificateCredential<\/em> class (which is a token credential that can provide an OAuth Token) with the details needed to authenticate against Azure Active Directory with a certificate, when using certificate-based authentication. This is how Azure Key Vault would allow the set of operations like wrapping and unwrapping of data encryption keys. To get those values, use the following steps:<\/p>\n<ol>\n<li>Select\u00a0Azure Active Directory\/App registrations\u00a0in Azure AD, select your application.<\/li>\n<li>Copy the Directory (tenant) ID and store it in your application app settings.<\/li>\n<\/ol>\n<p><img decoding=\"async\" width=\"818\" height=\"262\" class=\"wp-image-3203\" src=\"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-text-application-email-5.png\" alt=\"Graphical user interface, text, application, email Description automatically generated\" srcset=\"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-text-application-email-5.png 818w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-text-application-email-5-300x96.png 300w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-text-application-email-5-768x246.png 768w\" sizes=\"(max-width: 818px) 100vw, 818px\" \/><\/p>\n<p><img decoding=\"async\" width=\"818\" height=\"264\" class=\"wp-image-3205\" src=\"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-text-application-email-7.png\" alt=\"Graphical user interface, text, application, email Description automatically generated\" srcset=\"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-text-application-email-7.png 818w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-text-application-email-7-300x97.png 300w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-text-application-email-7-768x248.png 768w\" sizes=\"(max-width: 818px) 100vw, 818px\" \/><\/p>\n<h4>Step 3: Authentication: Two options<\/h4>\n<p>Let&#8217;s look at couple of authentication methods available for service principals: password-based authentication (application secret) and certificate-based authentication. We recommend using a certificate so this is shown as part of the sample code, but you can also create an application secret to get a token via <em>ClientCredential<\/em><\/p>\n<h5>Option 1: Upload a certificate.<\/h5>\n<p>You can use an existing certificate if you have one. Optionally, you can create a self-signed certificate for\u00a0<em>testing purposes only<\/em>. To generate a certificate, you can use Azure Key Vault.<\/p>\n<h5>To generate the certificate using Azure Key Vault:<\/h5>\n<ol>\n<li>On the Key Vault properties pages, select\u00a0Certificates.<\/li>\n<li>Click on\u00a0Generate\/Import.<\/li>\n<li>On the\u00a0Create a certificate\u00a0screen choose the following values:<\/li>\n<\/ol>\n<ul>\n<li>Method of Certificate Creation: Generate.<\/li>\n<li>Certificate Name: ExampleCertificate.<\/li>\n<li>Subject: CN=ExampleDomain<\/li>\n<li>Leave the other values to their defaults.<\/li>\n<\/ul>\n<p>Click Create. This may take some time to create and enable. Once that is done and you have received the message that the certificate has been successfully created, you may click on it on the list. You can then see some of the properties.<\/p>\n<p><img decoding=\"async\" width=\"1381\" height=\"954\" class=\"wp-image-3207\" src=\"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-text-application-email-9.png\" alt=\"Graphical user interface, text, application, email Description automatically generated\" srcset=\"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-text-application-email-9.png 1381w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-text-application-email-9-300x207.png 300w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-text-application-email-9-1024x707.png 1024w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-text-application-email-9-768x531.png 768w\" sizes=\"(max-width: 1381px) 100vw, 1381px\" \/><\/p>\n<h5>Export\/Download certificate from Azure Key Vault:<\/h5>\n<p>By clicking &#8220;Download in PFX\/PEM format&#8221; button, you can download the certificate. Once you have downloaded it, you need to install it under the current user on the client machine which will run always encrypted Azure Cosmos DB client. (The steps to create an Azure Key Vault are provided in the later section).<\/p>\n<p><img decoding=\"async\" width=\"930\" height=\"161\" class=\"wp-image-3209\" src=\"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-text-description-automa-2.png\" alt=\"Graphical user interface, text Description automatically generated\" srcset=\"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-text-description-automa-2.png 930w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-text-description-automa-2-300x52.png 300w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-text-description-automa-2-768x133.png 768w\" sizes=\"(max-width: 930px) 100vw, 930px\" \/><\/p>\n<h6>To upload the certificate:<\/h6>\n<ol>\n<li>Select\u00a0Azure Active Directory\/App registrations\u00a0in Azure AD, select your application.<\/li>\n<li>Select\u00a0Certificates &amp; secrets.<\/li>\n<li>Select\u00a0Upload certificate\u00a0and select the certificate.<\/li>\n<\/ol>\n<p><img decoding=\"async\" width=\"562\" height=\"196\" class=\"wp-image-3211\" src=\"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-application-description-3.png\" alt=\"Graphical user interface, application Description automatically generated\" srcset=\"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-application-description-3.png 562w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-application-description-3-300x105.png 300w\" sizes=\"(max-width: 562px) 100vw, 562px\" \/><\/p>\n<h5>Option 2: Create a new application secret.<\/h5>\n<p>If you choose not to use a certificate, you can create a new application secret.<\/p>\n<ol>\n<li>Select\u00a0Azure Active Directory.<\/li>\n<li>From\u00a0App registrations\u00a0in Azure AD, select your application.<\/li>\n<li>Select\u00a0Certificates &amp; secrets.<\/li>\n<li>Select\u00a0Client secrets -&gt; New client secret.<\/li>\n<li>Provide a description of the secret, and a duration. When done, select\u00a0Add.<\/li>\n<\/ol>\n<p>After saving the client secret, the value of the client secret is displayed.<\/p>\n<p><img decoding=\"async\" width=\"818\" height=\"170\" class=\"wp-image-3213\" src=\"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-text-application-descr-6.png\" alt=\"Graphical user interface, text, application Description automatically generated\" srcset=\"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-text-application-descr-6.png 818w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-text-application-descr-6-300x62.png 300w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-text-application-descr-6-768x160.png 768w\" sizes=\"(max-width: 818px) 100vw, 818px\" \/><\/p>\n<h4>Step 5: Create an Azure Key Vault using the Azure portal<\/h4>\n<p>Sign into the Azure portal at\u00a0<a href=\"https:\/\/portal.azure.com\/\">https:\/\/portal.azure.com<\/a>.<\/p>\n<h5>Create a key vault:<\/h5>\n<ol>\n<li>From the Azure portal menu, or from the\u00a0Home\u00a0page, select\u00a0Create a resource.<\/li>\n<li>In the Search box, enter\u00a0Key Vault.<\/li>\n<li>From the results list, choose\u00a0Key Vault.<\/li>\n<li>On the Key Vault section, choose\u00a0Create.<\/li>\n<li>On the\u00a0Create key vault\u00a0section provide the following information:<\/li>\n<li>Name: A unique name is required let us use\u00a0example-vault.<\/li>\n<li>Subscription: Choose a subscription.<\/li>\n<li>Under\u00a0Resource Group, choose\u00a0Create new\u00a0and enter a resource group name.<\/li>\n<li>In the\u00a0Region\u00a0pull-down menu, choose a region.<\/li>\n<li>Leave the other options to their defaults.<\/li>\n<li>After providing the information above, select\u00a0Create.<\/li>\n<\/ol>\n<p><img decoding=\"async\" width=\"1379\" height=\"962\" class=\"wp-image-3215\" src=\"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-application-description-5.png\" alt=\"Graphical user interface, application Description automatically generated\" srcset=\"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-application-description-5.png 1379w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-application-description-5-300x209.png 300w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-application-description-5-1024x714.png 1024w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-application-description-5-768x536.png 768w\" sizes=\"(max-width: 1379px) 100vw, 1379px\" \/><\/p>\n<h4>Step 6: Configure access policies on resources<\/h4>\n<p>Now that your client app is configured and you have your application ID, it is time to configure its access policy so you and your application can access the vault&#8217;s secrets (key encryption keys). The\u00a0<strong><em>get<\/em>,\u00a0<em>list<\/em>,\u00a0<em>sign<\/em>,\u00a0<em>verify<\/em>,\u00a0<em>wrapKey<\/em>, and\u00a0<em>unwrapKey<\/em><\/strong>\u00a0permissions are required for creating a new data encryption key which is used by Always encrypted Azure Cosmos DB to encrypt the data.<\/p>\n<ol>\n<li>In the\u00a0<a href=\"https:\/\/portal.azure.com\/\" target=\"_blank\" rel=\"noopener\">Azure portal<\/a>, navigate to your key vault and select\u00a0Access policies.<\/li>\n<li>Select\u00a0Add access policy, then select the key, secret, and certificate permissions you want to grant your application. Select the service principal you created previously.<\/li>\n<li>Select\u00a0Add\u00a0to add the access policy, then\u00a0Save\u00a0to commit your changes.<\/li>\n<\/ol>\n<p><img decoding=\"async\" width=\"1099\" height=\"491\" class=\"wp-image-3217\" src=\"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-text-application-email-11.png\" alt=\"Graphical user interface, text, application, email Description automatically generated\" srcset=\"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-text-application-email-11.png 1099w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-text-application-email-11-300x134.png 300w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-text-application-email-11-1024x457.png 1024w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-text-application-email-11-768x343.png 768w\" sizes=\"(max-width: 1099px) 100vw, 1099px\" \/><\/p>\n<h4>Step 7: Add a key to the Key Vault<\/h4>\n<p>To add a key to the vault, you just need to take a couple of additional steps. In this case, we add a key that will be used by Always Encrypted for Azure Cosmos DB. The key is called <em>ExampleKey<\/em>.<\/p>\n<ol>\n<li>On the Key Vault properties pages, select\u00a0Keys.<\/li>\n<li>Click on\u00a0Generate\/Import.<\/li>\n<li>On the\u00a0Create a key\u00a0screen choose the following values:\n<ul>\n<li>Options: Generate.<\/li>\n<li>Name: <em>ExampleKey<\/em>.<\/li>\n<li>Leave the other values to their defaults. Click\u00a0Create.<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n<p>Once that you have received the message that the key has been successfully created, you may click on it on the list.<\/p>\n<p><img decoding=\"async\" width=\"1140\" height=\"861\" class=\"wp-image-3219\" src=\"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-text-application-descr-8.png\" alt=\"Graphical user interface, text, application Description automatically generated\" srcset=\"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-text-application-descr-8.png 1140w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-text-application-descr-8-300x227.png 300w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-text-application-descr-8-1024x773.png 1024w, https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2021\/07\/graphical-user-interface-text-application-descr-8-768x580.png 768w\" sizes=\"(max-width: 1140px) 100vw, 1140px\" \/><\/p>\n<p>Make sure you note the Key Identifier which will be later used in the sample code, where we would use it as part of <em>EncryptionWrapMetaData<\/em> while creating a new data encryption key.<\/p>\n<p>There you go, you have registered an application and provided the desired permission for the application in the Key Vault.<\/p>\n<h2>How to build applications using Always Encrypted for Azure Cosmos DB<\/h2>\n<p>With Always Encrypted, the Azure Cosmos DB SDK has now exposed a bunch of new APIs that can help configure few things like the container-level encryption policies and corresponding data encryption keys. This is required before you start using your regular Azure Cosmos DB SDK API like <em>CreateItemAsync<\/em>(), <em>ReadItemAsync<\/em>() etc. which have remained unchanged with respect to function signature.<\/p>\n<p>To begin with, you must initialize your <em>KeyEncryptionKeyResolver<\/em>\u00a0 interface with either Azure Key Vault based <em>KeyResolver<\/em> or you can implement your own <em>IkeyEncryptionKeyResolver <\/em>\u00a0interface if necessary. However, we recommend using the <em>KeyResolver<\/em>\u00a0which comes implemented out of the box and relies on Azure Key Vault as a key store to store your key encryption key (which is used to encrypt your data encryption keys) and provide services like wrapping and unwrapping.<\/p>\n<h3><strong>TokenCredential to access Azure Key Vault services<\/strong><\/h3>\n<p>The <em>TokenCredential<\/em> class is the representation of a credential which can provide OAuth token used to prove an identity between Azure Key Vault and a registered application via Azure Active Directory.<\/p>\n<p>So, every call we make to Azure Key Vault to wrap or unwrap a data encryption key is through a token obtained by authenticating with Azure AD (Active Directory). This token is in turn used by Azure Key Vault to validate the application identified with Azure AD.<\/p>\n<p>So, the setup with respect to registering your application with Azure AD and Azure KeyVault were carried out to enable this support and thus obtain the <em>TokenCredential<\/em> to use Key Vault services.<\/p>\n<h3><strong>Enable Container with Encryption Capabilities<\/strong><\/h3>\n<p><script src=\"https:\/\/gist.github.com\/kr-santosh\/3306192529c8a35e64efcbf5a9af660c.js\"><\/script>\nThe <em>WithEncryption<\/em>() extension on <em>CosmosClient<\/em> enables encryption on containers created using this client.<\/p>\n<p>KeyEncryptionKeyResolver, which is passed as a parameter, internally provides all the functionality around managing securely your data encryption keys.<\/p>\n<p><em>keyCacheTimeToLive<\/em> optional parameter allows a user to set the cache time to live value that determines how long the unwrapped data encryption keys are cached. After expiry, we refresh the cache by making a call to Azure Key Vault to unwrap the wrapped data encryption keys that are stored in Azure Cosmos DB. By default, <em>keyCacheTimeToLive<\/em> is set to 1 hour.<\/p>\n<p>To configure a <em>ClientEncryptionPolicy<\/em>, you need to first create a data encryption key which you will use to encrypt your document properties.<\/p>\n<p>The <em>EncryptionKeyWrapMetadata<\/em> requires the key encryption key URL, which was earlier noted during the setup. This key is used to encrypt\/wrap the data encryption key created via this call and stored securely in Azure Cosmos DB.<\/p>\n<p><script src=\"https:\/\/gist.github.com\/kr-santosh\/6e7cd92f0c693b7389d919e7ddceca50.js\"><\/script>\nA client Encryption Policy is configured by providing a <em>ClientEncryptionIncludedPath<\/em>, which specifies:<\/p>\n<ul>\n<li>the property you want to encrypt.<\/li>\n<li>the data encryption key you want the policy to use for this property.<\/li>\n<li>the encryption type which is either Deterministic or Randomized.<\/li>\n<li>the encryption algorithm to use.<\/li>\n<\/ul>\n<p>If you are looking to run queries on encrypted properties, then Deterministic encryption type must be selected. The keys must be created first, before you create a <em>ClientEncryptionPolicy<\/em> which uses the keys.<\/p>\n<p>Currently, only top-level paths are supported and if the top-level path is an Object or say an Array type, then all the corresponding nested\/child properties are encrypted with the same Client Encryption Policy.\n<script src=\"https:\/\/gist.github.com\/kr-santosh\/c29090c110bb2655c4b2edde79e1f017.js\"><\/script>\nYou now have a container configured with an encryption policy and all documents created inside this container will have the corresponding properties specified in the policy, encrypted.<\/p>\n<h3><strong>Key Encryption Key (Master Key) Rotation via RewrapClientEncryptionKeyAync<\/strong><\/h3>\n<p>The <em>RewrapClientEncryptionKeyAync<\/em>(\u2026) can be used to rewrap an existing data encryption key, which is useful when you want to rotate the Key Encryption Key. If you want to revoke the existing Key Encryption Key, this must be done after the rewrap operation has completed. For example, one way to do it would be to use the <em>QueryIterator<\/em> available over <em>ClientEncryptionKey<\/em> and run a query to fetch all the keys with their Azure Key Vault key encryption key \u201dvalue\u201d field. Or if you have configured a unique name, you can use the \u201cname\u201d field to fetch all the keys which were encrypted using this key encryption key and rewrap them with a new key.<\/p>\n<p><script src=\"https:\/\/gist.github.com\/kr-santosh\/aeb25b1c5aa5acdd4e25fc2aa90f5f4a.js\"><\/script><\/p>\n<h3><strong>Running queries on Encrypted Documents<\/strong><\/h3>\n<p>Running queries on encrypted documents requires the use of parameterized query using an <em>AddParameterAsync(&#8230;)<\/em> extension provided with <em>QueryDefinition(&#8230;)<\/em>.<\/p>\n<p>Since the containers configured with an encryption Policy have properties that are encrypted, you cannot run queries with a WHERE clause without the use of <em>AddParameterAsync(&#8230;)<\/em> extension since that would result in sending plaintext data and failing, since the property might be encrypted.<\/p>\n<p><script src=\"https:\/\/gist.github.com\/kr-santosh\/2e105099fbe770d82cc4c9314d6f461b.js\"><\/script><\/p>\n<p>The third argument passed above in <em>AddParameterAsync(&#8230;)<\/em> should be the property path that was used in the client encryption policy to encrypt the property.<\/p>\n<h2><strong>Are there any known limitations?<\/strong><\/h2>\n<ul>\n<li>Encryption Policy &#8211; Client encryption policies are immutable, once a container is configured with a client encryption policy, we do not allow changing the client encryption policy.<\/li>\n<li>Query Support &#8211; There are limitations with respect to what type of queries are supported on encrypted documents. Client-side encryption relies on the encryption policy configured on the container to decrypt the encrypted properties in the document, when a document is read, the policy is retrieved, and paths are iterated over and for each of the included path, the corresponding property values in the document are decrypted. However, in certain cases for example when queries are performed on container where the query seeks to retrieve only a subset of the document, say, a particular nested\/child property of document, the decryption would fail since the document returned does not have the parent property name(included top-level path) in the name-value pair objects of the document, hence the corresponding value is not decrypted. So if you want to query on encrypted documents, make sure to design your queries in such a way that the document retrieved as part of the query results does include the top-level path.<\/li>\n<\/ul>\n<p style=\"padding-left: 40px;\">For example. Let us consider this encrypted document. This includes a property &#8220;Items&#8221; with nested properties, and since &#8220;\/Items&#8221; was part of the client encryption policy the property along with all its nested properties, &#8220;OrderQty&#8221;,&#8221;ProductId&#8221;, &#8220;UnitPrice&#8221;, &#8220;LineTotal&#8221; were encrypted.<\/p>\n<pre class=\"prettyprint\" style=\"padding-left: 40px;\">{\r\n  \"id\": \"myorderId1\",\r\n  \"ponumber\": \"PO18009186470\",\r\n  \"ttl\": 2592000,\r\n  \"OrderDate\": \"BQGG49bnwZHqo\/90VQTCXsL2ArsBfLdILZI0BIxqz1CNm1rm5IftOyknMTcjbA==\",\r\n  \"ShippedDate\": \"0001-01-01T00:00:00\",\r\n  \"AccountNumber\": \"Account1\",\r\n  \"SubTotal\": \"AwHCJH\/7s0W2tDQnGBuSq2+mexJiWyA0pP9Nwo6j7mg\",\r\n  \"TaxAmount\": 12.5838,\r\n  \"Freight\": \"AwHTstIOb5fb0Jcn8Rle8FOETrqxqtRBrXPazIQ8vmVUR2MGNLhf38tKe08tJ3YQ\",\r\n  \"TotalDue\": 985.018,\r\n  \"Items\": [\r\n    {\r\n      \"OrderQty\": \"BAFHrzTqEotjFYoUcq9aphO\/efTFgY6IWydLCrejEFrQlnej6dKlRyLThm\",\r\n      \"ProductId\": \"BAEWqWPCRgsPiR5N+n5QmjTy71fvdPT1AN3FOXFVSyPLzim8ZUlgK\",\r\n      \"UnitPrice\": \"AwF8o5vUXV6w9KNnm3J18gA2S2r3uSWPoFnXgIa+6XZSE4FPlt25rWs\",\r\n      \"LineTotal\": \"BAFHrzTqEotjFYoUcq9aphO\/efTFgY6IWydLCrejEFrQlnej6dKlRyLThm\"\r\n    },\r\n    {\r\n      \"OrderQty\": \"BAFHrzTqEotjFYoUcq9aphO\/efTFgY6IWydLCrejEFrQlnej6dKlRyLThm\",\r\n      \"ProductId\": \"BAHZh0aWe4pC1IqfE3YxgZIYQJfYC59T+4CRlJYtR \/xjE+Bq3pMSr7vZ\",\r\n      \"UnitPrice\": \"AwF8o5vUXV6w9KNnm3J18gA2S2r3uSWPoFnXgIa+6XZSE4FPlt25rW\",\r\n      \"LineTotal\": \"BAFHrzTqEotjFYoUcqadsfaphasdf\/ef6IWydLCrejEFrQlnej6dKlRyLThm\"\r\n    }\r\n  ],\r\n}\r\n<\/pre>\n<p style=\"padding-left: 40px;\">Now lets say you decide to write a query to fetch just the &#8220;ProductId&#8221; which is a child property of &#8220;Items&#8221;. This query would fail since the document fetched does not contain the top-level path &#8220;\/Items&#8221; here. So in order to perform query on encrypted documented, the queries have to be designed in such a way that the query result has documents with top-level paths as part of the document schema.<\/p>\n<pre class=\"prettyprint\" style=\"padding-left: 40px;\">{\r\n  \"_rid\": \"ngwiAOGDqDM=\",\r\n  \"Documents\": [\r\n    {\r\n      \"ProductId\": \" BAEWqWPCRgsPiR5N+n5QmjTy71fvdPT1AN3FOXFVSyPLzim8ZUlgK \"\r\n    },\r\n    {\r\n      \"ProductId\": \" BAHZh0aWe4pC1IqfE3YxgZIYQJfYC59T+4CRlJYtR \/xjE+Bq3pMSr7vZ \"\r\n    },    \r\n  ],\r\n  \"_count\": 2 \r\n}<\/pre>\n<p style=\"padding-left: 40px;\">The other limitation is in regards to using LINQ in query, we currently only support parameterized queries on encrypted properties only using <em>AddParameterAsync(&#8230;)<\/em> extension provided with <em>QueryDefinition(&#8230;).<\/em><\/p>\n<h2><strong>Next steps<\/strong><\/h2>\n<p>Try client-side encryption for Azure Cosmos DB by following this <a href=\"https:\/\/github.com\/Azure\/azure-cosmos-dotnet-v3\/tree\/master\/Microsoft.Azure.Cosmos.Samples\/Usage\/Encryption\" target=\"_blank\" rel=\"noopener\">sample<\/a>.<\/p>\n<p>Use Azure Cosmos DB emulator version 2.11.13.0 or higher to use Always Encrypted.<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Always encrypted for Azure Cosmos DB provides encryption support on the client-side to protect sensitive data by encrypting data such as credit card numbers or national identification numbers, before it gets transferred over the wire to be stored in Azure Cosmos DB. Always encrypted allows clients to protect JSON property values which are deemed sensitive, [&hellip;]<\/p>\n","protected":false},"author":64397,"featured_media":3257,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[12,14,667],"tags":[1786,1787],"class_list":["post-3188","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-announcements","category-core-sql-api","category-security","tag-always-encrypted","tag-keyvault"],"acf":[],"blog_post_summary":"<p>Always encrypted for Azure Cosmos DB provides encryption support on the client-side to protect sensitive data by encrypting data such as credit card numbers or national identification numbers, before it gets transferred over the wire to be stored in Azure Cosmos DB. Always encrypted allows clients to protect JSON property values which are deemed sensitive, [&hellip;]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/posts\/3188","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/users\/64397"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/comments?post=3188"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/posts\/3188\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/media\/3257"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/media?parent=3188"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/categories?post=3188"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/tags?post=3188"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}