{"id":1336,"date":"2020-06-15T09:00:18","date_gmt":"2020-06-15T16:00:18","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/cosmosdb\/?p=1336"},"modified":"2020-06-19T15:56:45","modified_gmt":"2020-06-19T22:56:45","slug":"new-string-function-performance-improvements-and-case-insensitive-search","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/cosmosdb\/new-string-function-performance-improvements-and-case-insensitive-search\/","title":{"rendered":"New string function performance improvements and case-insensitive search"},"content":{"rendered":"<p>Querying in Azure Cosmos DB just got even better! You now have an option for case-insensitive queries with the following string search system functions:<\/p>\n<ul>\n<li>Contains<\/li>\n<li>EndsWith<\/li>\n<li>StartsWith<\/li>\n<li>StringEquals<\/li>\n<\/ul>\n<p>Additionally, both Contains and EndsWith also have significant performance improvements. This update was rolled out to Azure Cosmos DB core (SQL) API accounts in our most recent service update. Customers using Azure Cosmos DB\u2019s API for MongoDB accounts can leverage the case-insensitive support and performance improvements through the <a href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/cosmos-db\/mongodb-feature-support-36#operators\">$regex<\/a> query evaluation operator.<\/p>\n<h2>Testing performance<\/h2>\n<p>Let\u2019s test the performance improvements by running some queries on a sample dataset with 8.5 million documents. I\u2019ve uploaded the dataset to a Cosmos container with 30,000 provisioned RU\u2019s.<\/p>\n<p>The dataset was generated using <a href=\"https:\/\/github.com\/bchavez\/Bogus\" target=\"_blank\" rel=\"noopener noreferrer\">Bogus <\/a>and contains a unique id identifier, a name, an address, a company, and a job:<\/p>\n<pre>{ \u00a0\u00a0 \r\n\"id\": \"ym570004cj\", \u00a0\u00a0 \r\n\"name\": \"Evans Kilback\", \u00a0\u00a0 \r\n\"address\": \"4561 Abernathy Station, Hahnmouth, Ecuador\",\r\n\"company\": \"Terry, Lind and Bogan\",\r\n\"job\": \"Administrator\" \r\n}<\/pre>\n<h3>Contains<\/h3>\n<p>Let\u2019s first do a search for all the people that live on a street named \u201cBrooks Street\u201d. We\u2019ll run the following query to check if the address property contains the word \u201cBrooks Street\u201d:<\/p>\n<pre>SELECT * FROM c \r\nWHERE CONTAINS(c.address, \"Brooks Street\")\r\n<\/pre>\n<p>The query returns 17 results with the following RU charges:<\/p>\n<p><strong>Original RU charge:<\/strong> 221,566.68 RUs<\/p>\n<p><strong>New RU charge:<\/strong> 224.99 RUs<\/p>\n<p>The performance improvement for <strong>Contains<\/strong>, gave this query over a 99% decrease in RU charge! Aside from indexing the property in the <strong>Contains<\/strong> system function, there are no other changes you need to make to see these improvements.<\/p>\n<p>&nbsp;<\/p>\n<h3>EndsWith<\/h3>\n<p>Let\u2019s look at an example with <strong>EndsWith<\/strong>. Here\u2019s a query that finds all the people that have a name that ends in \u201cLee\u201d:<\/p>\n<pre>SELECT * FROM c \r\nWHERE EndsWith (c.name, \"Lee\")<\/pre>\n<p>This query returns 0 results with the following RU charges:<\/p>\n<p><strong>Original RU charge:<\/strong> 198,649.32 RUs<\/p>\n<p><strong>New RU charge:<\/strong> 122.76 RUs<\/p>\n<p>&nbsp;<\/p>\n<h3>StartsWith<\/h3>\n<p>Additionally, you can take advantage of a new parameter in each of these system functions to get case-insensitive support. This parameter is optional and defaults to false when unspecified. The RU charge for Contains and EndsWith is the same regardless of whether they are case-insensitive.<\/p>\n<p>Here\u2019s an example with <strong>StartsWith<\/strong> to find the TOP 100 job titles that start with \u201cdeveloper\u201d whether the case matches or not:<\/p>\n<div>\n<pre>SELECT TOP 100 * FROM c\r\nWHERE StartsWith(c.job, \"developer\", true)<\/pre>\n<\/div>\n<p><strong>RU charge:<\/strong> 38.91 RUs<\/p>\n<p>&nbsp;<\/p>\n<h3>StringEquals<\/h3>\n<p>If you wanted to check for a full match, you could use <strong>StringEquals<\/strong>:<\/p>\n<div>\n<pre>SELECT TOP 100 * FROM c\r\nWHERE StringEquals(c.job, \"developer\", true)<\/pre>\n<\/div>\n<p><strong>RU charge:<\/strong> 38.89 RUs<\/p>\n<p>The RU charge for StartsWith and StringEquals is slightly higher with the case-insensitive option than without it. The case-insensitive option is available in the latest version of all Azure Cosmos DB SDK&#8217;s. For the .NET SDK, this is version 3.10 or later.<\/p>\n<p>In general, <strong>Contains<\/strong> and <strong>EndsWith<\/strong> will consume more RUs than <strong>StartsWith<\/strong> or <strong>StringEquals<\/strong>. The RU charge of <strong>Contains<\/strong> and <strong>EndsWith<\/strong> will increase as the cardinality of the property in the system function increases. Learn more about <a href=\"https:\/\/docs.microsoft.com\/azure\/cosmos-db\/sql-query-contains#remarks\" target=\"_blank\" rel=\"noopener noreferrer\"><strong>Contains<\/strong>\u00a0<\/a>and <a href=\"https:\/\/docs.microsoft.com\/azure\/cosmos-db\/sql-query-endswith#remarks\" target=\"_blank\" rel=\"noopener noreferrer\"><strong>EndsWith<\/strong><\/a> index utilization.<\/p>\n<h2>Next steps<\/h2>\n<p>We hope you try out these new query features! Here\u2019s a <a href=\"https:\/\/docs.microsoft.com\/learn\/modules\/access-data-with-cosmos-db-and-sql-api\/\" target=\"_blank\" rel=\"noopener noreferrer\">query lab<\/a>\u00a0with some sample data to get started. Beyond <a href=\"https:\/\/docs.microsoft.com\/azure\/cosmos-db\/index-policy#include-exclude-paths\" target=\"_blank\" rel=\"noopener noreferrer\">adding an index<\/a> for properties in the system functions, there is nothing else you need to do to leverage these significant improvements! If you are interested in additional text search functionality, you can leverage<a href=\"https:\/\/docs.microsoft.com\/azure\/search\/search-howto-index-cosmosdb\" target=\"_blank\" rel=\"noopener noreferrer\"> Azure Cosmos DB&#8217;s integration with Azure Cognitive Search<\/a>.<\/p>\n<p>&nbsp;<\/p>\n<p>If you have existing containers with these system functions, check out your <a href=\"https:\/\/docs.microsoft.com\/azure\/cosmos-db\/monitor-request-unit-usage#view-the-total-request-unit-usage-metric\" target=\"_blank\" rel=\"noopener noreferrer\">Request Unit (RU) consumption<\/a>. With these recent optimizations, your total RU consumption may have decreased and you may be able to decrease the amount of provisioned throughput on these containers.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Azure Cosmos DB now supports case-insensitive Contains, StartsWith, EndsWith, and StringEquals. Additionally, both Contains and EndsWith have had significant performance improvements. You can now store string data in Azure Cosmos DB without having to worry about matching the case in the query.<\/p>\n","protected":false},"author":12128,"featured_media":1455,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[12,14,818,19],"tags":[],"class_list":["post-1336","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-announcements","category-core-sql-api","category-query","category-tips-and-tricks"],"acf":[],"blog_post_summary":"<p>Azure Cosmos DB now supports case-insensitive Contains, StartsWith, EndsWith, and StringEquals. Additionally, both Contains and EndsWith have had significant performance improvements. You can now store string data in Azure Cosmos DB without having to worry about matching the case in the query.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/posts\/1336","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/users\/12128"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/comments?post=1336"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/posts\/1336\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/media\/1455"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/media?parent=1336"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/categories?post=1336"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/tags?post=1336"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}