{"id":3546,"date":"2021-11-02T08:00:52","date_gmt":"2021-11-02T15:00:52","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/cosmosdb\/?p=3546"},"modified":"2021-11-01T08:26:34","modified_gmt":"2021-11-01T15:26:34","slug":"query-performance-indexing-metrics","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/cosmosdb\/query-performance-indexing-metrics\/","title":{"rendered":"Optimize query performance with Azure Cosmos DB indexing metrics"},"content":{"rendered":"<p>You can now use the Azure Cosmos DB indexing metrics, announced as general available at Microsoft Ignite, to optimize query performance! The indexing metrics show two important pieces of information:<\/p>\n<ul>\n<li>Utilized indexes<\/li>\n<li>Recommended indexes for improving query performance<\/li>\n<\/ul>\n<p><strong>Enabling indexing metrics:<\/strong><\/p>\n<p>You can enable indexing metrics for a query by setting the\u00a0<strong>PopulateIndexMetrics<\/strong>\u00a0property to\u00a0true. When not specified,\u00a0<strong>PopulateIndexMetrics<\/strong>\u00a0defaults to\u00a0false. The indexing metrics are supported in the .NET SDK (version 3.21.0 or later) and Java SDK (version 4.19.0 or later).<\/p>\n<p>You should check the indexing metrics after you have already identified an expensive query, through methods such as using <a href=\"https:\/\/docs.microsoft.com\/azure\/cosmos-db\/cosmosdb-monitor-resource-logs#full-text-query\" target=\"_blank\" rel=\"noopener\">diagnostic logs<\/a>. We don\u2019t recommend proactively logging the indexing metrics because they won\u2019t change between different executions of the same query with the same indexing policy.<\/p>\n<p><strong>Index impact score:<\/strong><\/p>\n<p>The index impact score is the likelihood that an indexed path, based on the query shape, has a significant impact on query performance. In other words, the index impact score is the probability that, without that specific indexed path, the query RU charge would have been substantially higher.<\/p>\n<p>There are two possible index impact scores:\u00a0<strong>high<\/strong>\u00a0and\u00a0<strong>low<\/strong>. If you have multiple potential indexed paths, we recommend focusing on indexed paths with a\u00a0<strong>high<\/strong>\u00a0impact score.<\/p>\n<p>The only criteria used in the index impact score is the query shape. For example, in the below query, the indexed path\u00a0\/age\/?\u00a0would be assigned a\u00a0<strong>high<\/strong>\u00a0index impact score:<\/p>\n<pre>SELECT *\r\nFROM c\r\nWHERE c.age &gt; 30<\/pre>\n<p>The actual impact depending on the nature of the data, so not all indexes with <strong>high <\/strong>index impact scores will improve query performance. In the above example, the impact of an \/age\/? indexed path would be larger if fewer items matched the c.age &gt; 30 filter expression.<\/p>\n<p><strong>Examples:<\/strong><\/p>\n<p>To show example indexing metrics, we\u2019ve created an example container with about 9 million items. Here\u2019s an example item:<\/p>\n<pre><span style=\"font-size: 12pt;\">{<\/span>\r\n<span style=\"font-size: 12pt;\">\u00a0\u00a0\u00a0\u00a0\"id\":\u00a0\"qd471022kj\",<\/span>\r\n<span style=\"font-size: 12pt;\"> \u00a0\u00a0\u00a0\"name\":\u00a0\"Mark Brown\",<\/span>\r\n<span style=\"font-size: 12pt;\"> \u00a0\u00a0\u00a0\"address\": \"123 Cosmos DB Way\"<\/span>\r\n<span style=\"font-size: 12pt;\"> \u00a0\u00a0\u00a0\"company\":\u00a0\"Microsoft\",<\/span>\r\n<span style=\"font-size: 12pt;\"> \u00a0\u00a0\u00a0\"job\": \"Software Developer\",<\/span>\r\n<span style=\"font-size: 12pt;\">  \u00a0 \"_ts\": 1589840249<\/span>\r\n<span style=\"font-size: 12pt;\">}<\/span><\/pre>\n<p>The container has the default indexing policy of all paths indexed with a range index. We\u2019ll show the index metrics, both the utilized indexes and potential single indexes, for several different queries.<\/p>\n<p><strong>Query 1:<\/strong><\/p>\n<pre>SELECT *\r\nFROM c\r\nWHERE c.name = \u201cTim Sander\u201d\r\n\r\n<strong>Original RU charge:<\/strong> 2.79 RUs<\/pre>\n<p><strong>Index metrics:<\/strong><\/p>\n<pre>Index Utilization Information\r\n  Utilized Single Indexes\r\n \u00a0\u00a0 Index Spec: \/name\/?\r\n \u00a0\u00a0 Index Impact Score: High\r\n \u00a0\u00a0 ---\r\n  Potential Single Indexes\r\n  Utilized Composite Indexes\r\n  Potential Composite Indexes<\/pre>\n<p><strong>Explanation<\/strong>:<\/p>\n<p>In this simple example, the indexing metrics show that the query used the existing \/name\/? indexed path. Given the <strong>high<\/strong> index impact score, there\u2019s a high likelihood that this indexed path was critical for query performance. There aren\u2019t any recommendations for further improving query performance.<\/p>\n<p><strong>Query 2: <\/strong><\/p>\n<pre>SELECT AVG(c._ts)\r\nFROM c\r\nWHERE c.job = \u201cDeveloper\u201d<\/pre>\n<p><strong>Original RU charge:<\/strong> 222.35 RUs<\/p>\n<p><strong>Index metrics:<\/strong><\/p>\n<pre>Index Utilization Information\r\n  Utilized Single Indexes\r\n \u00a0\u00a0 Index Spec: \/job\/?\r\n \u00a0\u00a0 Index Impact Score: High\r\n \u00a0\u00a0 ---\r\n \u00a0\u00a0 Index Spec: \/_ts\/?\r\n \u00a0\u00a0 Index Impact Score: High\r\n \u00a0\u00a0 ---\r\n  Potential Single Indexes\r\n  Utilized Composite Indexes\r\n  Potential Composite Indexes\r\n \u00a0\u00a0 Index Spec: \/job ASC, \/_ts ASC\r\n \u00a0\u00a0 Index Impact Score: High<\/pre>\n<p><strong>Explanation<\/strong>:<\/p>\n<p>This query averages the timestamp employees with the job \u201cDeveloper\u201d. While the query used the existing indexed paths \/job\/? and \/_ts\/?, the indexing metrics show a recommended composite index with a high likelihood of impact. In this case, after adding a composite index on (\/job ASC, \/_ts ASC), the RU charge of the query is reduced from 222.35 RUs to 137.56 RUs.<\/p>\n<p>&nbsp;<\/p>\n<p><strong>Query 3: <\/strong><\/p>\n<pre>SELECT TOP 10 *\r\nFROM c\r\nWHERE c.job = \u201cDeveloper\u201d\r\nAND c._ts &gt; 1589840249\r\n\r\n<\/pre>\n<p><strong>Original RU charge:<\/strong> 31.3 RUs<\/p>\n<p><strong>Index metrics:<\/strong><\/p>\n<pre>Index Utilization Information\r\n  Utilized Single Indexes\r\n \u00a0\u00a0 Index Spec: \/job\/?\r\n \u00a0\u00a0 Index Impact Score: High\r\n \u00a0\u00a0 ---\r\n \u00a0\u00a0 Index Spec: \/_ts\/?\r\n \u00a0\u00a0 Index Impact Score: High\r\n \u00a0\u00a0 ---\r\n  Potential Single Indexes\r\n  Utilized Composite Indexes\r\n  Potential Composite Indexes\r\n \u00a0\u00a0 Index Spec: \/job ASC, \/_ts ASC\r\n \u00a0\u00a0 Index Impact Score: High<\/pre>\n<p><strong>Explanation<\/strong>:<\/p>\n<p>This filters to all developers and sorts by timestamp . While the query used the existing indexed paths \/job\/? and \/_ts\/?, the indexing metrics show a recommended composite index with a high likelihood of impact. In this case, after adding a composite index on (\/job ASC, \/_ts ASC), the RU charge of the query is reduced from RUs 31.3 to 23.7 RUs.<\/p>\n<p><strong>Next steps:<\/strong><\/p>\n<ul>\n<li><a href=\"https:\/\/docs.microsoft.com\/azure\/cosmos-db\/sql\/index-metrics\" target=\"_blank\" rel=\"noopener\">Learn more about using the indexing metrics to optimize high RU queries<\/a>.<\/li>\n<li><a href=\"https:\/\/docs.microsoft.com\/azure\/cosmos-db\/sql\/troubleshoot-query-performance\" target=\"_blank\" rel=\"noopener\">Troubleshoot query performance<\/a><\/li>\n<li><a href=\"https:\/\/docs.microsoft.com\/azure\/cosmos-db\/sql\/sql-query-getting-started\" target=\"_blank\" rel=\"noopener\">Getting started with SQL queries in Azure Cosmos DB<\/a><\/li>\n<\/ul>\n<p>Do you have any feedback about the indexing metrics? Share feedback directly with the Azure Cosmos DB engineering team: cosmosdbindexing [@] microsoft.com<\/p>\n","protected":false},"excerpt":{"rendered":"<p>You can now use the Azure Cosmos DB indexing metrics, now generally available, to optimize query performance! The indexing metrics show indexes that the query engine used and index recommendations for improving query performance.<\/p>\n","protected":false},"author":12128,"featured_media":61,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[14,818],"tags":[],"class_list":["post-3546","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-core-sql-api","category-query"],"acf":[],"blog_post_summary":"<p>You can now use the Azure Cosmos DB indexing metrics, now generally available, to optimize query performance! The indexing metrics show indexes that the query engine used and index recommendations for improving query performance.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/posts\/3546","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/users\/12128"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/comments?post=3546"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/posts\/3546\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/media\/61"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/media?parent=3546"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/categories?post=3546"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/tags?post=3546"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}