{"id":6600,"date":"2024-11-13T17:27:57","date_gmt":"2024-11-14T01:27:57","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/pix\/?p=6600"},"modified":"2024-12-16T15:07:07","modified_gmt":"2024-12-16T23:07:07","slug":"the-timing-capture-comparison-layout","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/pix\/the-timing-capture-comparison-layout\/","title":{"rendered":"The Timing Capture comparison layout"},"content":{"rendered":"

Starting with version 2408.05 of PIX on Windows<\/a>, the Timing Capture<\/a> comparison features have been moved from the Metrics view to a new dedicated Comparison Layout. This layout improves several comparison workflows, including the ability to compare data from multiple captures.<\/span><\/p>\n

\n
The Comparison Layout produces a statistical comparison of the average duration for the points that represent PIX event hierarchies in two selected ranges of time, or for the points above and below a metric’s budget. \u00a0Statistical comparisons help determine which portions of the event hierarchies had statistically different durations for the set of points being compared.<\/div>\n<\/div>\n
<\/div>\n
\n
The layout consists of a metric selection panel, graphs for the left and right set of points to be compared, a table showing the results of the comparison, and a histogram of the values of both the left and right sets of points for the selected row in the table. \u00a0The set of points for the right side of the comparison may come from a different capture than the points from the left side of the comparison. \u00a0This allows you to compare data from a capture taken on one type of Windows device with the data taken on a different device, for example.<\/div>\n
<\/div>\n<\/div>\n
\n
\n

Comparing groups of points for a metric<\/h3>\n
To produce a comparison for a metric, select Metrics<\/em> from the Comparison Item<\/strong> dropdown, select a metric from the selection panel or from the Recent Metrics<\/strong> dropdown, then select a range of time in both the left and right graphs. \u00a0For example, the following graph of a metric named Game::Render<\/strong> shows several ranges of time where the event durations are relatively consistent, but there are also ranges of time in which the event’s durations are significantly different between the baseline and experiment graphs. \u00a0A statistical comparison of the event durations in the two highlighted time ranges can be used to determine which portions of the Game::Render<\/strong> event hierarchy account for the differences in duration.<\/div>\n<\/div>\n<\/div>\n

\"Image<\/a><\/p>\n

\n
Note that each selected range of time must have at least two points for the metric being compared. \u00a0It’s recommended that you select ranges that have enough points to provide a good sample representation of your game’s behavior. The amount of data may vary depending on how noisy your source data is, but a good rule of thumb is to look for ranges with 50-100 data points. A warning indicator will be shown if the number of data points is below 20.<\/div>\n
<\/div>\n
The Left<\/strong> column in the table displays the average duration for the set of points in the range of time selected in the left graph. \u00a0The Right<\/strong> column displays the average duration for the set of points in the range of time selected in the right graph. The Delta<\/strong> and Delta %<\/strong> columns show the differences in duration between the left and right groups.<\/div>\n
<\/div>\n
The P-value<\/strong> column identifies whether the differences between the left and right groups are statistically significant or whether the differences are due to other factors such as noise, random sampling, or sampling bias. \u00a0The lower the P-value, the more statistically significant the differences are. \u00a0Those rows with the lowest P-values, are the rows you’ll likely want to investigate further by drilling further into the event hierarchy. \u00a0Rows that are statistically significant are colored in either red or green depending on whether the delta between the baseline and experiment groups is negative or positive. \u00a0The boldness of the red and green colors is relative to how close to 0 the P-value is. \u00a0Up and down arrows are also displayed in the P-value column to aid with accessibility. \u00a0Sorting by P-value is a convenient way to bring the most statistically significant points to the top of the table.<\/div>\n
<\/div>\n
The histogram at the bottom of the layout shows the distribution of the durations for the left and right groups for the selected row in the table. \u00a0There is generally a correlation between the P-value for a given row and its histogram. \u00a0The histograms for the left and right groups are largely overlapping for rows with high P-values, and vice versa.<\/div>\n
<\/div>\n
The number of points in the left and right groups also influences how to interpret the results. \u00a0The N<\/strong> columns in the table display the number of points for each group. \u00a0A yellow warning triangle is drawn in those cells with low sample counts. \u00a0Results may be biased in these cases.<\/div>\n<\/div>\n
<\/div>\n
\n
\n

Comparing the points above and below a metric’s budget<\/h3>\n
To compare the points above and below a metric’s budget, first create a metrics view configuration that includes the metrics and the budgets<\/a> to analyze. \u00a0Then, select Budgets<\/em> from the Comparison Item<\/strong> dropdown, the desired metrics view configuration from the Configuration<\/strong> dropdown, and the budget to analyze from the Budget<\/strong> dropdown.<\/div>\n
<\/div>\n
When comparing the points above and below a budget, an Above\/Below<\/strong> slider appears above both the left and right graphs. Ensure that Above<\/em> is selected for one graph, and Below<\/em> is selected from the other.<\/div>\n<\/div>\n<\/div>\n
<\/div>\n

\"Image<\/a><\/p>\n

\n
<\/div>\n
Note that any metric can be added to the comparison using the Selector panel.\u00a0 The list of metrics is not limited to only those that have budgets defined.\u00a0 This enables you to look at everything that is happening in a capture when a single metric is out of budget and compare it to what happens in the capture when the metric is within budget.<\/span><\/div>\n<\/div>\n
<\/div>\n
\n
\n
The comparison layout can also be accessed from the metrics view. \u00a0Select the calculator icon next to the budget to open the comparison layout.<\/div>\n<\/div>\n<\/div>\n
<\/div>\n
\"Image<\/a><\/div>\n
<\/div>\n
As always, please send us bug reports and feature requests using the PIX Feedback button in the upper right of the UI.<\/div>\n
<\/div>\n
Steven.<\/div>\n
<\/div>\n","protected":false},"excerpt":{"rendered":"

Starting with version 2408.05 of PIX on Windows, the Timing Capture comparison features have been moved from the Metrics view to a new dedicated Comparison Layout. This layout improves several comparison workflows, including the ability to compare data from multiple captures. The Comparison Layout produces a statistical comparison of the average duration for the points […]<\/p>\n","protected":false},"author":1915,"featured_media":6601,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-6600","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-pix"],"acf":[],"blog_post_summary":"

Starting with version 2408.05 of PIX on Windows, the Timing Capture comparison features have been moved from the Metrics view to a new dedicated Comparison Layout. This layout improves several comparison workflows, including the ability to compare data from multiple captures. The Comparison Layout produces a statistical comparison of the average duration for the points […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/pix\/wp-json\/wp\/v2\/posts\/6600","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/pix\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/pix\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/pix\/wp-json\/wp\/v2\/users\/1915"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/pix\/wp-json\/wp\/v2\/comments?post=6600"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/pix\/wp-json\/wp\/v2\/posts\/6600\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/pix\/wp-json\/wp\/v2\/media\/6601"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/pix\/wp-json\/wp\/v2\/media?parent=6600"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/pix\/wp-json\/wp\/v2\/categories?post=6600"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/pix\/wp-json\/wp\/v2\/tags?post=6600"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}