{"id":6303,"date":"2023-11-01T14:52:23","date_gmt":"2023-11-01T21:52:23","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/pix\/?p=6303"},"modified":"2023-11-01T16:20:47","modified_gmt":"2023-11-01T23:20:47","slug":"timing-capture-statistical-comparison-features","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/pix\/timing-capture-statistical-comparison-features\/","title":{"rendered":"Timing Capture Statistical Comparison Features"},"content":{"rendered":"

Starting with the 2310.30 version of PIX on Windows<\/a>, the Timing Capture Metrics view<\/a> can produce a statistical comparision of the average duration for the points that represent PIX event hierarchies in two selected ranges of time. \u00a0Statistical comparisions help determine which portions of the event hierarchies had statistically different durations for the set of points being compared.<\/span><\/p>\n

\n
For example, the following graph of a metric named Game::Render<\/strong> shows several ranges of time where the event durations are relatively consistent, but there are also ranges of time in which the event’s duration is significantly longer than the average. \u00a0A statistical comparison of the event durations in the two highlighted time ranges can be used to determine which portions of the Game::Render<\/strong> event hierarchy account for the differences in duration.<\/div>\n<\/div>\n
<\/div>\n
\n

\"Image<\/a><\/p>\n

\n
\n
To determine the statistical differences between two time ranges, select the first range of time, right click in that range for the event that you’d like to compare, then select Add graphed PIX CPU events in selected range to statistical analysis<\/strong>. \u00a0Repeat the process for the second range of time.<\/div>\n<\/div>\n<\/div>\n<\/div>\n
\"Image<\/a><\/div>\n
<\/div>\n
\n
\n
After two ranges of time have been added, the Statistical Comparision<\/strong> window appears. \u00a0The two ranges of time can be from different captures. \u00a0This allows you to compare data from a capture taken on one device with the data taken on a different device, for example. \u00a0Note that each selected range of time must have at least two points for the event being compared. \u00a0It’s recommended that you select ranges that have enough points to provide a good sample representation of your game’s behavior. The amount of data may vary depending on how noisy your source data is, but a good rule of thumb is to look for ranges with 50-100 data points. A warning indicator will be shown if the number of data points is below 20.<\/div>\n
<\/div>\n<\/div>\n
\"Image<\/a><\/div>\n<\/div>\n
<\/div>\n
\n
\n
The Baseline<\/strong> column in the table displays the average duration for the set of points in the first range of time. \u00a0The Experiment<\/strong> column displays the average duration for the set of points in the second range of time. The Delta<\/strong> and Delta %<\/strong> columns show the differences in duration between the baseline and experiment groups.<\/div>\n
<\/div>\n
The P-value<\/strong> column identifies whether the differences between the baseline and experiment groups are statistically significant or whether the differences are due to other factors such as noise, random sampling, or sampling bias. \u00a0The lower the P-value, the more statistically significant the differences are. \u00a0Those rows with the lowest P-values, are the rows you’ll likely want to investigate further, by drilling further into the event hierarchy. \u00a0Rows that are statistically significant are colored in either red or green depending on whether the delta between the baseline and experiment groups is negative or positive. \u00a0The boldness of the red and green colors is relatlive to how close to 0 the P-value is. \u00a0Up and down arrows are also displayed in the P-value column to aid with accessibility. \u00a0Sorting by P-value is a convenient way to bring the most statistically signifcant points to the top of the table.<\/div>\n
<\/div>\n
The histogram at the bottom of the Statistical Comparison<\/strong> window shows the distribution of the durations for the baseline and experiment groups for the selected row in the table. \u00a0There is generally a correlation between the P-value for a given row and it’s histogram. \u00a0The histograms for the baseline and experiment group are largely overlapping for rows with high P-values, and vice versa.<\/div>\n
<\/div>\n
The number of points in the baseline and experiment groups also influences how to interpret the results. \u00a0The N<\/strong> columns in the table display the number of points for each group. \u00a0A yellow warning triangle is drawn in those cells with low sample counts. \u00a0Results may be biased in these cases.<\/div>\n
<\/div>\n
Statistical analysis is integrated into the performance budgets feature in the Metrics view. \u00a0After establishing a budget and selecting a range of time, use the calculator icon to create a statistical comparison of the points above and the points below the budget.<\/div>\n<\/div>\n
<\/div>\n<\/div>\n
\n
\n
\n
\"Image<\/a><\/div>\n<\/div>\n<\/div>\n
<\/div>\n
Thanks for reading!<\/div>\n<\/div>\n
Steven.<\/div>\n","protected":false},"excerpt":{"rendered":"

Starting with the 2310.30 version of PIX on Windows, the Timing Capture Metrics view can produce a statistical comparision of the average duration for the points that represent PIX event hierarchies in two selected ranges of time. \u00a0Statistical comparisions help determine which portions of the event hierarchies had statistically different durations for the set of […]<\/p>\n","protected":false},"author":1915,"featured_media":6309,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-6303","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-pix"],"acf":[],"blog_post_summary":"

Starting with the 2310.30 version of PIX on Windows, the Timing Capture Metrics view can produce a statistical comparision of the average duration for the points that represent PIX event hierarchies in two selected ranges of time. \u00a0Statistical comparisions help determine which portions of the event hierarchies had statistically different durations for the set of […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/pix\/wp-json\/wp\/v2\/posts\/6303","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/pix\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/pix\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/pix\/wp-json\/wp\/v2\/users\/1915"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/pix\/wp-json\/wp\/v2\/comments?post=6303"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/pix\/wp-json\/wp\/v2\/posts\/6303\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/pix\/wp-json\/wp\/v2\/media\/6309"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/pix\/wp-json\/wp\/v2\/media?parent=6303"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/pix\/wp-json\/wp\/v2\/categories?post=6303"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/pix\/wp-json\/wp\/v2\/tags?post=6303"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}