{"id":2326,"date":"2016-12-16T19:14:31","date_gmt":"2016-12-16T19:14:31","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/pix\/?page_id=2326"},"modified":"2016-12-16T19:14:31","modified_gmt":"2016-12-16T19:14:31","slug":"file-io-captures","status":"publish","type":"page","link":"https:\/\/devblogs.microsoft.com\/pix\/file-io-captures\/","title":{"rendered":"File IO Captures"},"content":{"rendered":"<p><span style=\"color: #000000;font-family: Calibri\">File IO captures help you identify inefficiencies in your title\u2019s disk io patterns.<\/span><span><span style=\"color: #000000;font-family: Calibri\">\u00a0 <\/span><\/span><span style=\"color: #000000;font-family: Calibri\">Captures also include an analysis that helps you create a more optimal package layout.<\/span><\/p>\n<h3>Initiating a File IO Capture<\/h3>\n<p>To start a capture, click the File IO Trace button on your device connection tab.<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/41\/2019\/03\/file_io_capture_start.png\"><img decoding=\"async\" width=\"505\" height=\"625\" class=\"alignnone size-full wp-image-2335\" alt=\"file_io_capture_start\" src=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/41\/2019\/03\/file_io_capture_start.png\" \/><\/a><\/p>\n<p>When the capture starts running, you\u2019ll see a dialog that lets you either stop or cancel the capture.\u00a0 File IO captures can run for relatively long periods of time.\u00a0 It\u2019s common to use file io captures to profile a level load, or a play through of a significant part of your game, for example.<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/41\/2019\/03\/memory_stop.jpg\"><img decoding=\"async\" width=\"388\" height=\"127\" class=\"alignnone size-full wp-image-2145\" alt=\"memory_stop\" src=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/41\/2019\/03\/memory_stop.jpg\" \/><\/a><\/p>\n<p>File IO captures are ETW-based.\u00a0 When\u00a0you press the Stop button, the collection of ETW data will stop and PIX will open the capture.\nFile IO captures open to a tab called a landing page.\u00a0 This initial page provides a textual description of the rest of the tabs in the capture.<\/p>\n<p>The first tab you\u2019ll likely want to look at after the landing page is the Capture Summary tab.<\/p>\n<h3>The Capture Summary Tab<\/h3>\n<p>The capture summary provides some basic statistics about the files that were accessed during the capture.\u00a0 These statistics include the number of files, the amount of data that was accessed and the average throughput and disk utilization.<\/p>\n<p>Full callstacks are also provided for the Top 10 redundant reads.\u00a0 The Top 10 list is ordered by a cost which is defined as the number of bytes read multiplied by the read count.\u00a0 Each entry in the callstack includes a hyperlink to the source file for that function (if PIX could find the PDB).\u00a0 Clicking on the hyperlink will open the source file in the default editor registered for the file extension. To open the source file in Visual Studio, right click on the hyperlink and choose &#8220;Open In VS&#8221;.<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/41\/2019\/03\/file_io_summary.png\"><img decoding=\"async\" width=\"1101\" height=\"641\" class=\"alignnone size-full wp-image-2346\" alt=\"file_io_summary\" src=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/41\/2019\/03\/file_io_summary.png\" \/><\/a><\/p>\n<h3>The File IO Tab<\/h3>\n<p>The timeline tab provides data on all file accesses made by your title as the capture ran.\u00a0 The event list view has one row for every file access, ordered by time the access was initiated.\u00a0 By default, the events list includes columns that describe details such as the size, offset, start time and duration of the access.\u00a0 Click the Counters button to change the set of columns that are displayed.<\/p>\n<div>The event list also contains a Notes column that the profiler uses to point out potential inefficiencies in your file access patterns.\u00a0 The most common non-blank value you\u2019ll see in the Notes column is \u201cReverse Seek\u201d indicating that the offset of the current access is less than the offset of the previous access. In other words, you\u2019ve read backwards on the disk.<\/div>\n<div><\/div>\n<div>Selecting a row in the events list populates the Callstack view with the function name, source file and line number describing the location in your title that initiated the file access.<\/div>\n<div><\/div>\n<div>The bottom view in this tab is a timeline that shows the set of outstanding file IO operations at any point in time. The timeline includes a graph of IO throughput and disk utilization. The timeline and the events list are synchronized. If you select an IO event in the list, the corresponding section of the timeline will be highlighted, and vice versa.<\/div>\n<div>The timeline uses colors to show stages of an IO operation.\u00a0 Orange bars indicate the time when an IO operation is queued and purple bars indicate the time in which an operation is actually being serviced.<\/div>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/41\/2019\/03\/file_io_timeline.png\"><img decoding=\"async\" width=\"1429\" height=\"994\" class=\"alignnone size-full wp-image-2355\" alt=\"file_io_timeline\" src=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/41\/2019\/03\/file_io_timeline.png\" \/><\/a><\/p>\n<h3>The Usage Tab<\/h3>\n<p><span style=\"color: #000000;font-family: Calibri\">The Usage tab shows your accesses grouped by file. Presenting the data in this way makes it easy for you to see which files were read (and not read) during the capture. When compared with the files in a given section of your package layout, the data on the Usage tab can be used to adjust the files contained in that section. For example, you can use the file io profiler to determine which files you read during startup, then compare that set with the files contained in your launch section. If there are files contained in your launch section that don\u2019t appear in the capture, consider removing those files from the section in order to minimize the amount of data that must be installed or downloaded before your title can begin running. <\/span><\/p>\n<p><span style=\"color: #000000;font-family: Calibri\">The Usage tab also shows data about the accesses within each file. This data includes: <\/span><\/p>\n<ul>\n<li><span style=\"color: #000000;font-family: Calibri\">The percentage of each file that was read. This percentage is displayed in the % Accessed column. Large files for which only a small percentage is read are candidates for breaking into smaller files in order to reduce chunk size, if appropriate. <\/span><\/li>\n<li><span style=\"color: #000000;font-family: Calibri\">The number of accesses for each offset within the file. This data can be seen using a combination of the File Offset and Access Count columns. A high access count indicates that you\u2019re reading the same data from the file multiple times. More efficient file io performance can be obtained by optimizing your title to read data at a particular offset only once and caching the data in memory if possible.<\/span><\/li>\n<\/ul>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/41\/2019\/03\/file_io_usage.png\"><img decoding=\"async\" width=\"1045\" height=\"601\" class=\"alignnone size-full wp-image-2365\" alt=\"file_io_usage\" src=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/41\/2019\/03\/file_io_usage.png\" \/><\/a><\/p>\n<h3>Exporting Package Layout XML<\/h3>\n<p>The analysis performed by the Usage tab helps you determine which asset files should be grouped into the various content groups that define your package layout.\u00a0 PIX can be used to generate the XML that defines a content group in your layout.\u00a0 To generate the XML, select the group of files you\u2019d like to include in a content group and click the Export Content Group button.\u00a0 The XML will be opened in the application that is registered as the default handler for files with a .xml extension.\u00a0 You can then paste sections of the generated XML into the layout file used as input to the package build process.<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/41\/2019\/03\/fileio_export_button.jpg\"><img decoding=\"async\" width=\"1165\" height=\"726\" class=\"alignnone size-full wp-image-2995\" alt=\"\" src=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/41\/2019\/03\/fileio_export_button.jpg\" \/><\/a><\/p>\n<h3>Merging Multiple Captures for Analysis<\/h3>\n<p>The Usage tab makes it easy for you to see the percentage of each asset file that was accessed while the trace was running. This information can help you determine which asset files should belong in each content group in your package layout. When doing this analysis, it\u2019s often useful to view the data from multiple traces in a single analysis session. For example, if you have multiple testers playing through the same level of your game, it\u2019s likely that different testers might cause different assets to be read. By combining the data from individual traces you can improve the accuracy of the analysis presented on the Usage tab.<\/p>\n<p>To add an additional trace, select the Add Trace button and select the pix3 file you\u2019d like to add using the file selection dialog:<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/41\/2019\/03\/fileio_add_trace1.jpg\"><img decoding=\"async\" width=\"1387\" height=\"790\" class=\"alignnone size-full wp-image-2996\" alt=\"\" src=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/41\/2019\/03\/fileio_add_trace1.jpg\" \/><\/a><\/p>\n<p>After you\u2019ve selected a trace, PIX will create a new capture that contains the data from the original trace and the trace you just added. The contents of the two traces are merged and displayed in the File Usage event list.<\/p>\n<p>When merging data, PIX looks for asset files that have the same fully qualified files names. The reads of a given asset file from the multiple traces are analyzed and combined such that the unique data read in each capture is summed to produce a % Accessed value across all the captures. For example, consider the case where you have two captures that both read \u201cc:\\temp\\ilc\\Assets\\Level1.zip\u201d. If the first capture shows that 20% of that file was read and the second capture shows that a different 30% of that file was read, the merging of the two captures will show that 50% of the file was accessed. If the two captures include reads that are common to both, those reads will only be counted once in the merge. For example, if one capture reports that 20% of \u201cc:\\temp\\ilc\\Assets\\Level1.zip\u201d was read and another capture reports 30% was read, but 10% of those reads appear in both captures, the merged total would be 40% instead of 50%.<\/p>\n<p>The File Usage Capture List view lists all the traces that have been merged into your capture. The checkboxes next to each trace can be used to toggle whether a given trace is merged into the data presented in the File Usage events list.<\/p>\n<p>&nbsp;<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/41\/2019\/03\/fileio_merged_traces1.jpg\"><img decoding=\"async\" width=\"1058\" height=\"698\" class=\"alignnone size-full wp-image-3006\" alt=\"\" src=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/41\/2019\/03\/fileio_merged_traces1.jpg\" \/><\/a><\/p>\n<p>The File IO tab also includes a list of all captures that have been merged into the current capture. Selecting different captures from this list causes the contents of the Events, Timeline, and Event Callstack views to be populated with the data from the selected capture. Note that no merging of data is performed on the File IO tab. The data on that tab is meant to show data on individual reads, so the merge concept does not apply.<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/41\/2019\/03\/fileio_merged_traces2.jpg\"><img decoding=\"async\" width=\"1177\" height=\"696\" class=\"alignnone size-full wp-image-3015\" alt=\"\" src=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/41\/2019\/03\/fileio_merged_traces2.jpg\" \/><\/a><\/p>\n<h3>Mapping Reads within Archive Files to Specific Assets<\/h3>\n<p>The File IO tab shows you information about every read initiated by your title as the capture was running. By default, PIX will show reads as seen by the underlying file system. If your title packages individual assets into a composite file such as zip or pak, the file io profiler will show that your composite file was read, but it needs more information to determine exactly which asset within the composite file was accessed.<\/p>\n<p>PIX allows you to specify a mapping file that it uses to decode reads that occur within archive files, allowing you to identify the specific asset that was read. A mapping file consists of a list of entries, where each entry specifies the offset and size of a specific asset. A description for each asset must also be provided. PIX uses this offset+size information to map reads that occurred within the archive to the specific asset that was read.<\/p>\n<h4>Mapping File Format<\/h4>\n<p>Mapping files are in .csv format. This format should be easily producible either manually or from the portion of your build system that composes the archives.<\/p>\n<p>The first line of the mapping file must be the fully qualified file name of the composite, or archive, file that the entries in the rest of the mapping file apply to.<\/p>\n<p>Example first line:<\/p>\n<p style=\"padding-left: 30px\">g:\\assets\\Ice_King.DDS<\/p>\n<p>The remaining lines in the mapping file identify the individual assets within the archive. Assets are identified by the byte offset within the archive file where the asset is stored, the size of the asset in bytes and a description of the asset. The format of an asset line is<\/p>\n<p style=\"padding-left: 30px\">&lt;offset&gt;,&lt;size&gt;,&lt;description&gt;<\/p>\n<p>Example asset line:<\/p>\n<p style=\"padding-left: 30px\">28673, 256, sword<\/p>\n<p>The offset and size values on an asset lines can also be specified as hex numbers, for example:<\/p>\n<p style=\"padding-left: 30px\">0x7001, 0x100, sword<\/p>\n<p>Mapping files may also contain lines that are comments. PIX will ignore these lines when processing the mapping file. Comments lines start with a #.\nExample comment line:<\/p>\n<p style=\"padding-left: 30px\"># this line is a comment. Ignored by PIX.<\/p>\n<h4>Working with Mapping Files<\/h4>\n<p>The Mapping Files view on the File IO tab is used to manage the mapping files associated with your capture. To add a mapping file, click the Add Mapping File dialog and select your .csv file from the file selection dialog.<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/41\/2019\/03\/add_mapping_file.png\"><img decoding=\"async\" width=\"1385\" height=\"981\" class=\"alignnone size-full wp-image-3175\" alt=\"\" src=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/41\/2019\/03\/add_mapping_file.png\" \/><\/a><\/p>\n<p>The mapping files you add are listed on the view along with the name of the archive file to which they apply. If your mapping files contains invalid lines, PIX will populate a second list that contains the name of the mapping file that contained the invalid line, along with the line\u2019s contents.<\/p>\n<p>After you\u2019ve added a mapping file, PIX will update the contents of the Descriptions column in the event list based on contents of your mapping file. For example, adding a mapping file with the following contents:<\/p>\n<p style=\"padding-left: 30px\">g:\\assets\\Ice_King.DDS\n0, 256, armor\n28673, 256, sword\n40961, 256, mask\n45057, 256, bootL\n110593, 256, bootR\n114689, 256, hair\n180225, 256, helmet\n#this line is a comment\n208897, 256, sabre\n212993, 256, gloveR\n221185, 256, gloveL\nthis line contains a syntax error\n225281, 256, mail\n237569, 256, leash\n266241, 256, shield\n299009, 256, spike\n364545, 256, chain<\/p>\n<p>Will produce the following result:<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/41\/2019\/03\/with_mapping_file.png\"><img decoding=\"async\" width=\"1385\" height=\"981\" class=\"alignnone size-full wp-image-3185\" alt=\"\" src=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/41\/2019\/03\/with_mapping_file.png\" \/><\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>File IO captures help you identify inefficiencies in your title\u2019s disk io patterns.\u00a0 Captures also include an analysis that helps you create a more optimal package layout. Initiating a File IO Capture To start a capture, click the File IO Trace button on your device connection tab. When the capture starts running, you\u2019ll see a [&hellip;]<\/p>\n","protected":false},"author":1719,"featured_media":4769,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_acf_changed":false,"footnotes":""},"class_list":["post-2326","page","type-page","status-publish","has-post-thumbnail","hentry"],"acf":[],"_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/pix\/wp-json\/wp\/v2\/pages\/2326","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/pix\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/pix\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/pix\/wp-json\/wp\/v2\/users\/1719"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/pix\/wp-json\/wp\/v2\/comments?post=2326"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/pix\/wp-json\/wp\/v2\/pages\/2326\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/pix\/wp-json\/wp\/v2\/media\/4769"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/pix\/wp-json\/wp\/v2\/media?parent=2326"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}