GPU Captures: How we support placed and reserved resources

Austin Kinross

Austin

 

Summary

  • PIX on Windows supports placed and reserved (a.k.a. tiled) resources.
  • For most applications, PIX will “just work”, and it will capture/replay placed and reserved resources correctly.
  • In some cases, PIX needs hints from the application to correctly capture placed and reserved resources.
  • The best way you can help PIX is to use an aliasing barrier (with a non-NULL “after” resource) whenever your application starts using a placed and/or reserved resource that overlaps with a previously-used placed/reserved resource.

 

Recap: placed and reserved resources in D3D12

In D3D12, there are three types of resources:

  • Committed
  • Placed
  • Reserved (also known as “Tiled”)

A committed resource has its own dedicated backing memory that isn’t shared with any other resource. This makes them relatively easy for PIX to capture, so we won’t discuss them here.

Placed and reserved resources don’t have their own dedicated backing memory. Instead, applications must create ID3D12Heap objects that own the backing memory, and applications must define “mappings” between placed/reserved resources and the backing memory in heaps.

A reserved resource allows applications to map regions of the resource to regions of heaps. Different parts of a reserved resource can be mapped to different heaps. These mappings can be updated at any point in time via the ID3D12CommandQueue::UpdateTileMappings() API.

A placed resource is conceptually similar to a reserved resource with one mapping that can never be changed. Applications pass a heap and an offset into CreatePlacedResource(), and the application can’t change the heap or offset without destroying and recreating the resource.

 

Why are placed and reserved resources difficult for PIX to capture?

In D3D12, it’s legal for multiple placed and/or reserved resources to be mapped into the same part of a heap at once. These resources are also known as “aliased” or “overlapping” resources. It’s difficult for PIX on Windows to correctly capture overlapping resources, especially on some hardware.

In an ideal world, PIX would just serialize the following into its capture file:

  • The creation parameters for each ID3D12Resource and ID3D12Heap
  • The mappings between ID3D12Resources and ID3D12Heaps
  • The contents of ID3D12Heaps

Unfortunately, step 3 isn’t possible on a lot of modern GPUs. One reason is that some GPUs store additional compression metadata for placed/reserved resources in memory that wouldn’t be captured if PIX used this approach. Without this metadata, PIX wouldn’t correctly recreate the application’s resources and heaps at replay time.

A better approach would be for PIX to serialize the following into its capture file:

  • The creation parameters for each ID3D12Resource and ID3D12Heap
  • The mappings between ID3D12Resources and ID3D12Heaps
  • The contents of all ID3D12Resources (including overlapping resources)

This approach is still problematic though, because data inheritance between overlapping resources isn’t deterministic on some hardware. This means that PIX can’t capture the contents of multiple overlapping resources and recreate them at replay time without potentially corrupting some of them. Even worse, if PIX reads back the wrong placed resource at capture time then it could trigger a hardware decompression that invalidates the compression metadata of a different overlapping placed resource that the application may be using. This could cause corruption at capture time.

 

How does PIX capture placed and reserved resources?

PIX on Windows relies on a modified version of the “Simple Model” described here.

In this modified model, PIX re-uses the concept of “active” and “inactive” resources. There can only ever be one “active” resource at any part of an ID3D12Heap at any point in time. If an application starts using a resource (e.g. as a render target), then that resource becomes “active” and all other overlapping resources are marked as “inactive”. PIX tracks which resources are active at any given point in time.

PIX then serializes the following into its capture file:

  • The creation parameters for each ID3D12Resource and ID3D12Heap
  • The mappings between ID3D12Resources and ID3D12Heaps
  • The contents of all buffer resources (buffer resources don’t have compression metadata, so they aren’t susceptible to the corruption issues described above)
  • The contents of all “active” texture resources/subresources

This means that PIX won’t capture the contents of any “inactive” texture resources/subresources.

Resources are activated when they’re used as an API parameter to most command list APIs. For example, the source and destination resource will be activated when a Copy() operation in a command list is executed on the GPU. One exception is transition ResourceBarrier()s: transitioning a resource from one state to another does not activate that resource.

As-of August 2020, PIX tracks active/inactive placed resources at the resource level, but it tracks active/inactive reserved resources at the subresource level. For example, if a reserved resource has 3 mip levels, then it’s possible for mips 0 and 2 to be active while mip 1 is inactive. If capture is triggered in this state, then PIX will capture the contents of mips 0 and 2 but not mip 1. If a placed resource has 3 mip levels then all 3 mips will always have the same active/inactive state.

 

When does PIX need help capturing placed and reserved resources?

For most applications we’ve seen, PIX on Windows can correctly capture and replay their placed/reserved resource usage without any changes to the application.

Here are some examples of placed/reserved resource usage that may not “just work” in PIX on Windows:

  • Applications that aren’t correctly initializing their placed/reserved render targets (RT) and depth stencil (DS) resources after switching between overlapping resources. In D3D12, these resources must be explicitly initialized with a Clear(), Copy(), or DiscardResource() operation after the switch. If an application does this correctly, then PIX will see these initialization operations and it will use them to correctly track active resources. This rule isn’t enforced by the D3D12 debug layer though (as-of August 2020), so it is possible that a D3D12 application is breaking the rule without realizing it. In these cases, please make your application initialize the resources correctly.
  • Applications that are switching between overlapping non-RT/DS placed/reserved resources without using the resources in a command list API. For example, an application could create overlapping placed UAVs, create a descriptor to each UAV in a descriptor heap, and then dynamically index into the descriptor heap at runtime to choose which UAV to write to. In this scenario, it would be too computationally expensive for PIX to track the active UAV at capture time while maintaining a reasonable framerate. Note that transitioning the UAV to another state (e.g. SRV) is not enough to activate the resource. In these cases, PIX relies on hints from the application to help it track which resources are active. The best way to do this is to use an aliasing barrier with a non-NULL “after” resource.

0 comments

Comments are closed. Login to edit/delete your existing comments