Understanding GPU Virtual Addressing and Sparse Images/Buffers

Since the days of DirectX 11, and possibly even earlier, it's been possible to allocate memory on a GPU without actually using physical memory right away. But what does this mean?

Imagine you can create a buffer with a size of 64GB, even if your GPU only has 4GB of actual VRAM. How is this possible?

This works similarly to how virtual addresses work on a CPU. When you ask the operating system for memory, it doesn't immediately use real physical memory (RAM). Instead, it gives you a virtual address. The actual physical memory is only used when you start using that memory.

When you create a sparse buffer on a GPU, it only allocates a mapping table that looks something like this:

Page 0 = Address0 Page 1 = Address1 ...

If you try to read this memory before it is backed by real memory, it will return zero because the memory doesn't actually exist yet.

Next, you allocate real, physical memory. This memory is usually aligned in pages (typically 64KB on modern GPUs). For example, let's say we allocate 2 pages, which equals 128KB. Then, we can bind these pages to the virtual address.

You can tell the GPU: "Bind my BufferAddress + 1GB (16384 pages) to the start of my allocated data." The mapping table then updates like this:

Page 16383 = NULL [previous value] Page 16384 = AllocatedData + 0 Page 16385 = AllocatedData + 65536 Bytes Page 16386 = NULL [previous value]

After binding the real memory to the virtual address, you can read or write to it in your shaders, compute passes, etc. Essentially, your 64GB buffer only takes up the size of the mapping table plus the 128KB of allocated real memory.

Interseting discovery about Unity when using Vulkan.
See, Vulkan have two entry points to resolve functions: vkGetInstanceProcAddr and vkGetDeviceProcAddr.
vkGetInstanceProcAddr returns functions for a given VkInstance.
vkGetDeviceProcAddr on the other hand returns functions for a given VkDevice OR device child, e.g. VkQueue and VkCommandBuffer.
According to the docs, vkGetDeviceProcAddr is preferred when resolving device/-child functions, because returned address induce less overhead (probably because it doesn't need to resolve the child from VkInstance).

So, Unity requests vkGetDeviceProcAddr from Vulkan (or native plugin if hooked with InterceptVulkanInitialization). But actually never using it.
That means that all of the device/-child functions have an overhead to them. E.g. vkCmd functions, vkQueue functions etc. Those functions are actually used with insane frequency, in draw calls, uploading constants, binding buffer ranges etc.

The good thing is that we can reroute vkGetInstanceProcAddr to return pointers as vkGetDeviceProcAddr via native plugin. Which can give potential performance increase when events count is high.

How much performance? Well, you never know before you try. I think for 10K calls, say, on Android it could be measurable, like 0.5ms or something, but that's just my speculation.

Quick overview on using Vulkan-native features in Unity's HLSL:
(bonus point: adding Unsupported functionality in Unity's DXC)
https://meetemq.com/2024/06/14/using-vulkan-in-unity-shaders/

Meetemq

Using Vulkan in Unity shaders

Kept you waiting, huh? It’s been a long time, S… since my last post, mostly due to an enormous amount of work I made during this period. Ever wondered that you can utilize some Vulkan features for your good, but Unity seemingly doesn’t support them in HLSL?…

Начинаем через 3 минуты!

Все знают что произошло.
Тем кто сочувствует — мои соболезнования. Сам только отхожу.