How to modify texture from CUDA

[Alex Kordic]

Hi,

 

I would like to use opengl calls to modify texture using cuda kernels.

I would usually first make opengl context current to my thread, then get texture id, then register texture as cuda resource, run kernels, unregister cuda-texture and finally include my texture in render loop.

What would be performant method to do this in UltraEngine?

[Josh]

Ultra Engine uses Vulkan, not OpenGL.

There is a system for custom hooks that run in the rendering thread, but it is unofficial and not documented.

The World::AddHook method allows you to add a hook that will run either in the data transfer step, or in the render loop using HOOKID_RENDER or HOOKID_TRANSFER. @klepto2 has done some very interesting things with this:

The texture class has two undocumented commands that must ONLY be run in one of the hooks above:

 

•  VkImage Texture::GetImage()
•  VkSampler Texture::GetSampler()

I think this probably gives you the pieces you need to do what you want, but Vulkan is a difficult beast to tame. You might have better luck using compute shaders, if you are not committed to Cuda.

 

[klepto2]

while in theory it is possible to share resources between OpenGL and Vulkan i would not recommend that.

You can use this:

https://github.com/klepto2/UltraComputeShaderSample

it contains a lib which adds ComputeShaders to UltraEngine and a very basic sample. Note: as UltraEngine (as most Vulkan libs) is multithreaded the result of a compute shader is not available immediately but mostly in sync with the render pipeline.

[Alex Kordic]

 

Hi, thank you for the info. I will start learning about vulkan-cuda interop.

My goal is to produce "video texture" that gets updated each frame (if FPS is limited to 60). 

Ideally I would like to avoid image copy. I don't like CUDA >> system >> vulkan. For 4K resolution this is a lot of overhead.

 

[klepto2]

if you want something like that, it might be worth looking into: https://www.khronos.org/blog/an-introduction-to-vulkan-video

why do you want to use CUDA for this? do you need something special, or just the updated image from a video?

 

[Alex Kordic]

The decoded image is stored in CUDA memory in NV12 or planar YUV 4:2:0 format. memcpy() to texture is required. Color conversion to RGB24 is also required for opengl. Ideally i want color conversion where destination is texture memory.

Great pointer to vulkan-video extensions!

[klepto2]

with the info provided by josh you can try the apporach here:

https://github.com/nvpro-samples/gl_vk_simple_interop/blob/master/gl_vkpp.hpp