Texture::GetPixels()

[SpiderPig]

Now that I have this working I want to get the pixels of the texture rendered from the camera...

 

//Setup
auto texture = CreateTextureBuffer(512, 512);
auto cam2 = CreateCamera(world);
cam2->SetRenderTarget(texture);

//Loop
auto my_precious = texture->give_me_pixels()

 

I can see SetPixels() and SetSubPixles() but no GetPixels().  Can it be done yet?

[Josh]

Textures do not store their pixel data in system memory. They just load it up and send it to the GPU. The Pixmap class is meant for loading and manipulating image data on the CPU.

[SpiderPig]

Could it be retrieved from the GPU somehow?  Maybe through a shader uniform?

[SpiderPig]

Or can a texture be converted to a pixmap?  I need to perform some image processing on the CPU so using a shader wouldn't work.

[Josh]

Why not just load the image as a pixmap?

[SpiderPig]

I want to get the pixels from the render target of a camera per frame and do some stuff to it.  I'm doing it in Leadwerks but it's a bit slow with an extra camera or two.

[Josh]

If you retrieve a texture from the GPU into system memory, that will be extremely slow.

[SpiderPig]

I thought it would be.  I'll see if I can do what I want some other way.

[dugrosette]

The pixel data that makes up a texture is not saved in the system memory. They just load it and then pass it to the GPU to process. On the central processing unit, loading and manipulating picture data is the purpose of the Pixmap class. 

slope game

[SpiderPig]

On 5/15/2022 at 8:40 PM, Josh said:

If you retrieve a texture from the GPU into system memory, that will be extremely slow.

I'm thinking about giving this a go as it's not really a real time application I want this for.  How slow do you think this could be though for a 512x512 image rendered with a camera?  1FPS?

[Josh]

You can use Framebuffer:Capture to take screenshots and get a pixmap. I had to implement this in order to save the rendering results of the environment probes.

[SpiderPig]

Perfect!  I'll give it a go.  Thankyou.

[klepto2]

I have done something similar for my PBRGenerator, and will post the class to download data from the gpu soon. Syncing memory is always at a cost this goes for downloading and uploading the data. There are some tricks you can use to increase the speed a lot, but most of the time you just use staging buffers (these are buffers which are bound to eaither gpu or cpu sepending on the direction you want to transfer the data) 

1. create a buffer object 
2. map the buffer to the memory
3. commit the upload or download commands
4. unmap the buffer and retrieve the data (the buffer can be still mapped)

There are muliple ways to retrieve data from the gpu. The slowest is to use vkCmdCopyImage, then vkCmdBlitImage (slightly faster and already does format conversion and filtering, is not available for all formats). The fastest way is to use vkCmdCopyBuffer, this copies the pure gpu memory of a bound resource into the host memory. This is mostly used for data readbacks for compute shaders (shader storage buffers), but also useful for image synchronsation. 

The usage of snycing between gpu and host should be used carefully, it can slow down an application fast, and should only be considered for debugging or useful info, lets say a generated water heightmap, where you want the to calculate eg: the height an object is on. 

This being said, it would be nice to get access to the index or vertex buffer elements of a specific surface (speaking of particle systems etc.)

 

[Josh]

Fun fact: In Ultra there is only one vertex and index buffer. 

[klepto2]

I know, but you can map/unmap just a region of the buffer to a shader or upload and download to that particular region. we just need the offset of the surface and the actual buffer