Render-To-Texture with Vulkan in Leadwerks 5

Entry posted by Josh · January 6, 2020

3,530 views

In Leadwerks 4, render-to-texture was accomplished with the SetRenderTarget command, which allowed a camera to draw directly to a specified texture, while hiding the underlying framebuffer object (FBO). In the new engine we have a bit more explicit handling of this behavior. This is largely in part due to the use of Vulkan's bindless design, which greatly improves the context-binding design of OpenGL. The Leadwerks "Buffer" class was never documented or officially supported because the underlying OpenGL functionality made the system pretty messy, but the design of Vulkan simplifies this aspect of graphics.

We have seen that the Framebuffer classes replaces the LE4 context. I've added a TextureBuffer class which can be created similarly:

shared_ptr<TextureBuffer> CreateTextureBuffer(const int width, const int height, const int colorcomponents = 1, const bool depthcomponent = true, const int samples = 0);

Once a TextureBuffer is created, you can set a camera to target it for rendering:

camera->SetRenderTarget(texbuffer);

You can also apply its color component(s) to a material:

material->SetTexture(texbuffer->GetColorBuffer(0), TEXTURE_BASE);

You could also retrieve the depth buffer and apply that to a material, rendering the scene from the top down and using the depth in a rain or snow shader, for example.

This functionality will later be used to render the GUI system to a texture for use in VR or with in-game menus painted onto 3D surfaces.

Like everything with Vulkan, this involved a very long process of figuring out everything we need to use, discarding the things we don't, and packaging it up in a structure that is actually usable by the end user. However, once all that is done we have a very powerful system that is optimized for exactly the way modern GPUs work. Here is a small sample of some of my code, just to give you an idea of how complicated this stuff is:

		for (auto pair : visset->cameravislists)
		{
			auto cam = pair.first;
			clear[1].color = { cam->clearcolor.r, cam->clearcolor.g, cam->clearcolor.b, cam->clearcolor.a };

			auto light = dynamic_pointer_cast<RenderLight>(cam);
			if (light == nullptr and cam->rendertarget == nullptr) continue;
			renderpass[0] = device->shadowpass;
			renderpass[1] = device->renderpass[CLEAR_COLOR | CLEAR_DEPTH];							
			int faces = 1;
			if (light)
			{
				if (light->description.type == LIGHT_POINT) faces = 6;
			}
			if (MULTIPASS_CUBEMAP) faces = 1;
			for (int face = 0; face < faces; ++face)
			{
				renderPassBeginInfo.clearValueCount = 2;				
				if (light)
				{
					renderPassBeginInfo.renderPass = device->shadowpass->pass;
					if (light->description.type == LIGHT_POINT and MULTIPASS_CUBEMAP == true)
					{
						renderPassBeginInfo.renderPass = device->cubeshadowpass->pass;
					}
					renderPassBeginInfo.framebuffer = light->shadowbuffer[face]->framebuffer;
					renderPassBeginInfo.renderArea.extent.width = light->shadowbuffer[face]->size.width;
					renderPassBeginInfo.renderArea.extent.height = light->shadowbuffer[face]->size.height;
				}
				else
				{
					renderpass[0] = device->renderpass[CLEAR_COLOR | CLEAR_DEPTH];
					int cc = cam->rendertarget->CountColorTextures();
					renderPassBeginInfo.renderPass = device->rendertotexturepass[cc][int(cam->rendertarget->depthtexture != nullptr)]->pass;
					renderPassBeginInfo.framebuffer = cam->rendertarget->framebuffer;
					renderPassBeginInfo.renderArea.extent.width = cam->rendertarget->size.width;
					renderPassBeginInfo.renderArea.extent.height = cam->rendertarget->size.height;
				}
				vkCmdBeginRenderPass(commandbuffers[currentFrame]->commandbuffer, &renderPassBeginInfo, VK_SUBPASS_CONTENTS_INLINE);
				RecordDraw(currentFrame, cam, pair.second, renderpass[0], face);
				commandbuffers[currentFrame]->EndRenderPass();
				if (light) commandbuffers[currentFrame]->BindResource(light->shadowbuffer[face]);

				//Copy output to render texture
				if (cam->rendertarget)
				{
					for (int n = 0; n < cam->rendertarget->colortarget.size(); ++n)
					{
						if (cam->rendertarget->colortarget[n] != nullptr)
						{
							commandbuffers[currentFrame]->TransitionImageLayout(pair.first->rendertarget->colortexture[n], VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL, VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL, VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT, -1);
							commandbuffers[currentFrame]->TransitionImageLayout(pair.first->rendertarget->colortarget[n], VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT, -1);

							VkImageCopy regions = {};
							regions.dstOffset = {0u,0u,0u};
							regions.extent = { uint32_t(cam->rendertarget->colortarget[n]->size.x), uint32_t(cam->rendertarget->colortarget[n]->size.y), 1u};
							regions.srcOffset = regions.dstOffset;
							regions.dstSubresource.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT;
							regions.dstSubresource.baseArrayLayer = 0;
							regions.dstSubresource.layerCount = 1;
							regions.dstSubresource.mipLevel = 0;
							regions.srcSubresource = regions.dstSubresource;
							vkCmdCopyImage(commandbuffers[currentFrame]->commandbuffer, cam->rendertarget->colortexture[n]->vkimage, VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL, cam->rendertarget->colortarget[n]->vkimage, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, 1, &regions);
							commandbuffers[currentFrame]->TransitionImageLayout(pair.first->rendertarget->colortarget[n], VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL, VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT, -1);
						}
					}
				}
			}
		}

Below is a simple Lua program that sets up a scene with two cameras, and renders one camera to a texture buffer which is displayed on the middle box itself.

--Get the primary display
local displaylist = ListDisplays()
local display = displaylist[1];
if display == nil then DebugError("Primary display not found.") end
local displayscale = display:GetScale()

--Create a window
local window = CreateWindow(display, "Render to Texture", 0, 0, math.min(1280 * displayscale.x, display.size.x), math.min(720 * displayscale.y, display.size.y), WINDOW_TITLEBAR)

--Create a rendering framebuffer
local framebuffer = CreateFramebuffer(window);

--Create a world
local world = CreateWorld()

--Create second camera
local texcam = CreateCamera(world)
texcam:SetClearColor(1,0,1,1)

--Create a camera
local camera = CreateCamera(world)
camera:Move(0,0,-2)
camera:SetClearColor(0,0,1,1)

--Create a texture buffer
local texbuffer = CreateTextureBuffer(512,512,1,true)
texcam:SetRenderTarget(texbuffer)

--Create scene
local box = CreateBox(world)

local cone = CreateCone(world)
cone:SetPosition(2,0,0)
cone:SetColor(1,0,0,1)

local sphere = CreateSphere(world)
sphere:SetPosition(-2,0,0)
sphere:SetColor(0,1,0,1)

--Create render-to-texture material
local material = CreateMaterial()
local tex = texbuffer:GetColorBuffer()
material:SetTexture(tex, TEXTURE_BASE)
box:SetMaterial(material)

--Create a light
local light = CreateLight(world,LIGHT_DIRECTIONAL)
light:SetRotation(35,-55,0)

--Main loop
while window:Closed() == false do

	texcam:SetPosition(0,0,0)
	texcam:Turn(0,1,0)
	texcam:Move(0,0,-2)

	world:Update()
	world:Render(framebuffer)

end

Here is the result. Look how simple it is to control this powerful system!

1 Comment

Recommended Comments

Yue 768

Posted January 23, 2022

"In Leadwerks 4, render-to-texture was accomplished with the SetRenderTarget command, which allowed a camera to draw directly to a specified texture, while hiding the underlying framebuffer object (FBO). In the new engine we have a bit more explicit handling of this behavior. This is largely in part due to the use of Vulkan's bindless design, which greatly improves the context-binding design of OpenGL. The Leadwerks "Buffer" class was never documented or officially supported because the underlying OpenGL functionality made the system pretty messy, but the design of Vulkan simplifies this aspect of graphics."

I have been left with this information that points me in the right direction to create a mini map. The Hud Elements store example shows the Buffer command, but I guess with the camera command you can create a mini map without any problem.

The question Josh is can I apply a transparent texture to that rendering to make the mini map look round?

SetRenderTarget