Minimap Generation for Ultra

[StOneDOes]

One of the core things I want to tick off is automatic minimap generation, as my long term goal would involve a procedurally generated terrain. (For now I can manually take a screenshot if I want).

To me the ideal way of doing this would be to jump the camera to the center of the terrain, face it down on a 90 angle with an orthographic projection and fit the viewport correctly to the terrain dimensions. From here I would create a render target and render a single frame to this target. From here I should be able to retrieve the image data from the texture by mapping the resource. Here's the problem - I don't know how to map and unmap image resources in Vulkan (can probably look it up), and don't know how to get a pointer to the graphic device. I assume from here that I could feed the raw data into a Pixmap? Or is there a way to directly convert a Texture to a Pixmap without doing any mapping of resources?

After that I should be able to use the image as part of the UI and draw other shapes and indicators on top of it,

If anyone could offer some advice that would be appreciated, thanks

[Josh]

Interesting. We recently were dealing with this.

You're right that it would be unnecessary to try to retrieve texture data from Vulkan, convert to a pixmap, which would then convert back to a texture and send it back to Vulkan.

The simplest way is to just create a sprite and apply the texture to a material that is applied to the sprite, but if you are making something more complex you will want to use the GUI system. @SpiderPig did this with a custom widget, and just set a widgetblock's texture, but we both agreed it would be necessary to add a Widget::SetTexture() method. I will begin working on this today and post an update on the 1.0.1 branch when it is ready.

[SpiderPig]

Here's some basic code of what I did if it helps.  Widget->SetTexture() is not available yet as Josh said above but once it is it should work just like that.  My code is more complex and spread-out than what I've written here but the general idea is it should render a cube in the centre of the camera and display it on the widget.  You can move the camera any where in the world you want.

auto texture_buffer = CreateTextureBuffer(128, 128);

auto camera = CreateCamera(world);
camera->SetRenderLayers(2);
camera->SetFov(70.0f);
camera->SetRenderTarget(texture_buffer);

auto cube = CreateBox(world);
cube->SetRenderLayers(2);
cube->SetPosition(0.0f, 0.0f, 2.0f);

auto widget = CreatePanel(0, 0, 128, 128, ui->root);
widget->SetTexture(texture_buffer->GetColorAttachment());

 

[Josh]

I've now added support for Widget::SetTexture on the 1.0.1 branch:
https://www.ultraengine.com/learn/Widget_SetTexture

These changes are now available on the 1.0.1 branch. Please reinstall the client app to be able to detect and install the update, as I had to make some small changes to handle multiple branches:
https://ultraengine.github.io/files/UltraClient.exe

[Josh]

Here's something else that might be relevant to what you're doing:
https://www.ultraengine.com/learn/Camera_SetRealtime?lang=cpp
https://www.ultraengine.com/learn/Camera_Render?lang=cpp

By default, a camera constantly renders, but you can also set it to only refresh when it is manually triggered to do so. Since all rendering takes place on a separate thread, and may not match the frequency of the main thread, this provides precise control you wouldn't have just by hiding and showing the camera.

[StOneDOes]

Thanks so much for the fast support! And thanks for providing the other rendering recommendations. I'm downloading the update at the moment.

After that I will try and get something working. @SpiderPigthanks for the sample.

[StOneDOes]

After rendering a single frame to my render target, how do I reset the render target to the back buffer? Because I don't see a need to use a second camera for this. The main camera should be able to handle this just fine - its not a realtime visual minimap, just a topdown image of the world.

 

 

[Josh]

You could call Camera::SetRenderTarget with NULL as the argument. If you have any trouble with render synchronization, you can also use a second camera.

[StOneDOes]

Ok, so hypothetically this should work:

//Set camera for minimap snapshot
    camera->SetPosition( 0.f, 150.f, 0.f );
    camera->SetRotation( 90.f, 0.f, 0.f );
    camera->SetProjectionMode( CameraProjectionMode::PROJECTION_ORTHOGRAPHIC );
    camera->SetRealtime( false );

    //Create a render target and render a single frame
    auto renderTarget = CreateTextureBuffer( 128, 128 );
    camera->SetRenderTarget( renderTarget );
    camera->Render();

    //Setup the UI and assign the new minimap texture 
    auto ui = CreateInterface( window );
    auto widget = CreatePanel( 0, 0, 128, 128, ui->root );
    widget->SetTexture( renderTarget->GetColorAttachment() );

    //Reset the render target
    camera->SetRealtime( true );
    camera->SetRenderTarget( nullptr );

    //Main loop

 

However I run into this error before seeing anything:

Error: Widget::SetTexture can only be used with a 3D interface
Exception thrown at 0x00007FF70C845396 in SG23_d.exe: 0xC0000005: Access violation reading location 0x0000000000000378.

I'm not sure why this is the case, because I want a 2D UI, not a 3D one.

If I try using a second camera I do not hit an access violation, but I don't see a texture on screen at all (just the scene, no UI)

//Set camera for minimap snapshot
    auto minimapCamera = CreateCamera( world );
    minimapCamera->SetPosition( 0.f, 150.f, 0.f );
    minimapCamera->SetRotation( 90.f, 0.f, 0.f );
    minimapCamera->SetProjectionMode( CameraProjectionMode::PROJECTION_ORTHOGRAPHIC );
    minimapCamera->SetRealtime( false );

    //Create a render target and render a single frame
    auto renderTarget = CreateTextureBuffer( 128, 128 );
    minimapCamera->SetRenderTarget( renderTarget );
    minimapCamera->Render();

    //Setup the UI and assign the new minimap texture 
    auto ui = CreateInterface( window );
    auto widget = CreatePanel( 0, 0, 128, 128, ui->root );
    widget->SetTexture( renderTarget->GetColorAttachment() );

 

I would expect that if somehow my render to texture failed, that I would still see a grey square on screen? Must have got something wrong.

EDIT:

I tried just putting a button on the screen, and it seems that the button displays on the screen for about a second then once the terrain loads and appears the button is gone. It is caused by calling world->Render( framebuffer ); . 

[SpiderPig]

If you post a complete example I can give it a run for you if you like.  There might be something happening under the hood that's causing an issue with changing the camera's real-time and render target settings.  Otherwise I suggest using a second camera and then just delete that after the texture has been rendered too.  See if that works?

1 hour ago, St0neD0es said:

Error: Widget::SetTexture can only be used with a 3D interface
Exception thrown at 0x00007FF70C845396 in SG23_d.exe: 0xC0000005: Access violation reading location 0x0000000000000378.

Have you set up a UI camera?  This is required for rendering widgets on top of a 3D world.

[StOneDOes]

20 minutes ago, SpiderPig said:

Have you set up a UI camera?  This is required for rendering widgets on top of a 3D world.

Ah ok well I have not done that. There is a sample here https://www.ultraengine.com/learn/CreateInterface?lang=cpp but I'm sure that there is something else that I would have to do since I now have 2 cameras, right?

The snippet I posted is basically just in addition to the terrain sample. If I got back to single camera then perhaps I will post it.

[SpiderPig]

This is pretty much all you have to do for a UI camera.

auto font = LoadFont("Fonts/arial.ttf");
auto ui = CreateInterface(world, font, framebuffer->size);
ui->root->SetColor(0.0f, 0.0f, 0.0f, 0.0f);//Might need this as I think the root panel is not transparent by default?

auto camera = CreateCamera(world, PROJECTION_ORTHOGRAPHIC);
camera->SetPosition(float(framebuffer->size.x) * 0.5f, float(framebuffer->size.y) * 0.5f, 0);

After that any widget you create will be rendered to that ortho camera and will be drawn on top of your scene.

[StOneDOes]

Thanks for your help. The one other thing the UI camera also needs to do is Camera::SetClearMode() . I'll need to have a better read of the documentation next time.

And thanks Josh for adding the new function so quickly.

 

So now here it is. I guess the only other thing I want to do is delete the camera that snapshots the map. I'm assuming that there is correct engine function for this rather than me explicitly deleting it likely causing a crash?  I mean, its not rendering in realtime anyway so it doesn't really matter.

[reepblue]

20 minutes ago, St0neD0es said:

So now here it is. I guess the only other thing I want to do is delete the camera that snapshots the map. I'm assuming that there is correct engine function for this rather than me explicitly deleting it likely causing a crash?  I mean, its not rendering in realtime anyway so it doesn't really matter.

You can just point it to NULL or hide the camera until you need it again. If you're only rendering the camera once, it shouldn't cause an impact on performance.

[Josh]

11 hours ago, St0neD0es said:

However I run into this error before seeing anything:

Error: Widget::SetTexture can only be used with a 3D interface
Exception thrown at 0x00007FF70C845396 in SG23_d.exe: 0xC0000005: Access violation reading location 0x0000000000000378.

You can create an interface directly on a window, in which case the system drawing commands will be used (GDI+, Quartz, XRender). This is the way Ultra App Kit works. This provides the most responsive interface, and it also looks good because it is using the system font and text rendering settings (TrueType, etc.). This mode should be used for desktop applications (like the Ultra Engine editor).

You can also create an interface that gets rendered in Vulkan. This is considered a "3D interface" because it can be rendered flat on the screen, onto an object in the world (like the interactive panels in the more recent DOOM games), or floating in the air for a VR interface. With this mode, you can now display a texture directly on a panel without using a pixmap. Note that when you do this, you must manually feed events into the interface with the Interface::ProcessEvent method. This allows you do to things like map texture coordinates in the 3D world to GUI screen coordinates.

The exception above is my fault, and it will be fixed in the next update. However, that code would not work anyways. When you call Camera::Render it does not render immediately. All it does is store an instruction in a command buffer that tells the rendering camera to increment its number of frames it is supposed to render before it stops rendering. All rendering is on a separate thread, so that command buffer actually doesn't get executed until the next call to World::Render. (Well, it gets added to a stack of command buffers and then triggers a semaphore that tells the rendering thread a new command buffer is ready, the next time it cycles around and is ready for new instructions 🤪)

6 hours ago, reepblue said:

You can just point it to NULL or hide the camera until you need it again. If you're only rendering the camera once, it shouldn't cause an impact on performance.

It would be best to set realtime mode to false and just make a call to Render() whenever the camera refreshes. Otherwise it would be very hard to synchronize things with the rendering thread. When you call Camera::Render() you can be sure the camera will render one more time before it stops. The camera doesn't render immediately, but neither does the surface with the texture it is rendering to, so that latency doesn't matter.

6 hours ago, St0neD0es said:

I'm assuming that there is correct engine function for this rather than me explicitly deleting it likely causing a crash?

It's stored in a shared pointer, so there is no need to delete anything at all. Just set the variable to NULL and it will magically disappear. That goes for all objects in Ultra. 

But in this particular case, I would recommend holding onto the UI camera and just calling Render() whenever you want it refreshed.

[StOneDOes]

@Josh thank you for the detailed response. Very useful stuff.

Something else thats related that I also wanted to ask, can the UI system be used to draw basic shapes eg. square/rectangle? I noticed that you can use Widget::SetShape() but there is no way to set it to to be just an outlined rectangle instead of solid/filled? 

[SpiderPig]

You could probably use a panel with PANEL_BORDER style and set it's background colour to 0.0 alpha.  Or make a custom widget that draws only one block that is an outline.

[Dreikblack]

Also you can draw basically anything with Pixmap and WritePixel method https://www.ultraengine.com/learn/Pixmap_WritePixel?lang=cpp

For example this is how i draw a circle for my custom widget in Ultra App Kit:

void drawCircle(shared_ptr<Pixmap> pixmap, const int centerX, const int centerY, const int radius, unsigned int color) {
	int x = 0;
	int y = radius;
	int delta = 1 - 2 * radius;
	int error = 0;
	while (y >= 0) {
		drawPixel(pixmap, centerX + x, centerY + y, color);
		drawPixel(pixmap, centerX + x, centerY - y, color);
		drawPixel(pixmap, centerX - x, centerY + y, color);
		drawPixel(pixmap, centerX - x, centerY - y, color);
		error = 2 * (delta + y) - 1;
		if (delta < 0 && error <= 0) {
			++x;
			delta += 2 * x + 1;
			continue;
		}
		error = 2 * (delta - x) - 1;
		if (delta > 0 && error > 0) {
			--y;
			delta += 1 - 2 * y;
			continue;
		}
		++x;
		delta += 2 * (x - y);
		--y;
	}
}

 

[StOneDOes]

Is it possible to change the border colour of a widget? The other option is to go the way of writing pixels to a Pixmap.

As you can see the border on my minimap is black, and I would prefer white or maybe another colour.

[SpiderPig]

There is a WIDGETCOLOR_BORDER flag you can set when you use SetColor().  I've not tried it but should work.

[StOneDOes]

I see, perfect. Thanks man.

mViewSquare->SetColor( Vec4( 255.f, 255.f, 255.f, 0.f ), WIDGETCOLOR_BACKGROUND );
mViewSquare->SetColor( Vec4( 255.f, 255.f, 255.f, 255.f ), WIDGETCOLOR_BORDER );