As the first release of Ultra Engine approaches, it seems clear that the best way to maximize its usefulness is to make it as compatible as possible with Leadwerks. To that end, I have implemented the following features.
Native Loading of Leadwerks File Formats
Ultra Engine loads and saves DDS, glTF, and OBJ files. Other formats are supported by plugins, both first and potentially third-party, for PNG, JPG, BMP, TGA, TIFF, GIF, HDR, KTX2, and other files. Additionally, all Leadwerks fi
Midjourney is an AI art generator you can interact with on Discord. To use it, first join the Discord channel and enter one of the "newbie" rooms. To generate a new image, just type "/imagine" followed by the keywords you want to use. The more descriptive you are, the better. After a few moments four different images will be shown. You can upsample or create new variations of any of the images the algorithm creates.
And then the magic begins:
Here are some of the images I "
Autodesk 3ds Max now supports export of glTF models, as well as a new glTF material type. The process of setting up and exporting glTF models is pretty straightforward, but there are a couple of little details I wanted to point out to help prevent you from getting stuck. For this article, I will be working with the moss rocks 1 model pack from Polyhaven.
Getting geometry into 3ds Max is simple enough. I imported the model as an FBX file.
To set up the material, I opened the compa
I've been working hard getting all the rendering features to work together in one unified system. Ultra Engine, more than any renderer I have worked on, takes a lot of different features and integrates them into one physically-based graphics system. A lot of this is due to the excellent PBR materials system that Khronos glTF provides, and then there are my own features that are worked into this, like combined screen-space and voxel ray traced reflections.
Anyways, it's a lot of di
As I have stated before, my goal is not to build a marketplace of 3D models, but instead to just make sure our model loading code reliably loads all 3D models that are compliant with the glTF specification. I started testing more 3D models from Sketchfab, and found that many of them are using specular/gloss materials. At first I thought I could just fudge the result, but I wasn't getting very good results, and the Windows 10 3D Object Viewer was showing them perfectly. This made me very upset be
HDR skyboxes are important in PBR rendering because sky reflections get dampened by surface color. It's not really for the sky itself, but rather we don't want bright reflections to get clamped and washed out, so we need colors that go beyond the visible range of 0-1.
Polyhaven has a large collection of free photo-based HDR environments, but they are all stored in EXR format as sphere maps. What we want are cubemaps stored in a single DDS file, preferably using texture compression.
I'm wrapping up the terrain features now for the initial release. Here's a summary of terrain in Ultra Engine.
Terrains are an entity, just like anything else, and can be positioned, rotated, or scaled. Non-square terrains are supported, so you can create something with a 1024x512 or whatever resolution. (Power-of-two sizes are required.)
Ultra Engine includes an API that lets you modify terrain in real-time. I took something very complicated and distilled it down
I've actually been doing a lot of work to finalize the terrain system, but I got into tessellation, and another rabbit hole opened up. I've been thinking about detailed models in VR. Tessellation is a nice way to easily increase model detail. It does two things:
Curved surfaces get smoother (using point-normal triangles or quads)
A displacement map can be used to make small geometric detail to a surface.
These are really nice features because they don't require a lot of mem
Ultra Engine makes much better use of class encapsulation with strict public / private / protected members. This makes it so you can only access parts of the API you are meant to use, and prevents confusion about what is and isn't supported. I've also moved all internal engine commands into separate namespaces so you don't have to worry about what a RenderMesh or PhysicsNode does. These classes won't appear in the intellisense suggestions unless you were to add "using namespace UltraRender" to y
Happy Friday! I am taking a break from global illumination to take care of some various remaining odds and ends in Ultra Engine.
Variance shadow maps are a type of shadowmap filter technique that use a statistical sample of the depth at each pixel to do some funky math stuff. GPU Gems 3 has a nice chapter on the technique. The end result is softer shadows that run faster. I was wondering where my variance shadow map code went, until I realized this is something I only prototyped in OpenGL a
I'm finalizing the shaders, and I was able to pack a lot of extra data into a single entity 4x4 matrix:
Orthogonal 4x4 matrix
Per-entity texture mapping offset (U/V), scale (U/V), and rotation
Bitwise entity flags for various settings
Linear and rotational velocity (for motion blur)
All of that info can be fit into just 64 bytes. The shaders now use a lot of snazzy new function like this:
void ExtractEntityInfo(in uint
I have more than one light bounce working now, and it looks a lot nicer than single-bounce GI. The ambient light here is pure black. All light is coming off from the direct light, and bouncing off surfaces.
It will take some time to get the details worked out, and more bounces will require more memory. I'm actually kind of shocked how good looking it is. This is just a single 128x128x128 volume texture at 0.25 meters per voxel. Light leaks seem to be not a problem, even at that low re
Previously I wrote about introducing latency to the voxel cone step tracing realtime global illumination system. The idea here is to improve performance and quality, at the cost of a small delay when the GI calculation gets updated. The diffuse GI lighting gets cached so the final scene render is very fast.
Here's what a gradual GI update does. Of course, this will be running unseen in the background for the final version, but this shows what is actually happening:
My new video pro
Since previously determining that voxels alone weren't really capable of displaying artifact-free motion, I have been restructuring the GI system to favor speed with some tolerance for latency. The idea is that global illumination will get updated incrementally in the background over the course of a number of frames, so the impact of the calculation per frame is small. The new GI result is then smoothly interpolated from the old one, over a period of perhaps half a second. Here's a shot of the r
I have not used the engine outside the editor in a while, but I needed to for performance testing, so now I am back to real-time rendering. During the development of the GI system I broke most of the light types, so I had to spend a couple of days getting those to work again. While doing this, I decided to resolve some longstanding issues I have put off.
First, the PBR lighting will use a default gradient in place of the skybox, if no reflection map is set for the world. This is based off t
Before proceeding with multiple GI volumes, I decided to focus on just getting the lighting to look as close to perfect as possible, with a single stage.
Injecting the ambient light into the voxel data made flat-lit areas appear much more "3D", with color bleeding and subtle contours everywhere.
Lighting + albedo
Some adjustments to the way the sky color is sampled gave a more lifelike appearance to outdoor lighting.
Until now, all my experiments with voxel cone step tracing placed the center of the GI data at the world origin (0,0,0). In reality, we want the GI volume to follow the camera around so we can see the effect everywhere, with more detail up close. I feel my productivity has not been very good lately, but I am not being too hard on myself because this is very difficult stuff. The double-blind nature of it (rendering the voxel data and then using that data to render an effect) makes development ver
Adding emission into the cascaded voxel cone step tracing global illumination and dynamic reflections system (SEO ftw) was simple enough:
There's some slight trailing but it looks okay to me. There is a bit of a "glitch" in that when the emissive surface gets near the wall, the ambient occlusion kicks in, even though the sphere is self-illuminating. This happens because the emission color is mixed with the light voxel during the rasterization step. I could fix this by storing emis
I had to spend several weeks just eliminating light leaks and other artifacts, and getting the results I wanted in a variety of scenes. The results are looking good. Everyone who tries implementing this technique has problems with light leaks but I have fortunately been able to avoid this with careful planning:
Now that I have nice results with a single volume texture centered at the origin, it's time to add additional stages. The idea is to have a cascading series of volume textures
Finally, finally, finally, finally, for the first time since I started working on this feature several years ago, finally we have real-time global illumination with a second light bounce: Below you can see the direct light hitting the floor, bounding up to the ceiling, and then being reflected back down on the floor again.
Performance is still good and I have not started fine-tuning optimization yet. I was just trying to get the effect working at all, which was quite difficult to do,
Now that I have the downsampled reflection data working, I can start casting rays. The cone step tracing is not a 100% perfect representation of physical light, but it gives a very favorable balance of quality and performance. Somehow I came up with a few formulas that eliminate light leaks and other artifacts.
Quite honestly I did not think the results would be this good. Indoor / outdoor scenes with thin walls are very difficult to prevent light leaks in, but somehow it's working ve
For downsampling of GI voxel data, I found that a compute shader offers the best performance. The first step was to add support for compute shaders into Ultra Engine.
I've never used these before but I was able to get them working pretty quickly. I think the user API will look something like this:
//Load compute shader
auto module = LoadShaderModule("Shaders/Compute/test.comp.spv");
auto shader = CreateShader();
//Create work group
After testing and some discussion with other programmers, I decided to try performing voxelization on the GPU instead of the CPU. The downside is the memory usage is much higher than a sparse voxel octree, but I found that sparse voxel octrees were very slow when it came to soft reflections, although the results of the sharp raycast were impressive:
You can read the details of GPU voxelization here if you wish.
Initially I thought the process would require rendering the
I've got cone step tracing working now with the sparse voxel octree implementation. I actually found that two different routines are best when the surface is rough or smooth. For sharp reflections, and precise voxel raytracing works best:
For rough surfaces, cone step tracing can be used. There are some issues to work out and I need to revisit the downsampling routine, but it's basically working:
Here's a video showing the sharp raycast in motion. Performance is quite good
I've moved on to one of the final steps for voxel cone step tracing, which is downsampling the lit voxels in a way that approximates a large area of rays being cast. You can read more about the details of this technique here.
This artifact looks like a mirror that is sunken below the surface of some kind of frame. It was appearing because the mesh surface was inside the voxel, and neighboring voxels were being intersected. The solution was to move the ray starting point out of the voxel the