Jump to content

Smart Tessellation




I've actually been doing a lot of work to finalize the terrain system, but I got into tessellation, and another rabbit hole opened up. I've been thinking about detailed models in VR. Tessellation is a nice way to easily increase model detail. It does two things:

  • Curved surfaces get smoother (using point-normal triangles or quads)
  • A displacement map can be used to make small geometric detail to a surface.

These are really nice features because they don't require a lot of memory or disk space, and they're automatic. However, tessellation also has a nasty tendency to create cracks in geometry, so it tends to work best with organic models like rocks that are carefully wrapped. I was able to mitigate some of these problems with a per-vertex displacement value, which dampens displacement wherever there is a mesh seam:


However, this does not help round edges become rounder. With this cylinder, because the faces are not a continuous rounded surface, a crack appears at the sharp edges:


What if there was a way to properly tessellate mechanical shapes like this? Wouldn't it be great if all your models could just automatically gain new detail that scales with the camera distance, with no need for extra Lod versions?

Well, I had an idea how it might be possible, and a few hours later I came up with something. Here is the original model with no tessellation applied:



Here is the same model with my new and improved tessellation shader, suitable for organic and mechanical shapes:



A closeup view reveals perfectly aligned geometry with no gaps or cracks:


The distribution of polygons can probably be made more uniform with additional work. This feature uses two additional bytes in the vertex structure to control a normal for tessellated curves. I think it will be possible to automatically detect seams and assign tessellation normals in the final editor. If no separate tessellation normals are assigned, then the default tessellation behavior occurs. This is a very promising technique because it could allow you to add a lot of detail to any model very easily with barely any effort.

  • Like 3


Recommended Comments

We can also be smarter about the distribution of polygons, so tessellation doesn't need to create a lot of unnecessary polygons. The cylinder cap uses the minimal number needed to seal the gaps. If I used a separate mesh made up of quads for the sides, I could reduce the tessellation in one direction, so it would just create a lot of tall skinny polygons around the sides:


  • Like 2
Link to comment

Here it is using quads for the sides, creating only the polygons necessary to represent the detailed shape:


This will need further development, but it's extremely promising. This could provide unlimited detail independently from view location., and it does it without requiring more complex models. You really could model stuff at about this complexity and let the system upsample all the curves automatically. Even if you have to go through and fine-tune some vertices, it's not difficult because you can model things at such a simple resolution. This totally simplifies hard surface modeling:

It's the fulfillment of the promise that hardware tessellation initially failed to deliver. It kind of reminds me of "patches" for curved surfaces in the Quake 3 level editing tools.


I am wrapping this up now, but I think this feature will definitely play a big role in the future.

  • Like 1
Link to comment

If you create a bent cylinder like this:

auto model = CreateCylinder(world, 5, 10, 8, 3, 1);

float a;
auto piv = CreatePivot(NULL);
piv->SetPosition(10, 0, 0);

for (auto mesh : model->lods[0]->meshes)
    for (auto& vertex : mesh->vertices)
        vertex.position.y += 5.0f;
        a = -90.0f * (vertex.position.y / 10.0f);
        vertex.position.y = 0.0f;
        vertex.position = TransformPoint(vertex.position, model, piv);
        piv->SetRotation(0, 0, a);
        vertex.position = TransformPoint(vertex.position, piv, model);
        vertex.normal = TransformNormal(vertex.normal, piv, model);
        vertex.tangent = TransformNormal(vertex.tangent, piv, model);
        vertex.bitangent = TransformNormal(vertex.bitangent, piv, model);
        if (vertex.tessnormal != Vec3(0.0f)) vertex.tessnormal = TransformNormal(vertex.tessnormal, piv, model);

Here is the result. Not bad. I think the quad interpolation needs a little improvement, but it's doing what it should do:


  • Like 2
Link to comment

Can tessellation actually replace regular high poly geometry? Because I remember trying and realizing that basic displacement in Blender gives far better results than a tessellation shader with the same displacement map.

Tessellated stone has lighting like it's a flat plane even though there's crazy amount of geometry. And displacement in Blender with exactly the same height map and distance actually looks good. Both are based on the same flat mesh. That shader even had recalculation of displaced Normals and Tangents.



  • Like 1
Link to comment

@Genebris Maybe there is a problem with the lighting / normals in that tess shader. It also looks like the displacement map is barely being used, like the bricks are not sticking out very far.

  • Like 1
Link to comment

This is so cool. This mesh was processed automatically after creating a box, so the tessellation is curving the right way on the flat faces. I know now how to write a better algorithm for processing the mesh, and it will eliminate some of the extra polys you see here...


If I can just make a bevel work on that flat face, I will be totally satisfied with this. Real-life industrial objects tend to be simple variations of bevelled surfaces and cylinders because they are still limited by the manufacturing process.

  • Like 1
Link to comment
8 hours ago, IceBurger said:

What if the "extra faces" are needed for adding detail to the geometry?

If a displacement map is in use, then the face is fully tessellated. I think I can actually do some texture reads to figure out whether tessellation is needed or not in each area, so you should be able to do things like have just a few bricks sticking out of a wall, and only the area around the bricks gets tessellated.

  • Upvote 2
Link to comment

A simple mesh (16 polys) transforms into two different shapes with different tessellation parameters. Notice the straight cone still has a curved edge on the bottom:


  • Like 3
Link to comment

Here we have an arch shape that has tessellation settings automatically deduced from the mesh topography. Notice that all faces only tessellate in one direction. I estimate this mesh is about 1500 quads. If it were tessellated on two axes, like the arch above, it would be around 25,000 quads (50,000 triangles). So managing the tessellation carefully should make a huge different in performance.


  • Like 1
Link to comment

Here is a rounded arch. The tessellation settings were deduced from the mesh. The only part is missed is that center piece. There's no gap there, it just forms a straight edge along the center. There's not actually any indication in the mesh that part is supposed to be rounded, so I am leaning towards making the automatic calculation conservative, and then anything beyond that needs to be manually adjusted on a per-polygon basis. The mesh is only 34 quads, so manually adjusting some faces shouldn't be a problem.


  • Like 2
Link to comment
Add a comment...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Create New...