Hey Guys,
Been testing this in large open areas and found that although it works well in closed in and more close up areas there are some things that need to be changed to get it to work properly in a big open world scene with geographic topography made with the editor. If you don't know what I mean --> this is what happens if you use the shader outside with LE Editor hills
As you can see the shader is detecting the edges of polygons as edge lines which it shouldn't be and you can see that as you get to the horizon it starts to blacken out with a really thick black line. This is because the difference in depth on the depth buffer gets less and less the further away it gets. Therefore, I thought of a workaround to make the difference required to make a line dependant on the depth of the pixel in question... If you want to use the shader in the open planes with hills you'll need to change the following code:
From This:
fEdgeDepth = clamp((fDDX + fDDY - 0.01) * 10.0, 0.0, 1.0); //Line Number 24
To Something Like This:
if(fCurrent<=0.5){
fEdgeDepth = clamp((fDDX + fDDY - 0.001) * 10.0, 0.0, 1.0);
} else if(fCurrent>0.5 && fCurrent<=5.0){
fEdgeDepth = clamp((fDDX + fDDY - 0.02) * 10.0, 0.0, 1.0);
} else if(fCurrent>5.0 && fCurrent<=9.0){
fEdgeDepth = clamp((fDDX + fDDY - 5.0) * 10.0, 0.0, 1.0);
} else if(fCurrent>12 && fCurrent<=15.0){
fEdgeDepth = clamp((fDDX + fDDY - 7.0) * 10.0, 0.0, 1.0);
} else if(fCurrent>25 && fCurrent<=28.0){
fEdgeDepth = clamp((fDDX + fDDY - 12.0) * 10.0, 0.0, 1.0);
} else {
fEdgeDepth = clamp((fDDX + fDDY - 20.0) * 10.0, 0.0, 1.0);
}
DaDonik & I tested this in both small and open areas but Unfortunately this also results in losing a bit of detail outlining in the distance but the ultimate outline is still retained (Will have to hard-texture in the lines within the mesh to keep the detail <-- they do that in borderlands). So if your game is in-doors, the original shader code is still better. However, for the great outdoors you'll need to change it so something like this and tweak it to be as close to what you need as possible. I also tried using a function for a smoother transition rather than a big if statement but I didn't get a good result for some reason. Feel free to play around with the idea.
Here is the resulting outline generated after the changes
^^ As you can see the thick line in the distance is reduced and you only see black lines on the very border on the hills. You'll also notice that the vehicle has less inner outlines... This is unfortunately the trade off. However, those inner lines will become visible if you walk closer to the vehicle. Best to hard texture in the inner lines where possible