Jump to content
Search In
  • More options...
Find results that contain...
Find results in...

Normal buffer issues

Recommended Posts

I'm trying to make a cel shaded shader and am working on the outline part. Everything seems to work well except for this:




It seems to me like the normals past a certain depth are not uniform and I'm not really sure why. Or is this just Nyquist-like effect? If so, what would be the best way to handle this?


This is the shader I wrote (there's a simple lua script that binds the diffuse, depth and normal textures):


#version 400
uniform sampler2D texture1; //diffuse
uniform sampler2DMS texture2; //depth
uniform sampler2DMS texture3; //normal
uniform vec2 camerarange;
uniform bool isbackbuffer;
uniform vec2 buffersize;
out vec4 fragData0;
#define width 2
float DepthToZPosition(in float depth) {
return camerarange.x / (camerarange.y - depth * (camerarange.y - camerarange.x)) * camerarange.y;
void main(void)
vec2 texcoord = gl_FragCoord.xy/buffersize + 0.5/(buffersize*0.5);
if (isbackbuffer) texcoord.y = 1.0 - texcoord.y;
vec4 c = texture(texture1, texcoord);

//Line Detection
bool edge = false;
float depth = DepthToZPosition(texelFetch(texture2,ivec2(texcoord*buffersize),0).x);
float depth_temp = 0;
vec3 normal = texelFetch(texture3,ivec2(texcoord*buffersize),0).xyz;
vec3 normal_temp = vec3(0);

// Check adjacent pixels
for (int x=-1; x<2; x+=2)
for (int y=-1; y<2; y+=2) {
depth_temp = DepthToZPosition(texelFetch(texture2,ivec2(texcoord*buffersize)+ivec2(x*width,y*width),0).x);
normal_temp = texelFetch(texture3,ivec2(texcoord*buffersize)+ivec2(x*width,y*width),0).xyz;
normal_temp = normal_temp/length(normal_temp);

if ((abs(dot(normal_temp,normal)) < 1) && (abs(depth_temp-depth) > .1)) {
edge = true;


fragData0 = c;

if (edge) {
fragData0 = vec4(0,0,0,c.a);


Link to post
Share on other sites

Yeah I'm normalizing them, but after a certain depth, the normals seem to change a little. Maybe it's due to floating point precision? Either way, I was able to find a workaround by comparing the dot product of adjacent normals with a threshold that changes based on depth.

Link to post
Share on other sites

normals should be sampled like this I believe:

vec3 normal = normalize(texelFetch(texture3,ivec2(texcoord*buffersize),0).xyz * 2.0f - 1.0f);


also Leadwerks assign depth,diffuse,normals to texture0,1,2 default, so you don't really need a lua for the assigning if you use them in this order smile.png

  • Upvote 1
Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Create New...