BrokenPillar

I updated the files with the X and Y swapped in SwizzleTest.zip on the other thread (http://leadwerks.com/werkspace/index.php?/topic/1172-dxt5nm-and-3dc-dds-compression/page__st__20). It seems to fix the problem. This topic has always caused me grief in the past since it varies with each engine (which might explain why the normal looked correct in Unity) and with different software apps you use to bake normals.

Here are the updated files to look at the two cubes with the bumps side by side. It now includes the ability to put specular in the Red Channel and the fixed normal map: SwizzleTest.zip It has a mesh for a cube for all of you other non-software types that understand working with geometry better than drawing a cube and painting it with a material in code. You will need to unpack your shader.pak file first and then backup the standard mesh.frag shader somewhere else (this may be obvious, but just in case). You should then be able to copy the folders over and then just load the SwizzleTest map. The meshes are called DXT5Box.gmf and DXT5nmBox.gmf if you would rather just create your own scene and drop them in. They are supposed to be bumps, not holes. I'm not sure what is going on with the internal normal map calculations that you are referring to Niosop, but these are the versions of the files that gave me the best results (with the + instead of the - for now). Here is a shot of what I get when it is subtracted instead: Notice the pure black spots along his collar. They show up in different places around the model. To me it looks like it is calculating the Z to be too dark when the values are subtracted.

hmmm, just ran out of time to mess around with this today, but I will post some images of what I get when it is a subtract instead of an add...

Ok, here are my final results with the complete shader in place: Left Image is with a separate Spec Map, the image on the right is with the Spec in the Red channel of the DXT5NM. The Spec map gets pretty butchered in the red channel when compared to when it is stored in the Alpha (as expected) but it doesn't seem to have a big impact on the final product. Left Image is using an Uncompressed DDS, the image on the right is using the new DXT5NM method. Left image is the standard shader method, the image on the right is the new DXT5NM method with the spec in the Red channel. Both have exactly the same texture memory footprint. All textures are 1024x 1024 resolution. For anyone that is curious, this is what the new normal map looks like: I couldn't attach a DDS, so you can't see the alpha, but this is what the RGB channels look like I noticed a couple of errors in the example SwizzleTest.zip file I posted early, plus it doesn't have the Spec in Red Channel code integrated. I will clean it up and re-post it.

I edited my character comparison post to include a comparison of uncompressed vs the DXT5NM. I can barely tell any difference between the uncompressed and DXT5NM versions, but a huge difference between the standard DXT5 method and DXT5NM. Niosop, I notices that the derived z channel code you posted gives pure black spots on the model. Your code looks like this: normal.z = sqrt(1.0 - normal.x * normal.x - normal.y * normal.y); but in the website you linked to the two values are added rather than subtracted, like this: z = sqrt( 1 - x*x + y*y ); If I change the "-" to a "+" in my mesh.frag code, I no longer get the black spots and so the overall effect is improved. Is there a reason you switched it to a subtract instead of an add? Still need to make a version with the spec in the red channel...

getting the spec into the red channel isn't too bad. You can just copy the red channel into the alph, and then you can copy your spec into the Red and save it as a DXT5 instead of DXT5NM. The DXT5NM export just automatically copies your red channel into the alpha and then duplicates the Green into the Red and Blue, but still compresses using the same method as DXT5. I'll do a comparison against non-compressed a little later, and with the spec in the red channel.

Left image DXT5, Right image DXT5NM with new shader Apparently a little improved normal map compression goes a long way. One important note is that you have to save to the DXT5NM from the source image. You cannot just save out from an existing DXT5 map. This may be obvious, but I thought it was worth making clear.

Ok, I got the shader to work, and did an initial test with a significant difference in quality. Here is the first screenshot: the two columns on the left are using the standard mesh_diffuse_bumpmap.frag shader. The two columns on the right are with Niosop's new shader. I will clean up my files and post them all for other people that want to look for themselves. I will also try this out on a different mesh that you would actually see in a game (like a character or something) and post those results. Niosop (or anyone else that understands shader language better than me), any chance you could tweak the shader so that you can store the spec in the now unused Red or Blue channels so I can compare using that as well?

Thanks, I'll try to get some tests running later tonight when I get some time to work with the code you posted. It's all explained in great detail in the link in my first post, but the watered down version looks something like this: Source image on the left, compressed image using standard DXT1 or DXT5 method on the right. but with saving a normal map using the DXTnm setting and using the shader Niosop wrote you can get this: Source image on the left, compressed image using DXT5nm method on the right. and if we could get 3Dc working you could get this: Source image on the left, compressed image using 3Dc method on the right. The 3Dc method is the best quality, but it would take more work to implement and you wouldn't be able to store the specular with the normal (it is basically turning all 3 RGB channels into one block similar to the Alpha in a DTX5, so you are left with 2 channels that are the highest possible quality.) Any chance you could add that as well? It would be a better comparison to be able to see a model done the regular way (with diffuse, bump and spec) next to a model with DXT5nm (also with diffuse, bump and spec). Thanks

We discussed this briefly on a different thread, but that thread got a little long so I thought I would start this topic in a fresh one. This link is from nVidia's developer site and describes several methods to try and get better DDS compression results for normal maps: http://developer.nvidia.com/object/real-time-normal-map-dxt-compression.html I do not know much about writing shaders so I was hoping someone could write one for the DXT5nm and 3Dc methods described in this paper. I would then be happy to create some tests and record some video to compare the results so that people can decide whether it is worth using. for DXT5nm, I think you might be able to still store the spec in the red or green channels, but for the 3Dc you would have to save the spec somewhere else. Also, is 3Dc possible through shader language alone, or would something have to be added to the engine?

It's ok, I'm just glad Niosop figured out an objective test to figure it out. Plus now I know way more about DXT compression types than I did three days ago. When I first started reading this thread I was worried I might have to go and try to convince my work to let me convert our entire diffuse texture library into DXT5. I think I am going to start a new thread talking about trying out the DXT5nm and 3Dc compression methods.

wow. Smart test Niosop. Wish I would have thought about that yesterday, would have saved me some time and trouble. I just tried it in Photoshop from a PSD to a DDS the same way. I got the same results.

I did re-open them after saving and I get no difference no matter how far I zoom in on them. I have saved out the results, closed them and re-opended them, and then posted them. They are identical. The overall compression ratio difference is all contained within the Alpha channel. DXT1 is 8:1, DXT5 is 4:1, but all of the additional bit information is in the Alpha channel. I think you are assuming since their are more bits in DXT5 (meaning a better compression ratio) that the RGB channels have to be better. But this would only be the case if some of the additional bits went to the RGB channels, which they don't. The RGB channels are always 5:6:5, but the DXT1 has no Alpha while the DXT5 has an alpha with as much bit information as the entire RGB channels combined, giving it the higher overall compression ration of 4:1. None of it is in the RGB channels, which are done the same way for both. Nvidia says they are the same, not similar: http://developer.nvidia.com/object/real-time-ycocg-dxt-compression.html

I meant that it was a valid comparison between the two images I saved using both methods, not as a comparison against the images you displayed. Yes, and DXT5 is a 4:1 ratio, but the RGB channels in both methods are a fixed 5:6:5. All of the additional bit information in a DXT5 is in the alpha channel. I am just trying my best to help out here, VicToMeyeZR, not try to be confrontational or argumentative. I am new to the forum and community, but am not new to real time graphics. Every article that I can read or find on the topic says the same thing and every test I have performed shows the same thing, that there is no difference in bit quality or compression method in the RGB channels of a DTX1 and DXT5.

You can if you save from a screen dump to DXT1, then from the same screen dump to DXT5, if it is meant to compare the difference between the two methods. And why should I trust that article and my own tests that I have performed by saving several textures using both methods and comparing less than your opinion? or here it is again on nVidias site: http://developer.nvidia.com/object/real-time-ycocg-dxt-compression.html here it is again on Wikipedia: http://en.wikipedia.org/wiki/S3_Texture_Compression or from another forum thread on Polycount: http://boards.polycount.net/showthread.php?t=67519

I am not sure how or what you are saving with. Make sure that you are not saving with DXT1 with 1 bit alpha as that will reduce the RGB channel quality because in that case the 1 bit alpha is being stored at the expense of some of the information in the RGB channels. The RGB channels in DXT5 are saved using the exact same compression method as the RGB channels in DXT1 (without 1 bit alpha). This article explains it completely: http://www.fsdeveloper.com/wiki/index.php?title=DXT_compression_explained

Those images are the images from the doc, right? Their is no DXT5 image on that document for the object space normal map (that you are showing) that was saved with all three channels in RGB. The only DXT5 object space normal map image on that document was saved with one of the channels in the alpha. Again, read the document very closely. That paragraph is on top of the image that you showed for DXT5. The easier test though is just to save out an image as DXT1 and then as DXT5 (with no alpha) and compare. They are identical.

I would recommend learning how to use the stitch command and Relax UV tools. These two tools are huge time savers. Pelt mapping can be useful for organic meshes too. I usually assign the stitch command to a hot key while in the Edit UVs Window because I use it so much. If I remember right, the Next Gen Texture Techniques by Eat 3D goes into all 3 of these pretty well (stitch, relax, and pelt mapping), among other things. You can also get better results from Auto Unwrap by using the Pack UVs tool after they have been automatically unwrapped. I still would recommend learning to lay them out by hand, but it is an option if you don't have time to.

He didn't save anything, that is off of the article that Niosop linked to, and he should read the article a little more closely. The images that he compares are done two different ways. In the first, all 3 channels are stored in the RGB channels: In the second one, one of the channels is being stored in the alpha channel: I did listen to the people who developed the DXT format (That is where I got my information). The reason I am spending time on this is that it has a very large impact on how much memory will be used for textures. There is no difference in quality when you are using only the Red Blue and Green channels. You can also just save out any diffuse texture both ways and compare them yourself. Now whether it would be worth not using the alpha as a spec channel and creating a shader that can read the alpha in for one of the normal channels (like in DTX5nm) is a different matter.

Thanks for the compliments everyone. FYI Marley's Ghost, I kept the grunge on all of the objects in separate layers (diffuse, spec and bump) to make it easy to change the level of dirtiness, so if you want I can increase the opacity on them to make you feel more at home

(Deep breathe) Ok. Please bare with me here and read the entire post very carefully before responding and/or flaming me into oblivion for dragging this on. I know I do not have very much Leadwerks Community street cred, but I believe that this is an extremely important topic considering that DXT5 textures are exactly twice as large as DXT1, and that texture memory is such a huge piece of the overall memory footprint in a game. For example, a 2048 x 2048 texture saved as a DXT5 is 5.5 MB, whereas a 2048 x 2048 texture saved as a DXT1 is 2.75 MB. When added up over a few hundred textures (even if they aren't all 2048s), this becomes a significant consideration very quickly. My initial question in this thread was intended to figure out where I was wrong, which I assumed I was. However, after very carefully reading the document linked to by Niosop (http://developer.nvidia.com/object/real-time-normal-map-dxt-compression.html) and performing a few very specific tests, I am pretty confident that using DXT5 for diffuse textures with no need for an alpha channel is wasteful. So here is my understanding based on my reading and tests. DXT1 is 4 bpp and DXT5 is 8 bpp. So obviously DXT5 has twice as many bits per pixel as DXT1, which makes it sound like it would be better quality all around. The problem is that the additional 4 bits per pixel in DXT5 are entirely in the alpha channel. Or in other words, there is as much bit information in the alpha channel of a DXT5 texture as all three RGB channels combined. Here are my tests with side by side examples to compare. I took a source PSD document, then saved from the source into DXT1 and then from the source into DXT5 (I did not simply re-save the DXT1 as a DXT5). There is a noticeable difference between the source and both compressed textures, but I cannot see any difference at all between DTX1 and DXT5 (the artficating is even identical). I then did the same thing, but with the red channel only to see the specific difference (if any) between a single channel, but got the same results. In the last column, I copied the red channel into the alpha channel and then saved it as a DXT5. This is where the results where more interesting. The red channel, when copied into the alpha channel and saved there, is much higher quality than the one saved into the actual red channel of the texture. This, and the technical explanation of 3Dc compression of normal maps on nVidia's site, is what made me realize that all of the additional 4 bpp in a DXT5 are in the alpha channel. The other, more simple test that I tried was taking any of the diffuse terrain textures that were saved as DXT5 and re-saving them as DXT1, then putting them side by side and comparing. Again, I cannot tell a difference. The only difference that I could see is that the DXT5 has a blank white alpha channel and is twice as big (because it has 4 bpp stored in the blank white alpha channel). Please do the same tests yourself to see if you get the same results. Also, most of the textures that come with the SDK are actually DXT1 (or at least they are exactly half as big as their Dot3 counterparts that have the spec map in the alpha channel). This includes all of the Underground, weapons, FPS shooter hands, etc. The only ones that I could find that appear to be DXT5 are mostly terrain textures. Again, I am not trying to be argumentative or play gotcha with anyone, especially not the creator of an amazing piece of software that most definitely has already forgotten more about real time graphics than I will ever know. I just want to make 100% sure that everyone understands what exactly the differences are between DXT1 and DXT5 before committing to doubling the size of all of their textures that don't actually require an alpha channel. Here is another link that explains it better than I am (please if you read nothing else in this post, read the info in this link): http://www.fsdeveloper.com/wiki/index.php?title=DXT_compression_explained Also, according to this, saving it as a DXT1 with 1 bit alpha will result in poorer quality in the Red Green and Blue channels (not to be confused with standard DXT1 that doesn't store alpha information and handles the RGB channels the same as DXT5). (exhales)

I'm new to the community and a recent Leadwerks convert. I am working on a hobby indie title with a few friends of mine and we quickly realized that Leadwerks is easily the best engine out there for indie development. I updated my online porfolio to almost all rendered images out of Leadwerks and thought I would share a few as a way to introduce myself and what I do (3D modeling and texturing). Anyway, here are a few: One of the main characters, rigged skinned and (partially) animated for our indie title (the hand gun looks small because he is big, like around 7 feet) A fortified old playground/watch tower (with lua scripted joints for the swings, thanks to some help with the hinges on another thread ) Various house props

asdas Unless I am reading that wrong, DTX5 RGB compression is done the same way as DXT1 RGB compression (meaning both are lossy), and the only difference between the two is how the alpha channel information is saved. However, the advice to listen to the creator of the engine sounds like good advice to me, so I will go ahead and follow it... Thanks.

so if I am reading that article correctly, DXT1 is meant for color textures but DXT5 handles normal maps better. This works out with the current suggested practice of storing the spec map as the alpha in the normal map, although I agree that being able to use DXT5nm and 3Dc looks appealing (although it sounds like it would mean that you would no longer be able to store your spec as the alpha channel).

Sorry, I know how to get a dds with an interpolated alpha, my question was if there was a reason to save a texture using DXT5 when you do not want an alpha (such as diffuse textures with no transparency). Josh seemed to be indicating that you should never use DXT1, but I thought the only difference between DXT1 and DXT5 was that DXT5 is able to retain a 4 bpp alpha channel. I did not think there was any difference between DXT1 and DXT5 in terms of RGB channel compression artifacts. So I have always used DXT1 for textures without alpha channels, and DXT5 for textures with alpha channels. Josh's statement has me wondering if this is the wrong thing to do.