Non-instancing copy command

[Niosop]

As I understand it the current CopyEntity command creates a new instance of an existing entity. So if I wanted to swap out the texture on just one instance of a model there's currently no way to do this because changing the texture on one would change it for all instances of that model and even another call to LoadModel would just create another instance of the original. I suppose you could pass all the possible textures to a custom shader and use some per instance value like color to choose between them, but that could get unwieldy very quickly. Could an argument be added to CopyEntity that would allow creation of a totally separate, non-instanced copy of an entity so that per model properties (like the material) could be changed without affecting the other instances?

 

Or if I'm missing some method for this that's already in place, let me know

[Rick]

I agree. Having the same model but different textures is very common.

[omid3098]

You can do this by assigning a new material in object properties-appearence pannel into Leadwerks editor.

 

 

oops my bad it will change all instances, sorry

[Masterxilo]

and even another call to LoadModel would just create another instance of the original

You just need to copy (maybe also renaming would work) all the files and then load these, this will give you a new model which is no instance of any other entity.

Loading the same file does indeed always return just an instance.

 

I wrote these utility functions that do exactly this:

#include <string>
using namespace std;

TEntity LoadMeshUninstanced(str name, TEntity parent = 0)
{
// Prepare filename
static int loadedMeshes = 0; loadedMeshes++;
char buf[10];

string originalFilePath = AbstractPath(name);
originalFilePath.erase(originalFilePath.length()-4, 4);

string newFilePath = originalFilePath;
newFilePath += "_meshInstance";
newFilePath += itoa(loadedMeshes, buf, 10);

// # Copy all files
CopyFile((originalFilePath+".gmf").c_str(), (newFilePath+".gmf").c_str(), false);

// # Load model
TEntity mesh = LoadMesh(const_cast<str>((newFilePath+".gmf").c_str()), parent);

// # Delete temp files
DeleteFile((newFilePath+".gmf").c_str());

return mesh;
}

TEntity LoadModelUninstanced(str name, TEntity parent = 0)
{
// Prepare filename
static int loadedModels = 0; loadedModels++;
char buf[10];

string originalFilePath = AbstractPath(name);
originalFilePath.erase(originalFilePath.length()-4, 4);

string newFilePath = originalFilePath;
newFilePath += "_instance";
newFilePath += itoa(loadedModels, buf, 10);

// # Copy all files
// Models and lods
long hr = 1;
for(int lod = 0; hr != 0; lod++)
{
	string lodAdd = "";
	if(lod > 0)
	{
		lodAdd += "lod";
		lodAdd += itoa(lod, buf, 10);
	}

	hr = CopyFile((originalFilePath+lodAdd+".gmf").c_str(), (newFilePath+lodAdd+".gmf").c_str(), false);
}
// Other files
CopyFile((originalFilePath+".ini").c_str(), (newFilePath+".ini").c_str(), false);
CopyFile((originalFilePath+".lua").c_str(), (newFilePath+".lua").c_str(), false);
CopyFile((originalFilePath+".phy").c_str(), (newFilePath+".phy").c_str(), false);

// # Load model
TEntity model = LoadModel(const_cast<str>((newFilePath+".gmf").c_str()), parent);

// # Delete temp files
hr = 1;
for(int lod = 0; hr != 0; lod++)
{
	string lodAdd = "";
	if(lod > 0)
	{
		lodAdd += "lod";
		lodAdd += itoa(lod, buf, 10);
	}

	hr = DeleteFile((newFilePath+lodAdd+".gmf").c_str());
}
DeleteFile((newFilePath+".ini").c_str());
DeleteFile((newFilePath+".lua").c_str());
DeleteFile((newFilePath+".phy").c_str());

return model;
}

 

Usage example:

int main(void)
{
// Init
Initialize();
Graphics(640,480);
TEntity world = CreateWorld();
TEntity buffer = CreateBuffer(640,480,BUFFER_COLOR0|BUFFER_DEPTH|BUFFER_NORMAL|BUFFER_COLOR2);

// Cam
TEntity camera = CreateCamera();
MoveEntity(camera, Vec3(0,0,-2.5) );

// Light
TEntity light  = CreateSpotLight(15,0);
MoveEntity   (light,  Vec3(-1,5,-4) );
RotateEntity (light,  Vec3(45,0,0), 0);
SetShadowmapSize(light,512);
AmbientLight(Vec3(.05));

// Ground
TEntity plane  = CreateCube(0);
ScaleEntity  (plane,  Vec3(100,1,100) );
MoveEntity   (plane,  Vec3(0,-2.5,0) );

// Model
TEntity m1 = LoadModelUninstanced("abstract::oildrum.gmf");
PositionEntity   (m1,  Vec3(-1.5,0,0) );

TEntity m2 = LoadModelUninstanced("abstract::oildrum.gmf");
PositionEntity(m2,  Vec3(1.5,0,0) );
PaintEntity(GetChild(m2,1), LoadMaterial("abstract::oakbranch.mat"));

// Main Loop
while(!KeyHit(KEY_ESCAPE) && !AppTerminate())
{
	UpdateWorld();

	SetBuffer(buffer);
	RenderWorld();
	SetBuffer(BackBuffer());
	RenderLights(buffer);

	Flip(1);
}

// Terminate
return Terminate();
}

Notice both oil drums would have the "oakbranch.mat" applied if you used LoadModel.

[L B]

Lumooja once posted a way of creating 2 objects without being instances. It was in the old forum, for a guy who tried to apply an outline shader to a model that was hovered.

[Rick]

Do we have any way to make a copy of the model/mesh from memory instead of from file? That would be way cooler and cleaner as this "could" cause files to get left behind if something happens between making the file copy and deleting it. Although it's unlikely it would just be cleaner to be able to copy the memory of a loaded object already. I have to imagine there is a way to do this. I mean if you have a pointer to the loaded model in memory, we should be able to copy that memory to another location and get a pointer to it. I'm not that handy with low level memory stuff though, but it would be much cleaner that way.

 

Can't we do something with memcpy() in C/C++?

[Niosop]

Yes, I was thinking of taking this approach, the main problem with this is you can't use the zip file system without a whole bunch more hassle opening the zip, getting and renaming all objects, then recreating a new zip. So it might not be allowed by the Dexsoft license because you'd end up w/ unprotected files (unless the GMF format counts as protection). An in memory system would rock, or a way to change an instance of a model into an independent model. Maybe like Model:MakeUnique() or something.

[Rick]

An in memory system would rock

 

I have to imagine this is possible. I mean if we know the way it's stored in memory and we have a pointer to that memory, there must be a way to copy that memory to a new memory location. That would essentially create a new instance of that model and allow you to do whatever to it.

 

 

memcpy

 

function

<cstring>

 

void * memcpy ( void * destination, const void * source, size_t num );

 

Copy block of memory

Copies the values of num bytes from the location pointed by source directly to the memory block pointed by destination.

 

The underlying type of the objects pointed by both the source and destination pointers are irrelevant for this function; The result is a binary copy of the data.

 

So we need to get the size of a load model object in memory. I guess I just don't know how to get the engine to see this new model memory though. It would have to be able to be added to the world somehow. LoadModel() must do this. If we knew how it does that we could make this work I think.

 

Either way, this should 100% be added by Josh to the engine. It's such a basic thing to have.

[Niosop]

Can't just do a memcpy because the structure contains pointers to it's sub components. So even if you copy the parent structures data, it would point to the same child data. You'd need to do it recursively, and without more information on how stuff is laid out internally we can't do it from our side. Plus this might wreak havok w/ Blitzmax's garbage collection and stuff, I really don't know.

[Rick]

You'd need to do it recursively, and without more information on how stuff is laid out internally we can't do it from our side.

 

Where are the hackers at? I'm sure it can be figured out by some nice memory snooping tools. I wouldn't think that's a big deal to Josh.

[Niosop]

Would moving the material from per model to per instance cause a big performance hit? Some uniforms are passed per instance (like color, scale, etc), so is there something special about the textures or shaders that would break batching if they were per instance?

[Rick]

I think it would just mean you couldn't have a scene with a large amount of the same model because it would take up so much memory compared to the way it is now. I would say things like tree's but I think he made that neat trick with vegetation where you can have a ton of trees and it doesn't take a big hit.

[Masterxilo]

Some uniforms are passed per instance (like color, scale, etc), so is there something special about the textures or shaders that would break batching if they were per instance?

Nothing except the transformation matrix is passed per instance.

The colored is stored in the 4th row of it (which is actually a "hack").

[Niosop]

Yeah, if they can't share texture memory that would be a killer, but I would think that you could still share texture memory. What has to be the same for VBO to work? The vertex data, the normal data and the texture coordinates? I wouldn't think the actual texture data would have to be the same, but I'm just making wild guesses.

 

EDIT: Just saw Masterxilo's response. If the transformation matrix is the only thing that can change to still get the benefits of batching then we're back to only being able to do it by setting a material w/ the possible textures in texture slots and using something like the alpha channel info in the shader to choose among them.

[L B]

Where are the hackers at? I'm sure it can be figured out by some nice memory snooping tools. I wouldn't think that's a big deal to Josh.

 

For Josh it wouldn't, since he has access to the actual BMax members. For us, it would be quite a lot of trouble.

[TylerH]

@Niosop:

 

The only thing that has to be the same in a VBO is vertex data, normal data, and texture coordinate data. If you are using primary/secondary colors or other indices for the array pointers in the VBO, those must also be the same, but since the textures are passed as samplers to the shader per-mesh, they could be different.

 

From what I remember, this instancing behavior is something Leadwerks implemented in its own way using some kind of reference system vs. instances.

[Josh]

The reason instancing is used is because right now the engine does this:

 

-Make an array of mat4s for all the visible instances.

-Set the material/shader/textures.

-Set the vertex buffers

-Tell the GPU "take this array of matrices and render N copies of it". This is only one draw call that draws all instances of the object.

-Unset the vertex buffers

-Unset the material/textures/shader.

 

This makes it very very fast to draw lots of copies of an object.

[Rick]

But it causes issues when we want to have 1 model with different textures, which is a very common approach to things. Couldn't we have both worlds where by default the above you have happens, but if we choose to we can make them handled as if they were different models completely so they can have different textures? Us having to copy model files on disk is a pretty bad hack and can lead to issues with copies of model files cluttering up the players hard drive if issues happen between making the model and copying it.

 

Scenery and such is perfect for the way you have it, but a character is a very common place for the same model used with different textures.

 

Also, in my character object I'd love to have settings for changing textures in the editor. How easy it would be to populate a scene with what looks like so many different characters if we had this ability all from 1 model on disk. Scenes could come to life, because right now I'm tired of looking at woods.

[Josh]

You can actually do what you describe using a shader, without losing instancing. Put four different textures on one, and use the entity color to determine the texcoord offset for which texture is used. That's how I would do it.

[TylerH]

Why should we have to use a shader?

 

Why can't you just add support for non-instanced entities? Or atleast describe whether or not it is viable.

 

I am sick and tired of these hacky and backwards solutions to things that have a clear and concise way to be solved, it is just a bit irritating, and is a sign of ignorance and laziness in my honest opinion. If you want to know why people *cough* want source code, it is so they can fix things like this that you simply dodge in your response.

 

EDIT: Since I know people can't tell my tone properly from my posts, I just want to clarify this is in no way intended to be hostile. It is to be taken lightly, with a slight humorous undertone, but with a stern point about some general flaws that are causing many of the disagreements and quarrels around here.

[Niosop]

You can actually do what you describe using a shader, without losing instancing. Put four different textures on one, and use the entity color to determine the texcoord offset for which texture is used. That's how I would do it.

 

Cool, hadn't thought of using a texture offset. If we keep it to 4096x4096 texture sizes then we get 16 1k textures per texture sheet and could even use multiple textures. Red channel determines where on the texture sheet to look, green channel determines which texture sheet to use or something. Between the two of them that should be plenty. It's not as "clean" as non-instanced copies, but it should have better performance.

[Rick]

You can actually do what you describe using a shader, without losing instancing. Put four different textures on one, and use the entity color to determine the texcoord offset for which texture is used. That's how I would do it.

 

 

I have no idea how to do that. The problem is most models you buy come with 1 model and multiple textures and have coords setup already. Also, is there any limitations to your idea?

 

 

Tyler, I can't understand how anyone is supposed to read anything but anger in your post.

[wailingmonkey]

a shader like this would be handy for lots 'o folks. ( <- gentle nudge to shader-minded generous programmer-types)

[Niosop]

I'm going to need to learn shaders programming at some point so I'll take a shot at this. I think I see how to do it, I'll just add a #MULTI-TEXTURE define to mesh.frag that will use the alpha channel of the color to map up to 16 textures in a single texture sheet. Should have something to show shortly.

[Josh]

It's not a hacky solution. It's efficient and works really well with the way the hardware works.

 

Non-instanced copies might someday be added, but you should be aware you are asking me to increase bugs and decrease performance.