Assimp loading code

[Josh]

I am starting to investigate the assimp library. Posting some code to come back to when I am ready to continue:

    Assimp::Importer importer;
    auto d = CurrentDir();
    auto path = d.ToUtf8String() + "/Models/xxxxxxxxx.gltf";

    const aiScene* scene = importer.ReadFile(path, 0);

    if (!scene || scene->mFlags & AI_SCENE_FLAGS_INCOMPLETE || !scene->mRootNode) {
        // Error handling if the file fails to load
        return -1;
    }

    auto node = scene->mRootNode;
    
    auto name = node->mName;

    // Process meshes attached to the current node (if any)
    for (unsigned int i = 0; i < node->mNumMeshes; i++)
    {
        aiMesh* mesh = scene->mMeshes[node->mMeshes[i]];
        // Process the mesh as needed
    }

    // Recursively process child nodes
    for (unsigned int i = 0; i < node->mNumChildren; i++)
    {
        
    }

 

[Josh]

It doesn't appear the library supports loading files from memory, if the model consists of more than one file, like external textures, or in the case of glTF, an external binary file. I asked about it here but have not received a response yet:
https://github.com/assimp/assimp/issues/5511

[klepto2]

you can add your own IOSystem to the importer like this

Assimp::AndroidJNIIOSystem *ioSystem = new Assimp::AndroidJNIIOSystem(app->activity);
if ( nullptr != iosSystem ) {
  importer->SetIOHandler(ioSystem);
}

https://assimp-docs.readthedocs.io/en/latest/usage/use_the_lib.html#using-custom-io-logic-with-the-c-class-interface

[Andy90]

This is interesting. I dont know this. Maybe its possible with this to load files from a server.

[Josh]

55 minutes ago, Andy90 said:

This is interesting. I dont know this. Maybe its possible with this to load files from a server.

Yes, I believe this is what I was asking for.

[Josh]

I think the next step is to work this library into a standard Ultra C++ project so we can all test different files and see how well it works.

[Andy90]

Sounds good

[Josh]

Okay, here it is. I will let you guys experiment with this and see what you think:

modelimport.zip

[Josh]

It seems pretty easy to load models with this:

#include "UltraEngine.h"
#include "ComponentSystem.h"
#include "assimp/include/assimp/Importer.hpp"
#include "assimp/include/assimp/scene.h"
#include "assimp/include/assimp/postprocess.h"

using namespace UltraEngine;

shared_ptr<Model> AILoadModel(const aiScene* scene, aiNode* node, shared_ptr<World> world, shared_ptr<Entity> parent)
{
    auto model = CreateModel(world);
    model->SetParent(parent);
    model->name = std::string(node->mName.C_Str());

    // Process meshes attached to the current node (if any)
    for (unsigned int i = 0; i < node->mNumMeshes; i++)
    {
        MeshPrimitives mode = MeshPrimitives(0);
        aiMesh* aimesh = scene->mMeshes[node->mMeshes[i]];
        switch (aimesh->mPrimitiveTypes)
        {
        case aiPrimitiveType_POINT:
            mode = MESH_POINTS;
            break;
        case aiPrimitiveType_LINE:
            mode = MESH_LINES;
            break;
        case aiPrimitiveType_POLYGON:
            mode = MESH_QUADS;
            break;
        case aiPrimitiveType_TRIANGLE:
            mode = MESH_TRIANGLES;
            break;
        }
        if (mode != 0 and aimesh->HasPositions())
        {
            auto mesh = model->AddMesh(mode);
            Vec3 pos, norm;
            Vec2 texcoords;
            for (int v = 0; v < aimesh->mNumVertices; ++v)
            {
                pos.x = aimesh->mVertices[v].x;
                pos.y = aimesh->mVertices[v].y;
                pos.z = aimesh->mVertices[v].z;
                if (aimesh->HasNormals())
                {
                    norm.x = aimesh->mNormals[v].x;
                    norm.y = aimesh->mNormals[v].y;
                    norm.z = aimesh->mNormals[v].z;
                }
                if (aimesh->GetNumUVChannels() > 0)
                {
                    texcoords.x = aimesh->mTextureCoords[0]->x;
                    texcoords.y = aimesh->mTextureCoords[0]->y;
                }
                mesh->AddVertex(pos, norm, texcoords);
                if (aimesh->GetNumUVChannels() > 1)
                {
                    texcoords.x = aimesh->mTextureCoords[1]->x;
                    texcoords.y = aimesh->mTextureCoords[1]->y;
                    mesh->SetVertexTexCoords(v, texcoords);
                }
                if (mode == MESH_POINTS) mesh->AddIndice(v);
            }
            if (mode != MESH_POINTS)
            {
                for (int v = 0; v < aimesh->mNumFaces; ++v)
                {
                    switch (mode)
                    {
                    case MESH_LINES:
                        mesh->AddPrimitive(aimesh->mFaces[v].mIndices[0], aimesh->mFaces[v].mIndices[1]);
                        break;
                    case MESH_TRIANGLES:
                        mesh->AddPrimitive(aimesh->mFaces[v].mIndices[2], aimesh->mFaces[v].mIndices[1], aimesh->mFaces[v].mIndices[0]);
                        break;
                    case MESH_QUADS:
                        mesh->AddPrimitive(aimesh->mFaces[v].mIndices[3], aimesh->mFaces[v].mIndices[2], aimesh->mFaces[v].mIndices[1], aimesh->mFaces[v].mIndices[0]);
                        break;
                    }
                }
            }
            if (aimesh->mMaterialIndex)
            {
                auto aimtl = scene->mMaterials[aimesh->mMaterialIndex];
                
                aiString diffuseTexturePath, normalTexturePath, metalTexturePath, roughnessTexturePath, displacementTexturePath, occlusionTexturePath, specularTexturePath, glossTexturePath;
                
                if (aimtl->GetTextureCount(aiTextureType_DIFFUSE)) aimtl->GetTexture(aiTextureType_DIFFUSE, 0, &diffuseTexturePath);             
                if (aimtl->GetTextureCount(aiTextureType_NORMALS)) aimtl->GetTexture(aiTextureType_NORMALS, 0, &normalTexturePath);
                if (aimtl->GetTextureCount(aiTextureType_METALNESS)) aimtl->GetTexture(aiTextureType_METALNESS, 0, &metalTexturePath);
                if (aimtl->GetTextureCount(aiTextureType_DIFFUSE_ROUGHNESS)) aimtl->GetTexture(aiTextureType_NORMALS, 0, &roughnessTexturePath);
                if (aimtl->GetTextureCount(aiTextureType_SPECULAR)) aimtl->GetTexture(aiTextureType_SPECULAR, 0, &specularTexturePath);
                if (aimtl->GetTextureCount(aiTextureType_SHININESS)) aimtl->GetTexture(aiTextureType_SPECULAR, 0, &glossTexturePath);
                if (aimtl->GetTextureCount(aiTextureType_DISPLACEMENT)) aimtl->GetTexture(aiTextureType_DISPLACEMENT, 0, &displacementTexturePath);
                if (aimtl->GetTextureCount(aiTextureType_AMBIENT_OCCLUSION)) aimtl->GetTexture(aiTextureType_AMBIENT_OCCLUSION, 0, &displacementTexturePath);

                aiColor3D diffuseColor(0.f, 0.f, 0.f);
                aimtl->Get(AI_MATKEY_COLOR_DIFFUSE, diffuseColor);
                
                aiColor3D emissiveColor(0.f, 0.f, 0.f);
                aimtl->Get(AI_MATKEY_COLOR_EMISSIVE, emissiveColor);
            }
            mesh->UpdateBounds();
            if (aimesh->HasNormals() and aimesh->GetNumUVChannels() > 0) mesh->UpdateTangents();
        }
    }
    for (int n = 0; n < node->mNumChildren; ++n) AILoadModel(scene, node->mChildren[n], world, model);
    model->UpdateBounds();
    return model;
}

shared_ptr<Model> ImportModel(shared_ptr<World> world, const WString& path)
{
    Assimp::Importer importer;
    auto d = CurrentDir();
    
    const aiScene* scene = importer.ReadFile(path.ToUtf8String().c_str(), aiProcess_JoinIdenticalVertices | aiProcess_SortByPType);

    if (not scene or scene->mFlags & AI_SCENE_FLAGS_INCOMPLETE or not scene->mRootNode)
    {
        return nullptr;
    }

    auto model = AILoadModel(scene, scene->mRootNode, world, NULL);

    return model;
}

int main(int argc, const char* argv[])
{
    RegisterComponents();

    auto cl = ParseCommandLine(argc, argv);
    
    //Load FreeImage plugin (optional)
    auto fiplugin = LoadPlugin("Plugins/FITextureLoader");

    //Get the displays
    auto displays = GetDisplays();

    //Create a window
    auto window = CreateWindow("Ultra Engine", 0, 0, 1280 * displays[0]->scale, 720 * displays[0]->scale, displays[0], WINDOW_CENTER | WINDOW_TITLEBAR);

    //Create a framebuffer
    auto framebuffer = CreateFramebuffer(window);

    //Create a world
    auto world = CreateWorld();

    auto camera = CreateCamera(world);
    camera->AddComponent<CameraControls>();
    camera->Move(0, 0, -3);

    //Load a model
    auto model = ImportModel(world, "test.fbx");//load your own model here

    auto light = CreateDirectionalLight(world);
    light->SetRotation(34, 36, 0);

    //Main loop
    while (window->Closed() == false and window->KeyDown(KEY_ESCAPE) == false)
    {
        world->Update();
        world->Render(framebuffer);
    }
    return 0;
}

 

[Josh]

This is looking pretty good. Everything except animations is loading now.

I tried the Step file loading, but it failed on my first try, and there are a lot of posts saying it's not really supported, as I suspected.

#include "UltraEngine.h"
#include "ComponentSystem.h"
#include "assimp/include/assimp/Importer.hpp"
#include "assimp/include/assimp/scene.h"
#include "assimp/include/assimp/postprocess.h"
#include "assimp/include/assimp/pbrmaterial.h"

using namespace UltraEngine;

shared_ptr<Model> AILoadModel(const aiScene* scene, aiNode* node, shared_ptr<World> world, shared_ptr<Entity> parent, const std::vector<shared_ptr<Material> >& materials)
{
    auto model = CreateModel(world);
    model->SetParent(parent);
    model->name = std::string(node->mName.C_Str());

    // Process meshes attached to the current node (if any)
    for (unsigned int i = 0; i < node->mNumMeshes; i++)
    {
        MeshPrimitives mode = MeshPrimitives(0);
        aiMesh* aimesh = scene->mMeshes[node->mMeshes[i]];
        switch (aimesh->mPrimitiveTypes)
        {
        case aiPrimitiveType_POINT:
            mode = MESH_POINTS;
            break;
        case aiPrimitiveType_LINE:
            mode = MESH_LINES;
            break;
        case aiPrimitiveType_POLYGON:
            if (aimesh->mNumFaces > 0 and aimesh->mFaces[0].mNumIndices == 4) mode = MESH_QUADS;
            break;
        case aiPrimitiveType_TRIANGLE:
            mode = MESH_TRIANGLES;
            break;
        }
        if (mode != 0 and aimesh->HasPositions())
        {
            auto mesh = model->AddMesh(mode);
            Vec3 pos, norm;
            Vec2 texcoords;
            Vec4 color;
            std::array<int, 4> bones;
            std::array<float, 4> weights;
            for (int v = 0; v < aimesh->mNumVertices; ++v)
            {
                pos.x = aimesh->mVertices[v].x;
                pos.y = aimesh->mVertices[v].y;
                pos.z = aimesh->mVertices[v].z;
                if (aimesh->HasNormals())
                {
                    norm.x = aimesh->mNormals[v].x;
                    norm.y = aimesh->mNormals[v].y;
                    norm.z = aimesh->mNormals[v].z;
                }
                if (aimesh->GetNumUVChannels() > 0)
                {
                    texcoords.x = aimesh->mTextureCoords[0][v].x;
                    texcoords.y = 1.0f - aimesh->mTextureCoords[0][v].y;
                }
                mesh->AddVertex(pos, norm, texcoords);
                if (aimesh->HasVertexColors(0) and aimesh->GetNumColorChannels() == 4)
                {
                    color.r = aimesh->mColors[0][v].r;
                    color.g = aimesh->mColors[0][v].g;
                    color.b = aimesh->mColors[0][v].b;
                    color.a = aimesh->mColors[0][v].a;
                    mesh->SetVertexColor(v, color);
                }
                if (aimesh->GetNumUVChannels() > 1)
                {
                    texcoords.x = aimesh->mTextureCoords[1][v].x;
                    texcoords.y = 1.0f - aimesh->mTextureCoords[1][v].y;
                    mesh->SetVertexTexCoords(v, texcoords, 1);
                }
                if (aimesh->HasBones())
                {
                    for (int n = 0; n < Min(aimesh->mNumBones, 4); ++n)
                    {
                        //bones[n] = aimesh->mBones[n]->mNode;
                        weights[n] = aimesh->mBones[n]->mWeights->mWeight;
                    }
                    //mesh->SetVertexBones();
                }
                if (mode == MESH_POINTS) mesh->AddIndice(v);
            }
            if (mode != MESH_POINTS)
            {
                for (int v = 0; v < aimesh->mNumFaces; ++v)
                {
                    switch (mode)
                    {
                    case MESH_LINES:
                        mesh->AddPrimitive(aimesh->mFaces[v].mIndices[0], aimesh->mFaces[v].mIndices[1]);
                        break;
                    case MESH_TRIANGLES:
                        mesh->AddPrimitive(aimesh->mFaces[v].mIndices[0], aimesh->mFaces[v].mIndices[1], aimesh->mFaces[v].mIndices[2]);
                        break;
                    case MESH_QUADS:
                        mesh->AddPrimitive(aimesh->mFaces[v].mIndices[3], aimesh->mFaces[v].mIndices[2], aimesh->mFaces[v].mIndices[1], aimesh->mFaces[v].mIndices[0]);
                        break;
                    }
                }
            }

            mesh->SetMaterial(materials[aimesh->mMaterialIndex]);
            mesh->UpdateBounds();
            if (aimesh->HasNormals() and aimesh->GetNumUVChannels() > 0) mesh->UpdateTangents();
        }
    }
    for (int n = 0; n < node->mNumChildren; ++n) AILoadModel(scene, node->mChildren[n], world, model, materials);
    model->UpdateBounds();
    return model;
}

shared_ptr<Model> ImportModel(shared_ptr<World> world, const WString& path)
{
    Assimp::Importer importer;
    auto d = CurrentDir();
    
    const aiScene* scene = importer.ReadFile(path.ToUtf8String().c_str(), aiProcess_JoinIdenticalVertices | aiProcess_SortByPType);

    if (not scene or scene->mFlags & AI_SCENE_FLAGS_INCOMPLETE or not scene->mRootNode)
    {
        return nullptr;
    }

    std::vector<shared_ptr<Texture> > textures;
    if (scene->HasTextures())
    {
        for (unsigned int i = 0; i < scene->mNumTextures; ++i)
        {
            aiTexture* aitex = scene->mTextures[i];
            auto buffer = CreateBuffer(aitex->mWidth * aitex->mHeight * 4);
            memcpy(buffer->Data(), aitex->pcData, buffer->GetSize());
            auto pixmap = CreatePixmap(aitex->mWidth, aitex->mHeight, TEXTURE_RGBA, buffer);
            auto mipchain = pixmap->BuildMipchain();
            auto tex = CreateTexture(TEXTURE_2D, aitex->mWidth, aitex->mHeight, TEXTURE_RGBA, mipchain);
        }
    }

    std::vector<shared_ptr<Material> > materials;
    if (scene->HasMaterials())
    {
        for (unsigned int i = 0; i < scene->mNumMaterials; ++i)
        {
            auto mtl = CreateMaterial();
            
            auto aimtl = scene->mMaterials[i];
            aiString path;
            if (aimtl->GetTextureCount(aiTextureType_DIFFUSE) > 0)
            {
                if (aimtl->GetTexture(aiTextureType_DIFFUSE, 0, &path) == aiReturn_SUCCESS)
                {
                    auto tex = LoadTexture(std::string(path.C_Str()));
                    mtl->SetTexture(tex, TEXTURE_BASE);
                }
            }

            if (aimtl->GetTextureCount(aiTextureType_NORMALS) > 0)
            {
                if (aimtl->GetTexture(aiTextureType_NORMALS, 0, &path) == aiReturn_SUCCESS)
                {
                    auto tex = LoadTexture(std::string(path.C_Str()));
                    mtl->SetTexture(tex, TEXTURE_NORMAL);
                }
            }

            if (aimtl->GetTextureCount(aiTextureType_EMISSIVE) > 0)
            {
                if (aimtl->GetTexture(aiTextureType_EMISSIVE, 0, &path) == aiReturn_SUCCESS)
                {
                    auto tex = LoadTexture(std::string(path.C_Str()));
                    mtl->SetTexture(tex, TEXTURE_EMISSION);
                }
            }

            float metalness;;            
            if (AI_SUCCESS == aimtl->Get(AI_MATKEY_GLTF_PBRMETALLICROUGHNESS_METALLIC_FACTOR, metalness))
            {
                mtl->SetMetalness(metalness);
            }

            float roughness;;
            if (AI_SUCCESS == aimtl->Get(AI_MATKEY_GLTF_PBRMETALLICROUGHNESS_ROUGHNESS_FACTOR, roughness))
            {
                mtl->SetRoughness(roughness);
            }

            aiColor4D pbrSpecularGlossiness;
            if (AI_SUCCESS == aimtl->Get(AI_MATKEY_GLTF_PBRSPECULARGLOSSINESS, pbrSpecularGlossiness))
            {
                mtl->SetShaderFamily(LoadShaderFamily("Shaders/SpecularGloss.fam"));
                mtl->SetSpecular(Vec3(pbrSpecularGlossiness.r, pbrSpecularGlossiness.g, pbrSpecularGlossiness.b));
                mtl->SetGlossiness(pbrSpecularGlossiness.a);
            }

            int unlit;
            if (AI_SUCCESS == aimtl->Get(AI_MATKEY_GLTF_UNLIT, unlit) && unlit == 1)
            {
                mtl->SetShaderFamily(LoadShaderFamily("Shaders/Unlit.fam"));
            }

            aiColor4D diffuseColor(0.f, 0.f, 0.f, 0.f);
            aimtl->Get(AI_MATKEY_COLOR_DIFFUSE, diffuseColor);

            aiColor3D emissiveColor(0.f, 0.f, 0.f);
            aimtl->Get(AI_MATKEY_COLOR_EMISSIVE, emissiveColor);

            mtl->SetColor(diffuseColor.r, diffuseColor.g, diffuseColor.b, diffuseColor.a);
            mtl->SetEmission(emissiveColor.r, emissiveColor.g, emissiveColor.b);

            materials.push_back(mtl);
        }
    }

    auto model = AILoadModel(scene, scene->mRootNode, world, NULL, materials);

    //for (int n = 0; n < scene->mNumAnimations; ++n)
    //{
    //    aiAnimation* animation = scene->mAnimations[n];
    //    animation->mChannels[0].
    //}

    return model;
}

int main(int argc, const char* argv[])
{
    RegisterComponents();

    auto cl = ParseCommandLine(argc, argv);
    
    //Load FreeImage plugin (optional)
    auto fiplugin = LoadPlugin("Plugins/FITextureLoader");

    //Get the displays
    auto displays = GetDisplays();

    //Create a window
    auto window = CreateWindow("Ultra Engine", 0, 0, 1280 * displays[0]->scale, 720 * displays[0]->scale, displays[0], WINDOW_CENTER | WINDOW_TITLEBAR);

    //Create a framebuffer
    auto framebuffer = CreateFramebuffer(window);

    //Create a world
    auto world = CreateWorld();

    auto camera = CreateCamera(world);
    camera->AddComponent<CameraControls>();
    camera->Move(0, 0, 3);
    camera->Turn(0, 180, 0);
    camera->SetClearColor(0.25);

    //Load a model
    auto model = ImportModel(world, "test.gltf");//load your own model here

    auto light = CreateDirectionalLight(world);
    light->SetRotation(34, 36+180, 0);

    //Set environment maps
    const WString remotepath = "https://raw.githubusercontent.com/UltraEngine/Documentation/master/Assets";
    auto specmap = LoadTexture(remotepath + "/Materials/Environment/Storm/specular.dds");
    auto diffmap = LoadTexture(remotepath + "/Materials/Environment/Storm/diffuse.dds");
    world->SetEnvironmentMap(specmap, ENVIRONMENTMAP_SPECULAR);
    world->SetEnvironmentMap(diffmap, ENVIRONMENTMAP_DIFFUSE);

    //Main loop
    while (window->Closed() == false and window->KeyDown(KEY_ESCAPE) == false)
    {
        world->Update();
        world->Render(framebuffer);
    }
    return 0;
}

[Josh]

It looks like there is a very bad problem with bones in assimp. It looks like it quadruples the number of bones in the model.

[Josh]

This is what our FBX converter looks like:

[Josh]

Reported here: https://github.com/assimp/assimp/issues/5534

I am not going to continue on this path.

[Andy90]

I think its an issue with the model itself. Check this model Vampire.rar

 

[Josh]

That is a collada file.

[Josh]

This might be the solution, maybe:
https://github.com/assimp/assimp/issues/4035#issuecomment-1497239561

[Josh]

Seems to cause problems of its own. According to this, the Godot developers gave up and implemented their own FBX converter:
https://github.com/assimp/assimp/issues/2498

[Josh]

I am attaching the last version of my assimp model loader class so you can experiment with it if you want to.

AIModelLoader.cppAIModelLoader.h