ANGLE, c++, HoloLens

HoloLens: FBX Loading c++

final

Now, there are a few reasons why you might want the ability to load FBX files; maybe you want to load 3D model files dynamically or you are working on a native HoloLens app and just want a way to get 3D data in to render, animate, etc. If you are building in Unity you have the option of using Asset Bundles which you can host on a server and access via an API. There are a few reasons why you may not want this though; you have an API already supplying the data in another format or you are building an API to be shared amongst other apps which you don’t want to tie to Unity. Authoring the bundles also requires Unity which may or may not be convenient. If you want to load FBX files dynamically in a Unity built app and didn’t want to use Asset Bundles then you could use similar code to that which I will show here and wrap it up as a Unity custom plugin. I believe there are some offerings in the Unity Asset store which would achieve a similar goal but I haven’t used them so can’t vouch for going that route.

For one reason or another I have chosen to start this project as a native c++ app based on a template for ANGLE (Almost Native Graphics Layer Engine). This allows me to develop my rendering code with OpenGL ES 2.0 and have that run on a HoloLens. It does this by efficient translation of OpenGL to the equivalent Direct3D. I will come back to this in a subsequent blog post. You can find the project here.

The Microsoft fork of ANGLE has a branch named ms-holographic-experimental for experimental use on HoloLens. This branch has a template for Visual Studio 2015 so I used this as a starting point for my demo app.

vs-template

This is the output from that template when run on a HoloLens:

blank-template-running

So, it’s just a cube with per-vertex colours which floats in mid-air in front of you and spins. The interesting thing about it is that the rendering code is all OpenGL ES.

This snippet shows the Draw calls. Notice that the code will render for holographic and non-holographic apps and here you can see the difference.The else block is standard but look at the holographic part; two rendering targets are specified, one for each eye and there is a call to glDrawElementsInstancedANGLE which will instance the geometry saving performance when rendering the same scene to each eye.  

FBX SDK

If I cast my mind back many years I can recall working with a company called Kaydara on integration of motion control rig paths as a plugin to their software Filmbox which I believe is where the FBX format originates. It is now all over and seems to be about as close as we get to a standard although it is a proprietary format owned by Autodesk. Enter stage left, gltf https://github.com/KhronosGroup/glTF which is hopefully the open standard we are all moving towards.

You can download the FBX SDK from here http://usa.autodesk.com/adsk/servlet/pc/item?siteID=123112&id=10775847 including a UWP version.

That’s what I did and I added it to my project – here are the steps:

– Copy the include and lib folders from the FBX SDK install folder and paste them into a folder local to your project

– I usually copy the dll and pdb into the root of my project and use ‘show files’ and ‘include in project’ to add the dll to the project.

– Select the libfbxsdk.dll in your project and use the property window to mark it as content – ensuring that it will get copied to the output correctly

– Add the include folder as an additional include folder in your project properties – I add it so it is referenced locally (rather than full path name)

– In the Linker > input settings add the .lib file from the FBX SDK

Note. I have copied all of the dependencies locally and referenced them relatively – this is to make the solution self-contained

Now you should be able to add code like the following and have it compile and run:

Loading a File

Here’s some boilerplate code for loading a file:

This will give us everything we need via the scene to load meshes, materials, textures, animations, etc.

Loading a Mesh

Now an FBX file can consist of a complex hierarchy of objects including meshes so in order to load the whole model we’ll need to traverse the whole data structure and create vertex buffers for each mesh. To help traverse the scene we can use a recursive function to which we pass a function object to be called for each mesh:

we can pass a callback and query the scene data for information that we can use to populate our OpenGL buffers:

In a similar way we can get the vertex indices which define how the triangles are constructed. In the actual project I have refactored the code a bit to provide a Model class which has a list of meshes which in turn are responsible for the individual buffers for vertices, indices and normals, etc. and also responsible for issuing the Draw calls. At this point we have a model that renders but it has no material properties or lighting so it just looks like a blob.     

Materials

To make this code in any way useful we need to apply some materials or at least some colours to the meshes. When I first started the plan was to re-write the shaders and apply different materials to the meshes and have them rendered correctly. I have taken a shortcut in the interests of time as things started to get a bit more complicated with the FBX SDK when we start to dig into materials.This is what I have implemented so far:

Loop through the triangles of each mesh and discover the material applied to each. I then construct a buffer of vertex colours and derive those colours from the diffuse colour given in each material. I store the colour for each vertex and this allows me to continue to use the original fragment shader but still be able to differentiate between the different meshes in the model.

Here’s the snippet of code that collects the vertex colours:

Next Steps

The next steps would be to re-implement the shaders to have a standard set that could be mapped to the materials in the FBX and also implement a lighting model. Then import the animation and we’ll get close to having something useful.

Here’s the resulting model (of a HoloLens device) running on my HoloLens (the colours look a bit strange due to the per-vertex interpolation)

And all of the source code for the project can be found here https://github.com/peted70/hololens-fbx-viewer

Tagged , , , , , , ,

11 thoughts on “HoloLens: FBX Loading c++

  1. Hey! Thanks for the detailed tutorial. This is exactly what I had been looking for a long time.
    I made a Holographic Universal Windows Project in C++ to start and I have been trying to add the fbx sdk dependencies. I followed the steps you mentioned but for some reason the program throws a compile error “Error C2653 ‘FbxManager’: is not a class or namespace name”. I have tried to link it statically too, but it gives the same error. I would appreciate some guidance

      1. Got that solved. Was a problem with the dll file I was using.
        However, I now get this error when I try to initialize the importer with the fbx file:

        fbxsdk\fileio\fbximporter.cxx(497) : [pid0] ASSERTION FAILED: Unexpected file type!
        Debug Assertion Failed!

        I have placed the .fbx file in the Assets folder and have also included it to the project.

          1. Check to see if the fbx file is actually being deployed with the app first in the Assets folder.

  2. Hey I tried to change the fbx file to some of my models but I got error at the runtime
    It also get capture breakpoint when I try to change to cube.fbx…
    Why?

    1. From where is the exception thrown? And do you have any info about what the exception is? I guess since this was an illustrative sample I might not have been hugely careful about dealing with error conditions. Can you post your fbx somewhere?

Leave a Reply to Dew Drop - May 11, 2017 (#2477) - Morning Dew Cancel reply

Your email address will not be published. Required fields are marked *