HoloLens: FBX Loading c++
Posted on May 10, 2017
Now, there are a few reasons why you might want the ability to load FBX files; maybe you want to load 3D model files dynamically or you are working on a native HoloLens app and just want a way to get 3D data in to render, animate, etc. If you are building in Unity you have the option of using Asset Bundles which you can host on a server and access via an API. There are a few reasons why you may not want this though; you have an API already supplying the data in another format or you are building an API to be shared amongst other apps which you don’t want to tie to Unity. Authoring the bundles also requires Unity which may or may not be convenient. If you want to load FBX files dynamically in a Unity built app and didn’t want to use Asset Bundles then you could use similar code to that which I will show here and wrap it up as a Unity custom plugin. I believe there are some offerings in the Unity Asset store which would achieve a similar goal but I haven’t used them so can’t vouch for going that route.
For one reason or another I have chosen to start this project as a native c++ app based on a template for ANGLE (Almost Native Graphics Layer Engine). This allows me to develop my rendering code with OpenGL ES 2.0 and have that run on a HoloLens. It does this by efficient translation of OpenGL to the equivalent Direct3D. I will come back to this in a subsequent blog post. You can find the project here.
The Microsoft fork of ANGLE has a branch named ms-holographic-experimental for experimental use on HoloLens. This branch has a template for Visual Studio 2015 so I used this as a starting point for my demo app.
This is the output from that template when run on a HoloLens:
So, it’s just a cube with per-vertex colours which floats in mid-air in front of you and spins. The interesting thing about it is that the rendering code is all OpenGL ES.
This snippet shows the Draw calls. Notice that the code will render for holographic and non-holographic apps and here you can see the difference.The else block is standard but look at the holographic part; two rendering targets are specified, one for each eye and there is a call to glDrawElementsInstancedANGLE which will instance the geometry saving performance when rendering the same scene to each eye.
If I cast my mind back many years I can recall working with a company called Kaydara on integration of motion control rig paths as a plugin to their software Filmbox which I believe is where the FBX format originates. It is now all over and seems to be about as close as we get to a standard although it is a proprietary format owned by Autodesk. Enter stage left, gltf https://github.com/KhronosGroup/glTF which is hopefully the open standard we are all moving towards.
You can download the FBX SDK from here http://usa.autodesk.com/adsk/servlet/pc/item?siteID=123112&id=10775847 including a UWP version.
That’s what I did and I added it to my project – here are the steps:
- Copy the include and lib folders from the FBX SDK install folder and paste them into a folder local to your project
- I usually copy the dll and pdb into the root of my project and use ‘show files’ and ‘include in project’ to add the dll to the project.
- Select the libfbxsdk.dll in your project and use the property window to mark it as content – ensuring that it will get copied to the output correctly
- Add the include folder as an additional include folder in your project properties – I add it so it is referenced locally (rather than full path name)
- In the Linker > input settings add the .lib file from the FBX SDK
Note. I have copied all of the dependencies locally and referenced them relatively – this is to make the solution self-contained
Now you should be able to add code like the following and have it compile and run:
Loading a File
Here’s some boilerplate code for loading a file:
This will give us everything we need via the scene to load meshes, materials, textures, animations, etc.
Loading a Mesh
Now an FBX file can consist of a complex hierarchy of objects including meshes so in order to load the whole model we’ll need to traverse the whole data structure and create vertex buffers for each mesh. To help traverse the scene we can use a recursive function to which we pass a function object to be called for each mesh:
we can pass a callback and query the scene data for information that we can use to populate our OpenGL buffers:
In a similar way we can get the vertex indices which define how the triangles are constructed. In the actual project I have refactored the code a bit to provide a Model class which has a list of meshes which in turn are responsible for the individual buffers for vertices, indices and normals, etc. and also responsible for issuing the Draw calls. At this point we have a model that renders but it has no material properties or lighting so it just looks like a blob.
To make this code in any way useful we need to apply some materials or at least some colours to the meshes. When I first started the plan was to re-write the shaders and apply different materials to the meshes and have them rendered correctly. I have taken a shortcut in the interests of time as things started to get a bit more complicated with the FBX SDK when we start to dig into materials.This is what I have implemented so far:
Loop through the triangles of each mesh and discover the material applied to each. I then construct a buffer of vertex colours and derive those colours from the diffuse colour given in each material. I store the colour for each vertex and this allows me to continue to use the original fragment shader but still be able to differentiate between the different meshes in the model.
Here’s the snippet of code that collects the vertex colours:
The next steps would be to re-implement the shaders to have a standard set that could be mapped to the materials in the FBX and also implement a lighting model. Then import the animation and we’ll get close to having something useful.
Here’s the resulting model (of a HoloLens device) running on my HoloLens (the colours look a bit strange due to the per-vertex interpolation)
And all of the source code for the project can be found here https://github.com/peted70/hololens-fbx-viewer