problem importing an fbx file with animation

Beginners
1

Hi everyone.

So I’m trying to figure out a workflow for capturing the movements of a face and animating a mesh with the captured information. I’m using brekel pro face 2 and a kinect v2 for the capture (brekel.com/brekel-pro-face-2/) which has an option to save your recording to fbx format.

I want to render these face animations in a touch network but I’m having trouble with the fbx import. I get the same results if I drag and drop, use the import dialog or the python importFBX method.

The face mesh comes in fine, as do all the reference points as cross SOPs and I can see all the animation data playing in the generated exportAnim null CHOP, but the face doesn’t move.

I looked through the network and found at /project1/BrekelSample/Face1_Head/Face1_Face_Mesh that the disable_blendshapes switch SOP was set to bypass the the component doing the blending, so I changed that and could only see one point in the mesh being animated - the left eye brow.

Inside the blend shapes comp at /project1/BrekelSample/Face1_Head/Face1_Face_Mesh/blendshapes I can see there is a file in CHOP that has $TOUCH/Import/BrekelSample.fbx/Geo/Face1_Face_Mesh_blendshapes/Brow_Up_L.bchan loaded as the channel file.

My question is, is there meant to be one of these blendshapes components per point in the mesh to warp / animate? Or has the fbx been incorrectly parsed and the file in CHOP has loaded only part of the animation data? Or something else entirely?

The same fbx loaded into blender sets up the mesh and animation perfectly. I tried with my own capture exported from brekel to fbx, and the supplied brekel fbx example file and got the same result in both the stable and experimental builds.

I’ve attached a zip with the .toe, the directories and files that importing the fbx created and the example fbx that I’m trying to import if anyone wants to try to reproduce my issue.

I should point out that so I’m comfortable with python and touchdesigner, I’m a noob with 3D so I could be missing something simple here.

Any help is greatly appreciated!

C.
fbxProblem.zip (937 KB)

2

Has anyone else seen this happen with an fbx import? I haven’t had any luck messing with the imported network so far.

3

You’re right it looks like it’s only importing the first blendshape. I checked it out in Maya and it’s a deceiving setup because the face joints aren’t actually doing anything - there is no skinning on the model. The facial animation is done by blending between the 20 different blend-shapes. Each blendshape has a 0-1 value that is used to blend to that shape.

What it looks like to me is that initially this mesh was skinned to those joints but upon exporting from pro-face, it underwent some sort of conversion to a blendshape setup. This is actually a neat feature because then you can fine tune the facial expressions manually, but it is kinda deceiving because there is no skinning information and looks like it doesn’t import correctly to Touch. The only thing being used from the joints is the head rotation/translation which is just a matter of parenting.

You COULD just recreate this setup in touch using the blend sop, but you will have to import each blendshape then blend them based on the y-translate parameter for each of those locators lined up in front.

Is it possible to just change the animation export settings in pro-face to NOT export as blendshapes? I have been successful importing skinned meshes from other applications so I know it can work.

4

I was curious about this myself and attempted to recreate the same blendshape setup. I had to cheat and combine some of the blend-shapes together to get down to 16 total (the max amount for blend top).
blendshapeExample.toe (10.5 KB)

5

Oh hey!

Sorry, I didn’t get a notification that you’d replied to this! I just had a look at your example but it looks like you’ve created new .tog(s?) that weren’t uploaded with your .toe file. I’m assuming you were able to extract the individual blend shapes via maya where touch isn’t able to do that?

I tried relinking them to the mesh that comes from brekel face pro but all I have is the one face mesh and all I could get the network of blends to do is scale the whole face up and down.

I’m a novice with 3d software but I’ll see if I can dissect the .fbx in blendr.

Would you mind zipping up your version and attaching it here in the case that I can’t figure it out?

Thanks a lot!

C.

6

I just had a look at the .fbx in blender and I can see what I think are the blendshapes that I need to export individually? It’s late and I have to learn a lot about the interface so I’ll pick this up again soon.

7

Doh! - here’s the rar including togs

I tried to see what was up in Blender though I’ve never used it, and I can’t even find where the blendshapes are, they appear to be missing at least from the outliner. In maya, they were present in the outliner, though hidden. I simply selected just the blendshapes & exported as fbx(as static meshes), and imported into touch designer un-cheking “merge geometry”.

From there, I manually connected the blendshapes into a blend sop (had to merge some to get to 16), and used the y-translate parameter of each reference null (which you can see animating up and down driving the blendshape) to it’s intended blendshape number. It’s not perfect, the expressions don’t look as good as they do in maya and blender but you could probably fuss with the values.

Is there a way to change the fbx export settings in your brekel software? I would try and uncheck anything that says blendshape, to see if you can use the face rig which would be pretty cool.
fbxProblem.rar (1.19 MB)