SmartBody : Using Custom Skeletons and Animations

Any custom skeleton can be read and simulated by SmartBody as long as the skeleton conforms to one of the skeleton asset formats: .bvh, .asf/amc. COLLADA (.dae or .xml) or .fbx. Some SmartBody controllers need to know which joints are involved in a particular action in order to work properly. Thus, the controllers first verity whether or not the skeleton has those needed joints by looking for particular joint names. Thus, it is possible to create a mapping between the custom skeleton that doesn't use the standard SmartBody joint names, and those standard names by creating a joint map. Any number of joint mappings can exists and are managed by the SBJointMapManager:

jointMapManager = scene.getJointMapManager()

The jointMapManager holds all joint mapping sets, each of which can be dynamically created as follows:

mymap = jointMapManager.createJointMap("mymap")

Then, a mapping from the custom skeleton to the standard SmartBody skeleton can be done as follows:

mymap.setMapping("Hips", "base")

Wher 'Hips' is the name of a joint on the custom skeleton, and 'base' is the standard SmartBody joint name (see the section on The Standard SmartBody Skeleton for a list of joint names). The setMapping() method can then be called for each joint that needs to be mapped, for example:

mymap.setMapping("Back", "spine1")
mymap.setMapping("Chest", "spine2")
mymap.setMapping("Neck", "spine4")

Once a mapping has been created, it can be applied to either a skeleton or a motion by using the following commands:

mymap.setSkeleton(skeleton)

where 'skeleton' is the skeleton object that will be converted using the joint mapping. Note that after the setSkeleton() method is called, all references to joint names in the skeleton will assume the mapped names, and not the original names. Additionally, motions can be converted as follows:

 

 

where 'motion' is the motion object whose joints matching the joint names in the joint map. Again, after the setMotion() method is called, all joint names in the motion will now be known to SmartBody as the mapped names, and not the original names.

Note that the setSkeleton() and setMotion() methods should be called before those assets are used, preferably before the simulation has been started. Also note that all the SmartBody controllers combined use only a subset of the joints to operate, and thus a minimal mapping would only mapping this subset as described in Configuring Characters and replicated here:

JointRequired by:
baseSmartBody, all characters need this joint.
spine1gaze
spine2gaze
spine3gaze
spine4gaze, head movements
spine5gaze, head movements
skullbasegaze, head movements
face_top_parentgaze
eyeball_leftgaze, saccades, softeyes
eyeball_rightgaze, saccades, softeyes
l_sternoclavicularreaching
r_sternoclavicularreaching

 

Notwithstanding controller functionality, any skeletons and motions that have similar joint names can be automatically retargeted during runtime by copying motion curves over the joints. In other words, if your skeleton has the same joint names a motion, that motion can be applied to your skeleton automatically.

How To Export Skeleton/Animation From 3dsMax or Maya

Skeletons and motions can be exported from Maya or 3dsMax using COLLADA format. We recommend using OpenCollada plugin (http://www.opencollada.org/download.html) to export a Maya/3dsMax scene to COLLADA. Note that the current version of OpenCollada plugin does not support Maya 2012 yet, so you need to use the DAE_FBX plugin that came with Maya 2012.

The overall process is similar when exporting from either Maya and 3dsMax :

go to Export Menu, and select output file type as OpenCOLLADA, be sure to select Enable export for Animation. This will output a .dae file that contains both the skeleton and the corresponding motion. 

There are also a few requirements and caveats that need to be considered when exporting the animations :

  1. The first frame ( frame 0 ) needs to be the default bind pose ( usually called T-pose, depending on your skeleton setup ). This ensures that the output bone transformations would be correct. Otherwise, the bone transformations would be relative to the first frame of the animation and the motion would look strange when applied on the T-pose skeleton in SmartBody.

  2. 3dsMax allows you to name your joints with spaces, however it is preferable that the joint name does NOT contain any spaces. Some COLLADA exporter may not process this correctly ( i.e. The default Autodesk COLLADA exporter ) and may cause incorrect results when loading the file into SmartBody. If your skeleton contains joint names with spaces, you need to use OpenCOLLADA plugin for exporting. As it would take care of this by replace space (" ") with underline ("_").

  3. If the first frames of your motion is the T-pose, they need to be skipped when loading the motions in SmartBody. This can be done by either :
    1. Post-processing the loaded motion with Python command : 
       

      testMotion.trim(n,0)

       

      This command will trim the first n frames of testMotion so the resulting motions will not contain T-pose.

    2. Set a global attributes to trim the frames when loading a COLLADA motion :

      scene.setIntAttribute("colladaTrimFrames", n)


      This will automatically trim the first n frames when loading a COLLADA motion. Note that this attribute needs to be set before calling scene.loadAssets()

Use Skeleton/Character From Mixamo 

Mixamo ( http://www.mixamo.com ) let you quickly create 3D characters/animations and download it into various format. SmartBody provides the necessary script to quickly create a standard SmartBody character from mixamo character. This enable our users to create a custom character from mixamo, and then use SmartBody to produce various behaviors like locomotion/reach/gaze for the character.

The following is the step-by-step instructions to download a specific 'Football Player' character from mixamo website and get it working in SmartBody. The same procedure should work for other characters on their website.

  1. On the mixamo website, select the free character ‘Football Player’ in T-pose.
  2. Download the character T-pose into Collada (.dae) as well as FBX format. Here, we save the file name as player.dae and player.fbx
  3. If you just want to use SmartBody fltk-viewer :
    1. Put the Collada file into $SmartBodyDir /data/sbm-common/common-sk/ so SmartBody can load it as a skeleton. To use deformable mesh, you also need to put both the .dae file and the texture files into $SmartBodyDir/data/mesh/player directory.
    2. Modify the default-init-mixamo.py accordingly to accommodate for your character names. ( If you follow the instructions to download the Football Player and name is ‘player.dae’, the script should work out of box. )
    3. Run fltk-viewer with argument “-scriptpath ../../../../data/sbm-common/scripts  -script default-init-mixamo.py”

 

  1. If you want to use OgreViewer :
    1. Be sure to do step 3 correctly so SmartBody can load the skeleton.
    2. install Ogre Exporter for Maya. (http://www.ogre3d.org/tikiwiki/Maya+Exporter)
    3. Open the FBX file in Maya, and use Ogre Exporter to save out player.mesh, player.material, and player.skeleton.
    4. Put the ogre files in corresponding directory $SmartBodyDir/core/ogre-viewer/media.
    5. Modify the default-init-mixamo.py script. Specifically, make sure when you are creating a character, the character name is the same as your mesh name ( ex. scene.createCharacter(‘player’) ) This will enable the OgreViewer to use ‘player.mesh’ instead of the default ‘brad.mesh’.
    6. Run OgreViewer with the argument “-scriptpath ../../../../data/sbm-common/scripts  -script default-init-mixamo.py”

There is one pitfall when downloading the mixamo character. Sometimes they add a prefix to the joint name ( “Beast_ LeftUpLeg” instead of “LeftUpLeg” for example ). This will cause incorrect mapping for the bone names and thus the retarget will fail. For now the solution is to remove those joint name prefix manually from the file for this kind of characters, or modify the joint name mapping in  'mixamo-map.py' to accommodate for the prefix.