A SmartBody simulation consists of a scene, one or more characters, one or more pawns, and a number of assets, including animations, skeleton hierarchies, sounds, geometric meshes and textures, among others. Characters are controlled by a set of specialized controllers, which control various aspects of movement, such as idle posturing, facial animation, locomotion, gesturing, movement under physics and so forth. Every SmartBody object, such as a character, pawn, scene, motion or skeleton, has an associated set of dynamic attributes, which can be queried and changed in order to calibrate various features, or to toggle them on or off. SmartBody can be controlled directly through the underlying C++ interface, or through a high level scripting interface in Python. SmartBody can respond to Behavior Markup Language (BML) commands as a means to control and coordinate the behaviors of one or more characters.
A SmartBody simulation can either use a real-time clock, or can be set explicitly by an outside source.
SmartBody can connect to and receive messages from other cooperating components through the use of the Virtual Human Message system (VHMessage). The VHMessage system is an asynchronous messaging system that is built atop the ActiveMQ framework.
Character Controllers
The main focus of SmartBody is on the control and display of a virtual character. Each character has a set of controllers that operate together to perform various behaviors and functions for the character. The current group of controllers is:
| Controller | Purpose |
|---|---|
| World offset controller | Controls the position and orientation of the character in the 3D space |
| Posture scheduler | Schedules idle postures (standing, sitting, etc.) |
| Parameterized animation controller | Controls animations, blends, states and transitions between animation states. |
| Motion graph controller | Controls animations made using the motion graph. |
| Basic locomotion controller | Controls the locomotion of the character by using basic animation that moves but does not animate the character. |
| Single-animation locomotion controller | Controls the locomotion of a character using a single walk cycle (experimental) |
| Animation controller | Controls other animations (gestures, etc.) |
| Postprocess controller | Handles IK for footskate cleanup |
| Hand controller | Adds finger motion to gestures (experimental) |
| Reach scheduler | Handles arm and body movement for reaching/grabbing/touching/pointing motions |
| Grab scheduler | Handles fingers and hand movement for reaching/grabbing/touching/pointing motions, as well as any hand configurations |
| Breathing controller | Handles breathing of character |
| Gaze scheduler | Handles gaze |
| Eye saccade controller | Handles saccadic eye movements (fast movements of the eye) |
| Eyelid controller | Controls the eyelid position in response to the movement of the eyeball, or to change the appearance of the character |
| Head schedule | Handles head movements (nodding, shaking, etc.) |
| Face controller | Controls the movement of the face, including expressions and lip synchronization to speech |
| Physics controller | Handles physics for character, for ragdoll and dynamic controllers |
| Noise controller | Adds noise to the joints to simulate shaking and holding effects |
| Constraint controller | Manages positional and rotational constraints for the character. |
| Motion player | Playback of motion to override existing character state |
| Data receiver | Overrides character state from a outside source (for puppeteering from a Kinect sensor, for example) |
| Recorder | Stores the character state for persistence to a file. |
The controllers are executed in sequence, with some controllers overriding values from previous controllers, and others modifying values.
SmartBody Components
SmartBody uses a number of 3rd party libraries, including:
BOOST (www.boost.org)
FLTK (www.fltk.org)
FLU - FLTK Utility Widgets
SteerSuite (http://http://steersuite.cse.yorku.ca/)
Open Dynamics Engine (http://www.ode.org/)
ActiveMQ (http://activemq.apache.org/)
glew (http://glew.sourceforge.net/)
recast and detour (http://code.google.com/p/recastnavigation/)
zlib (http://www.zlib.net/)
festival (http://www.cstr.ed.ac.uk/projects/festival/)
Ogre (http://www.ogre3d.org/)
Pinocchio (http://www.mit.edu/~ibaran/autorig/pinocchio.html)
PolyVox (http://www.volumesoffun.com/polyvox-about/)