University of Southern CaliforniaUSC
USC ICT TwitterUSC ICT FacebookUSC ICT YouTube

Integrating SmartBody into a simple Unity 5 Project | General SmartBody Discussion | Forum

Avatar

Please consider registering
guest

sp_LogInOut Log In sp_Registration Register

Register | Lost password?
Advanced Search

— Forum Scope —




— Match —





— Forum Options —





Minimum search word length is 3 characters - maximum search word length is 84 characters

sp_Feed Topic RSS sp_TopicIcon
Integrating SmartBody into a simple Unity 5 Project
September 10, 2015
12:57 pm
Avatar
Member
Members
Forum Posts: 6
Member Since:
August 24, 2015
sp_UserOfflineSmall Offline

Hi all,

Has anybody here integrated SmartBody into a Unity 5 project? If yes, could you please provide me (novice C# & Unity developer) with detailed steps?

I understand SmartBody is essentially a DLL that could communicate with Unity. Is this correct?

Initially, I opened my own Unity 5 project and through Assets->Import Package->Custom Package, I imported 2 packages from the Virtual Human Toolkit installation directory (C:\vhtoolkit\bin\vhAssets) that were called vhAssetsPackage and vhAssetsTestScenePackage.

This was, of course, an attempt to integrate the Virtual Human Toolkit into my own Unity project, but due to several problems, I can no longer go ahead with this; e.g. I have to set the standalone build settings of Unity to .NET 2.0 so that the Virtual Human Toolkit would not throw an error while building, but then this stops my own project from running, etc...

So, anyway, I thought perhaps I should forget about integrating the entire Virtual Human Toolkit and see if I could simply integrate SmartBody alone, perhaps by importing some DLLs?

Is this possible?

Thank you,

 

A bit of context: I would like to be able to generate and use lip sync and nonverbal behavior in my own Unity 5 project by using simple C# commands. For example, if blah condition was met, the character will say his name (some audio file) and nods his head and puts up his hand. Obviously, these are not possible with Unity's MecAnim system, hence my attempt to incorporate SmartBody into Unity.

September 10, 2015
5:45 pm
Avatar
Admin
Forum Posts: 983
Member Since:
December 1, 2011
sp_UserOfflineSmall Offline

The Virtual Human Toolkit is maintained by a group here at the USC Institute for Creative Technologies and they have created an interface to Unity from SmartBody.

You can, of course, create your own interface. There is a C-style interface in the SmartBody code base (see sb/smartbody-c-dll.h) which could be connected to your interface in Unity. That interface looks like this:

 

SBAPI SBMHANDLE SBM_CreateSBM();

SBAPI bool SBM_Init( SBMHANDLE sbmHandle, const char * pythonLibPath, bool logToFile );
SBAPI bool SBM_Shutdown( SBMHANDLE sbmHandle );

SBAPI bool SBM_Update( SBMHANDLE sbmHandle, double timeInSeconds );
SBAPI bool SBM_UpdateUsingDelta( SBMHANDLE sbmHandle, double deltaTimeInSeconds ); // same as SBM_Update(), but pass in a delta time since the prev call.
SBAPI bool SBM_ProcessVHMsgs( SBMHANDLE sbmHandle, const char * op, const char * args );

SBAPI bool SBM_InitCharacter( SBMHANDLE sbmHandle, const char * name, SBM_CharacterFrameDataMarshalFriendly * character );
SBAPI bool SBM_GetCharacter( SBMHANDLE sbmHandle, const char * name, SBM_CharacterFrameDataMarshalFriendly * character );
SBAPI bool SBM_ReleaseCharacter( SBM_CharacterFrameDataMarshalFriendly * character );

// used for polling events. These must be called regularly, or else their queues will overflow
SBAPI bool SBM_IsCharacterCreated( SBMHANDLE sbmHandle, char * name, int maxNameLen, char * objectClass, int maxObjectClassLen );
SBAPI bool SBM_IsCharacterDeleted( SBMHANDLE sbmHandle, char * name, int maxNameLen );
SBAPI bool SBM_IsCharacterChanged( SBMHANDLE sbmHandle, char * name, int maxNameLen );
SBAPI bool SBM_IsVisemeSet( SBMHANDLE sbmHandle, char * name, int maxNameLen, char * visemeName, int maxVisemeNameLen, float * weight, float * blendTime );
SBAPI bool SBM_IsChannelSet( SBMHANDLE sbmHandle, char * name, int maxNameLen, char * channelName, int maxChannelNameLen, float * value );
SBAPI bool SBM_IsLogMessageWaiting( SBMHANDLE sbmHandle, char * logMessage, int maxLogMessageLen, int * logMessageType );
SBAPI bool SBM_IsBmlRequestWaiting( SBMHANDLE sbmHandle, char * charName, int maxNameLen, char * requestId, int maxRequestIdLength, char * bmlName, int maxBmlNameLength);
SBAPI void SBM_SendBmlReply(SBMHANDLE sbmHandle, const char * charName, const char * requestId, const char * utteranceId, const char * bmlText);

// python usage functions
SBAPI bool SBM_PythonCommandVoid( SBMHANDLE sbmHandle, const char * command );
SBAPI bool SBM_PythonCommandBool( SBMHANDLE sbmHandle, const char * command );
SBAPI int SBM_PythonCommandInt( SBMHANDLE sbmHandle, const char * command );
SBAPI float SBM_PythonCommandFloat( SBMHANDLE sbmHandle, const char * command );
SBAPI void SBM_PythonCommandString( SBMHANDLE sbmHandle, const char * command, char * output, int maxLen);

// DLL class accessors
SBAPI bool SBM_SBAssetManager_LoadSkeleton( SBMHANDLE sbmHandle, const void * data, int sizeBytes, const char * skeletonName );
SBAPI bool SBM_SBAssetManager_LoadMotion( SBMHANDLE sbmHandle, const void * data, int sizeBytes, const char * motionName );

SBAPI void SBM_SBDebuggerServer_SetID( SBMHANDLE sbmHandle, const char * id );
SBAPI void SBM_SBDebuggerServer_SetCameraValues( SBMHANDLE sbmHandle, double x, double y, double z, double rx, double ry, double rz, double rw, double fov, double aspect, double zNear, double zFar );
SBAPI void SBM_SBDebuggerServer_SetRendererIsRightHanded( SBMHANDLE sbmHandle, bool enabled );

SBAPI void SBM_SBMotion_AddChannel( SBMHANDLE sbmHandle, const char * motionName, const char * channelName, const char * channelType );
SBAPI void SBM_SBMotion_AddChannels( SBMHANDLE sbmHandle, const char * motionName, const char ** channelNames, const char ** channelTypes, int count );
SBAPI void SBM_SBMotion_AddFrame( SBMHANDLE sbmHandle, const char * motionName, float frameTime, const float * frameData, int numFrameData );
SBAPI void SBM_SBMotion_SetSyncPoint( SBMHANDLE sbmHandle, const char * motionName, const char * syncTag, double time );

SBAPI void SBM_SBJointMap_GetMapTarget( SBMHANDLE sbmHandle, const char * jointMap, const char * jointName, char * mappedJointName, int maxMappedJointName );

SBAPI void SBM_SBDiphoneManager_CreateDiphone(SBMHANDLE sbmHandle, const char * fromPhoneme, const char * toPhoneme, const char * name);
SBAPI void SBM_SBDiphone_AddKey(SBMHANDLE sbmHandle, const char * fromPhoneme, const char * toPhoneme, const char * name, const char * viseme, float time, float weight);

SBAPI void SBM_SBVHMsgManager_SetServer( SBMHANDLE sbmHandle, const char * server );
SBAPI void SBM_SBVHMsgManager_SetScope( SBMHANDLE sbmHandle, const char * scope );
SBAPI void SBM_SBVHMsgManager_SetPort( SBMHANDLE sbmHandle, const char * port );
SBAPI void SBM_SBVHMsgManager_SetEnable( SBMHANDLE sbmHandle, bool enable );

Ari

Forum Timezone: America/Los_Angeles

Top Posters:

jwwalker: 80

jyambao: 52

rbaral: 47

adiaz: 30

WargnierP: 29

lucky7456969: 28

mbarros: 28

avida.matt: 26

JonathanW: 24

laguerre: 23

Member Stats:

Guest Posters: 69

Members: 122101

Moderators: 3

Admins: 4

Forum Stats:

Groups: 1

Forums: 5

Topics: 531

Posts: 2495