University of Southern CaliforniaUSC
USC ICT TwitterUSC ICT FacebookUSC ICT YouTube

Getting started with smartbody-lib, part 2 | General SmartBody Discussion | Forum

Avatar

Please consider registering
guest

sp_LogInOut Log In sp_Registration Register

Register | Lost password?
Advanced Search

— Forum Scope —




— Match —





— Forum Options —





Minimum search word length is 3 characters - maximum search word length is 84 characters

sp_Feed Topic RSS sp_TopicIcon
Getting started with smartbody-lib, part 2
September 25, 2012
11:26 pm
Avatar
Member
Members
Forum Posts: 80
Member Since:
June 13, 2012
sp_UserOfflineSmall Offline

I'm not getting crashes anymore, but I haven't figured out how to get anything to actually happen. I have this initialization code:

xercesc_3_1::XMLPlatformUtils::Initialize();

initPython("/System/Library/Frameworks/Python.framework/Versions/2.6/lib");

vhcl::Log::g_log.AddListener( new vhcl::Log::StdoutListener );

mcuCBHandle& mcu = mcuCBHandle::singleton();
mcu.switch_internal_timer();

Then I create a character, apply a skeleton to it, start the simulation manager, call execBML to tell the character to look somewhere, and advance the simulation manager time. But no joints change their angle. Stepping through code in the debugger, it appears that execBML adds the command to a queue, but I can't see how to get it acted on. No warnings get logged. Help?

September 25, 2012
11:51 pm
Avatar
Member
Members
Forum Posts: 80
Member Since:
June 13, 2012
sp_UserOfflineSmall Offline

By the way, the brief section in the manual on using smartbody-lib suggests calling

sim->updateTime(timeInSeconds);

but SBSimulationManager doesn't have an updateTime method.

September 26, 2012
1:04 am
Avatar
Member
Members
Forum Posts: 80
Member Since:
June 13, 2012
sp_UserOfflineSmall Offline

I tried throwing in mcu.update(), a command that I saw in sbm_main.cpp, though I don't really know if I need it. Then I got this on the log:

WARNING: BML::parse_target(): Unimplented <0x8f97a1c 0x7ea52f0="x, y, z" ... />. Behavior ignored.

I tracked this down to the line that produced it in bml_target.cpp:

strstr << "WARNING: BML::parse_target(): Unimplented <"<<tagname<<" "<< BMLDefs::ATTR_TARGET<<"=\"x, y, z\" ... />. Behavior ignored.";

Apparently, tagname and ATTR_TARGET are wide strings saying "gaze" and "target" respectively, but basic_stringstream<char> does not understand wide strings, hence the messed-up output.

Here's what I was trying to do:

bmlMgr->execBML( "Luigi",
"<gaze target=\"3 0 0\" sbm:joint-range=\"CHEST BACK NECK\" "
"sbm:priority-joint=\"NECK\" />" );

I thought that gazing at a specific location would be about the simplest useful thing I could do with Smartbody, but it's not implemented?

September 26, 2012
1:50 am
Avatar
Member
Members
Forum Posts: 80
Member Since:
June 13, 2012
sp_UserOfflineSmall Offline

I guess I got the target=\"3 0 0\" thing from p. 107 of the manual, but that looks like an error. So I tried replacing "target" with "sbm:target-pos", and then got log messages:

WARNING: BML::parse_bml_gaze(): <0x85d401c> BML tag missing 0x1f61a8e0= attribute.
No gazing channels found.
MeCtGaze:: ERR: 'spine1' NOT FOUND in skeleton
MeCtGaze:: ERR: 'spine2' NOT FOUND in skeleton
MeCtGaze:: ERR: 'spine3' NOT FOUND in skeleton
MeCtGaze:: ERR: 'spine4' NOT FOUND in skeleton
MeCtGaze:: ERR: 'spine5' NOT FOUND in skeleton
MeCtGaze:: ERR: 'skullbase' NOT FOUND in skeleton
MeCtGaze:: ERR: 'eyeball_left' NOT FOUND in skeleton
MeCtGaze:: ERR: 'eyeball_right' NOT FOUND in skeleton
MeCtSaccade::initBufferIndex ERR: channel id not correct!

What that first line should say, apparently, is "<gaze> tag missing target= attribute". Hmm. What does "No gazing channels found." mean? And why can't it find any of my bones? I can find spine1 through spine5 with SBSkeleton::getJointByName. (I admit that the eyeballs are missing.)

By the way, the code in bml_gaze.cpp that produces this warning makes a more earnest attempt to handle wide strings than the previously mentioned code in bml_target.cpp, but it's obviously still off the mark. The problem is that it deals with strings of XMLCh, which is defined as uint16_t, and that's not the same as wchar_t on any platform but Windows.

September 26, 2012
5:06 pm
Avatar
Admin
Forum Posts: 983
Member Since:
December 1, 2011
sp_UserOfflineSmall Offline

The gaze controller is insisting that it can't find the right joints on your character, Luigi. Did you apply the skeleton to your character?

character = scene.getCharacter("Luigi")
skeleton = scene.createSkeleton("myskeleton")
character->setSkeleton(skeleton)

Are you sure that you are querying the skeleton that is connected to the character, and not just the one that has been loaded into the system?

Try this simpler gaze command:
bmlMgr->execBML( "Luigi", "" );

Did you get the simulation loop to work properly?

Ari

September 26, 2012
5:37 pm
Avatar
Member
Members
Forum Posts: 80
Member Since:
June 13, 2012
sp_UserOfflineSmall Offline

Ari Shapiro said

The gaze controller is insisting that it can't find the right joints on your character, Luigi. Did you apply the skeleton to your character?

character = scene.getCharacter("Luigi")
skeleton = scene.createSkeleton("myskeleton")
character->setSkeleton(skeleton)

Are you sure that you are querying the skeleton that is connected to the character, and not just the one that has been loaded into the system?

Yes, I definitely did character->setSkeleton. Later I do character->getSkeleton, and I get the same pointer back.

Try this simpler gaze command:
bmlMgr->execBML( "Luigi", "" );

Did you get the simulation loop to work properly?

I don't know what that "simpler gaze command" is supposed to do, so I wouldn't know whether it's working properly. I still get this message on the log:

MeCtSaccade::initBufferIndex ERR: channel id not correct!
September 26, 2012
5:45 pm
Avatar
Admin
Forum Posts: 983
Member Since:
December 1, 2011
sp_UserOfflineSmall Offline

Sorry, it looks like the forum editor ate my XML. This is the simple gaze command:

bmlMgr->execBML( "Luigi", "" );

The MeCtSaccade::initBufferIndex ERR: message is related to the saccade controller - it can't find any eyes, so the eyes can't shift around.

Ari

September 26, 2012
5:46 pm
Avatar
Admin
Forum Posts: 983
Member Since:
December 1, 2011
sp_UserOfflineSmall Offline

Wow, it did it again. Ok, third time is a charm:

bmlMgr->execBML( "Luigi", "" ); 
September 26, 2012
5:49 pm
Avatar
Admin
Forum Posts: 983
Member Since:
December 1, 2011
sp_UserOfflineSmall Offline

One more time 🙂

bmlMgr->execBML( "Luigi", "&ltgaze target=\"Luigi:l_toe\">" );

where l_toe is the name of any joint on the character.

September 26, 2012
6:13 pm
Avatar
Member
Members
Forum Posts: 80
Member Since:
June 13, 2012
sp_UserOfflineSmall Offline

With that gaze command, I no longer have the message about the saccade, but otherwise it's the same.

No gazing channels found.
MeCtGaze:: ERR: 'spine1' NOT FOUND in skeleton
MeCtGaze:: ERR: 'spine2' NOT FOUND in skeleton
MeCtGaze:: ERR: 'spine3' NOT FOUND in skeleton
MeCtGaze:: ERR: 'spine4' NOT FOUND in skeleton
MeCtGaze:: ERR: 'spine5' NOT FOUND in skeleton
MeCtGaze:: ERR: 'skullbase' NOT FOUND in skeleton
MeCtGaze:: ERR: 'eyeball_left' NOT FOUND in skeleton
MeCtGaze:: ERR: 'eyeball_right' NOT FOUND in skeleton
September 26, 2012
7:01 pm
Avatar
Member
Members
Forum Posts: 80
Member Since:
June 13, 2012
sp_UserOfflineSmall Offline

I looked at where the "NOT FOUND in skeleton" message is apparently generated, in MeCtGaze::controller_start_evaluate, which is looking in an array _toContextCh. It looks like that array gets set up in MeController::remap. I don't pretend to understand what's happening here, but it's looking at two channel arrays, contextChnls and localChnls. I added log statements and found that contextChnls contains all the bones but localChnls is empty. Does that give you any clue?

September 26, 2012
8:18 pm
Avatar
Admin
Forum Posts: 983
Member Since:
December 1, 2011
sp_UserOfflineSmall Offline

I just checked in a missing SBScene::update() implementation function. Can you update to revision 3754?

Here's a minimal main() function

int main( int argc, char **argv )
{
	XMLPlatformUtils::Initialize();
	mcuCBHandle& mcu = mcuCBHandle::singleton();
	
	SmartBody::SBScene* scene = mcu._scene;

	SmartBody::SBSimulationManager* sim = scene->getSimulationManager();

	sim->start();

	float time = 0.f;
	for (int x = 0; x setTime(time);
		scene->update();
	}
	sim->stop();

	return 0;
}

My first suspicion is that the simulation isn't advancing properly. Can you confirm that your time is updating?

Ari

September 26, 2012
8:34 pm
Avatar
Member
Members
Forum Posts: 80
Member Since:
June 13, 2012
sp_UserOfflineSmall Offline

Looks like the board software ate part of your code again. I suggest enclosing it with

and

. But I'll see if I can guess what you meant.

September 26, 2012
8:36 pm
Avatar
Admin
Forum Posts: 983
Member Since:
December 1, 2011
sp_UserOfflineSmall Offline

Ok, here's all my code:

#include <vhcl.h>
#include <sb/SBScene.h>
#include <sb/SBSimulationManager.h>
#include <sb/SBBmlProcessor.h>
#include <sb/SBSkeleton.h>
#include <sb/SBCharacter.h>
#include <sb/SBJoint.h>
#include <sbm/xercesc_utils.hpp>
#include <sbm/mcontrol_util.h>
#include <iostream>

int main( int argc, char **argv )
{
vhcl::Log::StdoutListener* listener = new vhcl::Log::StdoutListener();
vhcl::Log::g_log.AddListener(listener);

XMLPlatformUtils::Initialize();
mcuCBHandle& mcu = mcuCBHandle::singleton();

SmartBody::SBScene* scene = mcu._scene;

SmartBody::SBSimulationManager* sim = scene->getSimulationManager();

SBBmlProcessor* bml = scene->getBmlProcessor();
sim->start();

scene->addAssetPath("motion", "c:/users/shapiro/smartbody/trunk/data/sbm-common/common-sk");
scene->loadAssets();

SmartBody::SBCharacter* character = scene->createCharacter("test", "");

// skeleton assigned here
SmartBody::SBSkeleton* skeleton = scene->createSkeleton("common.sk");
character->setSkeleton(skeleton);

character->createStandardControllers();

float time = 0.f;
for (int x = 0; x < 10000; x++)
{
time += .016f;

sim->setTime(time);
scene->update();
if (time > 3.0f)
{
bml->execBML("test", "<gaze target=\"test:l_toe\"/>");
}
}
sim->stop();

return 0;
}

If I comment out the skeleton assignment code:

//SmartBody::SBSkeleton* skeleton = scene->createSkeleton("common.sk");
//character->setSkeleton(skeleton);

then I get the same errors you are getting.
Can you send me your code block and I can see if I can get it to work on my end?

Ari

September 26, 2012
9:29 pm
Avatar
Member
Members
Forum Posts: 80
Member Since:
June 13, 2012
sp_UserOfflineSmall Offline

We're making progress. The relevant difference between your code and mine is that you loaded a skeleton from a .sk file, while I was loading one from a .bvh file and then remapping the names, like so:

SmartBody::SBJointMapManager* mapMgr = scene->getJointMapManager();
SmartBody::SBJointMap* myMap = mapMgr->getJointMap("FFMap");
if (myMap == NULL)
{
myMap = mapMgr->createJointMap("FFMap");
InitializeFFJointMap( myMap );
}
SmartBody::SBSkeleton* theSkel = scene->createSkeleton( "/Volumes/Home/jwwalker/Desktop/skel.bvh" );
myMap->applySkeleton( theSkel );

The InitializeFFJointMap function contains statements such as

ioJointMap->setMapping( "Hips", "base" );
ioJointMap->setMapping( "Spine1", "spine1" );
ioJointMap->setMapping( "Spine2", "spine2" );
ioJointMap->setMapping( "Spine3", "spine3" );
ioJointMap->setMapping( "Spine4", "spine4" );
ioJointMap->setMapping( "Spine5", "spine5" );
ioJointMap->setMapping( "Neck", "skullbase" );
September 26, 2012
9:37 pm
Avatar
Member
Members
Forum Posts: 80
Member Since:
June 13, 2012
sp_UserOfflineSmall Offline

One more thing. If I hand-edit the .bvh file so that it contains base, spine1, ... spine5, and skullbase without any need for joint mapping, then the gaze controller still claims it can't find the bones.

September 26, 2012
10:23 pm
Avatar
Member
Members
Forum Posts: 80
Member Since:
June 13, 2012
sp_UserOfflineSmall Offline

OK, I think I have it. Using addAssetPath, loadAssets, createSkeleton( fileName ) works, whereas using just createSkeleton( fullPath ) does not work, although it produces a skeleton that seems to have all its joints. Thanks for the help.

September 26, 2012
11:23 pm
Avatar
Admin
Forum Posts: 983
Member Since:
December 1, 2011
sp_UserOfflineSmall Offline

OK, keep me posted on your progress.

Ari

September 27, 2012
1:52 am
Avatar
Member
Members
Forum Posts: 80
Member Since:
June 13, 2012
sp_UserOfflineSmall Offline

My current problem is that when I tell the character to look to the side, he bends to the side instead of twisting. I suppose this is some problem with coordinate systems or interpretation of the quaternions. Are the joint quaternions rotations relative to the parent joint, or relative to an absolute space?

September 27, 2012
9:24 pm
Avatar
Admin
Forum Posts: 983
Member Since:
December 1, 2011
sp_UserOfflineSmall Offline

I just submitted a change that lets you reorient your skeleton to face the z-axis (assuming this is the problem...)

Can you update to r3755 then add the following before you assign the skeleton to your character:

theSkel->setVec3Attribute("orientation", -90, 0, 0)

or whatever XYZ rotation is needed so that Y is up and Z is forward.

Ari

September 27, 2012
11:10 pm
Avatar
Member
Members
Forum Posts: 80
Member Since:
June 13, 2012
sp_UserOfflineSmall Offline

Thanks, however that didn't actually help. But I found an approach that gives reasonable-looking behavior, using some quaternion calculations to convert a rotation in absolute space into one relative to the parent bone. I already had this function around, because I had previously needed it in order to apply BVH animations to my characters.

There are still a couple of things I'm wondering about with regard to gazing. I positioned my character at (0,0,0) and had him facing forward, and then used the command

<gaze sbm:target-pos="-10 0 0" sbm:joint-range="BACK NECK" sbm:priority-joint="NECK" angle="0" />

I'd expect this to make him look exactly to my left, and he almost does, but falls short by maybe 30 degrees. Is there a reason for that? Maybe the gaze controller is expecting eyes to help, even though my skeleton has no eye bones and my BML command explicitly says not to use eyes?

Also, after doing this command, I logged out the values of the joints, and got:

spine1: (0.991442, 0.000894, -0.130500, 0.003390)
spine2: (0.991439, 0.001323, -0.130475, 0.004810)
spine3: (0.966158, 0.011138, -0.257622, 0.006770)
spine4: (0.996184, 0.002283, -0.086984, 0.006782)
spine5: (1.000000, 0.000000, 0.000000, 0.000000)
skullbase: (1.000000, 0.000000, 0.000000, 0.000000)

It looks like only the back and chest are being used, though I said to use the neck too.

September 27, 2012
11:25 pm
Avatar
Admin
Forum Posts: 983
Member Since:
December 1, 2011
sp_UserOfflineSmall Offline

Our original gaze controller did not make allowances for missing joints - it assumed that our common skeleton was being used.
In the past year, we have tweaked that controller to accommodate missing joints, but as you are experiencing, it doesn't fully compensate for those missing joints, and yes, it expects the eyes to participate.Once the top-most joint reaches its target, the other joints stop moving. This is on our 'todo' list.
However, there are joint limits that you could play with that might fix your problem:
If you look at the section called 'Configuring Gazing' (p 59) there are attributes that you can set on various joints that will expand the joint limits, such as:

character.setDoubleAttribute("gaze.limitHeadingChest", 30)

etc.

Once you make changes to the attributes, you will need to spawn a new gaze command for the new limits to take effect.
Let me know if those changes fix your problem.
Ari

September 27, 2012
11:26 pm
Avatar
Member
Members
Forum Posts: 80
Member Since:
June 13, 2012
sp_UserOfflineSmall Offline

P.S. I thought maybe the problem was the order of the joint range, so I tried sbm:joint-range="NECK BACK", but it didn't make any difference.

September 27, 2012
11:43 pm
Avatar
Member
Members
Forum Posts: 80
Member Since:
June 13, 2012
sp_UserOfflineSmall Offline

Just after creating my character, I added this:

testChar->setDoubleAttribute("gaze.limitHeadingEyes", 0.0);
testChar->setDoubleAttribute("gaze.limitHeadingChest", 30.0);
testChar->setDoubleAttribute("gaze.limitHeadingBack", 30.0);
testChar->setDoubleAttribute("gaze.limitHeadingNeck", 40.0);

But in the log, I got this:

Warning, Double Attribute gaze.limitHeadingEyes does not exist.
Warning, Double Attribute gaze.limitHeadingChest does not exist.
Warning, Double Attribute gaze.limitHeadingBack does not exist.
Warning, Double Attribute gaze.limitHeadingNeck does not exist.

September 27, 2012
11:55 pm
Avatar
Admin
Forum Posts: 983
Member Since:
December 1, 2011
sp_UserOfflineSmall Offline

Sorry; set the attributes after you call 'createStandardControllers()'.
Each controller has a set of attributes that get copied to the character (once the controller has been attacehd) and can be used to customize the controllers.

Ari

September 28, 2012
12:06 am
Avatar
Member
Members
Forum Posts: 80
Member Since:
June 13, 2012
sp_UserOfflineSmall Offline

OK, I got rid of the warnings, but the gaze controller still refuses to use the neck.

September 28, 2012
12:12 am
Avatar
Admin
Forum Posts: 983
Member Since:
December 1, 2011
sp_UserOfflineSmall Offline

Looking at the gaze code (me_ct_gaze.cpp), it looks like the neck is composed of three joints: "spine4", "spine5", "skullbase"
If you don't have a mapping for those three joints, the gaze controller might eliminate the neck from the gaze algorithm.
I'll ask my gaze expert and get back to you.

Ari

September 28, 2012
12:48 am
Avatar
Member
Members
Forum Posts: 80
Member Since:
June 13, 2012
sp_UserOfflineSmall Offline

I do have spine4, spine5, and skullbase, but as I said above, spine4 is getting used, while spine5 and skullbase are not used.

September 28, 2012
1:06 am
Avatar
Member
Members
Forum Posts: 80
Member Since:
June 13, 2012
sp_UserOfflineSmall Offline

I tried adding fake eyeball bones to my .bvh file that defines my skeleton. Now I no longer get warnings about missing eyeballs, but the gaze doesn't use them (even after removing the setting of gaze.limitHeadingEyes to 0), and still doesn't use spine5 or skullbase. And the total twist is still about 70 degrees, not the 90 degrees I was expecting.

September 28, 2012
1:30 am
Avatar
Member
Members
Forum Posts: 80
Member Since:
June 13, 2012
sp_UserOfflineSmall Offline

Ari Shapiro said (in reference to gaze limiting attributes)

Sorry; set the attributes after you call 'createStandardControllers()'.
Each controller has a set of attributes that get copied to the character (once the controller has been attacehd) and can be used to customize the controllers.

That didn't suffice, I also had to add

mcuCBHandle::singleton().createDefaultControllers();

to my initialization code. Without that, the gaze controller doesn't get created until the first time a gaze BML command is executed.

September 28, 2012
6:03 pm
Avatar
Member
Members
Forum Posts: 80
Member Since:
June 13, 2012
sp_UserOfflineSmall Offline

More testing results: If I use the BML command

<gaze sbm:target-pos="-10 0 0" />

and no gaze limit attributes, then all the bones get used, from spine1 to eyeballs, and the total twist angle is close to 90 degrees. If I impose the limit

testChar->setDoubleAttribute("gaze.limitHeadingEyes", 0.0);

then the eyeballs don't get used, but the total angle is about 77 degrees.

I conclude that neither sbm:joint-range nor gaze.limitHeadingEyes work the way I would expect and want them to work, i.e., making other joints take up the slack. But I can work with this, now that I understand it.

Oh, and by the way, if the eyeball joints are missing, then the gaze controller won't use spine5 or skullbase either. Weird.

October 1, 2012
8:22 pm
Avatar
Member
Members
Forum Posts: 80
Member Since:
June 13, 2012
sp_UserOfflineSmall Offline

If I stop the simulation and restart it later, then subsequent gaze commands fail to move the character. Am I missing a step, or do I really have to just keep the simulation going?

October 1, 2012
8:24 pm
Avatar
Admin
Forum Posts: 983
Member Since:
December 1, 2011
sp_UserOfflineSmall Offline

What do you mean by 'stop the simulation' - do you mean 'pause' it? not advance the time?
If you 'stop' the simulation, the simulation is finished and needs to be reset. Since you are controlling the SmartBody simulation time externally, you simply avoid taking the next simulation step - you can pause it indefinitely that way.

There is nothing (that I know of) that is different between the gaze and the other components - they all operate off of the simulation time.

Does this answer your question?

Ari

October 1, 2012
10:29 pm
Avatar
Member
Members
Forum Posts: 80
Member Since:
June 13, 2012
sp_UserOfflineSmall Offline

By "stop the simulation", I meant call SBSimulationManager::stop(). Yes, I see that I can pause by just not advancing the time. But what I wanted to be able to do was simulate, say, from time 0 to time 5, and later run again from time 0 onward. It looks like I can accomplish that by not stopping the simulation but rather calling SBSimulationManager::reset() before setting the time back to 0.

October 2, 2012
9:10 pm
Avatar
Admin
Forum Posts: 52
Member Since:
August 8, 2012
sp_UserOfflineSmall Offline

jwwalker said

More testing results: If I use the BML command

<gaze sbm:target-pos="-10 0 0" />

and no gaze limit attributes, then all the bones get used, from spine1 to eyeballs, and the total twist angle is close to 90 degrees. If I impose the limit

testChar->setDoubleAttribute("gaze.limitHeadingEyes", 0.0);

then the eyeballs don't get used, but the total angle is about 77 degrees.

I conclude that neither sbm:joint-range nor gaze.limitHeadingEyes work the way I would expect and want them to work, i.e., making other joints take up the slack. But I can work with this, now that I understand it.

Oh, and by the way, if the eyeball joints are missing, then the gaze controller won't use spine5 or skullbase either. Weird.

========================================================

I just fixed this problem in gaze controller. It is related to some incorrect logic for setting active joints when eyeballs are missing. Now all characters with no eye joints should gaze correctly. You can update to the latest version for this fix. Thanks !

Andrew

October 4, 2012
1:22 am
Avatar
Member
Members
Forum Posts: 80
Member Since:
June 13, 2012
sp_UserOfflineSmall Offline

Thanks, Andrew. When you say "I just fixed this problem", I assume you are referring to my comment that if the eye joints are missing, then spine5 and skullbase are not used. And yes, that is fixed. But it still seems to be the case that when eye joints are missing or limited, other joints do not take up the slack, so it's still not really practical to use a skeleton without eye joints.