University of Southern CaliforniaUSC
USC ICT TwitterUSC ICT FacebookUSC ICT YouTube

kinect v2 and capturing .dae | General SmartBody Discussion | Forum

Avatar

Please consider registering
guest

sp_LogInOut Log In sp_Registration Register

Register | Lost password?
Advanced Search

— Forum Scope —




— Match —





— Forum Options —





Minimum search word length is 3 characters - maximum search word length is 84 characters

sp_Feed Topic RSS sp_TopicIcon
kinect v2 and capturing .dae
January 19, 2015
2:39 am
Avatar
Member
Members
Forum Posts: 5
Member Since:
December 4, 2014
sp_UserOfflineSmall Offline

Hi everybody !!!! I am fairly new to smartbody and I have a little bit of knowledge of Python.  My question is : I need to record a .dae with the Kinect v2. I want to do like the kinect re-targeting program that was done previously: http://smartbody.ict.usc.edu/H.....apture.mp4. So my challenge create an avatar that the movement are recorded in .dae with a Kinect 2. Is it possible? Is work have been done with smartbody and kinect v2 ?

 

Any pointer will help me a lot...

 

Thanx,

 

Benoit

January 19, 2015
6:17 pm
Avatar
Admin
Forum Posts: 983
Member Since:
December 1, 2011
sp_UserOfflineSmall Offline

We haven't created an application that will connect to the Kinect v2 yet, but it would take the same form as the Kinect v1 version. This uses VHMSG (Virtual Human Message) library, which is an asynchronous communication library that works over ActiveMQ (http://activemq.apache.org/) and communicates with SmartBody.(the Kinect v1 version is located in the SmartBody SVN in lib/kinecttracker/src/main.cpp, here's a snippet from the code:

 

establish the vhmg connection to the messaging server, lib/kinecttracker/src/main.cpp:632:

if (vhmsg::ttu_open() == vhmsg::TTU_SUCCESS)
        vhmsg::ttu_notify2("receiver", "echo Kinect Connected...");

 

sends the base joint information, lib/kinecttracker/src/main.cpp:470:

XnSkeletonJointPosition base;
g_UserGenerator.GetSkeletonCap().GetSkeletonJointPosition(aUsers[i], XN_SKEL_TORSO, base);
std::stringstream command1;
command1 << "skeleton " << skeletonName << " kinect position 0 ";
command1 << base.position.X << " " << base.position.Y << " " << base.position.Z;
vhmsg::ttu_notify2("receiver", command1.str().c_str());

sends the info for the other joints, lib/kinecttracker/src/main.cpp:426:

 

std::stringstream command;

...

command << "skeleton " << skeletonName << " kinect rotations ";
for (int j = 0; j < 24; j++)
{
          matrixToQuat(jointInfo[j], w, x, y, z);
          command << w << " " << x << " " << y << " " << z << " ";
 }
vhmsg::ttu_notify2("receiver", command.str().c_str());

 

So in summary:

 

1) You need to get an application that connects to the Kinect v2 and tracks the character's body. The Kinect v2 sdk has such a program already.

2) Modify the application such that it uses the VHMSG library that is part of the SmartBody distribution (in lib/vhmsg/vhmsg-c)

3) For each character tracked in Kinect v2, for each joint/bone, send the bone/joint state via a VHMG to SmartBody. Messages look like this (from the code comments in mcontrol_callbacks.cpp:

// receiver echo <content>
// receiver enable
// receiver skeleton <skeletonName> <emitterType> position <joint-index/joint-name> <x> <y> <z>    
// receiver skeleton <skeletonName> <emitterType> positions <x1> <y1> <z1> <x2> <y2> <z2> ...                            24 Joints in total if emitterType == "kinect"
// receiver skeleton <skeletonName> <emitterType> rotation <joint-index/joint-name> <q.w> <q.x> <q.y> <q.z>
// receiver skeleton <skeletonName> <emitterType> rotations <q1.w> <q1.x> <q1.y> <q1.z> <q2.w> <q2.x> <q2.y> <q2.z>...    20 Joints in total if emitterType == "kinect"

4) On the character, set the 'receiverName' attribute to the name from the receiver message (skeletonName):

mycharacter.setStringAttribute("receiverName", "body1")

where 'body1' is the first skeleton, 'body2' is the second, etc (or whatever you decide to call it...)

Then you should be able to puppeteer the characters directly via joint information.

 

An even better way to do this would be to first puppeteer a Kinect v2 skeleton in SmartBody, then online retarget that information onto your own skeleton. We don't yet have a way to online retarget live streamed data (we do it for animations), but it wouldn't take much coding for us to adopt that capability.

 

Ari

January 20, 2015
8:48 pm
Avatar
Member
Members
Forum Posts: 5
Member Since:
December 4, 2014
sp_UserOfflineSmall Offline

Will it be possible to record the .dae ?  And is it possible to have a tutorial or a walkthrough to help me with this task.  I code in python but I am fairly new in the cpp arena....  Does the recording of the movement in .dae have been done ?

 

Thanx for answering...

 

benoit

January 20, 2015
9:24 pm
Avatar
Admin
Forum Posts: 983
Member Since:
December 1, 2011
sp_UserOfflineSmall Offline

In  Python, there are three functions on the character that let you save the performance:

 

mycharacter.startMotionRecord(frameRate)

mycharacter.stopMotionRecord(motionName, type) # where type is 'bvh' or 'skm', saves to a Motion

mycharacter.writeMotionRecord(file, type)# where type is 'bvh' or 'skm', saves to a file

 

If you wanted to export to a .dae as well, you (or someone...) would have to take the Motion data and convert it to .dae format, which probably isn't that hard. Alternatively, you could save to .bvh, then use some other converter to write it to .dae.

 

Ari

January 26, 2015
7:03 pm
Avatar
Member
Members
Forum Posts: 5
Member Since:
December 4, 2014
sp_UserOfflineSmall Offline

In what folder I can find those python script ?

 

Thank you,

 

Benoit

January 27, 2015
1:25 am
Avatar
Admin
Forum Posts: 983
Member Since:
December 1, 2011
sp_UserOfflineSmall Offline

Which scripts?

To send a python command? In sbgui, you can use the Window->Command Window

Is this what you mean?

 

Ari

Forum Timezone: America/Los_Angeles

Top Posters:

jwwalker: 80

jyambao: 52

rbaral: 47

adiaz: 30

WargnierP: 29

lucky7456969: 28

mbarros: 28

avida.matt: 26

JonathanW: 24

laguerre: 23

Member Stats:

Guest Posters: 69

Members: 122101

Moderators: 3

Admins: 4

Forum Stats:

Groups: 1

Forums: 5

Topics: 531

Posts: 2495