University of Southern CaliforniaUSC
USC ICT TwitterUSC ICT FacebookUSC ICT YouTube

Using Smartbody with Kinect | General SmartBody Discussion | Forum

Avatar

Please consider registering
guest

sp_LogInOut Log In sp_Registration Register

Register | Lost password?
Advanced Search

— Forum Scope —




— Match —





— Forum Options —





Minimum search word length is 3 characters - maximum search word length is 84 characters

sp_Feed Topic RSS sp_TopicIcon
Using Smartbody with Kinect
June 18, 2015
10:07 pm
Avatar
New Member
Members
Forum Posts: 2
Member Since:
June 18, 2015
sp_UserOfflineSmall Offline

Hi, 

I'm using smartbody with Kinect, and have successfully made the avatar repeat/mimic my movement which is captured by Kinect. Now I have a question. Can I add some textual information or even graphics (like arrows) in the scene? The information might be different for each frame.

Another question is, does the current version support the newest Kinect sensor (kinect 2, XBOX ONE)?

Thanks!

June 19, 2015
1:10 am
Avatar
Admin
Forum Posts: 983
Member Since:
December 1, 2011
sp_UserOfflineSmall Offline

1) There is an implementation in SmartBody in Windows of the CEGUI (http://cegui.org.uk/) library, which is a 2D drawing library in Python (and in C++). So in Python, you could code using the CEGUI API, something like this:

 

winManager = WindowManager.getSingleton()
staticText = winManager.createWindow("OgreTray/StaticText", "foo")
staticText.setText("this is some text")
System.getSingleton().getDefaultGUIContext().getRootWindow().addChild(staticText)

to create a 2D drawing, and you are limited only by whatever CEGUI could provide for you, so you could alter that data in a script that changes every frame. There are some examples under the data/examples/cegui folder, and we created a file called GUIUtil.py which simplifies some of the API.

 

2) I haven't ported the Kinect code to use the Kinect v2 yet, although it certainly wouldn't be hard to adapt the existing Kinect v2 body recognition demo the same way as for the Kinect v1. 

 

Ari

June 19, 2015
10:17 pm
Avatar
New Member
Members
Forum Posts: 2
Member Since:
June 18, 2015
sp_UserOfflineSmall Offline

Thanks for your help! For the first question, textual information in Python may not solve our problem. We use a python file which defines two still avatars. Then we have a .txt file in which each line defines the motion data (position or rotation) for each frame. For example,

receiver skeleton kinect1 kinect position 0 -0.0448584 0.433857 2.61957

or

receiver skeleton kinect1 kinect rotations 0.0139454 -0.0214925 0.999626 0.00951729 0.998403 -0.0516808 -0.0195215 0.0117794 0.995945 0.0899248 -0.000520614 0.00251146 0.99243 0.115477 0.00483237 -0.0415302 -0.38305 -0.0227072 0.054646 0.92183 0.972904 -0.0123402 -0.0416381 -0.227096 0.989051 0.124641 0.00987871 -0.0783898 0.999165 0.0103056 -0.000407686 -0.039527 0.371199 0.0229251 0.0572207 0.926505 0.981929 -0.0352633 0.0278288 0.183843 0.988071 0.139605 -0.00909568 0.0643757 0.998116 0.0204008 0.00118255 0.0578564 -0.392156 8.85866e-010 0 0.919899 0.940005 -0.0751833 0.0379788 -0.3306 0.998322 -0.0381425 -0.0016634 -0.0435369 0.865106 0.493382 0.0447672 -0.0784957 0.379678 0 -6.29191e-010 0.925119 0.917839 -0.115379 -0.0473319 0.376855 0.998762 0.0369683 -0.00123135 0.0332671 0.836933 0.547152 0.0070764 -0.0108242

Then in c++, we read a line from the .txt file and send it to smartbody via the following code:

 

getline(fp,s);  %fp reads the .txt file

command << s.c_str();

vhmsg::ttu_notify2("sbm",command.str().c_str());

 

So in each frame, the avatar moves according to the motion data in the .txt file.

Now we would like to add some textual information on the screen. The textual information is related with each frame. Since one frame include one "rotations" and "position" data, the textual information can be included as:

receiver skeleton kinect1 kinect rotations 0.0139454 -0.0214925 0.999626 0.00951729 0.998403 -0.0516808 -0.0195215 0.0117794 0.995945 0.0899248 -0.000520614 0.00251146 0.99243 0.115477 0.00483237 -0.0415302 -0.38305 -0.0227072 0.054646 0.92183 0.972904 -0.0123402 -0.0416381 -0.227096 0.989051 0.124641 0.00987871 -0.0783898 0.999165 0.0103056 -0.000407686 -0.039527 0.371199 0.0229251 0.0572207 0.926505 0.981929 -0.0352633 0.0278288 0.183843 0.988071 0.139605 -0.00909568 0.0643757 0.998116 0.0204008 0.00118255 0.0578564 -0.392156 8.85866e-010 0 0.919899 0.940005 -0.0751833 0.0379788 -0.3306 0.998322 -0.0381425 -0.0016634 -0.0435369 0.865106 0.493382 0.0447672 -0.0784957 0.379678 0 -6.29191e-010 0.925119 0.917839 -0.115379 -0.0473319 0.376855 0.998762 0.0369683 -0.00123135 0.0332671 0.836933 0.547152 0.0070764 -0.0108242

receiver skeleton kinect1 kinect position 0 -0.0448584 0.433857 2.61957  

"aaaaaaaaaaaaa"

receiver skeleton Dennis kinect rotations 0.00437766 -0.0252963 0.9996 -0.0118899 0.999489 -0.012951 -0.0245748 0.0158011 0.999312 0.0370444 -0.00148155 0.000630863 0.992818 0.11667 0.00309008 -0.0262953 -0.390074 -0.0212019 0.0499608 0.919183 0.968802 0.0100333 -0.0520203 -0.242106 0.991578 0.116515 0.00659911 -0.0561604 0.998863 -0.0180207 0.00079621 -0.0441325 0.399168 0.0231364 0.0530377 0.91505 0.973028 -0.0284903 0.0319208 0.226686 0.990067 0.133261 -0.00597782 0.0444125 0.996668 -0.0774175 -0.00198795 0.0255927 -0.386776 5.04961e-010 3.48928e-007 0.922174 0.938208 0.0322096 -0.0152667 -0.344232 0.998477 -0.0545463 0.000448914 0.00821743 0.997476 -0.00570067 -0.00040448 -0.070774 0.373694 -7.53049e-010 3.56443e-007 0.927552 0.922587 -0.0890579 -0.0356893 0.373668 0.998701 -0.0201249 0.00094309 0.0468008 0.832241 0.554255 0.00734478 -0.0110285

receiver skeleton Dennis kinect position 0 -0.040553 0.433123 2.61605

"bbbbbbbbbbb"

 

My question is, when I send the motion data to Smartbody via vhmsg::ttu_notify2("sbm",command.str().c_str()), how can I edit your code so that the words "aaaaaaaaaaaaa" will be shown on the screen exactly when the avatar is moving according to the motion data for this frame? Where is the motion data processed and how the avatars are control according to the data?

Thanks!

June 21, 2015
10:35 pm
Avatar
Admin
Forum Posts: 983
Member Since:
December 1, 2011
sp_UserOfflineSmall Offline

sbgui is integrated with the 2D drawing library, CEGUI, so you can use any CEGUI command (using a Python interface) to draw text on the screen. For example, to create a text box:

winManager = WindowManager.getSingleton()

myText = winManager.createWindow(

'OgreTray/StaticText'

, 'myTextBox')   

myText.setText(

"This is some text")

System.getSingleton().getDefaultGUIContext().getRootWindow().addChild(myText)

 

 

then create a function to update the text box:

def changeText(val):
   myText.setText(val)

then when you send a VHMSG, call the function directly:

 

sb changeText("aaaaaaaa")

That should work...

 

Ari

Forum Timezone: America/Los_Angeles

Most Users Ever Online: 733

Currently Online: Jamesnuh, gotselyuk1987mon, koenig62ramos, krygerhagen69, coxgeorge3, doyle68bekker, ludvigsen62abbott, shortstext78, cormierhopper5
66 Guest(s)

Currently Browsing this Page:
1 Guest(s)

Top Posters:

jwwalker: 80

jyambao: 52

rbaral: 47

adiaz: 30

WargnierP: 29

lucky7456969: 28

mbarros: 28

avida.matt: 26

JonathanW: 24

laguerre: 23

Member Stats:

Guest Posters: 67

Members: 100502

Moderators: 3

Admins: 4

Forum Stats:

Groups: 1

Forums: 5

Topics: 460

Posts: 2403