University of Southern CaliforniaUSC
USC ICT TwitterUSC ICT FacebookUSC ICT YouTube

Problems with sbdesktop | General SmartBody Discussion | Forum

Avatar

Please consider registering
guest

sp_LogInOut Log In sp_Registration Register

Register | Lost password?
Advanced Search

— Forum Scope —




— Match —





— Forum Options —





Minimum search word length is 3 characters - maximum search word length is 84 characters

sp_Feed Topic RSS sp_TopicIcon
Problems with sbdesktop
May 14, 2014
8:48 am
Avatar
Member
Members
Forum Posts: 15
Member Since:
September 1, 2012
sp_UserOfflineSmall Offline

Hi Ari,

 

First of all, hope you are doing well and great to see all the new things that has happend with SmartBody ;)

Been away for so long, but now back and ready to give SmartBody a go.

I downloaded the lastest stable version and building it is fine (Windows 7 64-bit and VS2010).

The sbgui is running and I can load the example scripts.

My problems begin with:

1. Had to modify MSSpeechRelay.cs line 356-361 because IVONA 2 voices e.Duration is always 0.5 whereas the AudioPosition is fine:

 

if (true) // ClickLearn

{ // ClickLearn

generateAudioReply.VisemeList.Add(newGenerateAudioReplyViseme(visemeIDMap[e.Viseme], e.AudioPosition.TotalSeconds, 1.0)); // ClickLearn

} // ClickLearn

elseif (visemeMappingType.Equals("facefx")) // ClickLearn

 

This is of course NOT the right fix here - should be controllable in some other way which value to use

2. The viseme handling is still us specific. So when I and having a locate da-DK the start value is 0,1234 instead of 0.1234. My fix is in the file TtsRelay.cs line 415 and 424:

xml.WriteAttributeString("start", v.Key.ToString("0.00000", System.Globalization.CultureInfo.InvariantCulture)); // ClickLearn

3. The simplesmartbody project has multiple issues. Tried to run it directly after build, but multiple files manually need to be copied (to output directory) for it to start. Also, the line 35 and 43 is off. I managed to get it running changeing them to:

 

initPython("../../Python27/Libs"); // ClickLearn ../

scene->setMediaPath("../../../../data"); // ClickLearn ../../../

4. sbdesktop has simular issues to simplesmartbody - but worse. I am really stuck here and hope you can tell me how to get it up and running.

Best regards,

 

Michael Olsen

May 15, 2014
1:44 am
Avatar
Admin
Forum Posts: 983
Member Since:
December 1, 2011
sp_UserOfflineSmall Offline

Hi Michael,

 

1 and 2. This line seems to work with the English voices:

 if (true) // ClickLearn
            { // ClickLearn

                generateAudioReply.VisemeList.Add(new GenerateAudioReplyViseme(visemeIDMap[e.Viseme], e.Duration.TotalSeconds, 1.0)); // ClickLearn
                Console.WriteLine("Total viseme duration: " + e.Duration.Seconds);

            } // ClickLearn

 

so use e.Duration.TotalSeconds, instead of e.AudioPosition.TotalSeconds.

 

The lip syncing scheme relies on having a set of animated curves for each pair of phonemes. In SmartBody, the English set already exists, but if you want to use German, we need to generate a new set of curves for that language (this is something that I would like to do). Also, the mappings, which are English-specific, need to be changed to accomodate this as well. If you have resources for this (a German-speaking animator) that data set could probably be generated in a few days time, and I'd be happy to help you to do that.

 

3. I'll have to fix the simplesmartbody program

 

4. sbdesktop is a desktop version of SmartBody rendered on a transparent window (think of the Microsoft Paper Clip on steroids...). I'll have to see why that isn't working either.

 

Ari

May 15, 2014
8:46 am
Avatar
Member
Members
Forum Posts: 15
Member Since:
September 1, 2012
sp_UserOfflineSmall Offline

Hi Ari,

 

Thank you for the fast reply.

It's not that I really need the sbdesktop, but found that it was the best starting point for what I want to make. I need to simulate talk + animation and save each frame from the scene into image files. I need to do this as fast as possible in terms of rendering - server side - without any displaying on screen.

Do you have any idea as of when you may have fixed the sbdesktop issues?

Best regards,

Michael Olsen

May 15, 2014
10:11 pm
Avatar
Admin
Forum Posts: 983
Member Since:
December 1, 2011
sp_UserOfflineSmall Offline

Did you download the SDK or are you building everything from the SVN source location?

 

I've put in a couple of fixes to simplesmartbody to work from the SVN source (as opposed to working from the SDK distributable)To get simplesmartbody working from SVN, you can change the line:

scene->setMediaPath(../data);

to

scene->setMediaPath("../../../../data")

Also, change the Working directory (right click on the simplesmartbody project, choose Debugging) to

$(ProjectDir)\..\..\sbgui\bin

Other than that, the code works. Also, I made a tweak to the sbdesktop code, which also should work.

Ari

Forum Timezone: America/Los_Angeles

Top Posters:

jwwalker: 80

jyambao: 52

rbaral: 47

adiaz: 30

WargnierP: 29

lucky7456969: 28

mbarros: 28

avida.matt: 26

JonathanW: 24

laguerre: 23

Member Stats:

Guest Posters: 69

Members: 122211

Moderators: 3

Admins: 4

Forum Stats:

Groups: 1

Forums: 5

Topics: 531

Posts: 2495