University of Southern CaliforniaUSC
USC ICT TwitterUSC ICT FacebookUSC ICT YouTube

TTS troubles and integration | General SmartBody Discussion | Forum

Avatar

Please consider registering
guest

sp_LogInOut Log In sp_Registration Register

Register | Lost password?
Advanced Search

— Forum Scope —




— Match —





— Forum Options —





Minimum search word length is 3 characters - maximum search word length is 84 characters

sp_Feed Topic RSS sp_TopicIcon
TTS troubles and integration
July 23, 2014
11:17 pm
Avatar
New Member
Members
Forum Posts: 1
Member Since:
July 23, 2014
sp_UserOfflineSmall Offline

Hi there. I've been doing a research on developing a TTS avatar system with real-time or preset list of texts. I am not too experienced with this, therefore i would like to apologize in advance for asking too trivial questions. 

 

I have ran your platform and imported your Brad character to test out how it all works. When trying to synthesize a speech through the BML editor I get the cmd line error "SpeechRequest has no visemes". I am guessing it is because I have not set any visemes to the correspond with the Phonemes or because there is no valid tts relay agent and I will get to that as well but I wanted to ask this in the meantime. 

My problem is that I cannot run the Ttsrelaygui due to a crash, simply not responding. When using VS2010 as a debugger it displays that there is a VHMSG library missing or cannot be found. I read all the relevant topics on the forum and tried after starting a ActiveMQ service (wrapper.exe), but it is still not working. I am using the r5615 Windows build.

I want to use cereproc as I have a registration for their cloud service but looking at their phoneme set, they have 46(45) phonemes and there are only 21 in your set. I think I came across a topic that was related to this in the forum, but what would be a general guideline in adding/editing phonemes in the software package. 

Another general question. A sample timings xml file from cereproc would be like this. Would i need to apply special formatting for it to work with the lip sync algorithms ?

And finally, i am planning on using Flash stage3D to implement it as a web application. I saw in the documentation that it is possible to integrate it as a rendering engine in flash, but that it also doesnt work with chrome. Is that still the case? Also I cannot find the trunk folder where the flash installation files are, there is no trunk folder at all.

<?xml version="1.0"?>

-<trans>

<phone end="0.011" start="0.000" name="sil"/>

<word end="0.011" start="0.011" name="voice"/>

<phone end="0.102" start="0.011" name="v"/>

<phone end="0.281" start="0.102" name="oi"/>

<phone end="0.371" start="0.281" name="s"/>

<mark end="0.371" start="0.371" name="cptk_0_0_5_5"/>

<word end="0.371" start="0.371" name="request"/>

<phone end="0.431" start="0.371" name="r"/>

<phone end="0.461" start="0.431" name="i"/>

<phone end="0.551" start="0.461" name="k"/>

<phone end="0.621" start="0.551" name="w"/>

<phone end="0.726" start="0.621" name="e"/>

<phone end="0.836" start="0.726" name="s"/>

<phone end="0.866" start="0.836" name="t"/>

<word end="2.845" start="2.845" name="metadata"/>

<phone end="2.935" start="2.845" name="m"/>

<phone end="3.015" start="2.935" name="e"/>

<phone end="3.075" start="3.015" name="t"/>

<phone end="3.115" start="3.075" name="@"/>

<phone end="3.215" start="3.115" name="d"/>

<phone end="3.336" start="3.215" name="aa"/>

<phone end="3.440" start="3.336" name="t"/>

<phone end="3.590" start="3.440" name="@"/>

<phone end="3.602" start="3.590" name="sil"/>

<phone end="3.614" start="3.602" name="sil"/>

</trans>

July 27, 2014
11:12 pm
Avatar
Admin
Forum Posts: 983
Member Since:
December 1, 2011
sp_UserOfflineSmall Offline

The SmartBody lip syncing scheme maps the set of English phonemes to a reduced set of 21 phonemes for efficiency with lip syncing (for example the 'b', 'm' and 'p' sounds are associated with the same reduced phoneme mapping. If you are using Windows, cerevoice has a SAPI mapping and you should be able to plug in the voice into Windows and the voice will appear when you use the TTSRelay. I've heard other people complain about this TTSRelay crash - not sure what it is. Have you tried downloading the SmartBody code from the SVN repository instead?

 

Regarding your sample timings, if you can convert that file into the file that SmartBody expects (SmartBody manual, page 173, section titled 'Synchronizing Speech using Prerecorded Audio') then it will work. The key is that you would have to do the phoneme mappings the way SmartBody does.

 

Yes, there is a flash version of SmartBody that we put together, athough it is a little out of date, and would need to be rebuilt using the latest code. If someone wanted to put in the effort to re-port the latest, I could support that. It would also be nice to connect it to Flash3D; our existing interface has a very lightweight scene editor and is not suitable for complex environments.

 

Ari

Forum Timezone: America/Los_Angeles

Top Posters:

jwwalker: 80

jyambao: 52

rbaral: 47

adiaz: 30

WargnierP: 29

lucky7456969: 28

mbarros: 28

avida.matt: 26

JonathanW: 24

laguerre: 23

Member Stats:

Guest Posters: 69

Members: 122101

Moderators: 3

Admins: 4

Forum Stats:

Groups: 1

Forums: 5

Topics: 531

Posts: 2495