University of Southern CaliforniaUSC
USC ICT TwitterUSC ICT FacebookUSC ICT YouTube

Adding new mixamo character | General SmartBody Discussion | Forum

Avatar

Please consider registering
guest

sp_LogInOut Log In sp_Registration Register

Register | Lost password?
Advanced Search

— Forum Scope —




— Match —





— Forum Options —





Minimum search word length is 3 characters - maximum search word length is 84 characters

sp_Feed Topic RSS sp_TopicIcon
Adding new mixamo character
April 3, 2014
1:07 am
Avatar
Member
Members
Forum Posts: 47
Member Since:
March 1, 2014
sp_UserOfflineSmall Offline

Hi,

 

I followed the user manual and tried to add new character(football player, as stated in the user manual). I was able to load the character as per the instructions provided in the manual. I am interested in doing lip sync of this new character. For other characters- Brad and Rachel there are many other related files(for ex: separate file for skeleton (.sk) and many other files).

I wonder, if I can make the lip sync by simply adding the .fbx and .dae file and accomodating the related changes in the init-default-mixamo.py or do I need to have each and every file as that are present for Rachel and Brad?

 

Thank you.

 

April 3, 2014
1:14 am
Avatar
Admin
Forum Posts: 983
Member Since:
December 1, 2011
sp_UserOfflineSmall Offline

In order to do lip sync on that (or any character) there needs to be a set of facial poses or blendshapes on the character. I'm not sure if that particular character has such facial poses, although you might be able to construct and export them with a 3D tool.

 

To do lip syncing, you will need to define a FaceDefinition (page 90 of the manual) and the set of lip sync poses (W, open, ShCh, etc.)  Once the facedefinition is set up, the character will be able to 'talk'.

Does this make sense?

 

Ari

 

 

April 3, 2014
2:12 pm
Avatar
Member
Members
Forum Posts: 47
Member Since:
March 1, 2014
sp_UserOfflineSmall Offline

Thanks Ari for the explanation. By the way, is it possible for you to let me know which tool you used to generate Brad and Rachel?

 

Thank you.

April 3, 2014
10:03 pm
Avatar
Admin
Forum Posts: 983
Member Since:
December 1, 2011
sp_UserOfflineSmall Offline

We used Maya.

 

There is also a project in the SmartBody source trunk called FbxToSBConverter which can convert .fbx files to our own internal format.

May 8, 2014
9:47 pm
Avatar
Member
Members
Forum Posts: 47
Member Since:
March 1, 2014
sp_UserOfflineSmall Offline

Hi,

I was trying to use the FbxToSBConverter.exe to convert the .fbx file into .skm file. I tried with the ChrBrad.fbx and ChrRachel.fbx. I just dragged the .fbx file on the FbxToSBConverter.exe as instructed in the forum https://groups.google.com/forum/#!topic/vhtoolkit/ZYorzm0eVCg.

But the .skm file generated was very small and didnot contain any animation related information. For instance the generated .skm file for ChrBrad.fbx was something like

# SKM Motion Definition
# FbxConverter.exe v1.0    Units: m

SkMotion

name "ChrBrad"

No animation data

I just wonder if there is anything missing.

Thank you.

May 9, 2014
6:24 pm
Avatar
Member
Members
Forum Posts: 3
Member Since:
May 9, 2014
sp_UserOfflineSmall Offline

Hello,

Dragging and dropping an fbx onto the FbxToSbConverter defaults to outputting an .skm (animation file.) If you want to generate an .sk (skeleton file) you need to provide arguments to the FbxToSbConverter

Pass arguments through the command line or a batch file like this

FbxToSBConverter.exe ChrBrad.fbx output .sk

For a list of all the arguments you can use, view this page

Keep in mind, ChrBrad.fbx is a brad's skeleton file and does not contain any animation information, which is why your output says "no animation data"

-Adam

May 9, 2014
7:11 pm
Avatar
Member
Members
Forum Posts: 47
Member Since:
March 1, 2014
sp_UserOfflineSmall Offline

Hi Adam,

Thanks for your reply.

I got your point. My intention of using FbxToSBconverter.exe was to get files like ChrBrad@open.skm, ChrBrad@PBM.skm, ChrBrad@ShCh.skm. I think these files are obtained from some .fbx files and gave a try with the .fbx files provided with SmartBody.

If I am in the wrong direction, can you please help me understand how the .skm files stated above are generated? Because I saw these files can be directly called from python scripts.

 

Thank you.

May 9, 2014
7:21 pm
Avatar
Member
Members
Forum Posts: 3
Member Since:
May 9, 2014
sp_UserOfflineSmall Offline

You are correct in thinking open, pbm, shch, etc are obtained from an fbx.  Artists create and animate these mouth shapes in a modeling program like Maya, export them to .fbx, and then drag and drop them on the FbxToSbConverter in order to get the output .skm which SmartBody can read.

-Adam

May 9, 2014
7:28 pm
Avatar
Member
Members
Forum Posts: 47
Member Since:
March 1, 2014
sp_UserOfflineSmall Offline

Is the .fbx corresponding to the animations(open,pbm..) available in svn?

 

Thank you.

May 12, 2014
6:25 pm
Avatar
Member
Members
Forum Posts: 47
Member Since:
March 1, 2014
sp_UserOfflineSmall Offline

I have some fbx files with animation which can be seen in other tools like Unity. But when I tried to convert these files using FbxToSbconverter.exe then the .skm file generated contains no animation. It contains the text 

No animation data

 

Thank you.

May 12, 2014
7:33 pm
Avatar
Member
Members
Forum Posts: 3
Member Since:
May 9, 2014
sp_UserOfflineSmall Offline

We typically use maya attributes as meta data in our animations in order to filter out joints that won't be used for optimization purposes.  If your animations don't use these attributes, you can force the FbxToSbConverter to output all channel information for all the joints in the animation by doing passing the "forceChannels" arg to the converter.

FbxToSBConverter.exe test.fbx forceChannels

after the conversion is over, you should see animation data in the output skm

May 13, 2014
6:59 pm
Avatar
Member
Members
Forum Posts: 47
Member Since:
March 1, 2014
sp_UserOfflineSmall Offline

Hi ict_adam,

Thanks for your response.

I added some animation on top of the ChrBrad.fbx file using Maya. After adding the forceChannels option to FbxToSbConverter.exe, I was able to get a .skm file whose contents are as shown below:

# SKM Motion Definition
# FbxConverter.exe v1.0 Units: m

SkMotion

name "ChrBrad@001_inner_brow_raiser"  # I gave this name myself

channels 8
JtLipUpperMid XPos
JtLipUpperMid YPos
JtLipUpperMid ZPos
JtLipUpperMid Quat
JtEyebrow02Lf XPos
JtEyebrow02Lf YPos
JtEyebrow02Lf ZPos
JtEyebrow02Lf Quat

frames 5
kt 0.000000 fr 0.000000 0.000000 0.000000 0.347852 -1.542662 -0.347852 0.000000 -0.000500 0.000000 0.287163 -1.761519 -0.232870
kt 0.041667 fr 0.000000 0.000000 0.000000 0.347852 -1.542662 -0.347852 0.000000 -0.001789 0.000000 0.287163 -1.761519 -0.232870
kt 0.083334 fr 0.000000 0.000000 0.000000 0.347852 -1.542662 -0.347852 0.000000 -0.003553 0.000000 0.287163 -1.761519 -0.232870
kt 0.125000 fr 0.000000 0.000000 0.000000 0.347852 -1.542662 -0.347852 0.000000 -0.005474 0.000000 0.287163 -1.761519 -0.232870
kt 0.166667 fr 0.000000 0.000000 0.000000 0.347852 -1.542662 -0.347852 0.000000 -0.007237 0.000000 0.287163 -1.761519 -0.232870

 

I manually added the followings to the end of the above

emphasis time: 0.033333
ready time: 0.033333
relax time: 0.033333
strokeStart time: 0.033333
stroke time: 0.033333

Then I modified the FacialMovementDemo.py file inside data\examples folder by adding the following Action Unit:

bradFace.setAU(1,  "both",  "ChrBrad@001_inner_brow_raiser")

I was hoping the FaceViewer window will be able to show this action unit and render my animation but these cannot be recognized by SmartBody. In my view my .skm file is similar(at least in format) to the ones already available. I dont know what might be misleading. Any help will be appreciated.

 

Thank you.

May 14, 2014
1:00 am
Avatar
Admin
Forum Posts: 983
Member Since:
December 1, 2011
sp_UserOfflineSmall Offline

A facial pose is only a single frame, so you can eliminate all the frames except for the first one.

Did you put the motion in the same location as the others?

 

Ari

May 14, 2014
4:21 am
Avatar
Member
Members
Forum Posts: 47
Member Since:
March 1, 2014
sp_UserOfflineSmall Offline

Hi Ari,

Thanks for your response.

If I understand it correctly, the other motions also have multiple frames. For instance the file ChrBrad@004_brow_lowerer_lf in the folder data\ChrBrad\face\ also has two frames. The portion of it is as mentioned below:

frames 2
kt 0.000000 fr 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 -0.000000 -0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 -0.000000 0.000000 0.000000 -0.001691 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 -0.009884 -0.008218 0.001050 -0.209480 -0.079591 0.184702 -0.010633 -0.009270 0.004343 0.000000 0.000000 0.000000 -0.009469 -0.004586 0.002958 0.000000 0.000000 0.000000 -0.002850 -0.003622 0.002807 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 -6.283185 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 -0.000196 -0.000004 0.000000 0.033072 -0.010295 -0.000054 0.000431 -0.000019 0.000000 0.072788 -0.022657 -0.000119 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 -0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
kt 0.033333 fr 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 -0.000000 -0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 -0.000000 0.000000 0.000000 -0.001691 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 -0.009884 -0.008218 0.001050 -0.209480 -0.079591 0.184702 -0.010633 -0.009270 0.004343 0.000000 0.000000 0.000000 -0.009469 -0.004586 0.002958 0.000000 0.000000 0.000000 -0.002850 -0.003622 0.002807 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 -6.283185 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 -0.000196 -0.000004 0.000000 0.033072 -0.010295 -0.000054 0.000431 -0.000019 0.000000 0.072788 -0.022657 -0.000119 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 -0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000

 

I placed the new skm file in the same location as other motions, which is inside data\ChrBrad\face\

Is there anything else that needs to be configured somewhere else?

 

Thank you.

May 16, 2014
5:55 pm
Avatar
Admin
Forum Posts: 983
Member Since:
December 1, 2011
sp_UserOfflineSmall Offline

That should be all you need to do.

Make sure that the name internal to the motion file:

 

name "ChrBrad@001_inner_brow_raiser"

 

matches the name of the .skm file:

 

ChrBrad@001_inner_brow_raiser.skm

 

I was able to get your data working on my system. It looks like SmartBody is unbothered by multiple frames; it ignores all but 1 frame.

 

Ari

May 16, 2014
6:56 pm
Avatar
Member
Members
Forum Posts: 47
Member Since:
March 1, 2014
sp_UserOfflineSmall Offline

Hi Ari,

I have the file FacialMovementDemo.py inside data\examples, and one of its line is 

bradFace.setAU(1,  "both", "ChrBrad@001_inner_brow_raiser")

I have the file  ChrBrad@001_inner_brow_raiser.skm in the path data\ChrBrad\face\ where other .skm files exist.

The file ChrBrad@001_inner_brow_raiser.skm contains the following:

# SKM Motion Definition
# FbxConverter.exe v1.0 Units: m

SkMotion

name "ChrBrad@001_inner_brow_raiser"

channels 8
JtLipUpperMid XPos
JtLipUpperMid YPos
JtLipUpperMid ZPos
JtLipUpperMid Quat
JtEyebrow02Lf XPos
JtEyebrow02Lf YPos
JtEyebrow02Lf ZPos
JtEyebrow02Lf Quat

frames 5
kt 0.000000 fr 0.000000 0.000000 0.000000 0.347852 -1.542662 -0.347852 0.000000 -0.000500 0.000000 0.287163 -1.761519 -0.232870
kt 0.041667 fr 0.000000 0.000000 0.000000 0.347852 -1.542662 -0.347852 0.000000 -0.001789 0.000000 0.287163 -1.761519 -0.232870
kt 0.083334 fr 0.000000 0.000000 0.000000 0.347852 -1.542662 -0.347852 0.000000 -0.003553 0.000000 0.287163 -1.761519 -0.232870
kt 0.125000 fr 0.000000 0.000000 0.000000 0.347852 -1.542662 -0.347852 0.000000 -0.005474 0.000000 0.287163 -1.761519 -0.232870
kt 0.166667 fr 0.000000 0.000000 0.000000 0.347852 -1.542662 -0.347852 0.000000 -0.007237 0.000000 0.287163 -1.761519 -0.232870

emphasis time: 0.033333
ready time: 0.033333
relax time: 0.033333
strokeStart time: 0.033333
stroke time: 0.033333

and thats it.  I cant see the bilateral AU for 1 in FaceViewer or in the left navigation panel.

I wonder why the same thing runs on yours and not on mine. Is there some setting to be enabled/disabled for this?

 

Thank you.

May 16, 2014
7:51 pm
Avatar
Admin
Forum Posts: 983
Member Since:
December 1, 2011
sp_UserOfflineSmall Offline

If you have a bilateral motion, you can't have an individual left or right one, so make sure you eliminate any other face definition that use AU 1. In other words, you can't have:

bradFace.setAU(1,  "both",  "ChrBrad@001_inner_brow_raiser_lf")
bradFace.setAU(1,  "right", "ChrBrad@001_inner_brow_raiser_rt")

SmartBody will get confused, and pick the 'right' one only.

 

Ari

May 16, 2014
8:38 pm
Avatar
Member
Members
Forum Posts: 47
Member Since:
March 1, 2014
sp_UserOfflineSmall Offline

Thanks Ari,

It worked finally. One thing what is still confusing me is the size of the .skm file corresponding to my animation and the one already present. Mine is small- just 8 channels and 5 frames but the one provided with Smartbody have 135 channels. I used ChrBrad.fbx to generate the animation. Well thats not an issue but just for curiosity.

 

Thank you.

May 16, 2014
8:42 pm
Avatar
Admin
Forum Posts: 983
Member Since:
December 1, 2011
sp_UserOfflineSmall Offline

Glad it worked!

If you traced channels listed to the data, you will notice that most of the channel data in the other files is actually zero. You've been very efficient, and only specified channel data that is relevant to your facial poses, which is why your file is much smaller.

 

Ari

Forum Timezone: America/Los_Angeles

Top Posters:

jwwalker: 80

jyambao: 52

rbaral: 47

adiaz: 30

WargnierP: 29

lucky7456969: 28

mbarros: 28

avida.matt: 26

JonathanW: 24

laguerre: 23

Member Stats:

Guest Posters: 69

Members: 122101

Moderators: 3

Admins: 4

Forum Stats:

Groups: 1

Forums: 5

Topics: 531

Posts: 2495