University of Southern CaliforniaUSC
USC ICT TwitterUSC ICT FacebookUSC ICT YouTube

How to transform the static mesh with the skeleton? | General SmartBody Discussion | Forum

Avatar

Please consider registering
guest

sp_LogInOut Log In sp_Registration Register

Register | Lost password?
Advanced Search

— Forum Scope —




— Match —





— Forum Options —





Minimum search word length is 3 characters - maximum search word length is 84 characters

sp_Feed Topic RSS sp_TopicIcon
How to transform the static mesh with the skeleton?
January 18, 2016
5:53 am
Avatar
Member
Members
Forum Posts: 28
Member Since:
November 5, 2015
sp_UserOfflineSmall Offline

I've written a DirectX renderer for Smartbody. It is now able to read in a .X file.

But the problem is now that the render is static, how do I manipulate the bones in order to move them

into correct positions.

I am currently using the DeformableMesh*=>SrModel=>Vertex Array, Index Array, Normal Array and Texture Coordinates Array etc.

Thanks

Jack

January 19, 2016
9:53 am
Avatar
Admin
Forum Posts: 980
Member Since:
December 1, 2011
sp_UserOfflineSmall Offline

Not sure I understand what you are asking. are you asking how do you make a character move?

January 21, 2016
2:38 am
Avatar
Member
Members
Forum Posts: 28
Member Since:
November 5, 2015
sp_UserOfflineSmall Offline

Hello Ari,

I'd like to know how I can manipulate the skin weights, bone transformations in order to move the character.

Currently, when I use the the data inside the DeformableMesh* => SrModel* => V, T and N buffers to render the mesh,

I render a full mesh but it doesn't move at all. I wonder how can I move the bones to transform the vertices etc of the

mesh. What I see in your opengl renderer, you tie the DeformableMesh::_deformPosBuf as a vertex buffer to the opengl

renderer and use DeformableMesh::SbmSubMesh::triBuf as an index buffer. When I am doing the same thing to the

DirectX renderer and use the .X file format, I just can see a malformed (a stick alike) character being rendered.

Thanks

Jack

January 21, 2016
9:28 am
Avatar
Admin
Forum Posts: 980
Member Since:
December 1, 2011
sp_UserOfflineSmall Offline

Still not sure I understand. If you send a command to move the character (say, nod the head using a BML command like:

 

bml.execBML('*', '<head type="NOD"/>")

a subset of joints will get a control signal and part of the character will move. You can animate the rest of the character using other commands. Is this what you are trying to do?

 

Ari

January 22, 2016
12:09 am
Avatar
Member
Members
Forum Posts: 28
Member Since:
November 5, 2015
sp_UserOfflineSmall Offline

Hello Ari,

I appreciate and am quite interested in the algorithms you and Dr Feng developed for automatic skeleton retargeting.

And I want to borrow your algorithm.

Nothing fancy about it. Semi-automation is just fine, and I am looking into simple bvh mapping and retargeting

http://www.arishapiro.com/mig1.....system.pdf

In this link, you talked about transferring the rotations and translations into the bones of the target skeleton.

When you said,

q' = qinit * RGInv * q * RG;

Are we referring to a reference skeleton or not?

The general steps I come up with without looking at your source code is

For DirectX

1) Resetting all combined transformation matrices to y-up for all bones

2) let's presume I have some bvh that outputs some global rotations of a bone

3) I put the tranformation matrix into qinit

4) I put this global rotation from the bvh to RG

5) I inverse this RG to get RGInv

6) Making a conjugate between y-up space and the bone basis (offset matrix) to get q

7) Make an inverse to make this back into local space

8) copy the transformations into each bone

9) Update the hierarchy

10) Render

Will this just work?

I am now trying and testing it, and want to have your comments.

Just to remark this that it doesn't require full-automation because it is quite application eccentric...

So Could you please lend me a hand?

Jack

January 22, 2016
3:09 pm
Avatar
Admin
Forum Posts: 52
Member Since:
August 8, 2012
sp_UserOfflineSmall Offline

Hi Jack,

The formula you refer to is to transfer the joint rotation q from canonical (reference) skeleton to q' of a custom skeleton. Thus both RG and RGInv should be inferred from the skeleton you are using. Similarly, qinit is intended to rotate the skeleton into T-pose ( if it's not already in T-pose) and should also be inferred from the same skeleton.

Assuming you want to transfer the pose from bvh to your skeleton, I think you should :

1. find local rotation q for each bone from bvh. (if your bvh is not in canonical pose initially, you may need to compensate for that as well ).

2. rotate some bones in your skeleton if necessary to make it into T-pose. Store those bone rotations as qinit

3. Infer the global rotations RG and RGInv from each joint at your skeleton.

4. apply the formula to obtain q', which can then be applied directly into your skeleton.

The above are the steps I did in SmartBody for automatic retargeting. Let me know if you have any questions regarding this process. Thanks.

Andrew

April 1, 2016
9:07 pm
Avatar
Member
Members
Forum Posts: 28
Member Since:
November 5, 2015
sp_UserOfflineSmall Offline

Hello Dr Feng,

[code]

Quaternion gsrcTemp = b.second->GetGlobalRotation();

                    

                    D3DXQUATERNION gsrc;
                    gsrc.x = gsrcTemp.x;
                    gsrc.y = gsrcTemp.y;
                    gsrc.z = gsrcTemp.z;
                    gsrc.w = gsrcTemp.w;

                    

                    // RG-1 (2)
                    FRAME* f = (FRAME*)D3DXFrameFind(mesh.GetFrameRoot(), b.first.Name);

                    D3DXQUATERNION gdst;
                    D3DXQuaternionRotationMatrix(&gdst, &f->matCombined);

                    //Quaternion gdst(gdstTemp);

                    // q (Change from OpenGL->DirectX) (3)
                    Quaternion protSrcTemp = b.second->GetRestRotation();
                    D3DXQUATERNION protSrc;
                    protSrc.x = protSrcTemp.x;
                    protSrc.y = protSrcTemp.y;
                    protSrc.z = protSrcTemp.z;
                    protSrc.w = protSrcTemp.w;

                    // qInit (1)
                    D3DXQUATERNION protDst;
                    D3DXQuaternionRotationMatrix(&protDst, &f->TransformationMatrix);
 
                    D3DXQuaternionInverse(&gdst, &gdst);
                    D3DXQuaternionInverse(&protSrc, &protSrc);

                    stringstream oss;
                    oss << "gsrc X:" << gsrc.x << " Y:" << gsrc.y << " Z:" << gsrc.z << " W:" << gsrc.w << endl;
                    oss << "protSrc X:" << protSrc.x << " Y:" << protSrc.y << " Z:" << protSrc.z << " W:" << protSrc.w << endl;
                    oss << "gdst X:" << gdst.x << " Y:" << gdst.y << " Z:" << gdst.z << " W:" << gdst.w << endl;
                    oss << "protDst X:" << protDst.x << " Y:" << protDst.y << " Z:" << protDst.z << " W:" << protDst.w << endl;
                    oss << "================" << endl;
                    OutputDebugString(oss.str().c_str());

                    D3DXQUATERNION resultQ = protDst * gdst * gsrc * protSrc;

[/code]

 

[code]

gsrc X:0.0853042 Y:0.0379734 Z:0.693659 W:0.714226
protSrc X:-0 Y:-0 Z:-0 W:1
gdst X:-0.707107 Y:-0.707107 Z:-2.87229e-010 W:-1.41422e-006
protDst X:0 Y:0.000399 Z:0 W:1
================
gsrc X:0.0843397 Y:0.0431162 Z:0.690231 W:0.717362
protSrc X:-0 Y:-0 Z:-0 W:1
gdst X:-0.0379638 Y:-0.0853138 Z:0.693678 W:0.714207
protDst X:0.0145295 Y:0.99549 Z:0.0334747 W:-0.0875667
================
gsrc X:0.0846339 Y:0.0438533 Z:0.689497 W:0.717986
protSrc X:-0 Y:-0 Z:-0 W:1
gdst X:-0.0805823 Y:-0.0273762 Z:-0.746041 W:0.660439
protDst X:0.994516 Y:-0.0605601 Z:-0.0380181 W:-0.0763131
================
gsrc X:0.0857059 Y:0.0440209 Z:0.689656 W:0.717693
protSrc X:-0 Y:-0 Z:-0 W:1
gdst X:-0.0775542 Y:-0.0106626 Z:-0.79746 W:0.598275
protDst X:0.986911 Y:-0.140869 Z:-0.0476919 W:-0.0623213
================
gsrc X:0.0858005 Y:0.0440563 Z:0.689669 W:0.717667
protSrc X:-0 Y:-0 Z:-0 W:1
gdst X:-0.0693706 Y:0.0231888 Z:-0.883593 W:0.462517
protDst X:0.95181 Y:-0.297756 Z:-0.0658274 W:-0.0325358
================
gsrc X:0.0858582 Y:0.044078 Z:0.689677 W:0.717651
protSrc X:-0 Y:-0 Z:-0 W:1
gdst X:-0.0456583 Y:0.0878104 Z:-0.982534 W:0.157603
protDst X:0.806155 Y:-0.583298 Z:-0.0946962 W:0.0300388
================
gsrc X:0.0859156 Y:0.0440995 Z:0.689685 W:0.717636
protSrc X:-0 Y:-0 Z:-0 W:1
gdst X:0.0134882 Y:0.182656 Z:-0.864295 W:-0.468462
protDst X:-0.279846 Y:0.942339 Z:0.119729 W:-0.139069
================
gsrc X:0.0859539 Y:0.0441139 Z:0.689691 W:0.717625
protSrc X:-0 Y:-0 Z:-0 W:1
gdst X:-0.0922816 Y:-0.135127 Z:-0.256295 W:0.952652
protDst X:0.85486 Y:0.492331 Z:0.0299552 W:-0.160996
================

[/code]

I am updating all local frames at the end (After all children's local frames are calculated),

and because of the update, the local matrix and the global matrix are changing,

and also the bvh data is changing, making the bone very unstable. Why?

I am keeping the scale and translational data from the original hierarchy...

 

Thanks

Jack

Forum Timezone: America/Los_Angeles

Most Users Ever Online: 733

Currently Online: huberdavidsen3, casey29english, kinneyreynolds06, andersen19eaton, kristensen00leblanc, bredahlbredahl9, bager59mayo, creechdonaldson71, eric506784, t9rrzpi307, g2pvwfg490, ray0963
100 Guest(s)

Currently Browsing this Page:
1 Guest(s)

Top Posters:

jwwalker: 80

jyambao: 51

rbaral: 47

adiaz: 30

WargnierP: 29

lucky7456969: 28

mbarros: 28

avida.matt: 26

JonathanW: 24

laguerre: 23

Member Stats:

Guest Posters: 65

Members: 52505

Moderators: 3

Admins: 4

Forum Stats:

Groups: 1

Forums: 5

Topics: 427

Posts: 2343