m y a n i m a t o r . c o m

So you want to do Hands in Motion Capture.

Here is what advice I can give you....

1.) * Don't do hands * - At least the way I did with overlapping figures and so many markers in such a close area...my session of motion capturing hands ended up being a very long process of more hours then I care to remember and still resulted in no really efficient data to use for my work.

2.) Okay you skipped my advice so here is what I learned...

All of the following information is an extension of the documentation already available on the ACCAD Motion Capture Webpage, and the Motion Capture Production Class Webpage.

3.) Planning your motion capture session with Hands

a.) Storyboard - Plan large that way you can scale it down when you begin your capture session. Below are the Storyboards I used for my capture session. As you will see only a section of these boards were actually captured during this session.

b.) Planning Sheet - Create a planning sheet that breaks down everything so that when you shoot you are ready to go. The planning sheet I created is located here.

c.) Marker File - Before you begin shooting you need to create a marker file for your hands so that Vicon system has a marker set. I created a pretty standard marker file, aside from the Pearl, which is available as HumanHandm.mkr. To see where the markers are placed take a look at this image.

4.) Do your capture session... referring to Recipe for Motion Capture for information.

5.) Vicon's Workstation - Following the capture session the next step is Workstation. Follow the instructions on auto labeling. I ended up with something like this following auto labeling.

Data after initial Autolabeling (Workstation)

6.) Cleaning in Workstation - This is the most time consuming process but once you are done hopefully you data will look a bit better like this...

Data after cleaning (Workstation)

7.) House of Move's Diva

	a.) Scripts - Fortunately thanks to Suba most of the scripts that you will use are already written and just require minor changes. However one script that I created was a Post-Workstation Script that found gaps in your marker and looked at the corresponding overlapping markers (up to 25) and swapped the data into any gaps. This became very useful as it helped Diva fill the gaps later. Post-Workstation Script - Currently proprietary to this marker set until Diva 2.0 comes out and I can add some customizing commands. You will also need the multiple_findgaps script as well. GenericHandCleaning Scripts - a compressed ZIP file of the various scripts that clean the data. (This will only work with the hand markers)
	b.) Creating the Skeleton - After cleaning your data and moving the markers you will naturally you will need to create a non-human skeleton. Below is an example of the layout that you should create for the hand skeleton.
	c.) Arranging the Skeleton - Use the Axis Offset to position the bones so that they resemble the hand skeleton structure so that it looks like this. Your skeleton will not match up in size or position to your marker data right away
	d.) Positioning the Skeleton - Using your TY you can change the length of the bones. Remember to select the bone above the one that you want to change the length. Use the rotation to position the bones accordingly to your Marker Data.
	e.) Making the Skeleton Work - Create your constraints, solvers, and weights for your skeleton and it should move correctly. Make sure that you solve for the root or else your hand won't be able to move. Skeleton Moving in Diva

8.) Alias's Maya

	a.) Getting the data to Maya - Depending on your data you may have to export different formats from Diva and bring them into Maya using the Dominatrix plug-in available on the Maya 4.0 CD. This is what I have found out importing hands data from the different formats. HDL (Standard Diva Format) - Transfers a wealth of information from Diva to Maya including Skeleton, Markers, and Character Nodes. Disadvantages are that every frame is keyed which makes it difficult to view your data curves in Maya. Also the skeleton solving is different then the solver in Diva. I've found for hands since there are many intricate movements that cause a finger to bend that this was very troublesome and did not work well for hand data. An example of this is this movie of the Marker Data (comprised of red spheres) next to the skeleton structure in Maya. - Marker Data to Skeleton Comparison AMC/ASF (Acclaim File Format) - The ASF file holds the skeleton data while the AMC file holds the motion information. You can choose your own frame rate when exporting your data which made the curves much easier to see in Maya. This file format worked extremely well in transitioning the solver in Diva accurately to Maya. However the AMC/ASF will only transfer over data that has a Skeleton attached to it. In my case the Pearl had no skeleton and did not transfer over. CSM (Character Studio) - Imports the marker data but nothing of the skeleton. This format allows you to downsample the keyed frame rate as well. BVH (BioVision Hierachy) - like the Acclaim file format it only imports the skeleton if it has data keyed to it. Again my Pearl was not imported in this format. This format allows you to downsample the keyed frame rate as well. C3D (Binary Motion File) - same as skipping Diva and coming in from Workstation. The marker data is imported but no skeleton. This format allows you to downsample the keyed frame rate as well.
	b.) Test Test and Test - Once you get your data into Maya quickly do a round binding of the skeleton to the geometry. You will be able to see quickly if it is going to work or not. This for me showed me obviously that some of the Degrees of Freedom I had checked in the Diva's Attributes were incorrect. Your geometry will form oddly due to the binding not adjusted but you will get the general idea. Ouch!
	c.) Adjusting the skeleton to the geometry (or vise-versa) - You should scale your hand geometry first to get it as close to the skeleton as possible. Next go to frame 0 and rotate all your joints into position. Key all the joints in only Rotation (x,y,z). You may also translate and scale the joints to nudge them into place since translation and scale is not keyed it will remain in that position according to the other markers. Since frame 0 is keyed you may always return to frame 0 as your basic pose. This is where you should bind your hand.
	d.) Fixing Jumps - Even with all that painstaking process in Workstation and Diva you will most likely encounter jumps in your markers/skeleton that need to be fixed. The best way to do this is to step through each joint and scrub through the time line while looking at the Graph Editor and the Perspective Window. When you see a joint jump away in the Perspective window it will match up with a jump in the Graph Data. Move the problematic frame(s) so that it matches its corresponding data. Do this through the whole session and it will save you further headaches.
	e.) Binding - This will probably be your most time consuming process in Maya. Fortunately if you have followed my advice and have completed step d above some of your headaches are already gone. You have a choice of doing a Smooth Bind or a Rigid Bind in Maya. While a Smooth bind allows more interactive control of the weights a rigid bind allows you to use more precise weighting. Video of Painting Weights (Smooth Bind)

9.) Why Mine didn't work and ways you can avoid it

	a.) The Models - I created my hand model in Sub Division surfaces which provided an excellent way to model a hand. Since I motion captured my own hands it made logical sense that the model should be of my hand. I placed my hand on the scanner and scanned my hand and created an imageplane on the top orthographic view in Maya. Sub Division Surfaces Hand Model While this model looked very nice and worked well when I attempted to bring it into Maya on the ACCAD systems they would not read the Sub Division Surfaces. I can only assume that this is due to our licensing and configuration. I was forced to convert the SubDs to Polygons which would only cause problems later when it came to painting weights. SubD to Polygon Hand Model
	b.) Binding - As stated above since I was working with a polygonal model this became a very tedious process that couldn't advance easily. Since a polygonal model is determined by the number of vertices, areas with few vertices became very difficult to weight. Here is a movie showing these problems after countless hours of weighting. The movie is simply titled Binding Hell .
	c.) Texturing - The last area I wanted to discuss in this information that also requires some time is the actual texturing of the hand(s). I worked with creating a skin like shader for all the faces of the hand. The finger nail was a Phong E shader with a high specular level. The Pearl was a composed of a Cloud shader for opacity and color. Changes in this area would be changing the color of the finger nails so they are less grey and more fleshy in their transparency. The Pearl also needs a glossy look to it which I could see as a highly transparent sphere outside the pearl geometry that consists of a Phong E shader with high specularity, high roughness, and lower size. Binding Hell Hand with Textures Applied
	d.) Conclusion - Overall I learned a lot in this experience and I can only hope that the information I provided here can be of use to people in the future who want to work with Motion Capture and hands. If Diva allowed you to limit the the degrees of freedom in angles not just yes or no that could also help out tremendously. If you'd like to see what 8 weeks in working in this process has brought me you can take a look at the movies below. Marker Data in Maya Both Hands (Right - bound : Left - Markers)