SmartBody : Running SmartBody on the web with Javascript

Browser Application 

This section explains how to use the complied Smartbody.js file to create an online application. 

• Construct the rendering html
  For an graphic application, first we need to create a rendering page. Emscripten provide a template html pageFeel(see src/shell.html) which create a basic html rendering page for you. If you look into the shell.html you will find there is Module object which is a global JavaScript object with attributes that Emscripten-generated code calls at various points in its execution. You can also provide an implementation of Module to control the execution of code. For example, to define how notification messages from Emscripten are displayed, developers implement the Module.print attribute. When generating only JavaScript (as opposed to HTML), no Module object is created by default, and the behaviour is entirely defined by the developer. When generating HTML, Emscripten creates a Module object with default methods, for more information, please refer to Module object. For our application, we currently use the shell.html, feel free to design your own web page layout.

  
  

• Data preparation and packaging

  For each application, there will be a certain amount of data such as character meshes, skeleton, behaviors set, motions and etc.

  Native code and “normal” JavaScript use quite different file-access paradigms. Portable native code usually calls synchronous file APIs in libc and libcxx, while JavaScript allows only asynchronous file access (except in web workers). In addition, JavaScript does not have direct access to the host file system when run inside the sandbox environment provided by a web browser. Emscripten provides a virtual file system that simulates the local file system, so that native code using synchronous file APIs can be compiled and run with little or no change.
  Emcc uses the file packager to package the files and generate the File System API calls that create and load the file system at run time. While Emcc is the recommended tool for packaging, there are cases where it can make sense to run the file packager manually. 
  We use the file packager to package files we need. The file packager generates a .data file and .js file. The .js file contains the code to use the data file, and must be loaded before loading your main compiled code.

  Suppose we are creating a HeadDemo which contains a character resources(face, mesh, skeleton and motions). The following command package a folder located at /data/ChrMaarten recursively to /ChrMaarten in the virtual file system and /data/ChrMaarten-Default-Motions recursively to /ChrMaarten/Motion, the '@' symbol is used in a path at build time to explicitly specify where the resource will be located in the virtual file system at runtime. For more inomation, please refer to Packaging Files

   

  python ../file_packager.py ./data-chrmaarten.data --preload ./data/ChrMaarten@/ChrMaarten --js-output=./data-chrmaarten.js
  python ../file_packager.py ./data-chrmaarten-default-motions.data --preload ./data/ChrMaarten-Default-Motions@/ChrMaarten/motions --js-output=./data-chrmaarten-default-motions.js

  Icon

  When packaging the data, you should always only package data that is actually needed to reduce the total data file size, which could shorten the downloading process. Also, try to abstract out some common part like a minimal set of behavior that could be reused by many characters in order to prevent duplicate data in the final data package.

  
  

• Converting Python script into Javascript

  As stated before, for SmartBody/Javascript, we use Embind instead of Boost:: python to bind C/C++ functions and classes to Javascript, so that the complied code could be used in Javascript. Thus, if you have a desktop application using smartbody, you could convert it into Javascript with some extra effort. Currently, we have converted partial of python scripts into Javascript. You could find the already converted scripts in the Smartbody SVN here.
  In case you need to convert some python code, there are a few things you need to keep in mind:
  1. Your application script should be embedded in a function called allReady()
  This function is called when all the data and scripts are loaded, the scene object is created successfully.

  function allReady(){
  	if(scene != null){
  		//Your application code goes here
  		var camera = Module.getCamera();
  		scene.getPawn('camera').setPosition(new Module.SrVec(0, -5, 0));
  	}else{
  		console.log("SBScene does not exist");
  	}
  }

  
  2. Always access objects through the Module object object, as shown above.   

  All symbols exposed by embind are available on the Emscripten Module object. In SBJavascript.cpp, we setup some global variables like scene, bml, so that you could use access them without Module dot operation in Javascript. Also notice to declare a new variable of the wrapper type you need to use new keyword. 

  //for those global variables do not use 'var' in Javascript
  EM_ASM(
  	scene = Module.getScene();
  	bml = scene.getBmlProcessor();
  	sim = scene.getSimulationManager();
  	assets = scene.getAssetManager();
  		
  	//wrapper for types, so that we could use SrVec in Javascript instead of Module.SrVec
  	SrVec = Module.SrVec;
  	StringVec = Module.StringVec;
  	VecMap = Module.VecMap;
  );
  /*
  PyRun_SimpleString("scene = getScene()");
  PyRun_SimpleString("bml = scene.getBmlProcessor()");
  PyRun_SimpleString("sim = scene.getSimulationManager()");
  PyRun_SimpleString("assets = scene.getAssetManager()");
  */

  3. Namespace

  Unlike Python script, if two files contains the same function name, for example, in BehaviorSetGestures.py  and BehaviorSetFemaleGestures.py, they both have a function called setupBehaviorSet. Javascript may not be able to differentiate them properly(that's is just my guess, not an expert of Javascript ), so if there are python scripts with the same function name, we would use namespace to tell them apart. Actually, we almost give every script a namespace, the namespace usually is just the file name. Also notice the function definition is different from python.

  //for BehaviorSetGestures.js
  var BehaviorSetGestures = {
  	setupBehaviorSet : function (){
  		//implementation
  	}
  }
  //for BehaviorSetFemaleGestures.js
  var BehaviorSetFemaleGestures = {
  	setupBehaviorSet: function (){
  		//implementation
  	}
  }
  //Then when you need to call them in your script 
  BehaviorSetGestures.setupBehaviorSet()
  BehaviorSetFemaleGestures.setupBehaviorSet()

  4. Vector operations

  There are minor differences between python and javascript vector access as listed below

  	Python	Javascript
  Insert	motions.append	motions.push_back
  Length/Size	len(motions)	motions.size()
  Access element	motions[i]	motions.get(i)
  Declaration		new Module.SrVec()

  5. No scene.run("foo.py")

  For Javascript version of Smartbody, the function scene.run() could only execute Javascript code pieces, like scene.run("var i = 1, j = 2; alert('result = ' + (i + j) ););

  6. Deriving from C++ classes in JavaScript

  Embind has its own rules to override functions and functions. Compare the scripts on left and right for override SBScript class, for information about Embind function/class override, please refer here

Put all of them together
After you creating your rendering page, packaging data needed and converted all necessary python script into Javascript, it is time to put them together and create your Web-based Smartbody application. The pictures below shows a web structure we are using for the demo page. 

One thing need to point out is if your data is different from the default expectation, for example we hold them in the data directory different from what the application webpage location, then we need to use Module.locateFile to tell the application to find the files at the specified location.

<script type='text/javascript'>
var Module = {
...
//file relocation
Module.locateFile = function (name) { 
	var extension = name.substr(name.lastIndexOf('.')+1);
	if(extension == 'data') 
		return '../data/' + name;
	else if(extension == 'mem')
		return '../' + name;
	else return name; 
},
...
}
</script>

Basically, your application will include a rendering page, data package and javascript file. All you need to do is adding the data files and other script files loading in the rendering page like below:

<!--Data files must be loaded first -->
<!-- Data files -->
<script async type="text/javascript" src="../data/data-chrbrad.js"></script>
<script async type="text/javascript" src="../data/data-chrrachel.js"></script>
<script async type="text/javascript" src="../data/data-female-gestures.js"></script>
<script async type="text/javascript" src="../data/data-gestures2.js"></script>
<!-- SmartBody Library -->
<script async type="text/javascript" src="../SmartBodyJS.js"></script>
<!-- Scripts -->
<script async type="text/javascript" src="../scripts/examples/GazeDemo.js"></script>
<script async type="text/javascript" src="../scripts/scripts/defaultViewer.js"></script>
<script async type="text/javascript" src="../scripts/scripts/zebra2-map.js"></script>
<script async type="text/javascript" src="../scripts/behaviorsets/BehaviorSetCommon.js"></script>
<script async type="text/javascript" src="../scripts/behaviorsets/BehaviorSetGestures.js"></script>
<script async type="text/javascript" src="../scripts/behaviorsets/BehaviorSetFemaleGestures.js"></script>
<!-- FPS counter -->
<script async type="text/javascript" src="../libs/fps/FPSCounter.js"></script>

All set, you could start your browser and run your demo.