Swing physics for Spark AR Studio: a tutorial

By Francesco Strada

Blog - March 26, 2020

For the purposes of this tutorial we will use a filter made for The Addams Family on Instagram as a reference, so that you will be able to see what can be accomplished with this guide.

The objective of this article is to show Spark AR users how to create a pseudo physics system that can be used to realistically animate 3D objects such as hair and earrings. With this step-by-step tutorial you will learn how to make these objects swing when the users move their head.

Unlike game-engines such as Unity and Unreal Engine 4, Spark AR Studio doesn’t come with a native physics system, but we can work around that by either using external Javascript libraries or by faking one using the position and rotation data from the FaceTracker. The former can be tested by importing the cannon.js file from the library, but we will explore the second option.

Step 0 – asset preparation

The first thing you want to do is to find or create a 3D model in your preferred 3D modelling software. For the purposes of this tutorial, we will be using the free tool Blender. 

While working with 3D for Spark AR, there are a few things you should keep in mind:

• Make sure you are using the correct units (centimetres) and scale. You can download these assets here and use them as a reference in Blender.
  
  


• Optimise your model as much as you can; the total number of triangles in your scene should be lower than 50,000.
  
  

Step 1 – rigging

Once we have a 3D model ready, we can create an armature for the parts that we want to swing:

1. In Blender, use the shortcut Shift-A and choose ‘Armature’. An armature object will appear.
   
   
   
   
2. Select it and press Tab on your keyboard to enter ‘Edit mode’.
   
   
   
   
3. Select the first bone and scale it to a reasonable size using the key S.
4. Toggle ‘X-ray view’ to see the model in transparency with the shortcut Alt-Z.
5. Using the G key, move the selected bone to the root of the model part where we want the swing motion to start (tip: press G and then X, Y or Z to move an object only on one axis).
6. Select the top sphere of the bone and move it lower than the lower sphere using G and then Z on your keyboard.
   
   
   
   
7. Extrude the bone pressing the E key to create a second bone and position it lower than the first one.
8. Let’s create as many bones as we like. The higher the number the more realistic our animation will look, but it will also have a bigger impact performance-wise. I will create a total of 8 bones going from the root to the lowest part of the braid.
9. Now we have to adjust the position of each bone, switching to camera views using the right Numpad (1: front-view; 3: side-view).
   
   
   
   
10. Once you are satisfied with the position of the bones, enter ‘Object mode’ with the Tab key or from the drop-down menu in the top left corner.
11. We are now ready to parent our model to the Armature. To do so, select the object first and the armature second, press Ctrl-P and select ‘With Automatic Weights’.
    
    
    
    
12. In most cases this is enough, but sometimes you might need to adjust the Weights. To check how your model will behave select the armature and select ‘Pose Mode’ from the top-left drop-down menu. You can make a few tests selecting the single bones and moving or rotating them, then undo these transformations. If you need to change something, select your object and enter ‘Weight Paint’, select the vertex group you want to modify, adjust the weight and paint with your mouse.
    
    
    
    

If you are satisfied with the armature and the way your model is controlled by it, you can repeat the same passages for as many parts of your model as you want to animate in this way.

Step 2 – export and import

1. Select the objects and the armature (press A to select everything) and export (File-> Export-> .fbx), check ‘Selected Objects’, choose position, name and hit ‘Export fbx’.
2. Open Spark AR studio and select the Face decoration template.
3. Delete the ‘deleteMe’ object from the Scene.
4. Drag the fbx file you created into the Spark AR Asset tab, then drag it from this tab to the Scene tab under the faceTracker0 object.
5. Select the model, move and scale it as needed with the transformation tools.
6. We can change the material applied to the objects.
   
   
   
   

Step 3 – swing logic

In this final step, we will use Patches to apply the swing motion to the armature of the hair.

1. Select the first bone you would like to apply the motion to, press the arrow beside ‘Rotation’ to bring it to the Patches editor.
   
   
   
   
2. We will add the current rotation of the bone to the one that will create the motion. To do so, right-click in the Editor and type ‘pack’, select ‘Pack’ and add to the three inputs an Add (right-click and type ‘add’).
3. Select the three Add and the Pack, and create a group (right-click and select Group or press Ctrl-G). 
4. Double click on the new group to edit it and create an input for each Add by left-clicking on the arrows and selecting ‘Create input to group’.
5. Create Output for group with a left-click on the arrow to the right of Pack.
6. Exit the group by clicking ‘Back to Main’, copy the three values you find in the bone rotation output yellow rectangle and paste them into the group inputs.
   
   
   
   
7. Now we want to create a new input for the group so that we can add the rotation of the user’s head to the original rotation. To do so, go back inside the group and create inputs for the rotation axis you want to influence. In my case, it’s the Z-axis.
8. We are now going to take the faceTracker rotation output and feed it to the group. Exit the group and drag the ‘faceTracker0’ object from the Scene tab to the Patches editor if you don’t already have it there.
9. Unpack the rotation by right-clicking and typing ‘unpack’, select it and connect the 3D rotation output to it. Drag the Z value to the last input of the group. The Z value is how much the users tilt their head; you will see that if you tilt your head the chosen bone, and consequently the model, will move with the tilt. This value alone is not enough to create a realistic swing.
   
   
   
   
10. Enter the group and detach the input from the Add; connect it to an Exponential Smoothing module (right-click, type ‘exponential’ and select it), then to a Multiply. We can use this Multiply to adjust the values; for now, I will type just ‘1’. You can already see the smoothing effect on the bone rotation; change the Exponential Smoothing value from 200 to 100.
    
    
    
    
11. Connect the Z input to a Negate and Add it to the first Exp. Smoothing module.
12. Create a new Exp. Smoothing from the external input Z with 200 as its value.
    
    
    
    
13. Make a new Negate from the input and connect it to a new Exp. Smoothing, where we will use 400 as value; multiply this by 1.2 (a value you can tweak later) and use another Exp. Smoothing with 400. 
14. Finally, add this last piece to the Exp. Smoothing with a 200 value and add this to the initial piece.
    
    
    
    
15. You can now see the swing motion in action on this first bone. We have to repeat this process for each bone: exit the group, select a lower bone, bring the rotation in the editor, copy the group, copy the first three values and paste them into the group inputs and connect the group to the rotation.
    
    
    
    

Now you have a swinging model that reacts to the head’s tilt by swinging upward at first and then going back down as real hair would do.

You can implement this same motion for back and forward tilt and even for the user’s movement from left to right. You will just need to multiply the input by a different value (200 for position).

This tutorial is complete and we have created a swinging 3D model in Spark AR using only Patches.

You can share questions and suggestions with Poplar Studio @wearepoplar and with me @augmentedidea on Instagram.

Thank you for following along!