3D face provides a 3D mesh that fits on the user’s face in real-time. It can be used to create different interactive face effects.
Before we get started, download the FaceMorphAsset package. You will see one texture folder and three FBX files.
Import designed 3D face morph model
You can import a 3D face morph model by going to + Add asset > Import > 3D Face Binding Mesh in the Assets panel, then choose the faceMorph_facMask.fbx file. Then you will see the faceMorph_facMask is imported in the Asset panel.
To use a mesh as 3D face mesh, you have to import it through the Import > 3D Face Binding Mesh, otherwise it will be treated as a normal mesh.
Import other 3D models and 2D textures
Next you can import other assets by going to + Add asset > Import > From Computer or by directly dragging the assets to the Assets panel (you have to drag individual files instead of the whole folder).
You will see all the needed assets have been imported into the Assets panel.
Adding 3D Face and face attachments to the scene
Drag the faceMorph_faceMask object from Assets panel to the Hierarchy panel. In the Inspector panel you will find its 3D Face Binding component and Deformation component.
Next you can also drag the the faceMorph_attachments object from Assets panel to the Hierarchy panel, and then put it under the previous faceMorph_faceMask object.
After nesting faceMorph_faceMask under faceMorph_customOccluder, we add an Occluder to the Asset panel by clicking + Add asset > Material > Occluder. Select the faceMorph_customOccluder object, find the Mesh Renderer in the Inspector panel, and change the Material to Occluder.
Then you will see the preview window like this.
Adjusting and assigning materials
You can then create a built in PBR material by clicking + Add asset > Material > Standard PBR. Find the material in the Assets panel and rename it to face_morph_mat.
In the Inspector panel, you need to check the Albedo Texture on and change the Texture to faceMorph_baseColor. In the same way, you can change the Normal texture to faceMorph_normal and the MRAO texture to faceMorph_MRAO. Then in Render State0 turn off the Depth Test and change the Blend Mode from normal to Transparent.
In order to show the face_morph_mat on top of others, you need to turn off Depth Test.
Next you can select the faceMorph_faceMask object, find its 3D Face Binding component and change the material to face_morph_mat.
You can set up the horn material the same way. The only difference is that you don’t need to assign the MRAO and normal texture. You will also notice that after you drag the attachments assets to the Hierarchy panel, it automatically generates a lambert material and assigns it to the attachments objects.
You can directly rename this lambert material to horn_mat and change its Albedo texture to horns_eye_baseColor and set its MRAORoughness to 0.3. After this step, the preview window will look like this.
Adding another base 3D face
You also need to add another 3d face which has a texture. To do this, first add a built-in Face Mesh through + Add asset > Mesh > Face, then select the face mesh, in the Inspector panel select Create face texture. It will create a Face Texture under the Textures folder.
Then you can drag the faceMorph_faceMask object from Assets panel to the Hierarchy panel, just like the previous step. But this time you need to put the faceMorph_faceMask above the previous one, and you can rename it to faceMorph_faceOriginal.
It will automatically create a Face Paint material, you can change the name to face_mat and change its Texture from DefaultTex to Face Texture you just created. Now you will see the result this:
Visual Scripting node
Now, you can set up the interaction by using visual scripting nodes. Open the Visual Scripting panel, add the Facial Movement Detection node, and change the Facial Movement to Mouth Open.
In the same way, add the Timer node, connect it to the Begin output of Facial Movement Detection node and set the Duration to 1.
Select the SkeletonAndRenderRoot object under the faceMorph_attachments object and in the Inspector panel, select SetPlayback from Animation component.
Select the faceMorph_faceMask object, in the Inspector panel, select Set alien_faceMorph_blendshape Target from Deformation component. In the same way, select the same blend shape target from faceMorph_faceOriginal object.
Now connect all the nodes together like this, and now every time you open your mouth, you will see the face morph and animation played.