Skip to main content

Face Morph

3D Face provides a 3D mesh that fits on the user’s face in real-time. It can be used to create different interactive face effects.

face morph demo

Prepare​

Download Assets​

Start by downloading the following package. You will see one texture folder and three FBX files.

↓ face_morph_assets.zip

Import a Designed 3D Face Morph Model​

You can import a 3D face morph model by clicking the Add button [+] > Import > 3D Face Binding Mesh in the Assets panel. Then choose the faceMorph_facMask.fbx file. You will see the faceMorph_facMask imported in the Asset panel.

import a 3d face binding mesh in the assets panel
note

To use a mesh as 3D face mesh, you have to import it by clicking Import > 3D Face Binding Mesh, otherwise it will be treated as a normal mesh.

Import Other 3D Models and 2D Textures​

Next you can import other assets by clicking the Add button [+] > Import > From Computer or by directly dragging the assets to the Assets panel (you have to drag individual files instead of the whole folder).

import other assets from computer in the assets panel

You will see all the needed assets have been imported into the Assets panel.

imported assets in the assets panel

3D Face​

Add 3D Face and Face Attachments to the Scene​

Drag the faceMorph_faceMask object from Assets panel to the Hierarchy panel. In the Inspector panel, you will find its 3D Face Binding component and Deformation component.

3d face binding and deformation components of the face mask object in the inspector panel

Next you can also drag the the faceMorph_attachments object from Assets panel to the Hierarchy panel, and then put it under the previous faceMorph_faceMask object.

facemorph object in the hierarchy panel

After nesting faceMorph_customOccluder under faceMorph_faceMask, add an Occluder to the Asset panel by clicking the Add button [+] > Material > Occluder. Select the faceMorph_customOccluder object, find the Mesh Renderer in the Inspector panel, and change the Material to Occluder.

select facemask_customoccluder in the assets panel
add an occluder material in the assets panel

Then you will see the preview window like this.

preview of 3d face with attachements

Adjust and Assign Materials​

You can then create a built in PBR material by clicking the Add button [+] > Material > Standard PBR in the Assets panel. Find the material in the Assets panel and rename it to face_morph_mat.

In the Inspector panel, go to Albedo, select the Texture On checkbox, and change the Texture to faceMorph_baseColor. In the same way, you can change the Normal texture to faceMorph_normal and the MRAO texture to faceMorph_MRAO. Then in Render State 0, deselect the Depth Test checkbox, and change the Blend Mode from Normal to Transparent.

face_morph_mat component properties in the inspector panel  face_morph_mat component properties in the inspector panel
tip

Tto show the face_morph_mat on top of others, you need to disable Depth Test.

Next you can select the faceMorph_faceMask object, find its 3D Face Binding component and change the material to face_morph_mat.

mesh renderer component of the facemorph_facemask object

You can set up the horn material the same way. The only difference is that you don’t need to assign the MRAO and normal texture. You will also notice that after you drag the attachments assets to the Hierarchy panel, it automatically generates a lambert material and assigns it to the attachments objects.

You can directly rename this lambert material to horn_mat and change its Albedo texture to horns_eye_baseColor and set its MRAORoughness to 0.3. After this step, the preview window will look like this.

preview with materials

Add Another Base 3D Face​

You also need to add another 3D face which has a texture. To do this, first add a built-in face mesh by clicking the Add button [+] > Mesh > Face. Then select the face mesh and in the Inspector panel, select Create face texture. It will create a Face Texture under the Textures folder.

add a face mesh  create face texture in the inspector panel

Then you can drag the faceMorph_faceMask object from Assets panel to the Hierarchy panel, just like you did in the previous step. However, this time you need to put the faceMorph_faceMask above the previous one, and you can rename it to faceMorph_faceOriginal.

facemask_faceoriginal in the hierarchy panel

It will automatically create a Face Paint material, you can change the name to face_mat and change its Texture from DefaultTex to Face Texture, which you just created. Now you will see the following result:

assign the face texture  preview with the face texture applied

Visual Scripting​

Now, you can set up the interaction by using Visual Scripting nodes. Open the Visual Scripting panel, add the Facial Movement Detection node, and change the Facial Movement to Mouth Open.

set facial movement of the facial movement detection node

In the same way, add the Timer node, connect it to the Begin output of the Facial Movement Detection node and set the Duration to 1.

connect the timer node to the output of the facial movement detection node

Select the SkeletonAndRenderRoot object under the faceMorph_attachments object and in the Inspector panel, select Set Playback from the Animation component.

set the animation property to set playback in the inspector panel

Select the faceMorph_faceMask object and in the Inspector panel, select Set alien_faceMorph_blendshape Target from the Deformation component. In the same way, select the same blend shape target from the faceMorph_faceOriginal object.

select Set alien_faceMorph_blendshape Target of the deformation component
connect the blend shape target nodes

Now connect all the nodes together like this. Now every time you open your mouth, you will see the face morph and animation play.

connect all nodes in the visual scripting panel