3D Face provides a 3D mesh that fits on the user’s face in real-time. It can be used to create different interactive face effects.
This example uses 3D Face to apply patterns on users’ faces by customizing textures and tweaking properties of the material of face shape
Adding a 3D Face
You can add a 3D face shape by going to + Add object > Face Effects > 3D Face or right-clicking in the Hierarchy panel, then clicking Face Effects > 3D Face.
A default 3D face shape will be generated to the face in the Preview panel.
Applying face paint material
You may notice that when we create a 3D Face, Face Paint is also generated in the Material section of the Assets panel and assigned on the 3D face automatically.
Face Paint is a built-in material that comes with the 3D face mesh. It is an unlit material that allows you to quickly swap patterns that appear on the face.
For more detailed properties about unlit materials, please refer to the Unlit material guide.
Then, you can import a customized image as texture and assign it to the material.
For texture that you want to keep the alpha channel, go to the Inspector panel, click Advance and check Premult.
Customizing face texture
If you want to customize textures for the 3D Face, download the FaceAssets package and use face_uv.png as reference of the key points and positions on the face.
Applying customized 3D face mesh
Adding a 3D Face
Add a 3D face by going to + Add object > Face Effects > 3D Face. A default 3D face will be generated on the face in the Preview panel.
Download the customized 3D face mesh:
You can then import the 3D face morph model by going to + Add asset > Import > 3D Face Binding Mesh in the Assets panel.
Choose 3D Face in the Hierarchy, then replace the Mesh property with your imported customized 3D face mesh.
You will see the Deformation component has automatically been assigned to 3D Face. That is because the mesh has blendshape build-in.
Extracting the face texture
If you want to extract real-time face texture and apply it to your 3D face mesh, select the default Face mesh in the Assets panel, then go to the Inspector panel to find Face Info. Click Create face texture to extract the face texture that you can find in the Assets panel.
Apply the face texture to the material of 3D Face.
Customizing 3D face mesh
The adjustedGeo_quads_01.fbx is the face shape mesh used in our facial tracking. Feel free to modify the model with blendshape.
You can modify the fbx using 3D modeling softwares such as Maya, you can change change its UV but do keep its vertex ID the same.
3D face related properties
3D Face Binding
Find the 3D Face Binding component in the Inspector panel when you navigate to 3D Face in the Hierarchy panel.
Which Face: The index of the face’s transform that the 3D face mesh will follow
Mesh: Indicates the 3d face mesh that is assigned to the 3D face
Material: Allows you to choose the material for the 3D face mesh
Cast Shadow: Check if the 3D face mesh will project shadow
This component will show automatically if the mesh contains blendshape.
Calculate Normal: If checked, the position & normal will be calculated by the blendShapes. If unchecked, only the position will be calculated by the blendShapes.
Weights: Click the arrow to hide or reveal blendshape channels.
ChannelName: Use this slider to adjust the weight of the channel. You can adjust between 0 and 1.
Face mesh properties
Which Face: 3D face supports a maximum of 5 faces. For each face, you can specify which face from the input video to attach the 3D face. You can add multiple face entities to create multiple face effects.
Fill Eyes: Enable/disable mesh with opened eyes
Fill Mouth: Enable/disable mesh with opened mouth
Create face texture: When click it, it will generate a dynamic face texture based on the tracking face.