3D Face provides a 3D mesh mask on your face in real-time. It can be used for creating 3D face masks, 3D makeup, makeup with light reflection, face morphs, and face swap effects.
Add 3D Face object
Add a 3D Face object by clicking + Add object > Face Effects > 3D Face in the Hierarchy panel.
You cannot modify the Transform of the 3D Face because it is tracking your face.
When creating a 3D Face object, it has a Mesh Renderer component and a 3D Face Binding component added by default. In the following sections, you’ll learn how to configure these components to make some 3D face effects.
Demo Effect : 3D Face Mask
Editing: face material + textureIn this example, you’ll learn how to make a simple 3D face mask with a customized pattern. Download the demo package:
Customize the Material of 3D Face
After you add 3D Face, a Face Paint material is generated in the Material folder in the Assets panel. It is assigned to 3D Face automatically.
Face Paint is a built-in material that comes with 3D Face. You can use the default Face Paint or create a new Face Paint material by clicking + Add asset > Material > Face Paint. For more detailed properties about Face Paint, please refer to Face Paint.
You can use Face Paint to create a face pattern or create and assign other advanced materials to 3D Face. For example, by using Standard PBR material, you can create a metal face mask that reacts to environmental lights.
Customize the Texture of 3D Face
Create the Face Texture
To customize textures for 3D Face, you can use any 2d design software to create a face image. You can download the FaceAssets package and use face_uv.png as a reference for the key points and positions on the face.
You can now import a customized image as a texture by clicking + Add Asset > Import > From Computer. You can also simply drag the image into the Assets panel. Refer to Image for the asset requirements.
Assign the Texture to the Material of the Face
First, click Face Paint Material in the Assets panel. Then drag your face texture into the Texture section to assign it to the material.
If you want to keep the alpha channel of your texture, select the texture under the Assets panel, then go to the Inspector panel, click Advance Setting and check Premult.
Demo Effect : 3D Face Morph
Editing: face mesh + material + texture
Customize 3D face Mesh
Download FaceAssets.zip to customize the mesh of the 3D Face.
After downloading this file, go and find the .fbx named as adjustedGeo_quads_01.fbx. This is the face mesh we used in our face tracking. Feel free to modify the .fbx using 3D modeling software. You can change its UV but keep the vertex ID the same. Be sure you don’t remove any faces or vertices when adjusting the face mesh.
You can adjust the blendshapes of the model to create interesting face morph effects.
Import 3D Face Mesh
Here is a face mesh with adjusted blendshapes. Import this 3D face morph model by clicking + Add asset > Import > 3D Face Binding Mesh in the Assets panel.
Face Mesh Info
Let’s take a look at Face Mesh Info by selecting Face in the Assets panel.
Which Face: 3D face supports a maximum of 5 faces. For each face, you can specify which face from the input video to attach the face with 3D Face binding. You can add multiple face objects to create multiple face effects.
Fill Eyes: Check this property to fill the face meshes eyes.
Fill Mouth: Check this property to fill the face meshes mouth.
Create face texture: When clicked, it will generate a dynamic face texture based on the tracked face.
Apply Face Mesh to 3D Face
Choose 3D Face object in the Hierarchy panel and replace the Mesh property with your imported customized 3D face mesh.
Configure Deformation Component
You will see the Deformation component has automatically been assigned to 3D Face. That is because the mesh has blendshapes. A Deformation component will show automatically if the mesh contains blendshapes.
- Calculate Normal: If checked, the position and normals will be calculated by the blendshapes. If unchecked, only the position will be calculated by the blendshapes.
- ChannelName: Use this slider to adjust the weight of the blendshape. You can adjust it between 0 and 1.
Apply Real-time Face Texture to 3D Face
If you want to extract real-time face texture and apply it to your 3D face mesh, select the default Face mesh in the Assets panel, then go to the Inspector panel to find Face Info. Click Create face texture to extract the face texture that you can find in the Assets panel.
Apply the face texture to the material of 3D Face.
Demo Effect: Swap Faces
In this example, you’ll learn how to swap your face with your friend’s face. Download demo package:
Create two 3D faces by clicking + Add object > Face Effects > 3D Face twice in the Hierarchy panel. You’ll see there are two face meshes and two face paint materials created in the Assets panel.
To apply one person’s face texture to another person’s face, you’ll first generate the face texture from the camera inputs for each face. Select the Face mesh in the Assets panel, then go to the Inspector panel to find Face Info. Click Create face texture to extract the face texture that you can find in the Assets panel. Same for Face(1) Mesh. However, in order to generate the Face texture out of the second face in the scene, you need to change Which Face value to 1, then click on Create face texture button.
Now, apply Face (1) Texture to the Face Paint material of 3D Face object. Apply Face Texture to the Face Paint (1) material of 3D Face (1) object.
Select 3D Face (1) object in the Hierarchy panel. Navigate to the Inspector panel, and change the Which Face property of the 3D Face Binding component to value 1. Now you should see the face swapped.
3D Face Binding properties
Which Face: Based on the order of faces it entered the scene, determine Which face the face mesh should follow. Remove this component to manually control the transforms of 3D Face.