Skip to main content
TikTok Effect House Platform Logo
Download
Log in
Tracking and detection

How to use Segmentation

The Segmentation function helps separate the selected body part and apply customized textures and effects to it. We provide segmentation textures like portrait, hair, head, cloth, as well as sky and building for exploring.

unaligned-image-0
unaligned-image-1
unaligned-image-2

Quick start

This example will show you how to setup Segmentation effects quickly.

Someone opening and closing their mouth

Adding segmentation effect

Go to the Hierarchy panel, click + Add object > Segmentation. This will generate a portrait segmentation effect which you can see in the preview.

Adding a Segmentation scene object

The segmentation object will automatically create two child objects along with it. The Background object has been automatically assigned the BackgroundTex texture.

The Effect House tool with the Portrait Segmentation scene object and its properties highlighted

The Portrait Segmentation object has been automatically assigned the Portrait Segmentation Texture.

The Effect House tool with the Background scene object and its properties highlighted

Changing the segmentation type

Besides portrait segmentation, you can also try to segment other parts. For example, let’s try to segment the cloth portion and apply a customized texture.

Select Portrait segmentation texture under the Assets panel and click the Type drop-down window in the Inspector panel. Choose Cloth to change the segment part.

Tip
Tip

Feel free to explore other types of segmentation such as Portrait, Hair, head, Cloth, Sky, and building segment types.

Changing which part of the body a Portrait Segmentation scene object targets

Then check Invert Mask to apply the image on the segment part.

Inverting a Portrait Segmentation scene object's texture
Tip
Tip

You can also create new segmentation textures by going to Assets panel, click + Add asset > Texture, choose the segmentation texture you want to apply to the Image component.

Adding more textures to a Portrait Segmentation scene object

Applying customized texture

You can add customized backgrounds to your effects using the Inspector panel when clicking on the Background object.

Adding a background to the project

Here, we are replacing the default texture with customized background texture.

Segmentation properties

When adding segmentation object, a segmentation object along with two child objects will be created. One is used for the background while the other is used for segmentation layer. Besides the objects, there are also default textures created in the assets. Let’s take a look at these related properties in detail.

Background

Background object

Background object is a 2D image that is automatically created with the segmentation object. It is used for the background of the segmentation.

Background objects will be automatically assigned the auto-created BackgroundTex.

Background object properties

Portrait segmentation

The Portrait Segmentation object is a 2D image that is automatically created with a segmentation object. It has been automatically assigned the Portrait Segmentation Texture.

Portrait segmentation object properties

Segmentation Texture

Portrait Segmentation Texture is automatically created and assigned as texture of Portrait Segmentation. It’s a type of segmentation texture, and there are few properties to customize the texture.

Segmentation texture properties

Currently supported segmentation types:

  • Portrait
  • Hair
  • Head
  • Cloth
  • Sky
  • Ground
  • Building
Note
Note

All these segmentation textures can be recreated by going to + Asset panel > + Add asset > Texture > Segmentation Texture

Adding a Segmentation Texture

Invert mask: Check to invert the mask.

Smoothness: This property determines how smooth the edge is. The larger the value, the smoother the edge.

Cutout On: This property turns on the cutout texture selection. If unchecked, the cutout texture should be greyed out and the segmentation texture will be the black/white mask.

Cutout Texture: This property will segment the selected cutout texture with the selected mask (with alpha) (left: input cutout texture, right: segmentation texture with cutout texture).

Performance

Graph of performance impact measured by resource usage and framerate

The Segmentation feature has a very high impact on performance, specifically on iOS devices. We strongly suggest not combining Segmentation with other features. Doing so will cause low FPS and may subsequently fail QA.

Extended Usage – Adding object in front or behind

You can also try to add 3D objects behind the segmentation layer or in front by changing the render order of the cameras or manipulating the layers of each object. You can also learn more about layering and camera render orders in our Camera guide.

For example, let’s add a sphere and put it behind the body. Go to Hierarchy panel, then choose + Add object > 3D > Sphere.

Adding a sphere in front of the preview model

Duplicate the 2D Background Camera in the Hierarchy panel and rename the copy to 2D Segmentation Camera.

Tip
Tip

Renaming is not critical, but it may help your organize your multiple cameras.

Renaming a 2D camera

Next we will duplicate the Background Canvas and rename the copy to Segmentation Canvas. We make these copies so that there will be a dedicated 2D camera for both canvases.

Creating a Segmentation Canvas

We will now disable the Portrait Segmentation object under the Background Canvas. After, disable the Background object under the Segmentation Canvas.

Disabling the Portrait Segmentation and Background objects

Click on the Segmentation Canvas to open the Inspector panel. Change the Layer to Layer 3. Click on 2D Segmentation Camera and change the Layer to Layer 3 to match the Segmentation Canvas.

Changing which layer the Segmentation Canvas exists on

Make sure that the sphere you added is at the same layer as the Camera. Update the Camera‘s Render Order to a number that is between the Render Order of the 2D Background Camera and the 2D Segmentation Camera.

Changing the camera's Render Order

Once you have updated the Render Order, the sphere can now be moved in between the segmentation and background canvases.

Moving the sphere in the scene
Tip
Tip

Please make sure that Auto sorting has been checked for all the three objects to guarantee the sorting order will follow the Hierachy top-down order. For detailed information about Sorting order, please refer to 3D objects.