Creating Lip Sync Effects

The Lip Sync model is an AI-powered tool that animates lips in an image to match movements from a reference video using a video-to-video (V2V) transformation process, creating fun, expressive, and lifelike results.

The key to creating great Lip Sync effects is starting with a strong reference video. By following the best practices below, you can create Lip Sync reference videos that ensure your AI-generated animations are stable, expressive, and visually appealing.

Zimomo song	AiScReam	Chinese POP	Jet2 Hoilday

Open AI Lip Sync

To open AI Lip Sync in AI Editor:

1. Open Effect House and navigate to the Tool Bar
2. Go to AI Editor
3. Select AI Lip Sync

Alternatively, click Add Object in the Hierarchy panel, go to AI Editor, and choose AI Lip Sync

Creating Effective Reference Videos for the Lip Sync Model

The quality of your Lip Sync reference video is important, as the AI only focuses on facial expressions and mouth movements. To achieve the best results, it helps to understand what the model looks for.

note

Avoid videos with multiple camera shots, large body movements, significant head and neck motion, extreme facial angles, flat expressions, head obstructions, or cluttered backgrounds. The model performs best when focused on a single, clear facial performance.

Illustration of a Good Reference Video

To get the best results, your reference video should feature a clear and centered subject:

• Keep the head centered in the frame
• Face the camera directly to avoid angular distortion
• Ensure the face is unobstructed and all features are clearly visible

Recording Techniques

After framing your subject correctly, follow these recording techniques to maintain stable motion and natural, expressive facial and mouth movements throughout the video.

Keep the Body Still

Avoid excessive movements of the head, neck, and shoulders to maintain video stability. Too much motion can cause distortions and reduce the quality of the generated effect.

Good example	Generated effect	Bad example	Generated effect

Keep the Face at a Frontal Angle

Maintain a straight, front-facing angle to prevent facial distortion or unnatural warping in the generated effect.

Good example	Generated effect	Bad example	Generated effect

Be Expressive

Use lively facial expressions and clear mouth movements to make your video more engaging and visually appealing.

Good example	Generated effect	Bad example	Generated effect

Keep the First Frame Natural

The algorithm relies on the first frame, so make sure your initial expression and mouth position look natural to ensure smooth and realistic transitions.

Good example	Generated effect	Bad example	Generated effect

Reference Video Examples

Here are some examples of different reference videos and their generated effects. Feel free to use these for inspiration when creating your own Lip Sync effect!

Name	Reference video	Generated effect	Generated effect	Generated effect	Generated effect
Zimomo song
Chinese POP
AiScReam
Jet2 Hoilday

Guidelines

When recording your reference video, please make sure your content adheres to the following guidelines:

1. Avoid including religious extremism, attacks on religious beliefs, or incitement of hatred or discrimination.
2. Avoid generating violent threats, abuse, terrorism, or incitement to harm individuals or groups.
3. Avoid generating obscene or pornographic material, sexual content involving minors, sexual violence, or adult sexual services.
4. Avoid generating illegal or dangerous content such as criminal activity guides, drug production, weapon manufacturing, hacking, or fraud.
5. Avoid generating instructions for illegal activities or fraudulent methods.
6. Avoid generating content that incites subversion, spreads false political information, or endangers social stability.
7. Avoid generating false medical advice that could lead to health risks.
8. Avoid inappropriate or illegal content involving minors.