Motion estimation lets VFX artists retime footage, generate intermediate frames and create believable motion blur. The challenge is preserving temporal continuity when objects overlap, disappear or move unpredictably.
1. What optical flow is
Optical flow describes apparent motion between image frames. A flow field can guide warping, interpolation, retiming and other temporal image operations.
+
MOTION ESTIMATION
+
FRAME B
↓
FLOW FIELD
↓
WARP / INTERPOLATE / RETIME
Flow is an estimate of image movement, not a perfect 3D description of the scene. Its reliability depends on texture, lighting, occlusion and the camera.
2. Motion blur
Motion blur is produced when a camera records movement during the exposure. In VFX it can be preserved from the plate, generated from motion data or simulated during compositing.
| Situation | Goal |
|---|---|
| Original plate | Preserve natural photographed blur. |
| CG element | Match the plate’s shutter behavior. |
| Retime | Keep blur consistent with the new motion. |
| Frame interpolation | Avoid sharp synthetic frames inside blurred motion. |
3. Optical flow
Flow algorithms compare consecutive frames and estimate where pixels or features move. Smooth surfaces can be difficult because they contain little visual information, while reflections and transparent objects may not follow simple motion.
| Scene condition | Flow difficulty |
|---|---|
| Textured object | Usually easier to estimate. |
| Fast movement | Large displacement can create errors. |
| Occlusion | Pixels disappear or appear unexpectedly. |
| Reflections | Reflected motion can differ from object motion. |
| Low texture | Motion becomes ambiguous. |
4. Frame interpolation
Frame interpolation creates a new frame between two existing frames. AI models may combine motion estimation with learned image synthesis to predict the missing moment.
↘
INTERMEDIATE FRAME
↗
FRAME 2
The generated frame is a prediction. It can contain plausible detail that was never captured by the camera, so high-value shots require careful inspection.
5. Retime and slow motion
Retime changes the temporal speed of a shot. For modest speed changes, conventional methods may be sufficient; for stronger slow motion, additional frames may be required.
| Method | Best for |
|---|---|
| Frame duplication | Simple timing changes where repeated frames are acceptable. |
| Frame blending | Soft transitions with some ghosting. |
| Optical flow | Motion-aware retiming. |
| AI interpolation | Large slow-motion conversions when the source supports it. |
Choose the method based on shot content rather than using the most advanced option automatically.
6. Occlusion and difficult motion
When an object passes in front of another, the hidden pixels are not directly observable. Interpolation systems must infer what happens behind the occluder.
| Problem | Typical artifact | Response |
|---|---|---|
| Object crossing | Warped edges. | Use masks or alternate interpolation treatment. |
| Disocclusion | Invented background detail. | Use clean plates or manual reconstruction. |
| Fast hand motion | Finger duplication. | Reduce interpolation strength or correct locally. |
| Fine hair | Shimmering strands. | Use motion-aware masking and cleanup. |
7. VFX applications
Temporal techniques are useful throughout VFX, especially when CG and live-action footage have different motion characteristics.
| Application | Use |
|---|---|
| CG integration | Generate matching motion blur. |
| Cleanup | Propagate repairs between frames. |
| Rotoscoping | Assist matte propagation. |
| Stabilization | Estimate motion for image warping. |
| Slow motion | Create intermediate frames. |
| Time effects | Retime selected regions or shots. |
8. Temporal artifacts
Temporal errors often become visible only when the shot plays. Common problems include ghosting, warping, texture crawling, duplicated limbs, melting edges and unstable details.
→ PLAY FULL SPEED
→ PAUSE ON FAST MOTION
→ CHECK OCCLUSIONS
→ CHECK EDGES
→ CHECK TEXT / FACES / HANDS
→ CHECK GRAIN
→ APPROVE
Faces, hands, text and high-contrast edges are particularly useful for spotting interpolation errors.
9. Complete Zgian temporal VFX workflow
↓
MOTION ANALYSIS
↓
CHOOSE RETIME METHOD
↓
OPTICAL FLOW / AI INTERPOLATION
↓
MASK DIFFICULT AREAS
↓
MOTION BLUR MATCH
↓
LOCAL REPAIR
↓
GRAIN / SHARPNESS MATCH
↓
TEMPORAL QC
↓
FINAL MASTER
- Inspect the source for motion and occlusion.
- Define the required speed change.
- Choose frame blending, optical flow or AI interpolation.
- Protect faces, hands, text and complex edges.
- Correct difficult regions.
- Match motion blur and camera characteristics.
- Restore grain and image texture.
- Review the entire sequence at normal speed.
10. Common mistakes
| Mistake | Why it fails | Better approach |
|---|---|---|
| Interpolating everything | Complex regions generate artifacts. | Use masks and selective treatment. |
| Ignoring shutter blur | Motion looks unnaturally sharp. | Match blur to the source. |
| Large speed changes from poor footage | Too much motion must be invented. | Use suitable source footage or accept limits. |
| Not checking text | Letters can warp or duplicate. | Inspect graphic and text regions. |
| Judging still frames | Temporal instability is hidden. | Review in motion. |
11. Final motion QC
- Check overall timing.
- Check faces.
- Check hands and fingers.
- Check hair and fine edges.
- Check occlusions and disocclusions.
- Check text and graphics.
- Check motion blur.
- Check warping and ghosting.
- Check grain consistency.
- Watch the full shot at normal playback speed.
How this fits the Zgian VFX + AI pipeline
AI ROTO → AI Rotoscoping 2.0
AI CLEANUP → AI Cleanup 2.0
AI TEMPORAL VFX → this guide
AI COMP → AI Compositing
AI VIDEO → AI Video Hub
Frequently asked questions
What is optical flow?
Optical flow is an estimate of how image regions move between frames. It can guide warping, retiming and frame interpolation.
What is frame interpolation?
It is the generation of one or more intermediate frames between existing frames to create smoother or slower motion.
Why does AI slow motion sometimes look strange?
The system must predict pixels that were never captured. Occlusion, fast motion, hair, hands and reflections can produce artifacts.
Should I add motion blur after interpolation?
Depending on the workflow, matching the original camera’s motion characteristics can help integrate generated frames, but excessive blur can hide rather than solve artifacts.
Is AI interpolation always better than optical flow?
No. Different shots favor different methods. A simpler optical-flow or frame-blending approach can be more reliable for some footage.
Continue the Zgian VFX + AI series
3D Generalist & VFX Professional · About the author →
