Different VFX shots need different tracking models. Point, planar, surface, object and 3D tracking each solve a specific motion problem, and the best workflow uses the simplest model that remains accurate.
1. What 3D tracking means
Tracking estimates how image features, surfaces, objects or the camera move through time. The result can drive graphics, cleanup patches, CG objects, digital characters or entire virtual environments.
→ TRACKING FEATURES
→ MOTION MODEL
→ SOLVE
→ CLEAN TRACK
→ ATTACH VFX
→ COMPOSITE
The most important decision is the motion model: use only as much complexity as the shot requires.
2. Point tracking
Point tracking follows distinctive image features such as corners, high-contrast details or textured points. It is often the foundation for camera solves and can also drive simple 2D effects.
| Good point | Risky point |
|---|---|
| Static corner | Moving reflection. |
| Strong texture | Animated texture. |
| Fixed environmental detail | Moving shadow. |
| High contrast feature | Temporary light flare. |
3. Planar tracking
Planar tracking models the motion and perspective of a relatively flat surface. It is especially useful for screens, signs, posters, walls and floors.
→ TRACK TEXTURE
→ ESTIMATE PERSPECTIVE
→ ATTACH GRAPHIC / PATCH
→ CHECK CORNERS
→ APPROVE
| Use | Example |
|---|---|
| Screen replacement | Phone, monitor or billboard. |
| Cleanup | Remove a logo or object from a wall. |
| Graphic insertion | Poster or sign replacement. |
| Set detail | Attach texture or VFX elements. |
4. Surface tracking
Surface tracking extends the idea of planar tracking to deforming or more complex surfaces. It is useful when a target changes shape rather than behaving like a rigid flat plane.
| Surface | Challenge |
|---|---|
| Clothing | Non-rigid deformation. |
| Face | Expression and depth changes. |
| Skin | Subtle deformation and lighting variation. |
| Flexible packaging | Shape changes over time. |
For performance-driven work, combine surface tracking with AI Face & Body Tracking where appropriate.
5. Object tracking
Object tracking estimates the movement of a distinct object. A rigid object can often be represented with translation and rotation, while more complex objects may need additional constraints.
| Object | Typical tracking data |
|---|---|
| Vehicle | Position, rotation and wheel relationships. |
| Prop | Translation and orientation. |
| Robot | Rigid body plus articulated parts. |
| CG replacement | Object transform aligned to plate. |
6. 3D tracking
3D tracking reconstructs a camera and scene relationship from image motion. It is required when CG must respond correctly to camera movement, perspective and parallax.
+
CAMERA MODEL
+
LENS
+
DEPTH RELATIONSHIPS
↓
3D CAMERA / SCENE SOLVE
↓
CG INTEGRATION
For a deeper camera-solving workflow, see AI Camera Tracking & Matchmoving 2.0.
7. Where AI helps
AI can reduce repetitive tracking work by finding features, identifying subjects and propagating tracks. It can also help distinguish static background features from moving objects.
| AI-assisted task | Benefit |
|---|---|
| Feature detection | Rapidly locate useful track candidates. |
| Segmentation | Exclude moving foreground objects. |
| Track propagation | Extend tracks through stable motion. |
| Outlier detection | Flag unstable tracks for review. |
| Object identification | Separate objects for targeted tracking. |
8. Track cleanup
Raw tracks are rarely perfect. Remove points that drift, jump, lock onto reflections or leave the target.
→ FILTER OUTLIERS
→ CHECK FRAME RANGE
→ CORRECT DRIFT
→ RE-SOLVE
→ TEST WITH VFX
Track cleanup should happen before downstream compositing. A small drift can become very obvious when a high-contrast CG object is attached to it.
9. VFX applications
| Application | Best tracking choice |
|---|---|
| Screen replacement | Planar or surface tracking. |
| Logo removal | Planar tracking plus cleanup. |
| CG prop | Object tracking. |
| Set extension | 3D camera tracking. |
| Digital human | 3D camera + performance tracking. |
| Particle effects | 3D camera/world-space tracking. |
10. Common mistakes
| Mistake | Why it fails | Better approach |
|---|---|---|
| Using planar tracking for a deforming surface | Surface corners drift. | Use surface/deformation-aware tracking. |
| Using 3D tracking for a simple screen | Unnecessary complexity. | Use planar tracking when appropriate. |
| Tracking reflections | Reflections can move differently from the object. | Use stable physical features. |
| Ignoring occlusion | Track jumps to another feature. | Split or restart the track. |
| Never testing the actual VFX | Small errors remain hidden. | Attach a test element early. |
11. Complete Zgian 3D tracking workflow
↓
CHOOSE TRACKING MODEL
↓
FEATURE / SURFACE TRACKS
↓
AI PROPAGATION
↓
OUTLIER CLEANUP
↓
SOLVE
↓
OBJECT / PLANAR / 3D VALIDATION
↓
ATTACH TEST VFX
↓
REFINE
↓
FINAL COMPOSITE QC
- Identify the physical motion problem.
- Choose point, planar, surface, object or 3D tracking.
- Create reliable tracks.
- Use AI propagation where it helps.
- Remove unstable tracks and solve again.
- Attach a simple test element.
- Check drift, perspective and occlusion.
- Approve the track before building the final VFX.
12. Final tracking QC
- Check track stability.
- Check target identity.
- Check occlusions.
- Check perspective.
- Check surface deformation.
- Check camera movement.
- Check object rotation.
- Check CG/VFX lock.
- Check the complete shot at normal speed.
- Review the final composite at full resolution.
How this fits the Zgian VFX + AI pipeline
3D / OBJECT TRACKING → this guide
PERFORMANCE TRACKING → Face & Body Tracking
AI ROTO → AI Rotoscoping 2.0
AI CLEANUP → AI Cleanup 2.0
AI TEMPORAL → Motion & Frame Interpolation
Frequently asked questions
What is point tracking?
Point tracking follows distinctive image features through a sequence and can provide motion data for compositing or camera solving.
What is planar tracking?
Planar tracking estimates the motion and perspective of a relatively flat surface, making it useful for screens, signs and cleanup patches.
What is object tracking?
Object tracking estimates the movement of a distinct object so a CG or VFX element can follow it.
When do I need 3D tracking?
Use 3D tracking when the VFX must respond to camera movement, perspective and depth rather than only a flat surface transform.
Can AI replace a tracking artist?
AI can accelerate repetitive steps, but choosing the correct tracking model and validating the final result still requires production judgment.
Continue the Zgian VFX + AI series
3D Generalist & VFX Professional · About the author →
