TAPE3D:TRACKING ALL PIXELS
EFFICIENTLY IN 3D

TAPE3D captures dense, long-range, 3D trajectories from casual videos in a feed-forward manner.

Comparison with 3D tracking approach: SceneTracker and SpatialTracker, and the 3D version of DOT

More results of 3D dense tracking can be found here.

Comparison with 2D tracking approach (lift 3D with depth): CoTracker and LocoTrack

Occlusion problem when using 2D track + Depth

Frontal view

Side view

(Note that all of the methods use the same video depth input)

The baseline (2D Tracker + Depth) struggles significantly with inconsistent video depth estimation, resulting in noticeable jittering effects. Additionally, it fails to accurately track objects during occlusion, as seen from the side view when the ball rolls behind the tree.

More qualitative results can be found here.

Application: Consistent video editting in 3D space

Application: Non-rigid structure from motion