This paper proposes a novel method for tracking fail-
ure detection. The detection is based on the Forward-
Backward error, i.e. the tracking is performed forward
and backward in time and the discrepancies between
these two trajectories are measured. We demonstrate
that the proposed error enables reliable detection of
tracking failures and selection of reliable trajectories
in video sequences. We demonstrate that the approach
is complementary to commonly used normalized cross-
correlation (NCC). Based on the error, we propose a
novel object tracker called Median Flow. State-of-the-
art performance is achieved on challenging benchmark
video sequences which include non-rigid objects.