The root of the issue was caused by the PredictMarkerPosition()
always returning false when tracking backwards. This was making
it so tracker always had to run brute initialization, which is
an expensive operation.
From own timing here:
- Tracking forward takes 0.667637 seconds
- Tracking backward used to take 2.591856 seconds
- Tracking backward now takes 0.827724 seconds
This is a very nice speedup, although the tracking backwards is
still somewhat slower. Will be investigated further as part of
a regular development.