Gold-Standard Error vs. Symmetric Transfer Error
Published:
When evaluating or optimizing homography (or other geometric) estimates, two error measures often appear: gold-standard (true geometric) error and symmetric transfer error. They are not the same.
1. Gold-standard (true geometric) error
Gold-standard error measures the Euclidean distance from noisy image correspondences to the exact geometric constraint (e.g., a homography). It minimizes
\[|x-\hat x|^2 + |x'-\hat x'|^2\]subject to \(\hat x’ = H\hat x\), allowing both image points to move optimally. This is the maximum-likelihood objective under Gaussian pixel noise.
2. Symmetric transfer error
Symmetric transfer error is a computational approximation, defined as
\[|x' - \pi(Hx)|^2 + |x - \pi(H^{-1}x')|^2\]It evaluates forward and backward reprojection without optimizing over corrected points.
3. These two errors are not the same
Symmetric transfer error matches the gold-standard error only to first order. They generally differ at second order, and their minima do not necessarily coincide.
4. Symmetric transfer error is a compromise
Symmetric transfer error is not a principled likelihood. It is widely used because it is:
- symmetric,
- measured in pixel space,
- inexpensive to compute,
- stable for small noise.
However, it is still an approximation.
5. Gold-standard error is algorithm-independent
Gold-standard error is valid for benchmarking any method (DLT, normalized DLT, Sampson, LM, bundle adjustment), regardless of what objective the algorithm optimized internally.
6. Optimization cost vs. evaluation metric
Optimization cost and evaluation metric must be separated. Algorithms may minimize algebraic, Sampson, or transfer errors for efficiency, but final comparison should ideally be done using geometric (gold-standard) error.
7. When the model and points are both optimized
If both the model \(H\) and image points are optimized, only the gold-standard geometric error remains meaningful. Symmetric transfer error becomes degenerate when image points are allowed to move.
8. With unlimited computational resources
The gold-standard geometric error is strictly superior for benchmarking:
- it is statistically correct,
- invariant to parameterization,
- and reflects the true geometric quality of the estimated model.
9. Terminology in practice
In practice, symmetric transfer error is commonly (and sometimes loosely) called “geometric error,” but this terminology is inaccurate unless the distance to the constraint manifold is explicitly minimized.
10. cv::findHomography() of opencv
It does NOT optimize the true Gold-standard error. It optimizes a one-sided reprojection error.
Why OpenCV Doesn’t Use Gold Standard
- Because Gold-standard requires:
- Optimizing corrected points
- Enforcing constraint per match
- Much heavier computation
- More complex Jacobians
- Iterative correction inside each residual
For practical computer vision (AR, stitching, tracking): Forward reprojection error is good enough.
From cheapest to most correct:
1️⃣ Algebraic error (DLT raw) 2️⃣ Forward reprojection error (OpenCV LM) 3️⃣ Symmetric transfer error 4️⃣ Gold-standard (true ML under Gaussian noise)
OpenCV stops at level 2.