The Caltech Tennis Dataset (CalTennis) is the first benchmark to use multi-view, real-world recordings of skilled human motion, capturing data underrepresented in existing pose datasets and more representative of downstream motion-reconstruction applications. It has significantly more depth variability and pose space coverage. Despite containing 10x more frames than other real-world benchmarks, it is also signifcantly cheaper than current MOCAP and real-world benchmark, requiring only every-day phones mounted on tripods.

To evaluate monocular pose estimators without ground-truth labels, we lift pose predictions from each camera into a shared global space-time reference frame:

CalTennis uses multi-view consistency as a label-free proxy for reconstruction error: a correct prediction must agree across views, and inter-view disagreement lower-bounds each model's true error. In addition to standard metrics (MPJPE, PA-MPJPE, PVE), we introduce four metrics that expose failure modes invisible to existing benchmarks.

• Translation error. L2 distance between per-view translation estimates for the same person. Exposes depth instability — the dominant failure mode we find across all models.
• Pose error. Mean per-joint position error relative to the pelvis, measuring orientation and pose consistency independently of translation.
• Footwork. Cross-view agreement of foot joint velocities and foot heights, exposing foot-skating and floating artifacts that are physically implausible.
• Stability. Cross-view disagreement of the distance from projected center of mass to the convex hull of grounded foot joints — models that disagree on whether a pose is balanced are unreliable for sport analysis.

We benchmark five state-of-the-art monocular 3D pose estimators on CalTennis: PromptHMR, WHAM, GVHMR, TRAM, and GENMO. No single model dominates across all dimensions. PromptHMR achieves the most consistent translation and joint-position estimates; WHAM excels at foot velocity consistency thanks to its ground-contact refinement step; GENMO is most consistent on foot height and stability. Across all models, performance on CalTennis is substantially worse than on existing benchmarks, reflecting the challenges of large depth range, fast athletic motion, and unscripted behavior.

We find that all models struggle with making consistent translation estimates, with average error ranging from 0.9m - 3.6m, with 75% of translation errors within a 1m window. As the poses contained in CalTennis span greater distances, pixel-level errors in pose estimates can result in more severe mistakes in translation estimates, an effect that is not obvious from current benchmarks. Qualitatively, we find that this results in a "pose drifting" effect, or oscillations in translation estimates along each camera's depth axis. Models are much more consistent when it comes to pose estimates, with about 11cm error between multi-view poses across all models. This suggests that these models are more ready for downstream applications involving pose estimates alone, rather than those dependent on accurate 3D identification of people in the scene.

We find that the most performant model depends on the metric and hardware limitations of the appplication. PromptHMR has the lowest translation inconsistency but also also the heaviest, while WHAM, which is second lowest, is also second lightest. These results highlight the trade-offs between static pose accuracy and temporal/physical consistency

An often-overlooked failure mode is shape inconsistency: the same person's estimated height, weight, and body proportions (SMPL-X β parameters) vary significantly across camera views and across models. All five evaluated models produce inconsistent shape predictions, with up to 20 cm of height disagreement across views. PromptHMR achieves the most consistent multi-view shape reconstruction, likely because it conditions on 2D bounding boxes and keypoints. Models that produce consistent trajectories typically predict shape parameters once per video from the first frame, which is insufficient for accurate biomechanical analysis. Our findings motivate a more nuanced, video-level approach to shape prediction.

Model failure modes become more clear when we project pose estimates from one camera view onto the other to visualize cross-view inconsistency. Low-disagreement frames correspond to stationary, “neutral” poses equally visible from both cameras. High-disagreement frames occur on distant or dynamic poses with partial occlusion — particularly at feet and hands during strokes and footwork.

No single model is best. Different models excel at different aspects: PromptHMR leads on translation and joint-position metrics but has the largest per-video variance; WHAM excels at foot skating and pose drift at the cost of translation accuracy; GENMO is the most internally consistent across views. The relative model ordering changes with each metric.

Three quantities remain unreliable across every model we tested: absolute distance and depth, ground-contact detection, and body shape (limb lengths, height, proportions). These are exactly the quantities that clinical biomechanical analysis, force and balance estimation, fine-grained sports analytics, pedestrian intent prediction, and forensic stride-length measurement most directly depend on.