Pose Anything Anywhere

Model-free Object Poses from Arbitrary References

ECCV 2026

Hongli Xu*1, Jiaqi Hu*1,3, Junwen Huang*†1,2, Boyang Zhong1, Peter KT Yu4,5, Nassir Navab1,2, Benjamin Busam1,2, Slobodan Ilic1,3
1Technical University of Munich 2Munich Center for Machine Learning (MCML) 3Siemens AG 4XYZ Robotics 5ROBOX

* Equal contribution. The first three authors are listed in random order. † Corresponding author.

Paper arXiv Video Code Weights
PANY teaser showing model-free object pose estimation from arbitrary references

PANY estimates 6D object poses from one or sparse arbitrary references, with RGB and RGB-D support.

Abstract

Estimating the 6D pose of unseen objects is a fundamental yet challenging problem for open-world robotics and embodied perception. Model-based methods are accurate but depend on CAD assets or heavy onboarding, while most model-free approaches are still limited to pairwise single-anchor matching and thus fail under occlusion and large viewpoint changes with low query–reference overlap. Therefore, we present PANY, a unified model-free framework that seamlessly supports both RGB and RGB-D inputs, operates on one or sparse pose-free reference views, and generalizes effectively to novel objects. Built on a multi-view transformer geometry backbone, PANY moves beyond pairwise matching by learning view-consistent geometry and cross-view alignment cues that remain stable under wide baselines and limited overlap. When additional unposed assist views are available, PANY aggregates them via pose-graph canonical registration to increase geometric coverage and reinforce the final pose. Extensive experiments show that PANY achieves state-of-the-art performance across multiple benchmarks, substantially outperforming existing model-free methods, improving pose accuracy by +12% on YCB-V and over +20% on LM-O. Furthermore, PANY consistently performs well under both single-reference and sparse-reference settings, demonstrating strong robustness in real-world environments.

Video

PANY Pipeline

PANY pipeline overview

Contributions

  • We formalize 6D pose estimation from arbitrary sparse and unordered references as a practical model-free setting, optionally using pose-free assist views without CAD models or an object-specific onboarding stage.
  • PANY adapts geometry foundation models to object-centric canonical alignment by learning geometry-aware and spatially consistent cross-view correspondences for robust pose reasoning under weak texture, occlusion, and wide baselines.
  • PANY introduces a multi-view inference procedure that aggregates unposed assist views through pose-graph canonical registration, improving robustness when no single reference view provides enough query overlap.

Visualization Results

Qualitative pose estimation examples for single-reference and sparse multi-view inference.

Single-view Inference

Single-view PANY inference visualization results

Multi-view Inference

Multi-view PANY inference visualization results

Citation

@inproceedings{xu2026pose,
  title={Pose Anything Anywhere: Model-free Object Poses from Arbitrary References},
  author={Xu, Hongli and Hu, Jiaqi and Huang, Junwen and Zhong, Boyang and Yu, Peter KT and Navab, Nassir and Busam, Benjamin and Ilic, Slobodan},
  booktitle={European Conference on Computer Vision (ECCV)},
  year={2026}
}