Abstract

Matrix manifolds play a fundamental role in machine learning, underpinning data representations (\textit{e.g.}, linear subspaces and covariance matrices) and optimization procedures. These manifolds follow Riemannian geometry, where intrinsic geometric structure plays an important role in geometric learning algorithms. However, traditional visualization methods based on Euclidean assumptions often fail to respect such non-Euclidean structure, leading to distortions in the resulting embeddings. To address this limitation, building upon the established Riemannian t-SNE paradigm, we develop three manifold-specific instantiations for the Grassmann, full-rank Correlation, and fixed-rank SPSD manifolds. By introducing Riemannian geodesics to define probability distributions between the original and target spaces, our method transforms high-dimensional manifold-valued data into low-dimensional embeddings, thereby respecting the intrinsic geometry of the data in these settings and reducing distortions caused by Euclidean approximations. This work provides a systematic empirical study of geometry-aware dimensionality reduction and visualization on the three involved matrix manifolds. Extensive experimental comparisons with existing visualization methods across synthetic and benchmarking datasets demonstrate the efficacy of our proposal in preserving geometric properties of the data. The source code and video presentation will be released at: \url{https://github.com/paradox-going/ManiReduce.