Publications

2026

1. CVPR

   

   Unified Spherical Frontend: Learning Rotation-Equivariant Representations of Spherical Images from Any Camera

   Mukai Yu, Mosam Dabhi, Liuyue Xie, and 2 more authors

   In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026

   Abs DOI Bib Video Website

   Modern perception increasingly relies on fisheye, panoramic, and other wide field-of-view (FoV) cameras, yet most pipelines still apply planar CNNs designed for pinhole imagery on 2D grids, where pixel-space neighborhoods misrepresent physical adjacency and models are sensitive to global rotations. Traditional spherical CNNs partially address this mismatch but require costly spherical harmonic transform that constrains resolution and efficiency. We present Unified Spherical Frontend (USF), a distortion-free lens-agnostic framework that transforms images from any calibrated camera onto the unit sphere via ray-direction correspondences, and performs spherical resampling, convolution, and pooling canonically in the spatial domain. USF is modular: projection, location sampling, value interpolation, and resolution control are fully decoupled. Its configurable distance-only convolution kernels offer rotation-equivariance, mirroring translation-equivariance in planar CNNs while avoiding harmonic transforms entirely. We compare multiple standard planar backbones with their spherical counterparts across classification, detection, and segmentation tasks on synthetic (Spherical MNIST) and real-world (PANDORA, Stanford 2D-3D-S) datasets, and stress-test robustness to extreme lens distortions, varying FoV, and arbitrary rotations. USF scales efficiently to high-resolution spherical imagery and maintains less than 1% performance drop under random test-time rotations without training-time rotational augmentation, and enables zero-shot generalization to any unseen (wide-FoV) lenses with minimal performance degradation.

   @inproceedings{yuUnifiedSphericalFrontend2025,
     title = {{{Unified Spherical Frontend}}: {{Learning Rotation-Equivariant Representations}} of {{Spherical Images}} from {{Any Camera}}},
     booktitle = {{IEEE}/{CVF} {Conference} on {Computer} {Vision} and {Pattern} {Recognition} ({CVPR})},
     author = {Yu, Mukai and Dabhi, Mosam and Xie, Liuyue and Scherer, Sebastian and Jeni, László A.},
     year = {2026},
     doi = {10.48550/arXiv.2511.18174},
     eventtitle = {{IEEE}/{CVF} {Conference} on {Computer} {Vision} and {Pattern} {Recognition} ({CVPR})},
     video = {https://youtu.be/Pt7SYRMXv0U},
   }

2. IROS Workshop

   

   MSO: Uncertainty-Aware Multi-Spectral Inertial Odometry for Extreme Environments

   Shibo Zhao, T Karan Balakumar, Avnish Patel, and 6 more authors

   In IROS 2026 Workshop on Data in Field Robotics: From State Estimation to Navigation, 2026

   Bib Video Website

   @inproceedings{zhaoMSOUncertaintyAware2026,
     title = {{MSO}: {Uncertainty-Aware Multi-Spectral Inertial Odometry for Extreme Environments}},
     author = {Zhao, Shibo and Balakumar, T Karan and Patel, Avnish and Maheshwari, Parv and Salian, Tejas and Jong, Andrew and Yu, Mukai and Wang, Wenshan and Scherer, Sebastian},
     booktitle = {IROS 2026 Workshop on Data in Field Robotics: From State Estimation to Navigation},
     year = {2026},
     video = {https://youtu.be/iDAmD-t83Uw},
     url = {https://jonasfrey96.github.io/data_in_field_robotics/},
   }

2023

1. IROS

   

   WIT-UAS: A Wildland-Fire Infrared Thermal Dataset to Detect Crew Assets from Aerial Views

   Andrew Jong, Mukai Yu, Devansh Dhrafani, and 4 more authors

   In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2023

   Abs DOI Bib

   We present the Wildland-fire Infrared Thermal (WIT-UAS) dataset for long-wave infrared sensing of crew and vehicle assets amidst prescribed wildland fire environments. While such a dataset is crucial for safety monitoring in wildland fire applications, to the authors’ awareness, no such dataset focusing on assets near fire is publicly available. Presumably, this is due to the barrier to entry of collaborating with fire management personnel. We present two related data subsets: WIT-UAS-ROS consists of full ROS bag files containing sensor and robot data of UAS flight over the fire, and WIT-UAS-Image contains hand-labeled long-wave infrared (LWIR) images extracted from WIT-UAS-ROS. Our dataset is the first to focus on asset detection in a wildland fire environment. We show that thermal detection models trained without fire data frequently detect false positives by classifying fire as people. By adding our dataset to training, we show that the false positive rate is reduced significantly. Yet asset detection in wildland fire environments is still significantly more challenging than detection in urban environments, due to dense obscuring trees, greater heat variation, and overbearing thermal signal of the fire. We publicize this dataset to encourage the community to study more advanced models to tackle this challenging environment. The dataset, code and pretrained models are available at https://github.com/castacks/WIT-UAS-Dataset.

   @inproceedings{jongWITUASWildlandFireInfrared2023,
     title = {{{WIT-UAS}}: {{A Wildland-Fire Infrared Thermal Dataset}} to {{Detect Crew Assets}} from {{Aerial Views}}},
     booktitle = {{{IEEE}}/{{RSJ International Conference}} on {{Intelligent Robots}} and {{Systems}} ({{IROS}})},
     author = {Jong, Andrew and Yu, Mukai and Dhrafani, Devansh and Kailas, Siva and Moon, Brady and Sycara, Katia and Scherer, Sebastian},
     year = {2023},
     pages = {11464--11471},
     issn = {2153-0866},
     doi = {10.1109/IROS55552.2023.10341683},
     url = {https://ieeexplore.ieee.org/document/10341683},
     eventtitle = {{{IEEE}}/{{RSJ International Conference}} on {{Intelligent Robots}} and {{Systems}} ({{IROS}})},
   }