Hybrid RGB–Depth Markerless Registration for High-Precision Augmented Reality Surgical Navigation
DOI:
https://doi.org/10.66238/jaeee23Keywords:
RGB-depth fusion, markerless registration, AR surgical guidance, multi-objective optimization, photometric consistencyAbstract
Depth-only registration may suffer from sparse geometric cues in texture-rich anatomical regions. This study proposes a hybrid RGB–depth alignment framework that combines geometric depth constraints with photometric consistency optimization. The method jointly minimizes depth residual error and RGB gradient alignment using a multi-objective optimization scheme. Evaluation included 20 cadaveric craniofacial datasets and 15 phantom models. The hybrid approach reduced mean registration error from 2.11 mm (depth-only) to 1.29 mm and improved pose stability by 22.4% over 60-second tracking intervals. Integration into a head-mounted AR system achieved 36 ms per frame processing time, supporting real-time surgical visualization.
References
1. Zhang, Y., & Wang, J. (2026). Design and Implementation of a Computer-Aided Full Lifecycle Quality Management System for Wind Farms in Upgrades, Renovations, and Subcontractor Supervision.
2. Ishida, H., Lu, Y., & Shirazian, P. (2026). Cavir: confidence-aware vision-based registration for image-guided surgery. International Journal of Computer Assisted Radiology and Surgery, 1-9.
3. Jiao, Y., Wang, A., Zhao, B., & Shi, T. (2026). The Impact of Visual Language Strategies in Public Art Creation on Community Spatial Perception and Public Behavior.
4. Maken, P., Rana, S. S., Gupta, A., & Kaur, H. (2024). Challenges for 3D Imaging for Craniofacial Applications. In Applications of Three-dimensional Imaging for Craniofacial Region (pp. 223-240). Singapore: Springer Nature Singapore.
5. Yang, Y., Leuze, C., Hargreaves, B., Daniel, B., & Baik, F. (2025). EasyREG: Easy Depth-Based Markerless Registration and Tracking using Augmented Reality Device for Surgical Guidance. arXiv preprint arXiv:2504.09498.
6. Mishra, R., Narayanan, M. K., Umana, G. E., Montemurro, N., Chaurasia, B., & Deora, H. (2022). Virtual reality in neurosurgery: beyond neurosurgical planning. International journal of environmental research and public health, 19(3), 1719.
7. Jiao, Y., Wang, A., Zhao, B., & Shi, T. (2026). Quantitative Study on the Construction and Application Effectiveness of Graffiti Wall Painting Teaching Models in Public Space Contexts.
8. Haenel, R., Semler, Q., Semin, E., Grussenmeyer, P., & Tabbone, S. (2022). Evaluation of low-cost depth sensors for outdoor applications. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 48, 101-108.
9. Liu, S., & Yim, J. (2025). Research on Generative AI Creation Systems Based on Visual Language Modeling: Human-Machine Collaboration and Cognitive Feedback Mechanisms. Available at SSRN 6139770.
10. Wolffsohn, J. S. (2025). Enhancing Clinical Assessment for Improved Ophthalmic Management (Doctoral dissertation, The University of Manchester (United Kingdom)).
11. Yu, X. (2026). AI-Driven Personalization across Domains for Local Categorical Query Understanding and Context-Aware Retrieval.
12. Kamal, U., Zunaed, M., Nizam, N. B., & Hasan, T. (2022). Anatomy-XNet: An anatomy aware convolutional neural network for thoracic disease classification in chest X-rays. IEEE Journal of Biomedical and Health Informatics, 26(11), 5518-5528.
13. Chen, J., Xiao, H., Zeng, Z., Du, X., & Xiao, R. (2026). Diffusion-step attention consistency for multivariate time series anomaly detection. Knowledge-Based Systems, 115716.
14. Boutsi, A. M. (2025). Photogrammetric Techniques and Algorithms Optimization for the Development of Interactive Augmented Reality Systems.
15. Wang, Y., Wen, Y., Wu, X., Wang, L., & Cai, H. (2024). Modulation of Gut Microbiota and Glucose Homeostasis through High-Fiber Dietary Intervention in Type 2 Diabetes Management.
16. Sandström, E., Zhang, G., Tateno, K., Oechsle, M., Niemeyer, M., Zhang, Y., ... & Tombari, F. (2025). Splat-slam: Globally optimized rgb-only slam with 3d gaussians. In Proceedings of the Computer Vision and Pattern Recognition Conference (pp. 1680-1691).
17. Wen, Y., Wu, X., Wang, L., Cai, H., & Wang, Y. (2025). Application of Nanocarrier-Based Targeted Drug Delivery in the Treatment of Liver Fibrosis and Vascular Diseases. Journal of Medicine and Life Sciences, 1(2), 63-69.
18. Di Giammarino, L., Giacomini, E., Brizi, L., Salem, O., & Grisetti, G. (2023). Photometric lidar and rgb-d bundle adjustment. IEEE Robotics and Automation Letters, 8(7), 4362-4369.
19. Wang, Y., Chen, J., Arias, R., Wang, Y., & Yin, X. (2026). Development and Validation of a Patient-Friendly Digital Assessment Platform for Precision Screening of Oral Anti-Obesity Medications (AOMs).
20. Kawaharazuka, K., Oh, J., Yamada, J., Posner, I., & Zhu, Y. (2025). Vision-language-action models for robotics: A review towards real-world applications. IEEE Access.
21. Gao, G., Ma, X., Lu, C., & Gao, R. (2026). Reliability Analysis and Application Research of SMS Communication Systems in Medical Notification Scenarios.
22. Bose, S. (2026). 3D Multi-View Stylization with Pose-Free Correspondences Matching for Robust 3D Geometry Preservation. arXiv preprint arXiv:2604.09639.
23. Wang, Y., Yin, X., Chen, J., & Wang, Y. (2026). Evidence-Based Study on Low-Burden Digital Phenotyping for Precision Screening of Oral Anti-Obesity Drug Efficacy.
24. Fischer, M. J., & Strong, E. B. (2025, October). Ar surgical navigation with surface tracing: comparing in-situ visualization with tool-tracking guidance for neurosurgical applications. In 2025 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) (pp. 1170-1179). IEEE.
25. Zhang, Y., Gu, W., & Wang, J. (2026). Construction of Wind Farm Asset Health Index Based on Multi-Dimensional Indicators and Analytic Hierarchy Process and Its Correlation with Operational Performance. Authorea Preprints.
26. Strong, E. B., Patel, A., Marston, A. P., Sadegh, C., Potts, J., Johnston, D., ... & Strong, E. B. (2025). Augmented reality navigation in craniomaxillofacial/head and neck surgery. OTO open, 9(2), e70108.
27. Chen, X., Xiao, H., Zeng, Z., Zhang, S., & Xiao, R. (2025). Fine-Grained Multivariate Time Series Anomaly Detection via Causal Inference. Knowledge-Based Systems, 114765.
28. Shankar, A., Jawad, L., & Pandya, A. (2025). An AI Approach to Markerless Augmented Reality in Surgical Robots. Robotics, 14(7), 99.
29. Wang, S. (2025). W-PCA Based Gradient-Free Proxy for Efficient Search of Lightweight Language Models. arXiv preprint arXiv:2504.15983.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Michael J. Thompson, Daniel K. Robertson, Emily R. Chen
This work is licensed under a Creative Commons Attribution 4.0 International License.
All articles published in the Journal of Advanced Electrical and Electronic Engineering (JAEEE) are distributed under the Creative Commons Attribution 4.0 International License (CC BY 4.0). Authors retain copyright of their work while granting the journal the right of first publication.
Under this license, users are permitted to read, download, copy, distribute, print, share, adapt, and reuse published material for both commercial and non-commercial purposes, provided appropriate credit is given to the original author(s), the source of publication is acknowledged, a link to the license is provided, and any changes made are clearly indicated.
The license does not permit users to imply endorsement by the original authors or the journal. No additional restrictions may be imposed that prevent others from exercising the rights granted under CC BY 4.0.
By submitting to the journal, authors confirm that their work is original, does not infringe upon third-party rights, and complies with ethical publishing standards.
