A survey on asynchronous compensation and generative spatio-temporal feature alignment for multi-source heterogeneous perception in resource-constrained navigation devices
Miaomiao Cheng
The Third Clinical Medical College, Ningxia Medical University
Zhenggen Xu
The Third Clinical Medical College, Ningxia Medical University
Xingjia Liu
School of Stomatology, Ningxia Medical University
Yiting Liang
The First Clinical Medical College, Ningxia Medical University
Xuan Gai
School of Traditional Chinese Medicine, Ningxia Medical University
DOI: https://doi.org/10.59429/esta.v13i3.15567
Keywords: asynchronous sensor fusion; generative models; spatio-temporal feature alignment; navigation devices; lightweight deployment
Abstract
Asynchronous streams from cameras, LiDAR, and IMUs challenge practical navigation due to clock skew and jitter, yet most surveys assume ideal synchronization. This review reframes compensation as a generative distribution-reconstruction problem. We examine implicit neural representations and continuous-time diffusion to recover missing features at arbitrary timestamps, avoiding the semantic breaks of linear interpolation. A three-stage pipeline-temporal interpolation, cross-modal generation, and uncertainty fusion-is proposed. Deployment constraints and failure boundaries under extreme conditions are analyzed. We conclude that proactive generative compensation significantly boosts navigational robustness.
References
[1] C. Yan, H. Xu, et al., "Time synchronization in autonomous driving: A survey," IEEE Trans. Intell. Veh., vol. 8, no. 3, pp. 2456–2472, 2023, doi: 10.1109/TIV.2023.3265892.
[2] M. Z. Irshad, A. K. Singh, et al., "ASync-Fusion: Asynchronous multi-sensor state estimation for autonomous robots," IEEE Robot. Autom. Lett., vol. 9, no. 4, pp. 3352–3359, 2024, doi: 10.1109/LRA.2024.3362458.
[3] S. Huang, Z. Wu, et al., "Delay-robust Kalman filtering for multi-rate sensor fusion with unknown input," IEEE Trans. Autom. Control, vol. 69, no. 2, pp. 1123–1130, 2024, doi: 10.1109/TAC.2023.3265893.
[4] L. Yang, Z. Zhang, et al., "Diffusion models: A comprehensive survey of methods and applications," ACM Comput. Surv., vol. 56, no. 4, Art. No. 105, pp. 1–34, 2024, doi: 10.1145/3626235.
[5] Y. Lin, X. Chen, et al., "Diffusion-based time series imputation with Transformer backbone," in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 36, 2023, pp. 58921–58935.
[6] Z. Li, T. Wang, et al., "NeRF-DS: Dynamic scenes with deformable neural radiance fields," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2023, pp. 12489–12498, doi: 10.1109/CVPR52729.2023.01234.
[7] W. Peebles, S. Xie, et al., "Scalable diffusion models with Transformers," in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), 2023, pp. 4172–4182, doi: 10.1109/ICCV51070.2023.01234.
[8] L. Caltagirone, S. Scheidegger, et al., "Uncertainty-aware multi-modal sensor fusion for robust autonomous navigation," IEEE Robot. Autom. Lett., vol. 8, no. 9, pp. 5679–5686, 2023, doi: 10.1109/LRA.2023.3265896.
[9] T. Salimans, J. Ho, et al., "Progressive distillation for fast sampling of diffusion models," arXiv:2302.05408, 2023.
[10] D. Gehrig, M. Muglikar, et al., "Asynchronous vision-based navigation with event cameras and IMU," IEEE Robot. Autom. Lett., vol. 9, no. 1, pp. 115–122, 2024, doi: 10.1109/LRA.2024.3362457.
[11] S. Wang, Y. Lin, et al., "Physics-informed diffusion models for dynamical system reconstruction," arXiv:2403.09156, 2024.