Authors:
Miao Zhang、Sherif Abdulatif、Benedikt Loesch、Marco Altmann、Marius Schwarz、Bin Yang
Paper:
https://arxiv.org/abs/2408.06772
Introduction
The rapid advancement of deep learning technology has significantly enhanced autonomous driving systems, particularly in 3D perception using multimodal sensors. While cameras and lidar sensors are commonly used, they face challenges under adverse weather and varying illumination conditions. Radar sensors, however, offer greater robustness and cost-effectiveness in such scenarios. This study investigates the domain shift in 4D radar-based object detection, focusing on how different environmental conditions, such as weather patterns and road types, impact 3D object detection performance. The findings underscore the importance of diverse data collection to address domain shifts effectively.
Related Work
Weather Effect on Radar Signals
Radar sensors transmit electromagnetic waves to capture range, angle, and velocity information. Compared to lidar, radar is more robust in adverse weather conditions due to its longer wavelength. However, challenges like interference, attenuation, and backscattering can degrade signal quality. Previous studies have shown that rain and snow significantly impact radar signals, while conditions like dust storms have minimal effect. Most research focuses on radar signal characteristics, with less emphasis on performance gaps in deep learning-based perception.
4D Radar Datasets
The latest 4D radar sensors provide additional elevation angle information, enhancing traditional 3D data. However, comprehensive open-source 4D radar datasets are limited. Common datasets like Waymo and KITTI provide lidar point clouds, while others like NuScenes and RADIATE offer 3D radar point clouds. The K-Radar dataset, with over 35k frames from various weather conditions, is particularly suitable for investigating domain shifts.
Domain Shift on 3D Object Detection
Domain shift, a well-researched topic in tasks like semantic segmentation and 2D object detection, occurs when a model trained on one dataset performs poorly on a different dataset. While lidar-based 3D object detection has been extensively studied, the impact of domain shift on radar object detection remains less explored. This study aims to fill this gap by focusing on environmental conditions like weather and road types.
Methodology
Dataset
Two datasets were used for this study: the K-Radar dataset and a self-collected Bosch-Radar dataset. The K-Radar dataset includes point clouds from seven different weather conditions, while the Bosch-Radar dataset contains over 180k frames from eight cities in Germany, covering three weather conditions and three road types.
Implementation Details
Two representative 3D object detectors, RTNH and SECOND, were used to validate the domain shift effect. Models were trained with data from different domains and validated on a fixed test set. Detection performance was measured using the AP3D and APBEV metrics from the KITTI dataset. The experimental workflow is illustrated below:
Weather Domain Shift
Results on K-Radar
The K-Radar dataset was used to investigate the effect of weather domain shifts. Models were trained on normal weather data and a mixed dataset encompassing all weather conditions. The evaluation results revealed significant domain gaps across five weather conditions, with snow scenarios exhibiting the most severe performance degradation. The average power of the point cloud in snow conditions was significantly lower than in normal weather, likely due to sensor blockages caused by ice or snow.
Results on Bosch-Radar Dataset
The Bosch-Radar dataset was used to investigate domain shift effects across three weather conditions. The results showed a significant domain shift effect from rain to normal and overcast conditions. Models trained on normal or overcast conditions adapted well to rain patterns. Increasing the dataset size did not effectively mitigate the effects of domain shift.
Comparison
Comparing the domain gaps in the two datasets revealed that weather domain shift effects vary between different datasets. The K-Radar dataset experienced significant domain shift effects in all adverse weather scenarios except overcast, while the Bosch-Radar data only exhibited adverse effects when transitioning from adverse to favorable weather conditions.
Road Domain Shift
The impact of domain shift across various road types was explored using the Bosch-Radar dataset. The results showed a unidirectional domain shift from highway to urban/rural areas. The urban-trained model underperformed on rural data, while the mixed-trained model demonstrated strong generalization ability. Collecting data from diverse road types is crucial for maintaining strong performance in complex scenarios.
Limitations
This study has several limitations. The conclusions are based on empirical analysis, and a theoretical analysis of domain shift under varying environmental conditions is needed. The datasets lack data on highly adverse weather conditions, such as snow. Additionally, the study focuses on environmental domain shifts involving radar point cloud data and does not cover behavior in radar spectrum data.
Conclusion
This paper presents an empirical study on the domain shift effect in radar-based object detection under diverse environmental conditions. The findings reveal that domain shifts occur across different weather conditions and road types, with the effect varying between datasets. This study contributes to a deeper understanding of domain shifts in radar perception systems and provides valuable insights for optimizing data collection strategies. Future work will investigate domain adaptation techniques to address challenges posed by environmental variations.
