Authors:
Pengkun Wei、Shuo Cheng、Dayou Li、Ran Song、Yipeng Zhang、Wei Zhang
Paper:
https://arxiv.org/abs/2408.10710
Introduction
Background
The automation of welding processes has become increasingly significant in industrial applications, driven by the need for efficiency and precision. Welding robots, as reported by the International Federation of Robotics (IFR), hold the second-largest market share among industrial robots globally. Despite advancements in in-welding monitoring and post-welding quality inspection, the pre-welding stage, particularly the recognition and localization of weld seams, remains a critical challenge. Traditional methods often focus on detecting single weld seams, which is time-consuming and inefficient for complex workpieces with multiple seams.
Problem Statement
The primary challenge addressed in this study is the efficient detection of multiple weld seams while maintaining sub-millimeter accuracy. Existing methods typically require multiple manual demonstrations and have limited fields of view, leading to inefficiencies. This paper proposes a novel framework that leverages both RGB images and 3D point clouds to detect multiple weld seams simultaneously, enhancing both efficiency and accuracy.
Related Work
Laser Sensor-Based Methods
Laser sensors are widely used in weld seam detection due to their high precision. However, their narrow perception range limits their application to single weld seams, reducing overall efficiency. Various methods have been proposed to enhance the accuracy of laser-based detection, including grayscale image feature extraction, self-updating template matching, and deep neural network-based feature extraction. Despite their precision, these methods are constrained by their limited field of view.
Other Sensor-Based Methods
Alternative approaches using RGB-D cameras and tactile exploration have been explored. RGB-D cameras offer a larger field of view, allowing for the detection of multiple weld seams. However, methods based on tactile exploration, while accurate, are impractical for industrial applications due to their prolonged exploration and computational times. The need for an efficient and accurate method to detect multiple weld seams in industrial settings remains pressing.
Research Methodology
Problem Statement
The proposed system takes an RGB image and a raw point cloud of the workpiece as input, acquired from the same shooting posture. The output is a robot-executable welding path with 6 degrees of freedom (DOF), represented by a set of points and poses in the world coordinate system.
Workpiece Edge Model
Weld seams are typically the junctions between metal plates, which can be flat or curved. The study models both linear and curved weld seams separately. Linear weld seams are represented by their starting and ending points in both the image and camera coordinate systems. Curved weld seams are modeled as a series of continuous edge segments, with their positions represented in both the image and camera coordinate systems.
Coarse Seam Detection Using VLM
The method filters out irrelevant point cloud data to focus on the regions containing weld seams. The Segment Anything Model (SAM) and its real-time variant, FastSAM, are used for coarse positioning. FastSAM segments the workpiece surfaces based on prompts, which can be manually or automatically generated. The intersection areas between surfaces indicate the approximate locations of weld seams, allowing for fine-grained point cloud extraction.
Fine Edge Extraction Based on Region-Growing
A region-growing algorithm is proposed for fine edge extraction from the point cloud data. The algorithm preprocesses the raw point cloud, segments the workpiece surfaces, and extracts the point cloud near the edges. The extracted edges are then fitted using the least squares method, and the welding path is generated based on the fitted edges and the calculated welding poses.
Experimental Design
Experimental Setup
The proposed method was validated using a UR5 robotic arm equipped with an RGB-D camera and a 3D-printed welding torch. Experiments were conducted on workpieces with both linear and curved weld seams to evaluate the system’s performance.
Effect of the Downsampling Parameter
Downsampling is used to reduce the density of point clouds, balancing accuracy and efficiency. The voxel grid downsampling method was employed, and experiments were conducted to determine the optimal grid size. The root mean square error (RMSE) was used to measure the accuracy of the generated welding paths compared to manually annotated ground truth points.
Quantitative Comparison
The proposed method was compared with existing methods in terms of efficiency and accuracy. The total time cost was divided into exploration time (data acquisition) and computation time (algorithm execution). The average time per weld seam was calculated to ensure a fair comparison. The effectiveness of the FastSAM-based coarse localization cropping method was also evaluated.
Application in Industry Scenario
The method was further validated in a real industrial scenario using a Fanuc M-10iD/8L robot arm. The task involved welding ribs to the main body of steel structures, which are commonly used in various fields such as bridges, buildings, and ships.
Results and Analysis
Downsampling Parameter Effect
The experiments demonstrated that setting the voxel grid size to 3 mm ensures sub-millimeter accuracy while maximizing efficiency. This balance is crucial for practical industrial applications where both precision and speed are required.
Quantitative Comparison
The proposed method outperformed existing methods in terms of average time per weld seam and RMSE. The inclusion of the FastSAM-based cropping module significantly reduced the number of point clouds processed and the computational time required.
Industrial Application
The method successfully identified and welded multiple seam types in a real industrial scenario, demonstrating its practical applicability and potential for widespread adoption in the industry.
Overall Conclusion
This paper presents a novel method for the coarse-to-fine detection of multiple weld seams in robotic welding. By leveraging RGB images and 3D point clouds, the proposed framework enhances both efficiency and accuracy. The method was validated through extensive experiments on various workpieces and in real industrial scenarios, showcasing its potential for practical applications. Future work will focus on reconstructing more complex weldments and further improving the detection and extraction of weld seams.
