机器视觉辅助机器人焊接全过程研究进展
Research Progress on the Application of Machine Vision in the Whole Process of Robot Welding
- 2025年55卷第9期 页码:1-18
收稿日期:2024-07-18,
修回日期:2024-09-24,
纸质出版日期:2025-09-20
DOI: 10.7512/j.issn.1001-2303.2025.09.01
移动端阅览
浏览全部资源
扫码关注微信
收稿日期:2024-07-18,
修回日期:2024-09-24,
纸质出版日期:2025-09-20
移动端阅览
机器视觉在机器人焊接中可以使整个焊接过程图像化,并具有较高的扫描精度、较快的检测速度、较好的场景适应性等优点,能结合焊接机器人的可编程性和数字化应用,提高机器人本体的灵活性,增强焊接机器人的场景适应能力。针对机器视觉辅助焊接全过程包括焊前焊缝识别、焊接过程焊缝跟踪、熔池检测以及焊后缺陷检测,总结了基于机器视觉的智能焊接机器人的研究进展,分析了机器视觉在弧焊、激光焊及电阻点焊机器人等不同焊接场景中的应用实际,阐述了机器视觉在不同算法及视觉硬件下的焊接应用效果;最后分析了机器视觉在机器人焊接领域包括虚拟技术与视觉相机的结合、焊接图像处理算法的优化、多机器人协作焊接平台的搭建、多样化传感器信息的融合以及焊接参数自主学习优化等未来应用。
Machine vision in robot welding can visualize the entire welding process and has advantages such as high scanning accuracy
fast detection speed
and good scene adaptability. By combing the programmability and digital application of welding robots to improve the flexibility of the robot body and enhance the scene adaptability of welding robots. The research progress of intelligent welding robots based on machine vision for the entire welding process
including pre weld seam recognition
welding seam tracking
melt pool detection
and post weld defect detection were summarized. The practical application of machine vision in different welding scenarios such as arc welding
laser welding
and resistance point welding robots
and elaborates on the welding application effects of machine vision under different algorithms and visual hardware were also analyzed. Finally
the future applications of machine vision in the field of robot welding were given
including the combination of virtual technology and visual cameras
optimization of welding image processing algorithms
construction of multi robot collaborative welding platforms
fusion of diverse sensor information
and autonomous learning optimization of welding parameters.
王文楷 , 石玗 , 张刚 , 等 . 基于激光视觉检测的焊缝轨迹离线规划 [J]. 电焊机 , 2023 , 53 ( 9 ): 55 - 60 .
WANG W K , SHI Y , ZHANG G , et al . Weld seam trajectories offline planning based on laser vision sensing [J]. Electric Welding Machine , 2023 , 53 ( 9 ): 55 - 60 .
Liu S , Qiu Z C , Zhang X M . Singularity and path-planning with the working mode conversion of a 3-DOF 3-RRR planar parallel manipulator [J]. Mechanism and Machine Theory , 2017 , 107 : 166 - 182 .
李鹤喜 , 石永华 , 王国荣 . 采用SVR-雅可比估计器的焊接机器人视觉导引 [J]. 华南理工大学学报(自然科学版) , 2013 , 41 ( 7 ): 19 - 25 .
LI H X , SHI Y H , WANG G R . Visual guidance of welding robot using SVR-Jacobian estimator [J]. Journal of South China University of Technology(Natural Science Edition) , 2013 , 41 ( 7 ): 19 - 25 .
LV X Q , Gu D X , Wang Y D , et al . Feature extraction of welding seam image based on laser vision [J]. IEEE Sensors Journal , 2018 , 18 ( 11 ): 4715 - 4724 .
Tian Y Z , Liu H F , Li L , et al . Automatic identification of Multi-Type weld seam based on vision sensor with Silhouette-Mapping [J]. IEEE Sensors Journal , 2020 , 21 ( 4 ): 5402 - 5412 .
Lu Z F , Fan J F , Hou Z X , et al . Automatic 3D seam extraction method for welding robot based on monocular structured light [J]. IEEE Sensors Journal , 2021 , 21 ( 14 ): 16359 - 16370 .
Liu H F , Tian Y Z , Li L , et al . One-shot,integrated positioning for welding initial points via co-mapping of cross and parallel stripes [J]. Robotics and Computer-Integrated Manufacturing , 2023 , 84 : 102602 .
Ou J , Zou L , Wan Q H , et al . Weld-seam identification and model reconstruction of remanufacturing blade based on three-dimensional vision [J]. Advanced Engineering Informatics , 2021 , 49 : 101300 .
Maiwald F , Roider C , Schmidt M , et al . Optical coherence tomography for 3D weld seam localization in Absorber-Free laser transmission welding [J]. Applied Sciences , 2022 , 12 ( 5 ): 2718 .
Tran C C , Lin C Y . An intelligent path planning of welding robot based on multisensor interaction [J]. IEEE Sensors Journal , 2023 , 23 ( 8 ): 8591 - 8604 .
Kim J W , Jeong J L , Chung M Y , et al . Multiple weld seam extraction from RGB-depth images for automatic robotic welding via point cloud registration [J]. Multimedia Tools and Applications , 2020 , 80 ( 6 ): 9703 - 9719 .
Li R , Gao H M . Denoising and feature extraction of weld seam profiles by stacked denoising autoencoder [J]. Welding in the World , 2021 , 65 ( 9 ): 1725 - 1733 .
Chen Y K , Shi Y H , Cui Y X , et al . Narrow gap deviation detection in Keyhole TIG welding using image processing method based on Mask-RCNN model [J]. The International Journal of Advanced Manufacturing Technology , 2021 , 112 : 2015 - 2025 .
He Y S , Yu Z H , Li J , et al . Discerning weld seam profiles from strong arc background for the robotic automated welding process via visual attention features [J]. Chinese Journal of Mechanical Engineering , 2020 , 33 ( 1 ): 21 .
Zhen Y , Fang G , Chen S B , et al . A robust algorithm for weld seam extraction based on prior knowledge of weld seam [J]. Sensor Review , 2013 , 33 ( 2 ): 125 - 133 .
巩书青 , 章国宝 , 朱宏伟 . 基于核相关滤波器的强噪声焊缝跟踪 [J]. 激光技术 , 2024 : 1 - 13 .
GONG S Q , ZHANG G B , ZHU H W . A weld seam tracking method under strong noise based on kernel correlationfilters [J]. Laser Technology , 2024 : 1 - 13 .
Zhang Z H , Zhang Y , Chen C , et al . Seam detection method design and automatic calibration based on an active light coaxial visual sensor [J]. Optics and Lasers in Engineering , 2023 , 160 : 107304 .
Li W M , Mei F , Hu Z , et al . Multiple weld seam laser vision recognition method based on the IPCE algorithm [J]. Optics & Laser Technology , 2022 , 155 : 108388 .
He Y S , Chen Y X , Xu Y L , et al . Autonomous detection of weld seam profiles via a model of Saliency-based visual attention for robotic arc welding [J]. Journal of Intelligent & Robotic Systems , 2016 , 81 ( 3-4 ): 395 - 406 .
Shao W J , Huang Y , Zhang Y . A novel weld seam detection method for space weld seam of narrow butt joint in laser welding [J]. Optics & Laser Technology , 2018 , 99 : 39 - 51 .
Wang X G , Chen T Y , Wang Y M , et al . The 3D narrow butt weld seam detection system based on the binocular consistency correction [J]. Journal of Intelligent Manufacturing , 2022 , 34 ( 5 ): 2321 - 2332 .
Zhang B R , Shi Y H , Gu S Y . Narrow-seam identification and deviation detection in keyhole deep-penetration TIG welding [J]. The International Journal of Advanced Manufacturing Technology , 2019 , 101 ( 5-8 ): 2051 - 2064 .
Rout A , Deepak B , Biswal B B , et al . Weld seam detection, finding, and setting of process parameters for varying weld gap by the utilization of laser and vision sensor in robotic arc welding [J]. IEEE Transactions on Industrial Electronics , 2022 , 69 ( 1 ): 622 - 632 .
Huang W , Kovacevic R . Development of a real-time laser-based machine vision system to monitor and control welding processes [J]. The International Journal of Advanced Manufacturing Technology , 2012 , 63 ( 1-4 ): 235 - 248 .
Liu J P , Jiao T , Li S , et al . Automatic seam detection of welding robots using deep learning [J]. Automation in Construction , 2022 , 143 : 104582 .
周海宇 , 刘兴德 , 蔚芳鑫 , 等 . 基于双目视觉的L型板几何参数测量研究 [J]. 吉林化工学院学报 , 2022 , 39 ( 3 ): 83 - 87 .
ZHOU H Y , LIU X D , WEI F X , et al . Research on geometric parameter measurement of L-shaped plate based on binocular vision [J]. Journal of Jilin Institute of Chemical Technology , 2022 , 39 ( 3 ): 83 - 87 .
Lei Y L , Zhang L , Deng H S , et al . Multi-Layer welding path correction of Medium-Thick plate based on vision system [J]. IEEE Access , 2023 , 11 : 104520 - 33 .
王飞 , 盛仲曦 , 陈弈 , 等 . 基于WebGL的焊接机器人仿真及多层多道路径规划 [J]. 焊接学报 , 2023 , 44 ( 1 ): 27 - 32 .
WANG F , SHENG Z X , CHEN Y , et al . Welding robot simulation and multi-layer and multi-channel path planning based on WebGL [J]. Transactions of the China Welding Institution , 2023 , 44 ( 1 ): 27 - 32 .
杨书娟 , 蒋毅 , 俞建峰 , 等 . 激光视觉引导的管道焊缝识别与跟踪系统 [J]. 激光与光电子学进展 , 2022 , 59 ( 16 ): 245 - 253 .
YANG S J , JIANG Y , YU J H , et al . Laser visual guided pipeline weld seam identification and tracking system [J], Laser & Optoelectronics Progress , 2022 , 59 ( 16 ): 245 - 253 .
Gao J , Li F , Zhang C , et al . A method of D-Type weld seam extraction based on point clouds [J]. IEEE Access , 2021 , 9 : 65401 - 65410 .
Fang W H , Xu X L , Tian X C . A vision-based method for narrow weld trajectory recognition of arc welding robots [J]. The International Journal of Advanced Manufacturing Technology , 2022 , 121 ( 11-12 ): 8039 - 8050 .
杨秀烨 , 方金祥 , 何鹏 . 自动焊接传感技术研究现状及发展趋势 [J]. 材料导报 , 2022 , 36 ( 3 ): 160 - 167 .
YANG X Y , FANG J X , HE P . Research status and development trend of automatic welding sensor technology [J]. Materials Reports , 2022 , 36 ( 3 ): 160 - 167 .
王中任 , 王小刚 , 刘德政 , 等 . 基于焊枪轮廓特征提取的焊接偏差测定方法 [J]. 焊接学报 , 2020 , 41 ( 7 ): 59 - 64+100 .
WANG Z R , WANG X G , LIU D Z , et al . Welding deviation measurement method based on welding torch contour feature extraction [J]. Transactions of the China Welding Institution , 2020 , 41 ( 7 ): 59 - 64+100 .
Ma H B , Wei S C , Sheng Z X , et al . Robot welding seam tracking method based on passive vision for thin plate closed-gap butt welding [J]. The International Journal of Advanced Manufacturing Technology , 2010 , 48 ( 9-12 ): 945 - 53 .
Shen H Y , Lin T , Chen S B , et al . Real-Time seam tracking technology of welding robot with visual sensing [J]. Journal of Intelligent & Robotic Systems , 2010 , 59 ( 3-4 ): 283 - 298 .
Xu Y L , Yu H W , Zhong J Y , et al . Real-time seam tracking control technology during welding robot GTAW process based on passive vision sensor [J]. Journal of Materials Processing Technology , 2012 , 212 ( 8 ): 1654 - 1662 .
Ye Z , Fang G , Chen S B , et al . Passive vision based seam tracking system for pulse-MAG welding [J]. The International Journal of Advanced Manufacturing Technology , 2013 , 67 ( 9-12 ): 1987 - 1996 .
Lu J , Yang A D , Chen X Y , et al . A seam tracking method based on an image segmentation deep convolutional neural network [J]. Metals , 2022 , 12 ( 8 ): 1365 .
Wang Q Y , Jiao W H , Wang P , et al . Digital Twin for Human-Robot Interactive Welding and Welder Behavior Analysis [J]. IEEE/CAA Journal of Automatica Sinica , 2021 , 8 ( 2 ): 334 - 343 .
谢忠奇 , 丁超 , 吴涛 , 等 . 基于被动视觉的窄间隙管道焊熔池特征分析 [J]. 热加工工艺 , 2024 , 53 ( 9 ): 18 - 22 .
XIE Z Q , DING C , WU T , et al . Analysis on weld pool characteristics of narrow gap pipe welding pool based on passive vision [J]. Hot Working Technology , 2024 , 53 ( 9 ): 18 - 22 .
郑太雄 , 黄帅 , 李永福 , 等 . 基于视觉的三维重建关键技术研究综述 [J]. 自动化学报 , 2020 , 46 ( 4 ): 631 - 652 .
ZHENG T X , HUANG S , LI Y F , et al . Key techniques for vision based 3D reconstruction:a review [J]. Acta Automatica Sinica , 2020 , 46 ( 4 ): 631 - 652 .
王仕仙 . 深度视觉技术在焊接轨迹识别中的研究综述 [J]. 装备制造技术 , 2023 ( 4 ): 309 - 312+316 .
WANG S X . A Summary of the Research on depth Vision Technology in Welding track recognition [J]. Equipment Manufacturing Technology , 2023 ( 4 ): 309 - 312+316 .
Du R Q , Xu Y L , Hou Z , et al . Strong noise image processing for vision-based seam tracking in robotic gas metal arc welding [J]. The International Journal of Advanced Manufacturing Technology , 2018 , 101 ( 5-8 ): 2135 - 2149 .
Ma Y K , Fan J F , Deng S , et al . Efficient and accurate start point guiding and seam tracking method for curve weld based on structure light [J]. IEEE Transactions on Instrumentation and Measurement , 2021 , 70 : 1 - 10 .
Fan J F , Deng S , Ma Y K , et al . Seam Feature Point Acquisition Based on Efficient Convolution Operator and Particle Filter in GMAW [J]. IEEE Transactions on Industrial Informatics , 2021 , 17 ( 2 ): 1220 - 1230 .
Liu C F , Shen J Q , Hu S S , et al . Seam tracking system based on laser vision and CGAN for robotic multi-layer and multi-pass MAG welding [J]. Engineering Applications of Artificial Intelligence , 2022 , 116 : 105377 .
Yang L , Fan J F , Huo B Y , et al . Image denoising of seam images with deep learning for laser vision seam tracking [J]. IEEE Sensors Journal , 2022 , 22 ( 6 ): 6098 - 6107 .
Xiao R Q , Xu Y L , Xu F J , et al . LSFP-Tracker: An autonomous laser stripe feature point extraction algorithm based on siamese network for robotic welding seam tracking [J]. IEEE Transactions on Industrial Electronics , 2024 , 71 ( 1 ): 1037 - 1048 .
黄健康 , 潘伟 , 孙天亮 , 等 . 不锈钢/碳钢TIG焊熔池表面流动行为 [J]. 焊接学报 , 2019 , 40 ( 8 ): 18 - 25 .
HUANG J K , PAN W , SUN T L , et al . Flow behavior of stainless steel/carbon steel TIG welding pool surface [J]. Transactions of the China Welding Institution , 2019 , 40 ( 8 ): 18 - 25 .
梁志敏 , 高旭 , 任政 , 等 . 基于变分立体匹配算法的GMAW熔池形貌三维重建 [J]. 焊接学报 , 2024 , 45 ( 2 ): 61 - 66+132 .
LIANG Z M , GAO X , REN Z , et al . Three-dimensional reconstruction of GMAW weld pool appearance based on variational stereo matching algorithm [J]. Transactions of the China Welding Institution , 2024 , 45 ( 2 ): 61 - 66+132 .
韩庆璘 , 李大用 , 李鑫磊 . 基于分区减光的电弧增材制造熔敷道尺寸主被动联合视觉检测 [J]. 焊接学报 , 2020 , 41 ( 9 ): 28 - 32 .
HAN Q L , LI D Y , LI X L . Bead geometry measurement for wire and arc additive manufacturing using active-passive composite vision sensing based on regional filter [J]. Transactions of the China Welding Institution , 2020 , 41 ( 9 ): 28 - 32 .
陈宸 , 周方正 , 李成龙 , 等 . 融合空间和通道特征的等离子弧焊熔池熔透状态预测方法 [J]. 焊接学报 , 2023 , 44 ( 4 ): 30 - 38+131 .
CHEN C , ZHOU F Z , LI C L , et al . Prediction method of plasma arc welding molten pool melting state based on spatial and channel characteristics [J]. Transactions of the China Welding Institution , 2023 , 44 ( 4 ): 30 - 38+131 .
Wang Z Z . Monitoring of GMAW weld pool from the reflected laser lines for Real-Time control [J]. IEEE Transactions on Industrial Informatics , 2014 , 10 ( 4 ): 2073 - 2083 .
Guo B , Shi Y H , Yu G Q , et al . Weld deviation detection based on wide dynamic range vision sensor in MAG welding process [J]. The International Journal of Advanced Manufacturing Technology , 2016 , 87 ( 9-12 ): 3397 - 410 .
Jiang C , Zhang F B , Wang Z M . Image processing of aluminum alloy weld pool for robotic VPPAW based on visual sensing [J]. IEEE Access , 2017 , 5 : 21567 - 21573 .
Yu R W , Han J , Zhao Z , et al . Real-Time Prediction of Welding Penetration Mode and Depth Based on Visual Characteristics of Weld Pool in GMAW Process [J]. IEEE Access , 2020 , 8 : 81564 - 81573 .
Liu Y K , Zhang Y M . Supervised learning of human welder behaviors for intelligent robotic welding [J]. IEEE Transactions on Automation Science and Engineering , 2017 , 14 ( 3 ): 1532 - 1541 .
Jiao Z Q , Qin H J , Gao X Y , et al . Image Processing and feature extraction for hull structure GMAW based on weld pool visual sensing [J]. Journal of Sensors , 2023 , 2023 : 1 - 23 .
Feng Y H , Chen Z Y , Wang D L , et al . DeepWelding: A deep learning enhanced approach to GTAW using multisource sensing images [J]. IEEE Transactions on Industrial Informatics , 2020 , 16 ( 1 ): 465 - 474 .
胡继涛 , 马晓锋 , 赵荣丽 , 等 . 基于轻量级DeepLabV3+网络的焊接熔池图像分割方法 [J]. 计算机集成制造系统 , 2024 : 1 - 17 .
HU J T , MA X F , ZHAO R L , et al . Welding pool image segmentation method based on lightweight DeepLabV3+network [J]. Computer Integrated Manufacturing Systems , 2024 : 1 - 17 .
樊丁 , 胡桉得 , 黄健康 , 等 . 基于改进卷积神经网络的管焊缝X射线图像缺陷识别方法 [J]. 焊接学报 , 2020 , 41 ( 1 ): 7 - 11+97 .
FAN D , HU A D , HUANG J K , et al . X-ray image defect recognition method for pipe weld based on improved convolutional neural network [J]. Transactions of the China Welding Institution , 2020 , 41 ( 1 ): 7 - 11+97 .
魏明贤 , 邬冠华 , 王俊涛 , 等 . 激光焊缝无损检测技术研究现状 [J]. 无损探伤 , 2022 , 46 ( 1 ): 25 - 30 .
WEI M X , WU G H , WANG J T , et al . Research status of laser weld nondestructive testing technology [J]. Nondestructive Testing , 2022 , 46 ( 1 ): 25 - 30 .
Li Y , Li Y F , Wang Q L , et al . Measurement and defect detection of the weld bead based on online vision inspection [J]. IEEE Transactions on Instrumentation and Measurement , 2010 , 59 ( 7 ): 1841 - 1849 .
Liu C H , Chen S , Huang J Q . Machine Vision-Based object detection strategy for weld area [J]. Scientific Programming , 2022 , 2022 : 1 - 12 .
Jia N N , Li Z Y , Ren J L , et al . A 3D reconstruction method based on grid laser and gray scale photo for visual inspection of welds [J]. Optics & Laser Technology , 2019 , 119 : 105648 .
Park J K , An W H , Kang D J . Convolutional neural network based surface inspection system for Non-patterned welding defects [J]. International Journal of Precision Engineering and Manufacturing , 2019 , 20 ( 3 ): 363 - 74 .
王浩 , 赵小辉 , 徐龙哲 , 等 . 结构光视觉辅助焊接的轨迹识别与控制技术 [J]. 焊接学报 , 2023 , 44 ( 6 ): 50 - 57+132 .
WANG H , ZHAO X H , XU L Z , et al . Research on trajectory recognition and control technology of structured light vision-assisted welding [J]. Transactions of the China Welding Institution , 2023 , 44 ( 6 ): 50 - 57+132 .
张臣 , 胡佩佩 , 朱新旺 , 等 . 基于高密度点云的激光焊接缺陷智能在线检测(特邀) [J]. 中国激光 , 2024 , 51 ( 4 ): 85 - 96 .
ZHANG C , HU P P , ZHU X W , et al . Intelligent online detection of laser welding defects based on high density point Clouds(Invited) [J]. Chinese Journal of Lasers , 2024 , 51 ( 4 ): 85 - 96 .
南方 , 李大华 , 高强 , 等 . 改进Steger算法的自适应光条纹中心提取 [J]. 激光杂志 , 2018 , 39 ( 1 ): 85 - 88 .
NAN F , LI D H , GAO Q , et al . Implementation of adaptive light stripe center extraction of improved Steger algorithm [J]. Laser Journal , 2018 , 39 ( 1 ): 85 - 88 .
夏攀 , 马飞 , 王中任 . 基于改进U-Net模型的焊缝特征提取 [J]. 激光与红外 , 2022 , 52 ( 8 ): 1259 - 1264 .
XIA P , MA F , WANG Z R . Weld feature extraction based on modified U-Net model [J]. Laser& Infrared , 2022 , 52 ( 8 ): 1259 - 1264 .
苏志威 , 黄子涵 , 邱发生 , 等 . 基于改进YOLOv8的航空铝合金焊缝缺陷检测方法 [J]. 航空动力学报 , 2024 : 1 - 10 .
SU Z W , HUANG Z H , QIU F S , et al . Weld defect detection of Aviation Aluminum alloy based on improved YOLOv8 [J]. Journal of Aerospace Power , 2024 : 1 - 10 .
郭波 , 石永华 . 基于视觉传感的CO2焊焊接偏差识别 [J]. 华南理工大学学报(自然科学版) , 2016 , 44 ( 7 ): 1 - 8 .
GUO B , SHI Y H . Vision-Based welding deviation detection for CO2 arc welding [J]. Journal of South China
University of Technology(Natural Science Edition) , 2016 , 44 ( 7 ): 1 - 8 .
Wang J Y , Mu C Y , Mu S , et al . Welding seam detection and location:Deep learning network-based approach [J]. International Journal of Pressure Vessels and Piping , 2023 , 202 ( 04 ): 104893 .
Li Hao , Wang X C , Liu Y , et al . A novel Robotic-Vision-Based defect inspection system for bracket weldments in a Cloud-Edge coordination environment [J]. Sustainability , 2023 , 15 ( 14 ): 10783 .
Zhu C X , Yuan H T , Ma G H . An active visual monitoring method for GMAW weld surface defects based on random forest model [J]. Materials Research Express , 2022 , 9 ( 3 ): 036503 .
胡丹 , 吕波 , 王静静 , 等 . 焊缝表面缺陷激光视觉传感HOG-SVM的检测方法 [J]. 焊接学报 , 2023 , 44 ( 1 ): 57 - 62+70 .
HU D , LV B , WANG J J , et al . Study on HOG-SVM detection method of weld surface defects using laser visual sensing [J]. Transactions of the China Welding Institution , 2023 , 44 ( 1 ): 57 - 62+70 .
王杰 , 张志芬 , 秦锐 , 等 . 基于机器学习的不锈钢薄板MIG焊焊穿缺陷识别 [J]. 电焊机 , 2023 , 53 ( 9 ): 70 - 77 .
WANG J , ZHANG Z F , QIN R , et al . Identification of MIG welding burn-through defects in stainless steel sheet based on machine learning [J]. Electric Welding Machine , 2023 , 53 ( 9 ): 70 - 77 .
相关作者
相关机构