Automatic location and defect identification of weld defects for large-scale components are the necessary condition to realize the automatic operations of weld grinding and repairing. Due to the characteristics of large-scale component weld and weld defect images, such as shape diversity and random gray distribution, the difficulty of image processing is increased. A method of weld location and defect recognition based on deep learning was proposed. The weld position was determined and the weld bead and unqualified defects were identified through the deep learning target detection method. The gas pore and pit defects were identified by the deep learning semantic segmentation method. The weld location and defect identification models were created and trained based on the FPN network structure, and the model optimization was completed by increasing the number of samples. The accuracy rate of weld location is 95%, the identification accuracy rate of weld bead is 98%, and the identification accuracy rate of gas pore and pit is about 91.8%.