Abstract: During welding processes, initial defects such as incomplete penetration and slag are easily generated. To ensure the safe operation of welding components, welded joints must be tested rigorously. Metal magnetic memory (MMM) technology, a new nondestructive testing in the 21st century, can detect macroscopic defects as well as early stress concentrations and hidden damages. However, the quantitative MMM testing is still a bottleneck for weld defects. To solve the bottleneck of quantitative inversion of weld defects by MMM testing, a quantitative inversion model was presented based on a support vector machine (SVM) method optimized with simulated annealing (SA) algorithm. Steel Q235 welded plate specimens, which were prefabricated with different sizes of incomplete penetration and slag defects, were tested. It is found that with the increase of weld damage degree, the peak-peak values of the tangential and normal magnetic field intensity exhibit nonlinear growth, as well as the change rates of the tangent and normal magnetic field intensity. In other words, the MMM feature parameters vary with the defect size, but the signals are scattered and uncertain. First, considering the finite, dispersive, and non-linear MMM signals, the MMM feature parameters data were normalized, and the MMM quantitative inversion model of weld defects was established based on SVM. Furthermore, the SVM parameters was optimized with SA so that the objective function of the model could reach the global optimal solution. Finally, considering the solution uncertainty when the three-dimensional sizes of weld defects were reversed from the MMM signals, a modified MMM multi-dimensional SVM inversion model was presented by constructing SVM multi-layer structures and optimized with SA. The results show that maximum inversion relative error of incomplete penetration defect size is 7.96%, and the defect of slag is 4.97%, which provides a new tool for quantitative MMM inversion and evaluation of weld defects.