Multimodal Large Model Guided Diffusion Model for Transformer Oil Leakage Image Generation

Abstract

Transformer oil leakage is a critical defect in power equipment inspection, yet its automatic detection remains challenging because real leakage samples are scarce, highly irregular in shape, variable in color, and easily confused with water stains, rust, shadows, and complex structural backgrounds. To alleviate the data scarcity problem and improve downstream segmentation performance, this paper proposes a multimodal semantic-heuristic diffusion framework, termed MSH-Diff, for automatic transformer oil-leakage image synthesis. The proposed framework exploits the visual reasoning capability and prior industrial knowledge of a multimodal large language model to identify potential leakage-prone components, such as flanges, valves, and radiator fin roots, through expert-role prompt engineering. The recognized component semantics are further mapped to spatial Gaussian anchors by analyzing cross-attention responses in the latent diffusion model, enabling leakage-center localization without manually drawn masks or key-point annotations. In addition, MSH-Diff automatically constructs dense scene descriptions involving illumination, viewpoint, and surface material, which are encoded as semantic constraints to enhance structural consistency and physical realism during diffusion sampling. Experimental results demonstrate that MSH-Diff achieves competitive image quality and diversity, with an FID of 38.14 and an IC-L score of 0.29. When the generated samples are incorporated into downstream semantic segmentation training, the mIoU of DeepLabV3+ increases from 60.09% to 62.31%, confirming the effectiveness of the proposed framework for industrial defect data augmentation.