|
|
1.INTRODUCTIONAs a crucial non-destructive testing device, Scanning Acoustic Microscope (SAM) can sensitively detect the morphological features and discontinuous defects inside the package without damaging the tested materials and components, and can effectively help the inspectors judge the quality and reliability of IC1. However, the above detection mode has the following defects:
At present, the IC internal morphology recognition and defect detection algorithms based on image processing and machine learning are impeded by the lack of SAM image resources. Therefore, we developed a Multi-category Scanning Acoustic Image (MSAI) dataset of SAM to aid the researchers in their training and evaluation processes. MASI dataset provides a relatively effective method to label the image data, and by introducing four typical convolutional neural networks (CNN), we have conducted the performance assessment and verification analysis of MASI database. This paper consists of two parts. In the first part, we introduce the procedure of developing MSAI, including image acquisition module, methods of original image processing and information annotation. In the second part, we evaluate this database with CNN for classification and usability testing. 2.THE MULTI-CATEGORY SCANNING ACOUSTIC IMAGE DATASET (MSAI)SAM image is a kind of data pattern that can reflect the internal morphology and defect location of IC. Therefore, the experimental design to obtain SAM images must meet the following conditions:
Based on these analyses, we designed a design procedure including image acquisition, image processing and image annotation to build MSAI database. 2.1IC sample descriptionThis paper mainly collects SAM images for IC of four packaging types, namely Ball Grid Array (BGA), Quad Flat No-leads (QFN), Small Outline Integrated Circuit (SOIC) and Small Outline Package (SOP). The above four IC packages are common electronic component types, and exist widely in all kinds of electronic products. The SAM images of BGA, QFN, SOIC and SOP can not only identify the morphological features of IC, but also provide a data base for defect detection algorithms and failure analysis. 2.2The analysis of MSAI database2.2.1IC Samples Description.In this paper, SAM images of four package types (BGA, QFN, SOIC and SOP) are collected, and each packaging mode contains 13 different products. Although the sample quantity for each product is different, the total sample number achieves a balance between each package type. In order to ensure the availability of the database, the data distribution of MASI dataset on the four packaging types is relatively average, and there is little difference in sample size between classes, that is, the BGA package contains 1147 samples, the QFN package contains 1231 samples, the SOIC package contains 930 samples, and the SOP package contains 1257 samples. In addition, the datasets with good classification effect often have the following two characteristics: (1) The similarity between classes objects (inter-class similarity) as small as possible; (2) The similarity of objects within the class (intra-class difference) as large as possible. Figure 1 presents the SAM images of different products in four packaging types, the horizontal axis shows the different product images from same packaging type, and the vertical axis shows the SAM images of different packaging types. 2.2.2Original Image Processing.In this paper, due to the design conditions and detection requirements of SAM equipment, the images of all layers are integrated to a TIFF comprehensive file. Furthermore, according to the scanning requirements, researchers place all samples into one tray for scanning, and obtain a grayscale image of an entire tray. Using the above initial image for recognition is bound to cause difficulty in model learning and low detection accuracy. Therefore, this paper will carry out two steps for image pre-processing: (1) The original TIFF file is converted into image sequence, and the C-scan image of each layer is extracted; (2) The image of substrate and lead frame is extracted and segmented to generate multiple single sample images. Figure 2a shows the original image collected by the SAM device, and Figure 2b shows the segmented image cluster. 2.2.3Category Labelling.Accurate and succinct labels are the basis of image recognition. Package type annotation of SAM images is one of the inevitable steps for recognition. Correct label information is the key to generate stable and reliable MSAI datasets. The criteria for labelling the packaging type were mainly based on IC product information. However, the product information provided by sample suppliers may be incomplete or missing, we have to take into account the comments of inspectors when labelling the package type. Two well-trained coders were involved in the analysis of labelling MSAI datasets. We processed the SAM images recordings in following steps: (1) The first step is to tag the SAM images by manufacturer identification information. This procedure was to collect the product information of the tested sample at the early stage of image acquisition, and indicate the packaging type of each product; (2) The second step is to let two well-trained inspectors mark the packaging type of all samples without knowing the product information; (3) The third step is to compare the above two labelling results, if the label information is the same, the packaging type of an IC product can be identified. 2.2.4.Profile of the Database.The developed database called MSAI contains 4565 SAM images from 52 IC products, and these samples are labelled with four packaging types, as listed in Table 1. The number of samples for each product is also shown in table 2, and the MSAI database has the following characteristics:
Table 1.Data distribution of MSAI.
Table 2.Architectures comparison: AlexNet, VGG-16, ResNet-101 and Inception-v4.
3.DATABASE EVALUATION, EXPERIMENT AND DISCUSSION3.1Methods of deep learningIn recent years, CNN has become one of the most popular deep learning frameworks8. It has the advantages of strong feature extraction ability, high recognition accuracy and outstanding feasibility. It also performs well in image related tasks, such as image classification, image retrieval and object detection9-11. In order to verify the separability and practicability of MSAI database, this paper used the above CNN to recognize SAM images, and Table 2 describes architecture characteristics of these CNN models. 3.2Experimental set-up and evaluation criteriaIn this paper, we described a new SAM image database that includes four packaging types and carried out classification experiments on it. To evaluate MSAI database, we used four CNN (AlexNet, VGG-16, ResNet-101 and Inception-v4) models mentioned in Table 2 to extract images features and classify these SAM images. Here, we used Leave-One-Image-Out (LOIO) cross-validation, i.e., in each fold, one SAM image was used as the test set and the others were used as the training set. After the analysis of 4564 folds, each sample had been used as the test set once, and the final recognition accuracy was calculated based on all results. This protocol was applied independently to each of the CNN models available. In order to train and test our network model and database, a training platform based on deep learning is built in this paper, as shown in Table 3. In addition, the learning rate of neural network model = le-3, mini batches = 16. Table 3.Deep learning configuration.
3.3Results of CNN methodIn this paper, we compare four CNN models (AlexNet, VGG-16, ResNet-101 and Inception-v4) for SAM image recognition on the MSAI database. Ideally, a robust model must classify accurately, regardless of whether image pre-processing is used or not. We analyze the model’s behavior by comparing the recognition accuracy between pre-processed and raw images. Table 4 shows the recognition accuracy of the original SAM image and CLAHE pre-processed image on different network models. Table 4.Accuracy of the models with/without pre-processing.
The experimental results show that compared with the two deeper network models of Inception-v4 and ResNet-101, AlexNet and VGG-16 get more optimal results and faster convergence speed. We also observed from the experimental results that the pre-processed images will not enhance the model prediction ability of AlexNet and VGG-16, indicating that the above two types of models have good fitting ability and generalization ability. The evaluation index in Table 5 further proves that the network models with relatively simple structure such as AlexNet and VGG-16 are more suitable for MSAI database. It is mainly because of the sample size of MSAI dataset was less prone to produce over-fitting results on models with few levels. However, networks with complex structure and strong expression ability tend to focus on interpreting training data at the expense of the description ability of future testing data, resulting in low prediction accuracy. Table 5Performance Comparison Between four CNN models.
Figure 3 shows the confusion matrices of two trained models (AlexNet and VGG-16) on raw testing sets, respectively. The result shows that the above two network models have good generalization ability in four packaging structures. 4.CONCLUSIONSIn this paper, we first create a novel Multi-category Scanning Acoustic Image (MSAI) dataset based on four IC packaging structures (BGA, QFN, SOIC and SOP), including 4565 SAM images of 52 products, which not only making up the deficiencies of database, but also provides a data basis for IC intelligent detection algorithm. Then, in order to verify the availability of MSAI dataset, this paper uses four typical network models: AlexNet, VGG-16, ResNet-101 and Inception-v4 to identify and classify the SAM images in MSAI dataset. Finally, according to the experimental results, AlexNet and VGG-16 network show high training accuracy and test accuracy in the packaging structure recognition experiment. ACKNOWLEDGMENTSThis work was supported in part by Guangdong Basic and Applied Research Foundation (Projects Numbers: 2021A1515110939). REFERENCESHsieh, M. C., Kang, K. T. and Choi, H. C.,
“Thin profile flip chip package-on-package development,”
in 11th Inter. Microsystems, Packaging, in Assembly and Circuits Technology Conf. (IMPACT),
143
–147
(2016). Google Scholar
Altmann, F. and Petzold, M.,
“Innovative failure analysis techniques for 3D packaging developments,”
IEEE Design & Test, 33
(3), 46
–55
(2016). https://doi.org/10.1109/MDAT.6221038 Google Scholar
Dong, C. Y. and Lee, K. Y.,
“Stress analysis of plastic IC package containing various interface desalinations using the boundary element method,”
Engineering Fracture Mechanics, 75
(1), 1
–16
(2008). https://doi.org/10.1016/j.engfracmech.2007.03.019 Google Scholar
Radhakrishnan, D. M. K. and He, J. Q.,
“Advanced package technologies for high-performance systems,”
Intel. Technology Journal, 9
(4), 259
–272
(2005). Google Scholar
Mehr, M. Y., Bahrami, A., et al.,
“An overview of scanning acoustic microscope, a reliable method for non-destructive failure analysis of microelectronic,”
in EuroSimE,
1
–4
(2015). Google Scholar
Chen, G. L., Yue, G. Q. and Hu, S. Z.,
“Research on SAM and its image processing,”
Acta Electronica Sinica, 60
(7), 7
–11
(1996). Google Scholar
Lemons, R. A. and Quate, C. F.,
“Acoustic microscope—Scanning version,”
Applied Physics Letters, 24
(4), 163
–165
(1974). https://doi.org/10.1063/1.1655136 Google Scholar
Li, X. J., Zhang, D. S. and Sun, L. L.,
“Light flame Detection method based on CNN in complex scene,”
Pattern Recognition and Artificial Intelligence, 34
(5), 415
–422
(2021). Google Scholar
Srinivas, A., Lin, T. Y. and Parmar, N.,
“Bottleneck transformers for visual recognition,”
in Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition,
16519
–16529
(2021). Google Scholar
Zhao, M., Ge, Y. L. and Ding, N.,
“Object detection in complex environment based on CenterNet algorithm,”
Journal of China Academy of Electronics Science, 16
(7), 654
–660
(2021). Google Scholar
Ge, Z., Liu, S. and Li, Z.,
“OTA: Optimal transport assignment for object detection,”
in Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition,
303
–312
(2021). Google Scholar
|
