3.1

Design of D-Res module

PP-YOLOE+ adopts an multi-scale feature map with three different scales to perform multi-scale feature integration which can adapt to targets of different sizes and have better detection effect on traditional natural images.However, due to the randomness of passenger luggage placement, after X-ray scanning, the contraband in the image will appear as multi-scale and multi-pose features, while ordinary items may be mistaken as contraband due to mutual occlusion and overlap between items.

To handle the above problem, we introduce the deformable convolutional DCNv2 that has modulation mechanism into the backbone network in this paper.Compared with the traditional convolution operation, DCNv2 can handle the spatial variation of target objects flexibly for improving the detection and recognition competence of the model. Since there is no fixed theoretical description for inserting deformable convolution in the network, the DBS module and the D-Res Block module are designed in this paper, and the D-Res module is obtained after replacing the backbone network CSPRes module, as shown in Figure 2. Each DBS module consists of DCNv2, BN and SiLU activation functions, for which n refers to the convolutional kernel size of DCNv2 and m refers to the sampling steps of the convolutional kernels.

Figure 2.

Design of D-Res module

3.2

Coordinate attention

Since security screening images are different from those in natural scenes, there is a large amount of noise and complex background information.This redundant information can hinder the overall network’s representational capability and affect the model’s overall detection of contraband.The coordinated attention mechanism is a method that can capture both channel relevance and spatial relevance in the feature map, and can adaptively adjust the weight relationship between different channels and different locations, so that the channels and locations with important information can receive more attention.It is shown that the coordination attention mechanism embedded into the backbone network and feature fusion network, shown on the CA module marked by the red solid line box in Figure 1.

The CA mechanism decomposes channel attention that captures features along two spatial directions from two one-dimensional feature encoding processes, one of which is responsible for capturing remote-dependent features and the other retains precise location information.The architecture is shown in Figure 3.

Figure 3.

Architecture of the coordination of attention mechanism

The CA attention mechanism can be made up of two steps: Coordinated information embedding and Coordinated attention generation. The specific implementation is as follows:

①Coordinate message embedding

As for the input feature map F ∈ R^C×H×W, the average pooling operation using the pooling kernels of size (H,1) and (1, W) along the width and height directions respectively is performed so that the feature maps and in the width and height directions are obtained, which are calculated as shown in Equation (1).

②Coordinate Attention generation

The feature maps which are encoded in the two directions of width and height are concatenated by the transformation of the information embedding step.Subsequently, the dimensionality is reduced to the original C / r by performing a 1×1 convolution operation in the channel dimension.After that, the batch normalized feature map will be fed into the Sigmoid activation function which results in a feature map of dimension 1 × (W + H) × C / r, as shown in Equation (2).

After that, the feature map f is subjected to a 1×1 convolution operation following the initial width and height so that the feature maps f^h and f^w with the same dimension as the initial ones are obtained. Then, they are processed by the Sigmoid activation function to obtain the attention weights g^h and g^w on the width and height directions, respectively, as shown in Equation (3).

Finally, the obtained attention weights are multiplied and weighted in accordance with the primal feature map so that a feature map y_c with attention weights is obtained, illustrated in Equation (4).

3.3

SIOU loss function

The PP-YOLOE+ model is calculated by adopting the GIOU loss function as the boundary box regression loss, although GIOU adds the minimum outer rectangle as the penalty term that is between the prediction box and the real box, and solves the problem that the gradient cannot be calculated when IOU is used as the loss function.However, there are still the following drawbacks: The first one is that when the prediction box and the real box overlap, the GIOU loss becomes degraded to IOU loss, which cannot properly reflect the position of the prediction box within the real box and the height-to-width ratio of the prediction box. The second is that the gradient of the GIOU loss function also becomes very large when the intersection area of prediction box and real box is close to 0, which leads to unstable training.

Therefore, it is introduced that the SIOU loss function replaced the pre-existing GIOU loss function.The parameters associated with the SIOU loss function are shown in Figure 4. Where, B^pred denotes that the real box with the location of the center point (b_cx, b_cy), B^gt denotes that the real box with the location of the center point , σ is the distance between the center of the real box and the predicted box, w and h denoted that the width and height of the predicted box, w^gt and h^gt denoted that the width and height of the real box, respectively. The figure contains two rectangular boxes, one is the rectangle that is formed at the center of the real box and the prediction box, which is represented by the dashed box in Figure 4, and whose height and width differences are denoted as c_h and c_w, respectively; In the other one is the outer solid wireframe, which represents a minimum outer rectangle enclosing the real and predicted boxes, its height and width differences are denoted as ch’ and cw’, respectively.

Figure 4.

Schematic diagram showing the parameters of the prediction box and the real box

The SIOU loss function that is made up of four cost functions, and they are: angle cost, distance cost, shape cost and IoU cost.

(1) Angle cost. Its calculation formula are shown in (5).

Among them

Where x is the sine function of angle α. The angle α can be obtained by taking the inverse function of the sine function. In the training process, if , then choose to minimize α, otherwise minimize β. Note that the angle loss will become 0 when the angle α is equal to or 0.

(2) Distance cost. In redefining the distance cost, the angle cost is to be considered, which is calculated by the formula shown in (9).

Among them

From this equation, it can be seen that the effect of distance cost is greatly reduced when the angle α tends to 0 degrees, and becomes more pronounced when the angle α tends to 90 degrees.

(3) Shape Cost. Its calculation formula is shown in (11).

Among them

where the role of the θ parameter as a factor that controls the concern for shape cost, avoiding paying excessive attention to shape loss, and reducing the movement of the prediction box, its value range is [2, 6].

(4) IoU cost. Its calculation formula is shown in (13).

In conclusion, ultimately the SIOU loss function can be defined in Equation (14).

4.1

Dataset source and production

For the training model, the dataset adopted is the OPIXray dataset¹² released by the Software Development Key Laboratory for Environment of Beijing University of Aeronautics and Astronautics in 2019.This dataset has a total of 8,885 X-ray images and 5 different knife categories. However, since the only contraband category labeled in this dataset is knives, the detection categories are single and limited, which is not suitable for the application.

Therefore, the following processing is taken to expand the dataset: in the first step, other contraband categories in this dataset are labeled using the labeling tool labelImg, including categories such as laptops, liquids, and cell phones; in the second step, considering the diversity of contraband categories in life, from the HiXray dataset¹³ and the PIDray dataset¹⁴, water, rechargeable batteries, cell phones, laptops, guns, lighters and aerosols. The new dataset, named OPI_MIXray, contains 12 categories of contraband and 30,676 images (training set:validation set:test set=8:1:1).

4.2

The experimental environment and hyperparameter settings

4.3

Model deployment

The model deployment framework uses Paddle-Inference inference framework. The framework is Baidu’s open source high-performance inference engine based on the PaddlePaddle training model, which supports embedded devices such as x86 and ARM. The deployment process is shown in Figure 5. First, Paddle-Inference 2.4.1 is installed on the embedded device Jeston Nano, the trained and improved model is converted into a model deployment file and downloaded to Jeston Nano, and finally the initialization of the deployment file is completed through the API functions provided by the Paddle-Infernece framework, the model is invoked and inference is performed The inference result is obtained, and the location detection and classification of contraband is completed.

Figure 5.

Model deployment inference flow chart

4.4

D-Res module introduces a comparison of location and attention mechanisms

In this paper, the CSPRes modules of C3, C4 and C5 stages of the backbone network output feature maps are replaced by D-Res modules, and the results under mAP@0.5 are all improved to different degrees than the baseline model, among which the C4 stage has the best results, it is shown in Table 1.

Table 1.

D-Res module and attention mechanism experiments

For the attention mechanism, we introduce ECA, CBAM and CA attention mechanisms respectively in this paper, which all have significantly improved the detection accuracy, among them the CA attention mechanism has the best effect, as shown in Table 1. Therefore, we choose to embed CA attention mechanism in the network in this paper, and replace the C4 stage CSPRes module with D-Res module at the output feature map of the backbone network, and adopt SIOU loss function to derive PP-YOLOE+_DCS model.

4.5

Ablation experiments

The ablation of experiments in this paper as shown in Table 2. Where D represents the D-Res module, C represents the CA attention mechanism, and S represents the SIOU loss function. The experimental results demonstrate that all three improvement methods proposed in this paper can improve the detection precision to different degrees compared with the baseline model PP-YOLOE+_S. By introducing the D-Res module, it makes the convolutional kernel better adaptable to contraband of different sizes, shapes and poses. By introducing the CA attention mechanism, it enables the network to acquire the context i southwest west of the region around the contraband in the security check image and reduce the influence of factors such as background and noise. By the replacement of the SIOU loss function, it reduces the localization error between the prediction box and the real box, which improves the detection accuracy and training speed of the model. Finally, with the combination of the three modules, mAP improves by 2.8% compared to the baseline model with only 0.24M additional number of parameters, and FLOPs and Latency both decrease to some extent.

Table 2.

The ablation experiment of the improved model

4.6

Comparison experiment

To further verify the improvement model PP-YOLOE+_DCS performance presented in this paper, the above model was selected for comparison experiments and also compared with other models of PP-YOLOE+ to better understand the relationship between model performance and scale. It can be seen from Table 3 that the improved method PP_YOLOE+_DCS introduced in this paper has reduced its inference delay by 10.6ms compared with that before the improvement, and good detection accuracy has been achieved in mAP@0.5:0.95 and mAP@0.5. This indicates that the improved method presented in this paper helps to improve the feature extraction ability of the model and makes it more capable of handling multi-scale and multi-attitude contraband. The improved model satisfies detection accuracy comparable to that of the complex model, making it more friendly for deployment on resource-limited devices and of practical application value.

Table 3.

Results of experiments with different models

4.7

Visualization of test results

Figure 6 shows the detection results for a partial test set (IoU=0.5). Among them, (a) and (c) show the detections results that are from the original PP-YOLOE+_S model, and (b) and (d) show the detections results that are from the improved PP-YOLOE+_DCS model. From the figures, it is observed that some targets with unclear shapes and contours are missed by the original model. And using the improved model, the missed targets can be detected (marked with red boxes). Therefore, the improved model PP-YOLOE+_DCS submitted in it can better capture the feature information of contraband and improve the recognition and localization of contraband.

Figure 6.

Comparison chart of test results