3.1

CA Module

The CA module of the coordinate attention mechanism was shown in Figure 2. When any intermediate feature tensor was input, C was the number of channels, H was the height of the input feature, and W was the width of the input feature. The CA module divides the input into a one-dimensional feature encoding process in two directions, one of which captures long-range dependencies in spatial directions, the other captures precise location information in spatial directions. The resulting feature maps were encoded into a pair of orientation-aware and position-sensitive feature maps. Respectively, so that the input feature map information could be supplemented to enhance the representation of keypoint locations. The final output transform tensor with precise position information had the same size as the input tensor X.

Figure 2.

Coordinate attention

3.2

C-Basicblock Module

Aiming at the problem that key point detection in complex background was easily affected by the surrounding environment, in order to enhance the extraction of key points location information in complex environment by HRNet feature extraction network. This paper introduced the location attention mechanism CA (Cooraditation Attention) module. The information was embedded into the channel attention, which enhanced the feature extraction network ability to obtain spatial coordinate information in complex backgrounds, which not only enables the network to obtain rich high-dimensional feature information but also obtains accurate location information. Thereby, improving the subsequent keypoints positioning and prediction effect, to enhance the robustness of the network. In the feature extraction part of the HRNet network, multiple coordinate attention CA modules were integrated to construct a C-Basicblock residual module for location feature extraction. The overall structure of the reconstructed feature extraction module C-Basicblock was shown in Figure 3.

Figure 3.

Residual block C-Basicblock

3.3

CA-HRNet

In order to improve the human pose estimation effect of the network in complex scenes, under the framework of deep learning, this paper used the high-resolution network HRNet as the basic network, and used the constructed C-Basicblock module to reconstruct HRNet, and finally obtained a high-resolution location-aware network. The high-resolution human pose estimation network CA-HRNet, and its overall network results were shown in Figure 4.

Figure 4.

Human pose estimation network CA-HRNet

Given an input data, the CA-HRNet network first preprocessed it, and then used the Bottleneck module, C-Basicblock module, Up Sample and Down sample operations of the high-resolution network to ensure sufficient extraction of features at different resolutions.And fully obtained the location information of key points, and finally performed feature fusion on the extracted relevant information to achieve accurate regression positioning of key points.

4.1

Image Preprocessing

In the model training process, the image was first preprocessed. The main preprocessing operations include cropping, flipping, and scale transformation. The cropping aspect ratio of the image was 256:196. The random flip strategy was used to randomly rotate the image (- 45°~ + 45°) and change the image scale, the scale change was 0.7:1:1.35 three different scales. In the model training process, the Adam optimization algorithm was used to update the network weights iteratively. After every 3.6 million iterations, the learning rate was reduced by two times. The initial learning rate was 5e-4, the batch size was set to 16, and the training period was set to 210 batches.

4.2

Dataset

The dataset used for model training in this paper was the MS COCO2017 dataset, which included a total of 57K images and 150K human instances, included 64115 training samples and 5000 validation samples, which were divided into training sets, validation sets, and test set as needed.Section headings.

4.3

Evaluation Indicators

In the MS COCO dataset, OKS (Object Keypoint Similarity) was used to represent the target keypoint similarity, whose value was distributed between (0,1), and the better prediction result when the value of OKS was close to 1. The formula for OKS was shown in Equation (1).

Where d_i represents the euclidean distance between each predicted key point and the real key point, which v_i means that the key point was visible, s represents the object scale, and k_i represents the control attenuation constant.

In order to evaluate the performance of the human pose estimation model, this paper adopted the main evaluation indicators of MS COCO data set AP, AP⁵⁰, AP⁷⁵, AP^M, AP^L, AR, AR⁵⁰, AR⁷⁵, AR^M, AR^L. Where AP value indicates the average precision at 10 OKS thresholds (OKS=0.50, 0.55, 0.60, …, 0.95), AP⁵⁰ indicates the detection precision at OKS=0.50, AP⁷⁵ indicates the detection precision at OKS=0.75, AP^M indicates medium The detection accuracy of objects, AP^L represents the detection accuracy of large objects. AR represents the average recall rate at 10 OKS thresholds, and each value of AR had the same meaning as AP.The predicted key point position and the marked key point position were calculated by formula (1) to calculate the OKS value, and then the corresponding AP and AR values could be calculated to obtain the final detection precision and recall rate.

4.4

Experimental Results and Analysis

4.4.1

Model Training

Figure 5 showed the accuracy change curve of the improved model training in this paper. It could be seen from the figure that when the model started training to about 165 batches, the model adaptively adjusted the learning rate, so that the accuracy increased and the change gradually became stable.

Figure 5.

Model training accuracy

4.4.3

Visual Analysis

The verification results of the algorithm in this paper on the COCO data set was shown in Figure 6. In Figures (a) and (b), when occlusion occurred in a complex environment, the model could ensure the detection of key points occluding the human body, and fully extracted the information of occlusion key points, so as to realized the positioning of key points of the human body. In Figure (c), when there were branches in the background that were similar in shape to the limbs, the model in this paper could to use the contextual semantic information and the model to infer the location of the occlusion key points according to the logical positions of the skeleton joints, which effectively avoids the influence of the branches. In Figures (d), (e), and (f), the crowded and complex scenes made it difficult to locate key points, but the model in this paper could still fully extract the spatial location information, so as to achieve accurate positioning in crowded and complex scenes. This verified the effectiveness of the algorithm in this paper.

Figure 6.

Validation results on COCO dataset