2.1

Data acquisition module

The point cloud is voxelized according to the method in Voxelnet. Referring to the grid division method of the point cloud in PV-RCNN and the characteristics of the experimental data set KIITI, the size of the unit voxel is set as the detection of the vehicle category., the paper clips the ground truth distribution of the point cloud at [0, 70.4], [−40, 40], [−3, 1]m along the X, Y, Z axes, which will get 352×400×40 Voxel collection.

The voxel index RCNN network model is constructed with reference to PV-RCNN. For the 3D backbone network, the 3D backbone network is mainly divided into four modules, and the number of convolution kernels is 16, 32, 48, 64 respectively. The point cloud is converted into gridded features, and the feature maps obtained by downsampling are stacked together along the Z axis and con-verted into BEV feature maps. The 2D backbone network consists of two modules: a feature extraction sub-network and a multi-scale feature fusion sub-network that upsamples and concatenates top-down features. Both modules con-sist of 5 convolutional layers with feature dimensions of (64, 128) respectively, the first module maintains the same resolution as the 3D backbone output in the X and Y axes, while the second module’s resolution rate is half of the former.

2.2

Voxel feature extraction and ROI pooling

The voxel index R-CNN designs a voxel ROI pooling layer to aggregate spatial information from 3D voxel features, and uses voxel grid space to represent points. The positions and features use non-empty voxel center coordinates respectively {v_i =(x_i, y_i, z_i)}, where the range of i, j, k is (0, N) and eigenvector represented by .

Figure 2

Spatial and Sectional Representations of Ball, Manhattan Distance, and Voxel Index Query

2.2.3

Voxel ROI pooling layer

First, based on the original voxel, the proposed region of fusion voxel features is divided into G×G×G regular sub-voxels, and the center point of the sub-voxel is the same as the original voxel. Because the 3D voxels are too sparse, and the space occupied by non-empty voxels is less than 3%, the maximum pooling layer in Fast-RCNN cannot be used to directly extract features for each sub-voxel. The features of the voxels are aggregated together. For example, given the center point g_i, the paper first uses the index difference query to obtain the set of adjacent voxels, and then uses the Point Net module to aggregate the features of these adjacent voxels:

Among them, the maximum pooling operation max(·) represents the aggregated feature vector ξ_i obtained along the number of channels, and σ(·) represents a layer of multilayer perceptron, represents the voxel feature of ,

v_i – g_i represents the relative coordinate between v_i and g_i. In order to improve the computational efficiency, this method does not aggregate the voxel features of the four stages in the 3D backbone network as in PV-RCNN, but only aggregates the voxel features of the latter two stages of the 3D backbone network. For each stage, set the size of the index difference to group voxels of multiple scales.

2.3

Loss function

Referring to the loss function used in PointPillars [12], the loss function of RPN is determined, which is composed of the object classification loss function and the bounding box regression loss function. The formula is:

Among them, p_i and σ_i represent the output of target classification and bounding box regression, respectively, and represent the classification label and regression target, respectively, means that only the regression loss of anchor points on foreground objects is calculated, and N_f represents the number of anchor points on foreground objects. The object classification function is expressed by Focal loss [13] as:

γ is a constant, when it is 0, L_cls is consistent with the cross-entropy loss function, and p^t is the probability of the current prediction.

The regression loss function is represented by Huber loss, and the formula is:

δ is a hyperparameter. When δ tends to 0, it will tend to mean absolute error; when δ tends to ∞, it will tend to mean square error.

2.4

Detection module

The detection module takes the region-of-interest features as input for bounding box regression. Specifically, the shared 2-layer MLP first converts ROI features into feature vectors, and then passes the vectorized features into two parallel branches: one for object bounding box regression and the other for confidence prediction. The bounding box regression branch predicts the difference from the 3D proposal region to the ground truth, and the confidence branch predicts the confidence associated with the IOU.

3.1

Training

The entire framework of Voxel Index R-CNN is optimized using the Adam optimizer. In the training phase, the network is set to train for 80 rounds, the number of samples inputted each time is 12, the learning rate is initialized to 0.01, and the cosine annealing strategy is used to update. In the detection module, the threshold for the foreground is set to 0.75, and the threshold for the background is set to 0.25. Threshold for bounding box regression is set to 0.6. In this paper, 128 regions of interest are randomly selected as the training samples of the detection module. Among the sampled ROIs, half are samples of, which coincide with the corresponding ground-truth boxes. In the inference stage, the paper first performs Non-Maximum Suppression on the RPN with a threshold of 0.7, and retains the first 80 proposal regions as the input of the detection module. After the bounding box regression prediction, NMS is applied again with a threshold of 0.1 to remove redundant predictions result.

3.2

Result

The results of Voxel Index R-CNN on the KITTI validation set for 3D target detection AP and BEV target detection AP are shown in Table 1, where the 9 threshold is 0.7.

Table 1

3D object detection AP and BEV object detection AP on KITTI val set

Figure 3 shows the 3D detection results of three of these scenes on the KITTI test set.

Figure 3

Detection effect on KITTI test set

The results of Voxel Index R-CNN are compared with several state-of-the-art methods on the KITTI test set. The results of the three detection difficulty levels of easy, medium and difficult are shown in Table 2.

Table 2.

Time domain index statistics of vibration signal

The comparison results show that the voxel index R-CNN in this paper achieves a good balance between accuracy and efficiency among all methods. Using the voxel index query method, the voxel index R-CNN achieved an average accuracy (AP) of 90.63% on the medium and easy difficulty of the Car class, and reached an average accuracy of 81.74% on the medium difficulty. The average accuracy rate is 81.62%, and the frame rate of detection processing is 28.5FPS. Specifically, voxel-indexed R-CNN achieves comparable accuracy with the best performing models PV-RCNN and Voxel R-CNN. Voxel-indexed R-CNN is similar in structure to PV-RCNN, but the detection frame rate Compared with 4.9 FPS higher, it proves that the use of voxel index query indeed reduces the computational loss; the detection result is 0.12% and 0.22% higher than that of Voxel R-CNN at the medium and difficult levels, respectively, indicating that the feature extraction module has a good effect on the ROI area. It is effective to perform feature enhancement. Furthermore, the performance of voxel-indexed R-CNN is greatly improved over existing voxel-based models.