2.1

Convolutional neural network

Convolutional Neural Convolutional Neural Network is inspired by the working mechanism of the visual cortex of the mammalian brain and is a common deep learning model. CNN models generally include input layers, convolutional layers, pooling layers, and fully connected layers ^{[12, 13]}. Among them, the main function of the convolution layer is to convolve the convolution kernel with the local receptive field of the input signal, extract the local receptive field features of the input signal, and use the convolution result of multiple input features as the output feature under the action of the activation function vector.

In the formula, and are the final output result and convolution operation result of the jth neuron in the ith feature surface of the lth layer respectively; x^l(j) is the lth The jth local receptive field of layer; is the weight vector of the ith convolution kernel of the lth layer; is the bias of the ith convolution kernel of the lth layer; f{·} is the activation function.

The ReLU function is a piecewise function that avoids the gradient saturation effect and is widely used in deep convolutional networks. Its expression is:

The role of the pooling layer is to downsample the feature map to filter out redundant parameters in the data. It has the advantages of feature invariance and preventing the network from overfitting. At present, the commonly used pooling methods are mean pooling and maximum pooling, whose expressions are respectively:

In the formula, is the pooling result of the neuron in the l+1h layer; w is the width of the pooling area; is the value of the t-th neuron in the i-th feature plane in the l-th layer.

It can be seen from equations (3) and (4) that mean pooling takes the average value of the receptive field of the feature surface as the output. The maximum pooling outputs the maximum value of the receptive field of each feature surface, extracts important features, and ignores secondary factors. The network structure of this paper adopts the maximum pooling layer.

After the input image is extracted as high-level informative features through the processing of convolutional layers and pooling layers, one or more fully connected layers are connected to classify the informative features. Each neuron in the fully connected layer is connected to each neuron in the previous layer and can receive all the local information of the previous layer. The first fully connected layer develops high-level feature vectors into one-dimensional vectors. The second fully connected layer uses the Softmax regression classifier to solve the classification problem and obtain the final recognition result. Its expression is:

In the formula, α is the parameter set of the training model; M is the total number of training set samples; (X_m, Y_m) is the training set samples of the model; D is the number of operating state categories of the synchronous motor; 1{Y_m=d} is the indicator function, when The function value is 1 when the parentheses are true, and 0 otherwise.

In this paper, the cross entropy loss function is used to calculate the error between the true label and the classification result, and its expression is:

In the formula, m is the size of the input mini-batch; j is the target category; P is the true label of the sample; Q is the classification result output by the model.

2.2

CNN model based on handwriting features

The contents of the power operation ticket include operation items, time, issuer, signature of recipient, etc. There are many handwritten fonts. Aiming at the characteristics of handwritten fonts with complex structure, various types and different writing styles ^[14], this paper proposes a text recognition method for power operation tickets based on CNN. On the basis of traditional CNN, a variety of handwriting features are introduced to replace the original image input, which breaks the limitation of traditional CNN on the learning of original image spatial features and improves the accuracy of handwritten font recognition.

2.2.1

Imaginary strokes

The imaginary Chinese character uses the degree of directional change to calculate the correlation between different strokes, and extracts the change characteristics between different strokes of the same Chinese character, and then realizes the recognition of handwritten fonts^[14]. The formula for calculating the degree of direction change dcd is:

Among them, θ is the number of angles (180≤θ≤180) formed by the connection between different strokes, , q is the stroke length, t=1/8.

In the process of writing Chinese characters, it includes the characteristics of starting a pen, dropping a pen, and connecting different strokes. If the connected strokes are shorter and the direction change is larger, it is a strong feature. Strong features can effectively identify the writing features of Chinese characters. Taking the word “作” in the electric power operation ticket as an example, its stroke change characteristics are shown in Figure 1. Among them, the feature pixels are marked by red pentagram. In this paper, by comparing the values of dcd of different pixels, the imaginary stroke matrix is calculated and used as the input of the convolutional neural network model.

Figure 1.

Variation of strokes of the word “作”

2.2.2

8-direction feature

Different from the characters composed of English letters, Chinese characters are mainly composed of horizontal (), vertical (|), apostrophe (/), and stroking (\), which have obvious directional characteristics ^[15]. Therefore, multidirectional features can be used to simulate strokes such as horizontal, vertical, slanting, and stroking of Chinese characters, and directional features such as 8, 16, and 32 are commonly used. This paper selects 8-direction features, as shown in Figure 2, respectively from 0°, 45°, 90°, 135°, 180°, 225°, 270°, 315° to calculate the handwriting gradient size, which can take into account the recognition speed and recognition accuracy. Assuming the start and end coordinates of a piece of handwriting (x₁, y₁) and (x₂, y₂), the gradient calculation formula is:

Figure 2.

Eight-direction characteristics of the word “作”

Among them, d₁ = |x₂ – x₁|, d₂ = |y₂ – y₁|,

2.3

CNN model based on handwriting features

The flow chart of the CNN text recognition model based on handwriting features proposed in this paper is shown in Figure 3. The imaginary strokes and 8-direction features of the text to be recognized are respectively used as the feature matrix input model for training, and the simple average method is used to calculate the classification results.

Figure 3.

Flow chart of CNN model based on handwriting features

The determination of the size of the convolution kernel has an important impact on the final recognition result. If the size of the convolution kernel is too small, it will increase the network depth and consume a lot of time; if the size of the convolution kernel is too large, the network structure will be too simple, and the features of handwritten text cannot be accurately extracted., and may contain too much redundant information. The specific structure of the CNN network in this paper is shown in Table 1. The size of the convolution kernel of the first layer is slightly larger than that of the other layers. By extracting short-term features and expanding the receptive field, the useful features of handwritten fonts are effectively extracted and useless information is filtered out; the rest The use of smaller convolution kernels in each layer can reduce network parameters, deepen the number of network layers, enhance computing power and avoid overfitting. In the table, N indicates that the dimensional model of handwriting features is a 6-layer structure, including 4 alternating convolutional layers, maximum pooling layers, 1 fully connected layer and 1 output layer.

Table 1.

CNN network structure

3.1

Experimental results

The experimental verification is carried out with 40,000 images of electric power operation tickets collected by a power supply company of State Grid Power Co., Ltd. as a dataset. Among them, 80% of the images are selected as the training set and 20% as the test set. The dataset includes 1000 commonly used Chinese characters written by 100 writers, each from 100 writers. The experimental environment of this paper is Intel(R) Core(TM) i5-4590, CPU@3.30GHz, RAM24.00GB microcomputer platform, the language used is Python3.7, and the deep learning framework is tensorflow2.7.0. In the training process, the number of iteration steps is 100, and the number of samples in each batch is 100.

In the training phase of the CNN model, the Adam optimization algorithm is used to update the weights, and the learning rate is set to 0.001. The learning rate of the traditional stochastic gradient descent algorithm does not change during the training process, maintaining a single learning rate to update the weights, while the Adam optimization algorithm designs independent adaptive learning for different parameters by calculating the first-order and second-order moment estimates of the gradient It has strong robustness to hyperparameters and is widely used in the field of deep learning. In order to avoid overfitting the data, the Droupout regularization rule is introduced in the fully connected layer, and the dropout ratio after each convolutional layer is set to 0, 0, 0.05, and 0.1 respectively during the experiment.

Figures 4-5 show the change of loss function and classification accuracy of CNN text recognition based on handwriting features.

Figure 4.

Accuracy change graph

Figure 5.

Loss function change graph

As shown in Figure 4-5, as the number of iterations increases, the loss function gradually decreases and converges at 40 iterations, and the accuracy gradually increases, and finally stabilizes at 95.9% at 40 iterations, which can recognize handwriting more accurately Electricity operation ticket.

3.2

Comparison of different algorithms

In order to verify the effectiveness of the method proposed in this paper, a traditional CNN (CNN-none), a CNN model containing only imaginary strokes (denoted as CNN-hy), and a CNN with only directional features (denoted as CNN-di) were selected for comparison. The power operation ticket recognition accuracy of different algorithms is shown in Table 2.

Table 2.

Different algorithms text recognition accuracy

As shown in Table 2, the CNN-di+hy method proposed in this paper significantly outperforms other algorithms. Compared with the CNN-none method, the recognition accuracy is increased by 15.67%, and the accuracy is increased by 8.4% and 6.92% compared with CNN-hy and CNN-dy, respectively. It shows that the information obtained based on different handwriting features has strong complementarity, and the fusion of different handwriting features can obtain better classification results.