|
Word embedding is a widely used method for representing words in contemporary natural language processing, where words are depicted as low-dimensional dense vectors. However, the features of each dimension in the vector are difficult to interpret. Most existing interpretable word embedding methods enhance the interpretability of word embeddings through orthogonal or sparse transformations. Still, the semantics of each dimension need to be determined according to the knowledge base or manually assigned after transformation. Embedded topic model (ETM) can automatically acquire interpretable semantic space of words, but this topic space tends to be ambiguous and redundant. To address this issue, this paper proposes an Explainable Text Representation method with a Learnable and Explicit Semantic Space (ETRLESS), which autonomously learns an orthogonal explicit semantic space, allowing words and documents to be represented as interpretable vectors within this space. It obtains the document embedding representation through a BiLSTM model, initializes topic embeddings through pre-training ETM model, and imposes orthogonal constraints on the topic embedding to obtain a more interpretable topic semantic space. By using the reconstruction loss of documents, the document-topic distribution loss, and the orthogonal loss of topic embeddings as optimization objectives, it employs the backpropagation algorithm to learn an interpretable, orthogonal explicit topic semantic space and word representations based on this space. The results demonstrate that the embeddings generated by the ETRLESS model have clear semantic information in each dimension and maintain performance in downstream tasks.
|
