Text Generation Using LSTM

Abstract

Due to their capacity to learn dependencies over time, long short-term memory (LSTM) units on sequence-based models are utilized in classification, question-answering systems, and translation tasks. By learning language models with grammatically stable syntaxes, LSTM networks are providing impressive results in natural language generation for text generation models. The network, on the other hand, does not learn about the context. Regardless of pragmatics, the network only learns the input-output function and produces text from a set of input words. There is no semantic consistency among the sentences that are generated because the model is trained in a context that does not exist. A context vector and a predetermined set of input words are used to train the proposed model to produce text. A context vector, like a paragraph vector, understands the sentence's semantic meaning (context). In this work, several approaches to extracting the context vectors are suggested. In addition to the input-output sequences, context vectors are also trained alongside the inputs when a language model is being trained. The model learns the relationship between the target word, the context vector, and the input words because of this structure. A well-trained model will generate text based on the provided context given a set of context terms. Two variants of the model—word importance and word clustering—have been tested in light of the nature of computing context vectors. The appropriate embeddings between various domains are also examined in the word clustering method. The results are judged by how closely the generated text matches the given context in terms of semantics