Adapting Transformer Networks for Document Summarization and Sentiment Analysis

Abstract

Due to the increasing number of text-based content sources, the demand for effective sentiment analysis and document summarization techniques has been increasing. Several transformer-based models, including “ELECTRA, BERT, XLNet, RoBERTa, DistilBERT, and ALBERT” have emerged as promising alternatives to traditional methods. This paper aims to study the effectiveness of the different transformer models for performing sentiment analysis and document summarization on the Yelp dataset. The paper aims to analyze the various transformer models' performance on the tasks, identify their weaknesses, and suggest possible improvements. It also thoroughly studies the Yelp dataset, which has over 5 million reviews. The paper introduces the different transformer models that are used for performing document summarization and analysis on the Yelp dataset. We then perform evaluation on these models using various metrics to measure their performance. Some of these include ROUGE, F1-score, AUC-ROC, and accuracy. According to the paper's experimental results, the RoBERTa and BERT models perform better than the other transformer models when it comes to document summarization. In addition, we identified the weaknesses and strengths of each model. We suggest implementing domain-specific training and fine-tuning techniques to improve their performance. The results of the experiment revealed that the RoBERTa and BERT models perform better than the other ones when it comes to document summarization. We also found that the models have weaknesses and strengths, and we suggest using domain-specific training and fine-tuning techniques to improve their performance. The paper contributes to the literature related to the use of transformer-based models in sentiment analysis and documents summarization by providing an extensive analysis of the different models' performance in the Yelp dataset. It also suggests various modifications to improve their capabilities.