Improved hierarchical classifiers for multi-way sentiment analysis

Sentiment Analysis (SA) is field in computational linguistics concerned with determining the sentiment conveyed in a piece of text towards certain entities (such as people, organizations, products, services, events, etc.) using NLP tools. The considered sentiments can be as simple as positive vs. negative. A more fine-grained approach known as Multi-Way Sentiment Analysis (MWSA) is based on ranking systems, such as the 5-star ranking system. In such systems, rankings close to each other can be confusing; thus, some researchers have suggested that using Hierarchical Classifiers (HCs) can yield better results compared with traditional Flat Classifier (FCs). Unlike FCs, which try to address the entire classification problem at once, HCs employ some kind of tree structures where the nodes are simple “core” classifiers customized to address a subset of the classification problem. This study aims to explore extensively the use of HCs to address MWSA by studying six different hierarchies. We compare these hierarchies using four well-known core classifiers (SVM, Decision Tree, Naive Bayes, and KNN) using many measures such as Precision, Recall, F1, Accuracy and Mean Square Error (MSE). The experiments are conducted on the Large Arabic Book Reviews (LABR) dataset, which consists of 63K book reviews in Arabic. The results show that using some of the proposed HCs yield significant improvements in accuracy. Specifically, while the best Accuracy and MSE for FC are 45.77% and 1.61, respective-ly, the best accuracy and MSE for an HC are 72.64% and 0.53, respectively. Also, the results show that, in general, KNN(k-nearest neighbors) benefitted the most from using hierarchical classification.