Question to question similarity analysis using morphological, syntactic, semantic, and lexical features

In the digitally connected world that we are living in, people expect to get answers to their questions spontaneously. This fact increased the burden on the Question/Answer platforms such as Stack Overflow and many others. A promising solution to this problem is to detect if a question being asked similar to a question in the database and present the answer of the detected question to the user. To address this challenge, we propose a novel Natural Language Processing (NLP) approach that detects if two Arabic questions are similar or not using their extracted morphological, syntactic, semantic, lexical features. Our approach involves several phases including Arabic text processing, novel feature extraction, and text classifications. To conduct our experiments, we used a real-world questions dataset consisting of 4,000 pairs of Arabic questions in which our approach achieved 78.2% accuracy using XGBoost model on the best features selected by the Random Forest feature selection technique. This high accuracy shows the ability of our approach to correctly detect the similarity between two Arabic questions.