A Multimodal Deep Learning Approach for Characterizing the Targeted Entity in Political Memes into: Hero, Villain, Victim and Foolish

Memes are one of the most widely distributed materials in social media, greatly influencing society as they influence people's opinions and thoughts since they carry visual and textual information that can be easily shared and launched over the Internet without censorship or supervision. Memes can be misused to convey certain ideas, spread misinformation, or cyberbully others. Sometimes, memes tend to glorify or victimize the entities they refer to, whereas, most of the time, memes vilifies or sillynize them. Detecting the targeted entity (entities) of the meme and the role in which each entity in the meme is characterized is a common problem to address. To this end, this article we create (i) the WarElect dataset of political memes and (ii) the CHAT-WELECT model (a multimodal model that classifies memes based on their topic, predicts the targeted entity in the meme (Task 1) and characterizes the role(s) of each entity (Task 2). The CHAT-WELECT model achieves outstanding results of 93.46% accuracy and 93.47 F1_score for Task 1, and 82.43% accuracy, 81.24 F1_score and 17.57 hamming loss for Task 2.