YMAI at SemEval-2022 Task 5: Detecting Misogyny in Memes using VisualBERT and MMBT MultiModal Pre-trained Models

Mohammad Habash; Yahya Daqour; Malak Abdullah; Mahmoud Al-Ayyoub

doi:10.18653/v1/2022.semeval-1.108

YMAI at SemEval-2022 Task 5: Detecting Misogyny in Memes using VisualBERT and MMBT MultiModal Pre-trained Models

Mohammad Habash
, Yahya Daqour
, Malak Abdullah
, Mahmoud Al-Ayyoub

Jordan University of Science and Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

3 Scopus citations

Abstract

This paper presents a deep learning system that contends at SemEval-2022 Task 5. The goal is to detect the existence of misogynous memes in sub-task A. At the same time, the advanced multi-label sub-task B categorizes the misogyny of misogynous memes into one of four types: stereotype, shaming, objectification, and violence. The Ensemble technique has been used for three multi-modal deep learning models: two MMBT models and VisualBERT. Our proposed system ranked 17^th place out of 83 participant teams with an F1-score of 0.722 in sub-task A, which shows a significant performance improvement over the baseline model's F1-score of 0.65.

Original language	English
Title of host publication	SemEval 2022 - 16th International Workshop on Semantic Evaluation, Proceedings of the Workshop
Editors	Guy Emerson, Natalie Schluter, Gabriel Stanovsky, Ritesh Kumar, Alexis Palmer, Nathan Schneider, Siddharth Singh, Shyam Ratan
Publisher	Association for Computational Linguistics (ACL)
Pages	780-784
Number of pages	5
ISBN (Electronic)	9781955917803
DOIs	https://doi.org/10.18653/v1/2022.semeval-1.108
State	Published - 2022
Externally published	Yes
Event	16th International Workshop on Semantic Evaluation, SemEval 2022, co-located (hybrid) with The 2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics, NAACL 2022 - Seattle, United States Duration: 14 Jul 2022 → 15 Jul 2022

Publication series

Name	SemEval 2022 - 16th International Workshop on Semantic Evaluation, Proceedings of the Workshop

Conference

Conference	16th International Workshop on Semantic Evaluation, SemEval 2022, co-located (hybrid) with The 2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics, NAACL 2022
Country/Territory	United States
City	Seattle
Period	14/07/22 → 15/07/22

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 5 Gender Equality

Access to Document

10.18653/v1/2022.semeval-1.108

Cite this

Habash, M., Daqour, Y., Abdullah, M., & Al-Ayyoub, M. (2022). YMAI at SemEval-2022 Task 5: Detecting Misogyny in Memes using VisualBERT and MMBT MultiModal Pre-trained Models. In G. Emerson, N. Schluter, G. Stanovsky, R. Kumar, A. Palmer, N. Schneider, S. Singh, & S. Ratan (Eds.), SemEval 2022 - 16th International Workshop on Semantic Evaluation, Proceedings of the Workshop (pp. 780-784). (SemEval 2022 - 16th International Workshop on Semantic Evaluation, Proceedings of the Workshop). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/2022.semeval-1.108

Habash, Mohammad ; Daqour, Yahya ; Abdullah, Malak et al. / YMAI at SemEval-2022 Task 5 : Detecting Misogyny in Memes using VisualBERT and MMBT MultiModal Pre-trained Models. SemEval 2022 - 16th International Workshop on Semantic Evaluation, Proceedings of the Workshop. editor / Guy Emerson ; Natalie Schluter ; Gabriel Stanovsky ; Ritesh Kumar ; Alexis Palmer ; Nathan Schneider ; Siddharth Singh ; Shyam Ratan. Association for Computational Linguistics (ACL), 2022. pp. 780-784 (SemEval 2022 - 16th International Workshop on Semantic Evaluation, Proceedings of the Workshop).

@inproceedings{190f9472601c424e90c2c43905b96765,

title = "YMAI at SemEval-2022 Task 5: Detecting Misogyny in Memes using VisualBERT and MMBT MultiModal Pre-trained Models",

abstract = "This paper presents a deep learning system that contends at SemEval-2022 Task 5. The goal is to detect the existence of misogynous memes in sub-task A. At the same time, the advanced multi-label sub-task B categorizes the misogyny of misogynous memes into one of four types: stereotype, shaming, objectification, and violence. The Ensemble technique has been used for three multi-modal deep learning models: two MMBT models and VisualBERT. Our proposed system ranked 17th place out of 83 participant teams with an F1-score of 0.722 in sub-task A, which shows a significant performance improvement over the baseline model's F1-score of 0.65.",

author = "Mohammad Habash and Yahya Daqour and Malak Abdullah and Mahmoud Al-Ayyoub",

note = "Publisher Copyright: {\textcopyright} 2022 Association for Computational Linguistics.; 16th International Workshop on Semantic Evaluation, SemEval 2022, co-located (hybrid) with The 2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics, NAACL 2022 ; Conference date: 14-07-2022 Through 15-07-2022",

year = "2022",

doi = "10.18653/v1/2022.semeval-1.108",

language = "English",

series = "SemEval 2022 - 16th International Workshop on Semantic Evaluation, Proceedings of the Workshop",

publisher = "Association for Computational Linguistics (ACL)",

pages = "780--784",

editor = "Guy Emerson and Natalie Schluter and Gabriel Stanovsky and Ritesh Kumar and Alexis Palmer and Nathan Schneider and Siddharth Singh and Shyam Ratan",

booktitle = "SemEval 2022 - 16th International Workshop on Semantic Evaluation, Proceedings of the Workshop",

address = "Australia",

}

Habash, M, Daqour, Y, Abdullah, M & Al-Ayyoub, M 2022, YMAI at SemEval-2022 Task 5: Detecting Misogyny in Memes using VisualBERT and MMBT MultiModal Pre-trained Models. in G Emerson, N Schluter, G Stanovsky, R Kumar, A Palmer, N Schneider, S Singh & S Ratan (eds), SemEval 2022 - 16th International Workshop on Semantic Evaluation, Proceedings of the Workshop. SemEval 2022 - 16th International Workshop on Semantic Evaluation, Proceedings of the Workshop, Association for Computational Linguistics (ACL), pp. 780-784, 16th International Workshop on Semantic Evaluation, SemEval 2022, co-located (hybrid) with The 2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics, NAACL 2022, Seattle, United States, 14/07/22. https://doi.org/10.18653/v1/2022.semeval-1.108

YMAI at SemEval-2022 Task 5: Detecting Misogyny in Memes using VisualBERT and MMBT MultiModal Pre-trained Models. / Habash, Mohammad; Daqour, Yahya; Abdullah, Malak et al.
SemEval 2022 - 16th International Workshop on Semantic Evaluation, Proceedings of the Workshop. ed. / Guy Emerson; Natalie Schluter; Gabriel Stanovsky; Ritesh Kumar; Alexis Palmer; Nathan Schneider; Siddharth Singh; Shyam Ratan. Association for Computational Linguistics (ACL), 2022. p. 780-784 (SemEval 2022 - 16th International Workshop on Semantic Evaluation, Proceedings of the Workshop).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - YMAI at SemEval-2022 Task 5

T2 - 16th International Workshop on Semantic Evaluation, SemEval 2022, co-located (hybrid) with The 2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics, NAACL 2022

AU - Habash, Mohammad

AU - Daqour, Yahya

AU - Abdullah, Malak

AU - Al-Ayyoub, Mahmoud

PY - 2022

Y1 - 2022

N2 - This paper presents a deep learning system that contends at SemEval-2022 Task 5. The goal is to detect the existence of misogynous memes in sub-task A. At the same time, the advanced multi-label sub-task B categorizes the misogyny of misogynous memes into one of four types: stereotype, shaming, objectification, and violence. The Ensemble technique has been used for three multi-modal deep learning models: two MMBT models and VisualBERT. Our proposed system ranked 17th place out of 83 participant teams with an F1-score of 0.722 in sub-task A, which shows a significant performance improvement over the baseline model's F1-score of 0.65.

AB - This paper presents a deep learning system that contends at SemEval-2022 Task 5. The goal is to detect the existence of misogynous memes in sub-task A. At the same time, the advanced multi-label sub-task B categorizes the misogyny of misogynous memes into one of four types: stereotype, shaming, objectification, and violence. The Ensemble technique has been used for three multi-modal deep learning models: two MMBT models and VisualBERT. Our proposed system ranked 17th place out of 83 participant teams with an F1-score of 0.722 in sub-task A, which shows a significant performance improvement over the baseline model's F1-score of 0.65.

UR - https://www.scopus.com/pages/publications/85137593347

U2 - 10.18653/v1/2022.semeval-1.108

DO - 10.18653/v1/2022.semeval-1.108

M3 - Conference contribution

AN - SCOPUS:85137593347

T3 - SemEval 2022 - 16th International Workshop on Semantic Evaluation, Proceedings of the Workshop

SP - 780

EP - 784

BT - SemEval 2022 - 16th International Workshop on Semantic Evaluation, Proceedings of the Workshop

A2 - Emerson, Guy

A2 - Schluter, Natalie

A2 - Stanovsky, Gabriel

A2 - Kumar, Ritesh

A2 - Palmer, Alexis

A2 - Schneider, Nathan

A2 - Singh, Siddharth

A2 - Ratan, Shyam

PB - Association for Computational Linguistics (ACL)

Y2 - 14 July 2022 through 15 July 2022

ER -

Habash M, Daqour Y, Abdullah M, Al-Ayyoub M. YMAI at SemEval-2022 Task 5: Detecting Misogyny in Memes using VisualBERT and MMBT MultiModal Pre-trained Models. In Emerson G, Schluter N, Stanovsky G, Kumar R, Palmer A, Schneider N, Singh S, Ratan S, editors, SemEval 2022 - 16th International Workshop on Semantic Evaluation, Proceedings of the Workshop. Association for Computational Linguistics (ACL). 2022. p. 780-784. (SemEval 2022 - 16th International Workshop on Semantic Evaluation, Proceedings of the Workshop). doi: 10.18653/v1/2022.semeval-1.108

YMAI at SemEval-2022 Task 5: Detecting Misogyny in Memes using VisualBERT and MMBT MultiModal Pre-trained Models

Abstract

Publication series

Conference

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this