Overview of the Mowjaz Multi-Topic Labelling Task

Mahmoud Al-Ayyoub; Haitham Seelawi; Mohamed Zaghlol; Hussein T. Al-Natsheh; Samer Suileman; Ali Fadel; Riham Badawi; Ahmed Morsy; Ibraheem Tuffaha; Mohannad Aljarrah

doi:10.1109/ICICS52457.2021.9464604

Overview of the Mowjaz Multi-Topic Labelling Task

Mahmoud Al-Ayyoub
, Haitham Seelawi
, Mohamed Zaghlol
, Hussein T. Al-Natsheh
, Samer Suileman
, Ali Fadel
, Riham Badawi
, Ahmed Morsy
, Ibraheem Tuffaha
, Mohannad Aljarrah

Jordan University of Science and Technology
Mawdoo3 Ltd

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

15 Scopus citations

Abstract

Multilabel text classification is an important task in Natural Language Processing (NLP). One use case of such a task is in categorizing news articles, where each article may belong to one or more classes. In this work, we present the ICICS2021 Mowjaz Multi-Topic Labelling Task. Given a piece of news, systems participating in this task are expected to select its topic(s). The systems are evaluated based on the F1 score measure. In total, 46 teams registered on the task's CodaLab page. Out of them, 28 teams submitted 309 runs. The results are surprisingly high. Moreover, they are very close to each other with all teams having systems achieving F1 scores ranging between 0.7965 and 0.8567. Most of these systems used deep learning models, such as Recurrent Neural Networks (RNN), coupled with pretrained word embeddings such as BERT-based models. Few of them experimented with traditional machine learning models such as Support Vector Machine (SVM) and Naive Bayes (NB).

Original language	English
Title of host publication	2021 12th International Conference on Information and Communication Systems, ICICS 2021
Editors	Mohammad Alsmirat, Abdallah Almaaitah, Yaser Jararweh, Jaime Lloret Mauri
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	502-508
Number of pages	7
ISBN (Electronic)	9781665433518
DOIs	https://doi.org/10.1109/ICICS52457.2021.9464604
State	Published - 24 May 2021
Externally published	Yes
Event	12th International Conference on Information and Communication Systems, ICICS 2021 - Virtual, Valencia, Spain Duration: 24 May 2021 → 26 May 2021

Publication series

Name	2021 12th International Conference on Information and Communication Systems, ICICS 2021

Conference

Conference	12th International Conference on Information and Communication Systems, ICICS 2021
Country/Territory	Spain
City	Virtual, Valencia
Period	24/05/21 → 26/05/21

Keywords

AraBERT
AraVec
Arabic BERT
GRU
GigaBERT
LSTM
Multi-label Text Classification
RNN
SVM

Access to Document

10.1109/ICICS52457.2021.9464604

Cite this

Al-Ayyoub, M., Seelawi, H., Zaghlol, M., Al-Natsheh, H. T., Suileman, S., Fadel, A., Badawi, R., Morsy, A., Tuffaha, I., & Aljarrah, M. (2021). Overview of the Mowjaz Multi-Topic Labelling Task. In M. Alsmirat, A. Almaaitah, Y. Jararweh, & J. L. Mauri (Eds.), 2021 12th International Conference on Information and Communication Systems, ICICS 2021 (pp. 502-508). Article 9464604 (2021 12th International Conference on Information and Communication Systems, ICICS 2021). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICICS52457.2021.9464604

Al-Ayyoub, Mahmoud ; Seelawi, Haitham ; Zaghlol, Mohamed et al. / Overview of the Mowjaz Multi-Topic Labelling Task. 2021 12th International Conference on Information and Communication Systems, ICICS 2021. editor / Mohammad Alsmirat ; Abdallah Almaaitah ; Yaser Jararweh ; Jaime Lloret Mauri. Institute of Electrical and Electronics Engineers Inc., 2021. pp. 502-508 (2021 12th International Conference on Information and Communication Systems, ICICS 2021).

@inproceedings{d523da3e3917402eb705325bed0a5738,

title = "Overview of the Mowjaz Multi-Topic Labelling Task",

abstract = "Multilabel text classification is an important task in Natural Language Processing (NLP). One use case of such a task is in categorizing news articles, where each article may belong to one or more classes. In this work, we present the ICICS2021 Mowjaz Multi-Topic Labelling Task. Given a piece of news, systems participating in this task are expected to select its topic(s). The systems are evaluated based on the F1 score measure. In total, 46 teams registered on the task's CodaLab page. Out of them, 28 teams submitted 309 runs. The results are surprisingly high. Moreover, they are very close to each other with all teams having systems achieving F1 scores ranging between 0.7965 and 0.8567. Most of these systems used deep learning models, such as Recurrent Neural Networks (RNN), coupled with pretrained word embeddings such as BERT-based models. Few of them experimented with traditional machine learning models such as Support Vector Machine (SVM) and Naive Bayes (NB).",

keywords = "AraBERT, AraVec, Arabic BERT, GRU, GigaBERT, LSTM, Multi-label Text Classification, RNN, SVM",

author = "Mahmoud Al-Ayyoub and Haitham Seelawi and Mohamed Zaghlol and Al-Natsheh, \{Hussein T.\} and Samer Suileman and Ali Fadel and Riham Badawi and Ahmed Morsy and Ibraheem Tuffaha and Mohannad Aljarrah",

note = "Publisher Copyright: {\textcopyright} 2021 IEEE.; 12th International Conference on Information and Communication Systems, ICICS 2021 ; Conference date: 24-05-2021 Through 26-05-2021",

year = "2021",

month = may,

day = "24",

doi = "10.1109/ICICS52457.2021.9464604",

language = "English",

series = "2021 12th International Conference on Information and Communication Systems, ICICS 2021",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "502--508",

editor = "Mohammad Alsmirat and Abdallah Almaaitah and Yaser Jararweh and Mauri, \{Jaime Lloret\}",

booktitle = "2021 12th International Conference on Information and Communication Systems, ICICS 2021",

address = "United States",

}

Al-Ayyoub, M, Seelawi, H, Zaghlol, M, Al-Natsheh, HT, Suileman, S, Fadel, A, Badawi, R, Morsy, A, Tuffaha, I & Aljarrah, M 2021, Overview of the Mowjaz Multi-Topic Labelling Task. in M Alsmirat, A Almaaitah, Y Jararweh & JL Mauri (eds), 2021 12th International Conference on Information and Communication Systems, ICICS 2021., 9464604, 2021 12th International Conference on Information and Communication Systems, ICICS 2021, Institute of Electrical and Electronics Engineers Inc., pp. 502-508, 12th International Conference on Information and Communication Systems, ICICS 2021, Virtual, Valencia, Spain, 24/05/21. https://doi.org/10.1109/ICICS52457.2021.9464604

Overview of the Mowjaz Multi-Topic Labelling Task. / Al-Ayyoub, Mahmoud; Seelawi, Haitham; Zaghlol, Mohamed et al.
2021 12th International Conference on Information and Communication Systems, ICICS 2021. ed. / Mohammad Alsmirat; Abdallah Almaaitah; Yaser Jararweh; Jaime Lloret Mauri. Institute of Electrical and Electronics Engineers Inc., 2021. p. 502-508 9464604 (2021 12th International Conference on Information and Communication Systems, ICICS 2021).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Overview of the Mowjaz Multi-Topic Labelling Task

AU - Al-Ayyoub, Mahmoud

AU - Seelawi, Haitham

AU - Zaghlol, Mohamed

AU - Al-Natsheh, Hussein T.

AU - Suileman, Samer

AU - Fadel, Ali

AU - Badawi, Riham

AU - Morsy, Ahmed

AU - Tuffaha, Ibraheem

AU - Aljarrah, Mohannad

PY - 2021/5/24

Y1 - 2021/5/24

N2 - Multilabel text classification is an important task in Natural Language Processing (NLP). One use case of such a task is in categorizing news articles, where each article may belong to one or more classes. In this work, we present the ICICS2021 Mowjaz Multi-Topic Labelling Task. Given a piece of news, systems participating in this task are expected to select its topic(s). The systems are evaluated based on the F1 score measure. In total, 46 teams registered on the task's CodaLab page. Out of them, 28 teams submitted 309 runs. The results are surprisingly high. Moreover, they are very close to each other with all teams having systems achieving F1 scores ranging between 0.7965 and 0.8567. Most of these systems used deep learning models, such as Recurrent Neural Networks (RNN), coupled with pretrained word embeddings such as BERT-based models. Few of them experimented with traditional machine learning models such as Support Vector Machine (SVM) and Naive Bayes (NB).

AB - Multilabel text classification is an important task in Natural Language Processing (NLP). One use case of such a task is in categorizing news articles, where each article may belong to one or more classes. In this work, we present the ICICS2021 Mowjaz Multi-Topic Labelling Task. Given a piece of news, systems participating in this task are expected to select its topic(s). The systems are evaluated based on the F1 score measure. In total, 46 teams registered on the task's CodaLab page. Out of them, 28 teams submitted 309 runs. The results are surprisingly high. Moreover, they are very close to each other with all teams having systems achieving F1 scores ranging between 0.7965 and 0.8567. Most of these systems used deep learning models, such as Recurrent Neural Networks (RNN), coupled with pretrained word embeddings such as BERT-based models. Few of them experimented with traditional machine learning models such as Support Vector Machine (SVM) and Naive Bayes (NB).

KW - AraBERT

KW - AraVec

KW - Arabic BERT

KW - GRU

KW - GigaBERT

KW - LSTM

KW - Multi-label Text Classification

KW - RNN

KW - SVM

UR - https://www.scopus.com/pages/publications/85113858393

U2 - 10.1109/ICICS52457.2021.9464604

DO - 10.1109/ICICS52457.2021.9464604

M3 - Conference contribution

AN - SCOPUS:85113858393

T3 - 2021 12th International Conference on Information and Communication Systems, ICICS 2021

SP - 502

EP - 508

BT - 2021 12th International Conference on Information and Communication Systems, ICICS 2021

A2 - Alsmirat, Mohammad

A2 - Almaaitah, Abdallah

A2 - Jararweh, Yaser

A2 - Mauri, Jaime Lloret

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 12th International Conference on Information and Communication Systems, ICICS 2021

Y2 - 24 May 2021 through 26 May 2021

ER -

Al-Ayyoub M, Seelawi H, Zaghlol M, Al-Natsheh HT, Suileman S, Fadel A et al. Overview of the Mowjaz Multi-Topic Labelling Task. In Alsmirat M, Almaaitah A, Jararweh Y, Mauri JL, editors, 2021 12th International Conference on Information and Communication Systems, ICICS 2021. Institute of Electrical and Electronics Engineers Inc. 2021. p. 502-508. 9464604. (2021 12th International Conference on Information and Communication Systems, ICICS 2021). doi: 10.1109/ICICS52457.2021.9464604

Overview of the Mowjaz Multi-Topic Labelling Task

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this