TY - GEN
T1 - JUSTers at SemEval-2020 Task 4
T2 - 14th International Workshops on Semantic Evaluation, SemEval 2020, co-located with COLING 2020
AU - Fadel, Ali
AU - Al-Ayyoub, Mahmoud
AU - Cambria, Erik
N1 - Publisher Copyright:
© 2020 14th International Workshops on Semantic Evaluation, SemEval 2020 - co-located 28th International Conference on Computational Linguistics, COLING 2020, Proceedings. All rights reserved.
PY - 2020
Y1 - 2020
N2 - In this paper, we describe our team's (JUSTers) effort in the Commonsense Validation and Explanation (ComVE) task, which is part of SemEval2020. We evaluate five pre-trained Transformer-based language models with various sizes against the three proposed subtasks. For the first two subtasks, the best accuracy levels achieved by our models are 92.90% and 92.30%, respectively, placing our team in the 12th and 9th places, respectively. As for the last subtask, our models reach 16.10 BLEU score and 1.94 human evaluation score placing our team in the 5th and 3rd places according to these two metrics, respectively. The latter is only 0.16 away from the 1st place human evaluation score.
AB - In this paper, we describe our team's (JUSTers) effort in the Commonsense Validation and Explanation (ComVE) task, which is part of SemEval2020. We evaluate five pre-trained Transformer-based language models with various sizes against the three proposed subtasks. For the first two subtasks, the best accuracy levels achieved by our models are 92.90% and 92.30%, respectively, placing our team in the 12th and 9th places, respectively. As for the last subtask, our models reach 16.10 BLEU score and 1.94 human evaluation score placing our team in the 5th and 3rd places according to these two metrics, respectively. The latter is only 0.16 away from the 1st place human evaluation score.
UR - https://www.scopus.com/pages/publications/85099082408
U2 - 10.18653/v1/2020.semeval-1.66
DO - 10.18653/v1/2020.semeval-1.66
M3 - Conference contribution
AN - SCOPUS:85099082408
T3 - 14th International Workshops on Semantic Evaluation, SemEval 2020 - co-located 28th International Conference on Computational Linguistics, COLING 2020, Proceedings
SP - 535
EP - 542
BT - COLING 2020 - The International Workshop on Semantic Evaluation, Proceedings of the 14th Workshop
A2 - Herbelot, Aurelie
A2 - Zhu, Xiaodan
A2 - Palmer, Alexis
A2 - Schneider, Nathan
A2 - May, Jonathan
A2 - Shutova, Ekaterina
PB - International Committee for Computational Linguistics
Y2 - 12 December 2020 through 13 December 2020
ER -