Overview of the PAN@FIRE 2020 Task on the Authorship Identification of SOurce COde

Ali Fadel; Husam Musleh; Ibraheem Tuffaha; Mahmoud Al-Ayyoub; Yaser Jararweh; Elhadj Benkhelifa; Paolo Rosso

doi:10.1145/3441501.3441532

Overview of the PAN@FIRE 2020 Task on the Authorship Identification of SOurce COde

Ali Fadel
, Husam Musleh
, Ibraheem Tuffaha
, Mahmoud Al-Ayyoub
, Yaser Jararweh
, Elhadj Benkhelifa
, Paolo Rosso

Jordan University of Science and Technology
Duquesne University
University of Staffordshire
Polytechnic University of Valencia

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

4 Scopus citations

Abstract

Authorship identification is essential to the detection of undesirable deception of others' content misuse or exposing the owners of some anonymous malicious content. While it is widely studied for natural languages, it is rarely considered for programming languages. Accordingly, a PAN@FIRE task, named Authorship Identification of SOurce COde (AI-SOCO), is proposed with the focus on the identification of source code authors. The dataset consists of crawled source codes submitted by the top 1,000 human users with 100 correct C++ submissions or more from the CodeForces online judge platform. The participating systems are asked to predict the author of a given source code from the predefined list of code authors. In total, 60 teams registered on the task's CodaLab page. Out of them, 14 teams submitted 94 runs. The results are surprisingly high with many teams and baselines breaking the 90% accuracy barrier. These systems used a wide range of models and techniques from pretrained word embeddings (especially, those that are tweaked to handle source code) to stylometric features.

Original language	English
Title of host publication	FIRE 2020 - Proceedings of the 12th Annual Meeting of the Forum for Information Retrieval Evaluation
Editors	Prasenjit Majumder, Mandar Mitra, Surupendu Gangopadhyay, Parth Mehta
Publisher	Association for Computing Machinery
Pages	4-8
Number of pages	5
ISBN (Electronic)	9781450389785
DOIs	https://doi.org/10.1145/3441501.3441532
State	Published - 16 Dec 2020
Externally published	Yes
Event	12th Annual Meeting of the Forum for Information Retrieval Evaluation, FIRE 2020 - Hyderabad, India Duration: 16 Dec 2020 → 20 Dec 2020

Publication series

Name	ACM International Conference Proceeding Series

Conference

Conference	12th Annual Meeting of the Forum for Information Retrieval Evaluation, FIRE 2020
Country/Territory	India
City	Hyderabad
Period	16/12/20 → 20/12/20

Keywords

authorship-identification
datasets
source-code

Access to Document

10.1145/3441501.3441532

Cite this

Fadel, A., Musleh, H., Tuffaha, I., Al-Ayyoub, M., Jararweh, Y., Benkhelifa, E., & Rosso, P. (2020). Overview of the PAN@FIRE 2020 Task on the Authorship Identification of SOurce COde. In P. Majumder, M. Mitra, S. Gangopadhyay, & P. Mehta (Eds.), FIRE 2020 - Proceedings of the 12th Annual Meeting of the Forum for Information Retrieval Evaluation (pp. 4-8). (ACM International Conference Proceeding Series). Association for Computing Machinery. https://doi.org/10.1145/3441501.3441532

Fadel, Ali ; Musleh, Husam ; Tuffaha, Ibraheem et al. / Overview of the PAN@FIRE 2020 Task on the Authorship Identification of SOurce COde. FIRE 2020 - Proceedings of the 12th Annual Meeting of the Forum for Information Retrieval Evaluation. editor / Prasenjit Majumder ; Mandar Mitra ; Surupendu Gangopadhyay ; Parth Mehta. Association for Computing Machinery, 2020. pp. 4-8 (ACM International Conference Proceeding Series).

@inproceedings{988b5baa25aa440e9fcc0a4383b2968b,

title = "Overview of the PAN@FIRE 2020 Task on the Authorship Identification of SOurce COde",

abstract = "Authorship identification is essential to the detection of undesirable deception of others' content misuse or exposing the owners of some anonymous malicious content. While it is widely studied for natural languages, it is rarely considered for programming languages. Accordingly, a PAN@FIRE task, named Authorship Identification of SOurce COde (AI-SOCO), is proposed with the focus on the identification of source code authors. The dataset consists of crawled source codes submitted by the top 1,000 human users with 100 correct C++ submissions or more from the CodeForces online judge platform. The participating systems are asked to predict the author of a given source code from the predefined list of code authors. In total, 60 teams registered on the task's CodaLab page. Out of them, 14 teams submitted 94 runs. The results are surprisingly high with many teams and baselines breaking the 90\% accuracy barrier. These systems used a wide range of models and techniques from pretrained word embeddings (especially, those that are tweaked to handle source code) to stylometric features.",

keywords = "authorship-identification, datasets, source-code",

author = "Ali Fadel and Husam Musleh and Ibraheem Tuffaha and Mahmoud Al-Ayyoub and Yaser Jararweh and Elhadj Benkhelifa and Paolo Rosso",

note = "Publisher Copyright: {\textcopyright} 2020 ACM.; 12th Annual Meeting of the Forum for Information Retrieval Evaluation, FIRE 2020 ; Conference date: 16-12-2020 Through 20-12-2020",

year = "2020",

month = dec,

day = "16",

doi = "10.1145/3441501.3441532",

language = "English",

series = "ACM International Conference Proceeding Series",

publisher = "Association for Computing Machinery",

pages = "4--8",

editor = "Prasenjit Majumder and Mandar Mitra and Surupendu Gangopadhyay and Parth Mehta",

booktitle = "FIRE 2020 - Proceedings of the 12th Annual Meeting of the Forum for Information Retrieval Evaluation",

address = "United States",

}

Fadel, A, Musleh, H, Tuffaha, I, Al-Ayyoub, M, Jararweh, Y, Benkhelifa, E & Rosso, P 2020, Overview of the PAN@FIRE 2020 Task on the Authorship Identification of SOurce COde. in P Majumder, M Mitra, S Gangopadhyay & P Mehta (eds), FIRE 2020 - Proceedings of the 12th Annual Meeting of the Forum for Information Retrieval Evaluation. ACM International Conference Proceeding Series, Association for Computing Machinery, pp. 4-8, 12th Annual Meeting of the Forum for Information Retrieval Evaluation, FIRE 2020, Hyderabad, India, 16/12/20. https://doi.org/10.1145/3441501.3441532

Overview of the PAN@FIRE 2020 Task on the Authorship Identification of SOurce COde. / Fadel, Ali; Musleh, Husam; Tuffaha, Ibraheem et al.
FIRE 2020 - Proceedings of the 12th Annual Meeting of the Forum for Information Retrieval Evaluation. ed. / Prasenjit Majumder; Mandar Mitra; Surupendu Gangopadhyay; Parth Mehta. Association for Computing Machinery, 2020. p. 4-8 (ACM International Conference Proceeding Series).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Overview of the PAN@FIRE 2020 Task on the Authorship Identification of SOurce COde

AU - Fadel, Ali

AU - Musleh, Husam

AU - Tuffaha, Ibraheem

AU - Al-Ayyoub, Mahmoud

AU - Jararweh, Yaser

AU - Benkhelifa, Elhadj

AU - Rosso, Paolo

PY - 2020/12/16

Y1 - 2020/12/16

N2 - Authorship identification is essential to the detection of undesirable deception of others' content misuse or exposing the owners of some anonymous malicious content. While it is widely studied for natural languages, it is rarely considered for programming languages. Accordingly, a PAN@FIRE task, named Authorship Identification of SOurce COde (AI-SOCO), is proposed with the focus on the identification of source code authors. The dataset consists of crawled source codes submitted by the top 1,000 human users with 100 correct C++ submissions or more from the CodeForces online judge platform. The participating systems are asked to predict the author of a given source code from the predefined list of code authors. In total, 60 teams registered on the task's CodaLab page. Out of them, 14 teams submitted 94 runs. The results are surprisingly high with many teams and baselines breaking the 90% accuracy barrier. These systems used a wide range of models and techniques from pretrained word embeddings (especially, those that are tweaked to handle source code) to stylometric features.

AB - Authorship identification is essential to the detection of undesirable deception of others' content misuse or exposing the owners of some anonymous malicious content. While it is widely studied for natural languages, it is rarely considered for programming languages. Accordingly, a PAN@FIRE task, named Authorship Identification of SOurce COde (AI-SOCO), is proposed with the focus on the identification of source code authors. The dataset consists of crawled source codes submitted by the top 1,000 human users with 100 correct C++ submissions or more from the CodeForces online judge platform. The participating systems are asked to predict the author of a given source code from the predefined list of code authors. In total, 60 teams registered on the task's CodaLab page. Out of them, 14 teams submitted 94 runs. The results are surprisingly high with many teams and baselines breaking the 90% accuracy barrier. These systems used a wide range of models and techniques from pretrained word embeddings (especially, those that are tweaked to handle source code) to stylometric features.

KW - authorship-identification

KW - datasets

KW - source-code

UR - https://www.scopus.com/pages/publications/85100403582

U2 - 10.1145/3441501.3441532

DO - 10.1145/3441501.3441532

M3 - Conference contribution

AN - SCOPUS:85100403582

T3 - ACM International Conference Proceeding Series

SP - 4

EP - 8

BT - FIRE 2020 - Proceedings of the 12th Annual Meeting of the Forum for Information Retrieval Evaluation

A2 - Majumder, Prasenjit

A2 - Mitra, Mandar

A2 - Gangopadhyay, Surupendu

A2 - Mehta, Parth

PB - Association for Computing Machinery

T2 - 12th Annual Meeting of the Forum for Information Retrieval Evaluation, FIRE 2020

Y2 - 16 December 2020 through 20 December 2020

ER -

Fadel A, Musleh H, Tuffaha I, Al-Ayyoub M, Jararweh Y, Benkhelifa E et al. Overview of the PAN@FIRE 2020 Task on the Authorship Identification of SOurce COde. In Majumder P, Mitra M, Gangopadhyay S, Mehta P, editors, FIRE 2020 - Proceedings of the 12th Annual Meeting of the Forum for Information Retrieval Evaluation. Association for Computing Machinery. 2020. p. 4-8. (ACM International Conference Proceeding Series). doi: 10.1145/3441501.3441532

Overview of the PAN@FIRE 2020 Task on the Authorship Identification of SOurce COde

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this