ACTI at EVALITA 2023: Automatic Conspiracy Theory Identification Task Overview

Giuseppe Russo; Niklas Stoehr; Manoel Horta Ribeiro

ACTI at EVALITA 2023: Automatic Conspiracy Theory Identification Task Overview

Giuseppe Russo, Niklas Stoehr, Manoel Horta Ribeiro

Research output: Contribution to journal › Conference article › peer-review

Abstract

English. Automatic Conspiracy Theory Identification (ACTI) is a new shared task proposed for the first time at the EVALITA 2023 evaluation campaign. ACTI is based on a new, manually labeled dataset of comments scraped from conspiratorial Telegram channels and consists of two subtasks: (1) identifying conspiratorial content (conspiratorial content classification); and (2) classifying content into specific conspiracy theories (conspiratorial category classification). A total of 15 teams participated in the task with 81 submissions. In this task summary, we discuss the data and task, and outline the best-performing approaches that are largely based on large language models. We conclude with a brief discussion of the application of large language models to counter the spread of misinformation on online platforms.

Original language	English (US)
Journal	CEUR Workshop Proceedings
Volume	3473
State	Published - 2023
Externally published	Yes
Event	8th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop, EVALITA 2023 - Parma, Italy Duration: Sep 7 2023 → Sep 8 2023

All Science Journal Classification (ASJC) codes

General Computer Science

Keywords

Computational Social Science
Conspiracy Theory
Content Moderation
Large Language Models

Cite this

@article{6f840654f41843c9b1e6678a4c43bcca,

title = "ACTI at EVALITA 2023: Automatic Conspiracy Theory Identification Task Overview",

abstract = "English. Automatic Conspiracy Theory Identification (ACTI) is a new shared task proposed for the first time at the EVALITA 2023 evaluation campaign. ACTI is based on a new, manually labeled dataset of comments scraped from conspiratorial Telegram channels and consists of two subtasks: (1) identifying conspiratorial content (conspiratorial content classification); and (2) classifying content into specific conspiracy theories (conspiratorial category classification). A total of 15 teams participated in the task with 81 submissions. In this task summary, we discuss the data and task, and outline the best-performing approaches that are largely based on large language models. We conclude with a brief discussion of the application of large language models to counter the spread of misinformation on online platforms.",

keywords = "Computational Social Science, Conspiracy Theory, Content Moderation, Large Language Models",

author = "Giuseppe Russo and Niklas Stoehr and Ribeiro, {Manoel Horta}",

note = "Publisher Copyright: {\textcopyright} 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).; 8th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop, EVALITA 2023 ; Conference date: 07-09-2023 Through 08-09-2023",

year = "2023",

language = "English (US)",

volume = "3473",

journal = "CEUR Workshop Proceedings",

issn = "1613-0073",

publisher = "CEUR-WS",

}

TY - JOUR

T1 - ACTI at EVALITA 2023

T2 - 8th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop, EVALITA 2023

AU - Russo, Giuseppe

AU - Stoehr, Niklas

AU - Ribeiro, Manoel Horta

PY - 2023

Y1 - 2023

N2 - English. Automatic Conspiracy Theory Identification (ACTI) is a new shared task proposed for the first time at the EVALITA 2023 evaluation campaign. ACTI is based on a new, manually labeled dataset of comments scraped from conspiratorial Telegram channels and consists of two subtasks: (1) identifying conspiratorial content (conspiratorial content classification); and (2) classifying content into specific conspiracy theories (conspiratorial category classification). A total of 15 teams participated in the task with 81 submissions. In this task summary, we discuss the data and task, and outline the best-performing approaches that are largely based on large language models. We conclude with a brief discussion of the application of large language models to counter the spread of misinformation on online platforms.

AB - English. Automatic Conspiracy Theory Identification (ACTI) is a new shared task proposed for the first time at the EVALITA 2023 evaluation campaign. ACTI is based on a new, manually labeled dataset of comments scraped from conspiratorial Telegram channels and consists of two subtasks: (1) identifying conspiratorial content (conspiratorial content classification); and (2) classifying content into specific conspiracy theories (conspiratorial category classification). A total of 15 teams participated in the task with 81 submissions. In this task summary, we discuss the data and task, and outline the best-performing approaches that are largely based on large language models. We conclude with a brief discussion of the application of large language models to counter the spread of misinformation on online platforms.

KW - Computational Social Science

KW - Conspiracy Theory

KW - Content Moderation

KW - Large Language Models

UR - http://www.scopus.com/inward/record.url?scp=85173556036&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85173556036&partnerID=8YFLogxK

M3 - Conference article

AN - SCOPUS:85173556036

SN - 1613-0073

VL - 3473

JO - CEUR Workshop Proceedings

JF - CEUR Workshop Proceedings

Y2 - 7 September 2023 through 8 September 2023

ER -

ACTI at EVALITA 2023: Automatic Conspiracy Theory Identification Task Overview

Abstract

All Science Journal Classification (ASJC) codes

Keywords

Other files and links

Fingerprint

Cite this