crowdsourcing

Beyond Seeing Is Believing: On Crowdsourced Detection of Audiovisual Deepfakes

Workshop Talk - The 6th Workshop on Reducing Online Misinformation through Credible Information Retrieval (ROMCIR 2026), held as part of The 48th European Conference on Information Retrieval (ECIR 2026). Delft, The Netherlands.

The Magnitude of Truth: On Using Magnitude Estimation for Truthfulness Assessment

Conference Talk - The 48th International ACM SIGIR Conference on Research and Development in Information Retrieval. Padua, Italy.

The Magnitude of Truth: On Using Magnitude Estimation for Truthfulness Assessment

Evaluating the truthfulness of online content is critical for combating misinformation. This study examines the efficiency and effectiveness of crowdsourced truthfulness assessments through a comparative analysis of two approaches: one involving …

Deploying Complex Crowdsourcing Tasks Without Fear: An Introduction to Crowd_Frame

Demo Presentation - The 3rd Italian Conference on Big Data and Data Science (ITADATA 2024). Pisa, Italy.

Social Computing

Course in the Bachelor's Degree in Internet of Things, Big Data & Web at the University of Udine

Crowdsourced Fact-checking: Does It Actually Work?

There is an important ongoing effort aimed to tackle misinformation and to perform reliable fact-checking by employing human assessors at scale, with a crowdsourcing-based approach. Previous studies on the feasibility of employing crowdsourcing for …

Fact-Checking at Scale with Crowdsourcing Experiments and Lessons Learned

In this paper, we present our journey in exploring the use of crowdsourcing for fact-checking. We discuss our early experiments aimed towards the identification of the best possible setting for misinformation assessment using crowdsourcing. Our …

How Many Crowd Workers Do I Need? On Statistical Power When Crowdsourcing Relevance Judgments

To scale the size of Information Retrieval collections, crowdsourcing has become a common way to collect relevance judgments at scale. Crowdsourcing experiments usually employ 100-10,000 workers, but such a number is often decided in a heuristic way. …

The Effects of Crowd Worker Biases in Fact-Checking Tasks

Due to the increasing amount of information shared online every day, the need for sound and reliable ways of distinguishing between trustworthy and non-trustworthy information is as present as ever. One technique for performing fact-checking at scale …

Crowd_Frame: A Simple and Complete Framework to Deploy Complex Crowdsourcing Tasks Off-the-Shelf

Conference Talk - The 15th ACM International Conference on Web Search and Data Mining. February 15th, 2022. Phoenix, U.S.A. Held remotely as a pre-recorded contribution due to the COVID-19 pandemic.