Publications

38 publications

2026

Beyond Seeing Is Believing: On Crowdsourced Detection of Audiovisual Deepfakes

Michael Soprano, Andrea Cioci, Stefano Mizzaro

6th Workshop on Reducing Online Misinformation through Credible Information Retrieval (ROMCIR 2026), held in conjunction with ECIR 2026.

Materials PDF DOI

Large Language Models as Assessors: On the Impact of Relevance Scales

Riccardo Zamolo, Riccardo Lunardi, Michael Soprano, Gianluca Demartini, Stefano Mizzaro, Kevin Roitero

Advances in Information Retrieval: 48th European Conference on Information Retrieval (ECIR 2026). Conference Rank: CORE A; GGS A-.

Materials DOI

Analyzing AI Evaluation Benchmarks Through Information Retrieval and Network Science

Gaia Simeoni, Michael Soprano, Riccardo Lunardi, Kevin Roitero, Stefano Mizzaro

Advances in Information Retrieval: 48th European Conference on Information Retrieval (ECIR 2026), Lecture Notes in Computer Science, vol. 16484, Springer, Cham. Conference Rank: CORE A; GGS A-.

Dataset DOI

Large Language Models for Combinatorial Optimization: A Systematic Review

Francesca Da Ros, Michael Soprano, Luca Di Gaspero, Kevin Roitero

ACM Computing Surveys. Journal Ranks: Journal Citation Reports (JCR) Q1 (2024), Scimago (SJR) Q1 (2024).

PDF DOI

2025

AIDME: A Scalable, Interpretable Framework for AI-Aided Scoping Reviews

Michael Soprano, Sandip Modha, Kevin Roitero, Eddy Maddalena, Marco Viviani, Gabriella Pasi, Stefano Mizzaro

Proceedings of the 2025 International ACM SIGIR Conference on Innovative Concepts and Theories in Information Retrieval (ICTIR)

Materials PDF DOI

The Magnitude of Truth: On Using Magnitude Estimation for Truthfulness Assessment

Michael Soprano, Denis Eduard Tapu, David La Barbera, Kevin Roitero, Stefano Mizzaro

Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval. Conference Rank: GGS A++, Core A.

Materials PDF DOI

PILs of Knowledge: A Synthetic Benchmark for Evaluating Question Answering Systems in Healthcare

Riccardo Lunardi, Michael Soprano, Paolo Coppola, Vincenzo Della Mea, Stefano Mizzaro, Kevin Roitero

Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval. Conference Rank: GGS A++, Core A.

Dataset PDF DOI

Efficiency and Effectiveness of LLM-Based Summarization of Evidence in Crowdsourced Fact-Checking

Kevin Roitero, Dustin Wright, Michael Soprano, Isabelle Augenstein, Stefano Mizzaro

Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval. Conference Rank: GGS A++, Core A.

Dataset PDF DOI

In Crowd Veritas: Leveraging Human Intelligence To Fight Misinformation

Michael Soprano

Revised version of the PhD thesis, defended on 22 May 2023; arXiv:2506.09221.

Materials Code PDF DOI

Agent-Based Healthcare Chatbots for Regional System Services: A Case Study in the Friuli-Venezia Giulia Region

Riccardo Lunardi, Kevin Roitero, Michael Soprano, Paolo Coppola, Vincenzo Della Mea, Stefano Mizzaro

Intelligent Health Systems – From Technology to Data and Knowledge.

DOI

Search Trajectory Networks Applied to a Real-World Parallel Batch Scheduling Problem

Francesca Da Ros, Luca Di Gaspero, Marie-Louise Lackner, Nysret Musliu, Michael Soprano

Applications of Evolutionary Computation (EvoApplications 2025). Conference Rank: GGS B.

DOI

Evaluation of Crowdsourced Peer Review using Synthetic Data and Simulations

Michael Soprano, Eddy Maddalena, Francesca Da Ros, Maria Elena Zuliani, Stefano Mizzaro

Proceedings of the 21st Conference on Information and Research Science Connecting to Digital and Library Science.

Materials PDF

2024

Report on the Hands-On PhD Course on Responsible AI from the Lens of an Information Access Researcher

Damiano Spina, Kevin Roitero, Stefano Mizzaro, Vincenzo Della Mea, Francesca Da Ros, Michael Soprano, Hafsa Akebli, Alex Falcon, Mehdi Fasihi, Alessio Fiorin, David La Barbera, Daniele Lizzio Bosco, Riccardo Lunardi, Alberto Marturano, Zaka-Ud-Din Muhammad, Francesco Nascimben, Moritz Nottebaum, Massimiliano Pascoli, Mihai Horia Popescu, Laura Rasotto, Mubashara Rehman, Francesco Taverna, Biagio Tomasetig, Alessandro Tremamunno

SIGIR Forum

Materials PDF DOI

Report on the 14th Italian Information Retrieval Workshop (IIR 2024)

Eddy Maddalena, Stefano Mizzaro, Kevin Roitero, Marco Viviani, David La Barbera, Sandip Modha, Gabriella Pasi, Francesca Da Ros, Michael Soprano

SIGIR Forum

PDF DOI

Enhancing Fact-Checking: From Crowdsourced Validation to Integration with Large Language Models

Kevin Roitero, Michael Soprano, David La Barbera, Eddy Maddalena, Stefano Mizzaro

Proceedings of the 14th Italian Information Retrieval Workshop.

PDF

Crowdsourcing Statement Classification to Enhance Information Quality Prediction

Jaspreet Singh, Michael Soprano, Kevin Roitero, Davide Ceolin

Proceedings of the 6th Multidisciplinary International Symposium on Disinformation in Open Online Media (MISDOOM 2024). Münster, Germany.

DOI

Longitudinal Loyalty: Understanding The Barriers To Running Longitudinal Studies On Crowdsourcing Platforms

Michael Soprano, Kevin Roitero, Ujwal Gadiraju, Eddy Maddalena, Gianluca Demartini

ACM Transactions on Social Computing. Journal Rank: Scimago (SJR) Q2 (2023).

Materials PDF DOI

Crowdsourced Fact-checking: Does It Actually Work?

David La Barbera, Eddy Maddalena, Michael Soprano, Kevin Roitero, Gianluca Demartini, Davide Ceolin, Damiano Spina, Stefano Mizzaro

Information Processing & Management. Journal Ranks: Journal Citation Reports (JCR) Q1 (2023), Scimago (SJR) Q1 (2023).

Materials DOI

Cognitive Biases in Fact-Checking and Their Countermeasures: A Review

Michael Soprano, Kevin Roitero, David La Barbera, Davide Ceolin, Damiano Spina, Gianluca Demartini, Stefano Mizzaro

Information Processing & Management. Journal Ranks: Journal Citation Reports (JCR) Q1 (2022), Scimago (SJR) Q1 (2022).

DOI

2023

Towards a Conversational-Based Agent for Health Services

Michael Soprano, Kevin Roitero, Vincenzo Della Mea, Stefano Mizzaro

Proceedings of the Italia Intelligenza Artificiale - Thematic Workshops co-located with the 3rd CINI National Lab AIIS Conference on Artificial Intelligence.

PDF

Fact-Checking at Scale with Crowdsourcing: Experiments and Lessons Learned

David La Barbera, Michael Soprano, Kevin Roitero, Eddy Maddalena, Stefano Mizzaro

Proceedings of the 13th Italian Information Retrieval Workshop.

PDF

How Many Crowd Workers Do I Need? On Statistical Power When Crowdsourcing Relevance Judgments

Kevin Roitero, David La Barbera, Michael Soprano, Gianluca Demartini, Stefano Mizzaro, Tetsuya Sakai

ACM Transactions on Information Systems. Journal Ranks: Journal Citation Reports (JCR) Q1 (2021), Scimago (SJR) Q1 (2021).

Materials Code PDF DOI

2022

Transparent assessment of information quality of online reviews using formal argumentation theory

Davide Ceolin, Giuseppe Primiero, Michael Soprano, Jan Wielemaker

Information Systems. Journal Ranks: Journal Citation Reports (JCR) Q2 (2021), Scimago (SJR) Q1 (2021).

Code DOI

A Neural Model to Jointly Predict and Explain Truthfulness of Statements

Erik Brand, Kevin Roitero, Michael Soprano, Afshin Rahimi, Gianluca Demartini

Journal of Data and Information Quality, July 2022. Journal Ranks: Journal Citation Reports (JCR) Q3 (2021), Scimago (SJR) Q2 (2021).

Code PDF DOI

The Effects of Crowd Worker Biases in Fact-Checking Tasks

Tim Draws, David La Barbera, Michael Soprano, Kevin Roitero, Davide Ceolin, Alessandro Checco, Stefano Mizzaro

Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency.

Materials PDF DOI

Crowd_Frame: A Simple and Complete Framework to Deploy Complex Crowdsourcing Tasks Off-the-Shelf

Michael Soprano, Kevin Roitero, Francesco Bombassei De Bona, Stefano Mizzaro

Proceedings of the 15th ACM International Conference on Web Search and Data Mining (WSDM 2022). Virtual Event, AZ, USA. Conference Ranks: GGS A+, Core A\.

Code PDF DOI

2021

The Many Dimensions of Truthfulness: Crowdsourcing Misinformation Assessments on a Multidimensional Scale

Michael Soprano, Kevin Roitero, David La Barbera, Davide Ceolin, Damiano Spina, Stefano Mizzaro, Gianluca Demartini

Information Processing & Management. Journal Ranks: Journal Citation Reports (JCR) Q1 (2021), Scimago (SJR) Q1 (2021).

Dataset DOI

E-BART: Jointly Predicting and Explaining Truthfulness

Erik Brand, Kevin Roitero, Michael Soprano, Gianluca Demartini

Proceedings of the 2021 Truth and Trust Online Conference

Code PDF

Can The Crowd Judge Truthfulness? A Longitudinal Study on Recent Misinformation About COVID-19

Kevin Roitero, Michael Soprano, Beatrice Portelli, Massimiliano De Luise, Damiano Spina, Vincenzo Della Mea, Giuseppe Serra, Stefano Mizzaro, Gianluca Demartini

Personal and Ubiquitous Computing. Journal Ranks: Journal Citation Reports (JCR) Q2 (2020), Scimago (SJR) Q1 (2021).

Dataset PDF DOI

Assessing the Quality of Online Reviews Using Formal Argumentation Theory

Davide Ceolin, Giuseppe Primiero, Jan Wielemaker, Michael Soprano

Proceedings of the 21st International Conference on Web Engineering. Conference Ranks (2021): GGS B-, Core B.

Code DOI

2020

The COVID-19 Infodemic: Can the Crowd Judge Recent Misinformation Objectively?

Kevin Roitero, Michael Soprano, Beatrice Portelli, Damiano Spina, Vincenzo Della Mea, Giuseppe Serra, Stefano Mizzaro, Gianluca Demartini

Proceedings of the 29th ACM International Conference on Information and Knowledge Management. Conference Ranks: GGS A+, Core A.

Dataset DOI

Can The Crowd Identify Misinformation Objectively? The Effects of Judgment Scale and Assessor's Background

Kevin Roitero, Michael Soprano, Shaoyang Fan, Damiano Spina, Stefano Mizzaro, Gianluca Demartini

Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. Conference Rank: GGS A++, Core A.

Dataset Code DOI

2019

Bias and Fairness in Effectiveness Evaluation by Means of Network Analysis and Mixture Models

Michael Soprano, Kevin Roitero, Stefano Mizzaro

Proceedings of the 10th Italian Information Retrieval Workshop.

PDF

HITS Hits Readersourcing: Validating Peer Review Alternatives Using Network Analysis

Michael Soprano, Kevin Roitero, Stefano Mizzaro

Proceedings of the 4th Joint Workshop on Bibliometric-enhanced Information Retrieval and Natural Language Processing for Digital Libraries co-located with the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval.

PDF