Large Language Models as Assessors: On the Impact of Relevance Scales
Poster - The 48th European Conference on Information Retrieval (ECIR 2026). Delft, The Netherlands.
riccardo-zamolo
Riccardo Lunardi
•
Mar 30, 2026 13:30 — 14:30
•
1 min read
Analyzing AI Evaluation Benchmarks Through Information Retrieval and Network Science
Poster - The 48th European Conference on Information Retrieval (ECIR 2026). Delft, The Netherlands.
gaia-simeoni
Large Language Models as Assessors: On the Impact of Relevance Scales
Traditionally, relevance judgments have relied on human annotators, but recent advances in Large Language Models (LLMs) have prompted growing interest in their use as a proxy for …
riccardo-zamolo
Analyzing AI Evaluation Benchmarks Through Information Retrieval and Network Science
Many analyses have been performed on Information Retrieval (IR) evaluation benchmarks. Benchmarking also plays a central role in evaluating the capabilities of Large Language …
gaia-simeoni
PILs of Knowledge: A Synthetic Benchmark for Evaluating Question Answering Systems in Healthcare
Patient Information Leaflets (PILs) provide essential information about medication usage, side effects, precautions, and interactions, making them a valuable resource for Question …
riccardo-lunardi
Agent-Based Healthcare Chatbots for Regional System Services: A Case Study in the Friuli-Venezia Giulia Region
This paper presents an agent-based model for interacting with regional healthcare system services in the Friuli-Venezia Giulia (FVG) region. The model is designed to streamline and …
riccardo-lunardi
Report on the Hands-On PhD Course on Responsible AI from the Lens of an Information Access Researcher
While the concept of responsible AI is becoming more and more popular, practitioners and researchers may often struggle to characterize responsible practices in their own work. …
damiano-spina