Abstract
Assessing the truthfulness of information is a critical task in fact-checking and is typically performed using binary or coarse ordinal scales (2–6 levels), though fine-grained scales (e.g., 100 levels) have also been explored. Magnitude Estimation (ME) takes this approach further by allowing assessors to assign any value in the range (0, +∞). However, it introduces challenges, including the need to aggregate assessments from individuals with different interpretations of the scale. Despite these challenges, successful applications in other domains suggest its potential suitability for truthfulness assessment. We conduct a crowdsourcing study by collecting assessments on claims sourced from the PolitiFact fact-checking organization using ME. To the best of our knowledge, this is the first systematic investigation of ME in the context of truthfulness assessment. Our results show that while aggregation methods significantly impact assessment quality, optimal aggregation strategies yield accuracy and reliability comparable to traditional scales. More importantly, ME captures subtle differences in truthfulness, offering richer insights than conventional coarse-grained scales.