Lost in Conversation? Hermeneutics, uncertainty and large language models

Abstract

Technical uncertainty estimation methods do have a role to play in tackling the epistemic calibration challenges that stem from LLMs’ opaque ‘epistemic facade’. Excessive focus on the latter methods however risks concealing the pervasive ways in which the uncertainty frames embodied by LLMs stand to reshape the larger cultural patterns and norms that structure our navigation of ethically fraught domains. The fruitfulness of the conversations that contribute to refining the intuitions at the heart of our morally-loaded stances not only depends on our awareness of our moral fallibility. It also depends on the way that awareness is translated into discursive markers of epistemic humility. In such contexts, signalling uncertainty creates conversational space for genuine exchange across difference. Given LLMs’ rapidly growing role in these conversational spaces, there is an urgent need to widen the research frame when it comes to LLMs’ uncertainty communication capabilities. Can LLMs be ‘taught’ to linguistically manifest moral uncertainty in ways that not only prevent unwarranted epistemic confidence, but also foster a ‘spirit of enquiry’? To make progress in the above line of work requires us to understand the unique dynamics of human-LLM linguistic collaborations, as they co-constitute new forms of sense-making. What does it mean for human processes of reality construction and meaning negotiation to become intertwined with language models? While it is informed by those broad philosophical questions, the concern at the heart of this paper is met with possible ways forward when it comes to designing participatory interfaces that incentivise collective feedback about an LLMs’ uncertainty communication strategies.

Keywords: Generative AI, Large language models, uncertainty, communication of uncertainty, Dewey, spirit of enquiry