Xanadu, GPT, and Beyond: An Adventure of the Mind

Abstract

This article recounts an intellectual journey that began in curiosity about the structure of Coleridge’s “Kubla Khan” in the late 1960s and has led to an interest in large language models at the present time. A close analysis of the poem revealed its two parts each to have a nested structure (think of a matryoshka doll) that suggested the operation of an underlying computational process (nested loops). That led to the study of computational linguistics (semantic networks), followed by neuroscience (Karl Pribram’s neural holography), and cultural evolution. In the 2010s I began following work digital humans had been doing with machine learning. When GPT-3 was released in 2020 I was ready, though it took me awhile to establish a link, however tentative, between that conceptual universe and that of “Kubla Khan.”

Keywords: poetics, Kubla Khan, computational linguistics, machine learning, GPT, tranformers, vector semantics, neurscience