Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

arXiv:2208.05720 · Electronic Proceedings in Theoretical Computer Science

A Model of Anaphoric Ambiguities using Sheaf Theoretic Quantum-like Contextuality and BERT

Kin Ian Lo, Mehrnoosh Sadrzadeh, Shane Mansfield

Published Aug 9, 2022• 4 citations
arXivHTMLPDF

Abstract

Ambiguities of natural language do not preclude us from using it and context helps in getting ideas across. They, nonetheless, pose a key challenge to the development of competent machines to understand natural language and use it as humans do. Contextuality is an unparalleled phenomenon in quantum mechanics, where different mathematical formalisms have been put forwards to understand and reason about it. In this paper, we construct a schema for anaphoric ambiguities that exhibits quantum-like contextuality. We use a recently developed criterion of sheaf-theoretic contextuality that is applicable to signalling models. We then take advantage of the neural word embedding engine BERT to instantiate the schema to natural language examples and extract probability distributions for the instances. As a result, plenty of sheaf-contextual examples were discovered in the natural language corpora BERT utilises. Our hope is that these examples will pave the way for future research and for finding ways to extend applications of quantum computing to natural language processing.

DOI
10.4204/eptcs.366.5

Related papers

Quantum-like contextuality in large language models

Kin Ian Lo, Mehrnoosh Sadrzadeh, Shane Mansfield

Contextuality is a distinguishing feature of quantum mechanics, and there is evidence that it is a necessary condition for quantum advantage in the context of magic state distillation. To harness it, researchers have been asking whether similar phenomena arise in other domains. The answer has been affirmative, e.g. in behavioural sciences. However, one has to adopt frameworks that take some degree of signalling into account. Two such frameworks exist: (i) a signalling-corrected sheaf-theoretic model and (ii) the Contextuality-by-Default (CbD) framework. This paper provides the first large-scale experimental evidence for contextuality in the large language model BERT. We constructed a linguistic schema modelled over a contextual quantum scenario, instantiated it in the Simple English Wikipedia, and extracted probability distributions for the instances. This led to the discovery of 77 118 sheaf-contextual and 36 938 948 CbD contextual instances. We prove that these contextual instances arise from semantically similar words by deriving an equation that relates degrees of contextuality to the Euclidean distance of BERT’s embedding vectors. Our linguistic schema is a variant of the coreference resolution challenge. These results suggest that quantum methods may be advantageous in the field of natural language processing.

arXiv:2412.16806•Proceedings of the Royal Society A Mathematical Physical and Engineering Sciences•Aug 1, 2025•4 citations

On the Quantum-like Contextuality of Ambiguous Phrases

Daphne Wang, Mehrnoosh Sadrzadeh, Samson Abramsky +1 more

Language is contextual as meanings of words are dependent on their contexts. Contextuality is, concomitantly, a well-defined concept in quantum mechanics where it is considered a major resource for quantum computations. We investigate whether natural language exhibits any of the quantum mechanics' contextual features. We show that meaning combinations in ambiguous phrases can be modelled in the sheaf-theoretic framework for quantum contextuality, where they can become possibilistically contextual. Using the framework of Contextuality-by-Default (CbD), we explore the probabilistic variants of these and show that CbD-contextuality is also possible.

arXiv:2107.14589•Jul 19, 2021•14 citations

Quantum Contextuality in Natural Language

Kin Ian Lo

Quantum computing has demonstrated computational advantages over classical computing, yet its applications to natural language processing (NLP) remain in their early stages. Quantum contextuality, a fundamental feature of quantum mechanics, has been identified as a resource for achieving these advantages. Meanwhile, ambiguities in human languages present challenges in building systems capable of understanding and generating natural language. Many of these ambiguities can be addressed by considering the context in which they arise. This thesis investigates the connection between quantum contextuality and ambiguities in natural language. It introduces a framework that models ambiguities in language as a form of measurement, analogous to physical experiments. The findings reveal that contextuality is present in natural language and that its degree is linked to the reasoning capabilities of large language models (LLMs).

•Oct 28, 2025•0 citations

Generalised Winograd Schema and its Contextuality

Kin Ian Lo, Mehrnoosh Sadrzadeh, Shane Mansfield

Ambiguities in natural language give rise to probability distributions over interpretations. The distributions are often over multiple ambiguous words at a time; a multiplicity which makes them a suitable topic for sheaf-theoretic models of quantum contextuality. Previous research showed that different quantitative measures of contextuality correlate well with Psycholinguistic research on lexical ambiguities. In this work, we focus on coreference ambiguities and investigate the Winograd Schema Challenge (WSC), a test proposed by Levesque in 2011 to evaluate the intelligence of machines. The WSC consists of a collection of multiple-choice questions that require disambiguating pronouns in sentences structured according to the Winograd schema, in a way that makes it difficult for machines to determine the correct referents but remains intuitive for human comprehension. In this study, we propose an approach that analogously models the Winograd schema as an experiment in quantum physics. However, we argue that the original Winograd Schema is inherently too simplistic to facilitate contextuality. We introduce a novel mechanism for generalising the schema, rendering it analogous to a Bell-CHSH measurement scenario. We report an instance of this generalised schema, complemented by the human judgements we gathered via a crowdsourcing platform. The resulting model violates the Bell-CHSH inequality by 0.192, thus exhibiting contextuality in a coreference resolution setting.

arXiv:2308.16498•Electronic Proceedings in Theoretical Computer Science•Aug 23, 2023•5 citations

Logical and topological contextuality in quantum mechanics and beyond

Giovanni Carù

The main subjects of this thesis are non-locality and contextuality, two fundamental features of quantum mechanics that constitute valuable resources for quantum computation. Our analysis is based on Abramsky & Brandenburger's sheaf theoretic framework, which captures both these phenomena in a unified treatment and in a very general setting. This high-level description transcends quantum physics and allows to precisely characterise the notion of contextuality as the apparent paradox realised by data being locally consistent but globally inconsistent. More specifically, we aim to develop a deeper understanding of so-called logical forms of contextuality, i.e. situations where the phenomenon can be witnessed using purely logical arguments, disregarding probabilities. The sheaf theoretic description of logical contextuality has recently inspired the development of a topological treatment of the phenomenon based on sheaf cohomology. In this thesis, we embark on a detailed analysis of the cohomology of contextuality, exposing key shortcomings in the current methods, and introducing an (almost) complete cohomological characterisation of logical forms of contextuality. More specifically, we show that, in its current formulation, sheaf cohomology does not constitute a complete invariant for contextuality, not even in its strongest forms, and that higher cohomology groups cannot be used to study the phenomenon. Then, we solve these issues by introducing a novel construction, which derives refined versions of the presheaves describing empirical models to expose their deeper extendability properties, resulting in a sheaf cohomological invariant which is applicable to the vast majority of empirical models, and conjectured to work universally. We propose a general theory of contextual semantics using the language of valuation algebras. In particular, we give a general definition of contextual behaviour as a fundamental gap between local agreement and global disagreement of information sources. Not only does this formalism aptly capture and generalise the known instances of contextuality beyond quantum theory, but it also provides inspiration for further applications of the phenomenon, and paves the way for the transfer of results and techniques between different fields. We give a prime example of this potential by developing faster algorithms to detect contextuality based on mainstream methods of generic inference. Finally, we turn our attention back to instances of contextuality in quantum physics, and study strong contextuality in multi-qubit states. We give a complete combinatorial characterisation of All-vs-Nothing proofs of strong contextuality in stabiliser quantum mechanics. This allows to produce the complete list of all stabiliser states exhibiting this kind of contextuality, which consitututes an important resource in certain models of quantum computation. Then, we extend our search for strongly contextual behaviour beyond stabiliser states, and identify the minimum quantum resources needed to realise strong non-locality. Additional results include a partial classification of strongly non-local models comprised of three-qubit states and local projective measurements, and the introduction of a new infinite family of strongly non-local three-qubit states.

•Jan 1, 2019•3 citations

Sheaf-Theoretic Semantics and Quantum Contextuality in Large Language Models: A Unified Categorical Framework for Understanding Semantic Coherence and Hallucination Phenomena

Pranay M. Mahendrakar

Life of Research AbstractThis paper introduces a novel theoretical framework for understanding semantic processing in Large LanguageModels through the lens of sheaf theory and quantum contextuality. We propose that the semantic structure ofLLM-generated text can be rigorously modeled as a sheaf over the topological space of prompt contexts, wherelocal semantic sections may fail to glue into globally consistent interpretations. This mathematical formalizationreveals a deep structural parallel between LLM behavior and quantum mechanical systems exhibitingcontextuality, as characterized by the Kochen-Specker theorem. Our central thesis posits that hallucinationphenomena in LLMs arise fundamentally from cohomological obstructions to the existence of global semanticsections, analogous to how quantum systems exhibit measurement outcomes that cannot be explained by preexisting hidden variables. We develop a complete categorical framework using topos-theoretic methods, introduceformal definitions of semantic presheaves and their associated Grothendieck topologies, and demonstrate howCech cohomology groups can quantify the degree of semantic inconsistency in model outputs. This work bridgesabstract mathematics with practical AI interpretability, offering new diagnostic tools for understanding when andwhy language models produce incoherent or factually inconsistent outputs.

•Dec 28, 2025•0 citations
Ambiguities of natural language do not preclude us from using it and context helps in getting ideas across. They, nonetheless, pose a key challenge to the development of competent machines to understand natural language and use it as humans do. Contextuality is an unparalleled phenomenon in quantum mechanics, where different mathematical formalisms have been put forwards to understand and reason about it. In this paper, we construct a schema for anaphoric ambiguities that exhibits quantum-like contextuality. We use a recently developed criterion of sheaf-theoretic contextuality that is applicable to signalling models. We then take advantage of the neural word embedding engine BERT to instantiate the schema to natural language examples and extract probability distributions for the instances. As a result, plenty of sheaf-contextual examples were discovered in the natural language corpora BERT utilises. Our hope is that these examples will pave the way for future research and for finding ways to extend applications of quantum computing to natural language processing.
Contextuality is a distinguishing feature of quantum mechanics, and there is evidence that it is a necessary condition for quantum advantage in the context of magic state distillation. To harness it, researchers have been asking whether similar phenomena arise in other domains. The answer has been affirmative, e.g. in behavioural sciences. However, one has to adopt frameworks that take some degree of signalling into account. Two such frameworks exist: (i) a signalling-corrected sheaf-theoretic model and (ii) the Contextuality-by-Default (CbD) framework. This paper provides the first large-scale experimental evidence for contextuality in the large language model BERT. We constructed a linguistic schema modelled over a contextual quantum scenario, instantiated it in the Simple English Wikipedia, and extracted probability distributions for the instances. This led to the discovery of 77 118 sheaf-contextual and 36 938 948 CbD contextual instances. We prove that these contextual instances arise from semantically similar words by deriving an equation that relates degrees of contextuality to the Euclidean distance of BERT’s embedding vectors. Our linguistic schema is a variant of the coreference resolution challenge. These results suggest that quantum methods may be advantageous in the field of natural language processing.
Language is contextual as meanings of words are dependent on their contexts. Contextuality is, concomitantly, a well-defined concept in quantum mechanics where it is considered a major resource for quantum computations. We investigate whether natural language exhibits any of the quantum mechanics' contextual features. We show that meaning combinations in ambiguous phrases can be modelled in the sheaf-theoretic framework for quantum contextuality, where they can become possibilistically contextual. Using the framework of Contextuality-by-Default (CbD), we explore the probabilistic variants of these and show that CbD-contextuality is also possible.
Quantum computing has demonstrated computational advantages over classical computing, yet its applications to natural language processing (NLP) remain in their early stages. Quantum contextuality, a fundamental feature of quantum mechanics, has been identified as a resource for achieving these advantages. Meanwhile, ambiguities in human languages present challenges in building systems capable of understanding and generating natural language. Many of these ambiguities can be addressed by considering the context in which they arise. This thesis investigates the connection between quantum contextuality and ambiguities in natural language. It introduces a framework that models ambiguities in language as a form of measurement, analogous to physical experiments. The findings reveal that contextuality is present in natural language and that its degree is linked to the reasoning capabilities of large language models (LLMs).
Ambiguities in natural language give rise to probability distributions over interpretations. The distributions are often over multiple ambiguous words at a time; a multiplicity which makes them a suitable topic for sheaf-theoretic models of quantum contextuality. Previous research showed that different quantitative measures of contextuality correlate well with Psycholinguistic research on lexical ambiguities. In this work, we focus on coreference ambiguities and investigate the Winograd Schema Challenge (WSC), a test proposed by Levesque in 2011 to evaluate the intelligence of machines. The WSC consists of a collection of multiple-choice questions that require disambiguating pronouns in sentences structured according to the Winograd schema, in a way that makes it difficult for machines to determine the correct referents but remains intuitive for human comprehension. In this study, we propose an approach that analogously models the Winograd schema as an experiment in quantum physics. However, we argue that the original Winograd Schema is inherently too simplistic to facilitate contextuality. We introduce a novel mechanism for generalising the schema, rendering it analogous to a Bell-CHSH measurement scenario. We report an instance of this generalised schema, complemented by the human judgements we gathered via a crowdsourcing platform. The resulting model violates the Bell-CHSH inequality by 0.192, thus exhibiting contextuality in a coreference resolution setting.
The main subjects of this thesis are non-locality and contextuality, two fundamental features of quantum mechanics that constitute valuable resources for quantum computation. Our analysis is based on Abramsky & Brandenburger's sheaf theoretic framework, which captures both these phenomena in a unified treatment and in a very general setting. This high-level description transcends quantum physics and allows to precisely characterise the notion of contextuality as the apparent paradox realised by data being locally consistent but globally inconsistent. More specifically, we aim to develop a deeper understanding of so-called logical forms of contextuality, i.e. situations where the phenomenon can be witnessed using purely logical arguments, disregarding probabilities. The sheaf theoretic description of logical contextuality has recently inspired the development of a topological treatment of the phenomenon based on sheaf cohomology. In this thesis, we embark on a detailed analysis of the cohomology of contextuality, exposing key shortcomings in the current methods, and introducing an (almost) complete cohomological characterisation of logical forms of contextuality. More specifically, we show that, in its current formulation, sheaf cohomology does not constitute a complete invariant for contextuality, not even in its strongest forms, and that higher cohomology groups cannot be used to study the phenomenon. Then, we solve these issues by introducing a novel construction, which derives refined versions of the presheaves describing empirical models to expose their deeper extendability properties, resulting in a sheaf cohomological invariant which is applicable to the vast majority of empirical models, and conjectured to work universally. We propose a general theory of contextual semantics using the language of valuation algebras. In particular, we give a general definition of contextual behaviour as a fundamental gap between local agreement and global disagreement of information sources. Not only does this formalism aptly capture and generalise the known instances of contextuality beyond quantum theory, but it also provides inspiration for further applications of the phenomenon, and paves the way for the transfer of results and techniques between different fields. We give a prime example of this potential by developing faster algorithms to detect contextuality based on mainstream methods of generic inference. Finally, we turn our attention back to instances of contextuality in quantum physics, and study strong contextuality in multi-qubit states. We give a complete combinatorial characterisation of All-vs-Nothing proofs of strong contextuality in stabiliser quantum mechanics. This allows to produce the complete list of all stabiliser states exhibiting this kind of contextuality, which consitututes an important resource in certain models of quantum computation. Then, we extend our search for strongly contextual behaviour beyond stabiliser states, and identify the minimum quantum resources needed to realise strong non-locality. Additional results include a partial classification of strongly non-local models comprised of three-qubit states and local projective measurements, and the introduction of a new infinite family of strongly non-local three-qubit states.
Life of Research AbstractThis paper introduces a novel theoretical framework for understanding semantic processing in Large LanguageModels through the lens of sheaf theory and quantum contextuality. We propose that the semantic structure ofLLM-generated text can be rigorously modeled as a sheaf over the topological space of prompt contexts, wherelocal semantic sections may fail to glue into globally consistent interpretations. This mathematical formalizationreveals a deep structural parallel between LLM behavior and quantum mechanical systems exhibitingcontextuality, as characterized by the Kochen-Specker theorem. Our central thesis posits that hallucinationphenomena in LLMs arise fundamentally from cohomological obstructions to the existence of global semanticsections, analogous to how quantum systems exhibit measurement outcomes that cannot be explained by preexisting hidden variables. We develop a complete categorical framework using topos-theoretic methods, introduceformal definitions of semantic presheaves and their associated Grothendieck topologies, and demonstrate howCech cohomology groups can quantify the degree of semantic inconsistency in model outputs. This work bridgesabstract mathematics with practical AI interpretability, offering new diagnostic tools for understanding when andwhy language models produce incoherent or factually inconsistent outputs.