When statistical models are used in social sciences, there is no presumption that actual reality is stochastic. Even if actual reality is deterministic, hidden and unobservable variables require a use of effective (in ordinary language, approximate) models, with number of variables potentially unrestricted. From the quantum reconstruction point of view, quantum mechanics can be interpreted as a statistical information-processing framework - therefore, the aforementioned view in social sciences can naturally be transported to quantum physics - this especially so given prevalent uses of effective theories. This naturally gives us the common framework behind hidden variables interpretations unconstrained by particular laws of deterministic motion. The measurement problem is cleanly identified as arising from missing hidden variables. However, hidden variables interpretations do not imply that a final UV-complete theory of physics has to be usable - the theory may be uncomputable, with inability to generate predictions. In such a case, an infinite sequence of effective quantum theories would have to be used. Therefore, even if hidden variables interpretations are correct, supremacy of the orthodox quantum mechanics framework may be upheld, which is supported by a combination of quantum reconstruction and computational complexity arguments. This allows us to bypass considerable difficulties of hidden variables interpretations in satisfying Lorentz invariance. Connections to entropic gravity are then explored.
When statistical models are used in social sciences, there is no presumption that actual reality is stochastic. Even if actual reality is deterministic, hidden and unobservable variables require a use of effective (in ordinary language, approximate) models, with number of variables potentially unrestricted. From the quantum reconstruction point of view, quantum mechanics can be interpreted as a statistical information-processing framework - therefore, the aforementioned view in social sciences can naturally be transported to quantum physics - this especially so given prevalent uses of effective theories. This naturally gives us the common framework behind hidden variables interpretations unconstrained by particular laws of deterministic motion. The measurement problem is cleanly identified as arising from missing hidden variables. However, hidden variables interpretations do not imply that a final UV-complete theory of physics has to be usable - the theory may be uncomputable, with inability to generate predictions. In such a case, an infinite sequence of effective quantum theories would have to be used. Therefore, even if hidden variables interpretations are correct, supremacy of the orthodox quantum mechanics framework may be upheld, which is supported by a combination of quantum reconstruction and computational complexity arguments. This allows us to bypass considerable difficulties of hidden variables interpretations in satisfying Lorentz invariance. Connections to entropic gravity are then explored.
The standard, or Copenhagen, formulation of quantum mechanics postulates that the complete specification of a quantum state is given by the corresponding state vector (completeness). A different approach is possible, assuming instead the incompleteness of the theory. Additional parameters, called “hidden variables” since they are not empirically known, would be needed to completely characterize the quantum state. The knowledge of these hidden variables would allow the precise determination of the values for the observables of the quantum system. In 1952 David Bohm, starting from such an assumption, has proposed a hidden variables formulation of quantum mechanics that is empirically equivalent to standard quantum mechanics, but offers a more rational and coherent picture of reality. Bohm's model integrates the ordinary quantum theory by introducing particle coordinates as hidden variables. Therefore, particles are distinguishable and describe trajectories in space or in configuration space that are causally determinate. In this context it is possible to explain double slit experiments and interference phenomena in terms of particle trajectories. Quantum probabilities become epistemic: the probabilistic nature of physical predictions is not an intrinsic characteristic of nature, but depends on our ignorance of the exact value of the hidden variables. Since Bohm's interpretation is, both epistemologically and ontologically, a natural extension of classical mechanics to the quantum domain, the visualization of physical processes is still possible, and the corresponding picture of reality is more intuitive. Bohm's interpretation clearly also presents limits and weaknesses: a Lorentz invariant formulation of the model is still lacking, and all the observables result to be dependent on the global context (contextuality), with the only exception of position observables.
The nondeterministic character of quantum measurement can, and should, be taken to imply a deeper 'hidden variable' description of a system, which reproduces quantum theory when the unknown values of the variables are marginalised over. Differences in measurements on identically prepared systems then represent differences in the hidden variables of the systems. Not to seek the hidden variables, as the Copenhagen interpretation of quantum mechanics arbitrarily instructs, is to give up all hope of improvement in advance, and is contrary to the purposes of science. Correspondingly, it can never be proven that hidden variables don't exist; the most that can be done is to place hypothetical restrictions on them and prove that the resulting theory cannot be marginalised into quantum theory - implying that the hidden variables violate the restriction. The best known example is Bell's theorem, which rules out all local hidden-variable theories and explains why the variables are so coy: it is not easy to isolate and influence something nonlocal. A promising way forward is to proceed from the Dirac theory of the electron in its natural, geometric language of Clifford algebra. This approach, due to Hestenes, suggests that the free electron is in fact a massless point particle executing zitterbewegung at the speed of light, with radius the electron Compton wavelength. This proposal improves the correspondence of the physics with the formalism and resolves many paradoxes; it also hints that the hidden variables are concerned with those parameters needed to define the electron orbit completely. These ideas all represent a demystification of quantum theory. In the 19th century, microscopes advanced to the point at which a new phenomenon, today called Brownian motion after its chronicler, could be seen. The small particles, newly made visible ('dust' particles), were observed to jiggle around in a manner the scientists could not predict, for no apparent reason. Motion of any individual dust particle could not be predicted; however, statistical details of the motion, such as the average (drift) velocity or the mean square velocity, were reproducible. The scientists of the day, probably without realising it, made an assumption: that there was a reason for the jiggling, and that if they could find this, they could predict the precise motion of individual dust particles. They never said Maybe the dust particles just behave like that and there isn't a why; maybe we can go no further and this is the end of the line. 79
The paper argues that far from challenging-or even refuting-Bohm's quantum theory, the no-hidden-variables theorems in fact support the Bohmian ontology for quantum mechanics. The reason is that (i) all measurements come down to position measurements; and (ii) Bohm's theory provides a clear and coherent explanation of the measurement outcome statistics based on an ontology of particle positions, a law for their evolution and a probability measure linked with that law. What the no-hidden-variables theorems teach us is that (i) one cannot infer the properties that the physical systems possess from observables; and that (ii) measurements, being an interaction like other interactions, change the state of the measured system.
In a recent paper (arXiv:1111.3328), Pusey, Barrett and Rudolph claim to prove that statistical interpretations of quantum mechanics do not work. In fact, their proof assumes that all statistical interpretations must be based on hidden variable realism. Effectively, the authors demand from the start that reality must be decided by mathematics, and not by measurements. If this unjustified assumption is dropped, the quantum formalism has a natural statistical interpretation that fully explains the paradox presented by the authors. It is therefore possible to conclude that the paradox actually supports the statistical interpretation, demonstrating once more that quantum mechanics should not be explained by measurement independent realities that are never observed and therefore lie beyond the reach of empirical tests.
Abstract Hidden variables are hypothetical parameters that, if known, would deterministically specify quantum measurement outcomes, aiming to restore determinism and causality typical of classical mechanics. This work provides a modern and pedagogical exposition of John von Neumann’s 1932 proof on the incompatibility of hidden variables with quantum mechanics. We detail his two main conclusions: (1) dispersion-free ensembles (as defined within the framework of hidden variable theories) are incompatible with quantum mechanics; and (2) the quantum state provides a complete description of physical reality. We critically analyze the key assumptions in von Neumann’s proof, highlighting later challenges—particularly John Bell’s critique and the development of contextual hidden-variable theories by Kochen and Specker. This discussion clarifies von Neumann’s historical role as a pioneer in this debate, while showing how modern hidden-variable theories, such as Bohmian mechanics, remain compatible with quantum predictions. Conversion tables are included between this article’s notation and von Neumann’s original book.
The Statistical Interpretation of quantum theory is formulated for the purpose of providing a sound interpretation using a minimum of assumptions. Several arguments are advanced in favor of considering the quantum state description to apply only to an ensemble of similarily prepared systems, rather than supposing, as is often done, that it exhaustively represents an individual physical system. Most of the problems associated with the quantum theory of measurement are artifacts of the attempt to maintain the latter interpretation. The introduction of hidden variables to determine the outcome of individual events is fully compatible with the statistical predictions of quantum theory. However, a theorem due to Bell seems to require that any such hidden-variable theory which reproduces all of quantum mechanics exactly (i.e., not merely in some limiting case) must possess a rather pathological character with respect to correlated, but spacially separated, systems.