The success of Newton's mechanic, Maxwell's electrodynamic, Einstein's theories of relativity, and quantum mechanics is a strong argument for the space-time continuum. Nevertheless, doubts have been expressed about the use of a continuum in a science squarely based on observation and measurement. An exact science requires that qualitative arguments must be reduced to quantitative statements. The observability of a continuum can be reduced from qualitative arguments to quantitative statements by means of information theory. Information theory was developed during the last decades within electrical communications, but it is almost unknown in physics. The closest approach to information theory in physics is the calculus of propositions, which has been used in books on the frontier of quantum mechanics and the general theory of relativity. Principles of information theory are discussed in this book. The ability to think readily in terms of a finite number of discrete samples is developed over many years of using information theory and digital computers, just as the ability to think readily in terms of a continuum is developed by long use of differential calculus
The success of Newton's mechanic, Maxwell's electrodynamic, Einstein's theories of relativity, and quantum mechanics is a strong argument for the space-time continuum. Nevertheless, doubts have been expressed about the use of a continuum in a science squarely based on observation and measurement. An exact science requires that qualitative arguments must be reduced to quantitative statements. The observability of a continuum can be reduced from qualitative arguments to quantitative statements by means of information theory. Information theory was developed during the last decades within electrical communications, but it is almost unknown in physics. The closest approach to information theory in physics is the calculus of propositions, which has been used in books on the frontier of quantum mechanics and the general theory of relativity. Principles of information theory are discussed in this book. The ability to think readily in terms of a finite number of discrete samples is developed over many years of using information theory and digital computers, just as the ability to think readily in terms of a continuum is developed by long use of differential calculus
At this point in time, two major areas of physics, statistical mechanics and quantum mechanics, rest on the foundations of probability and entropy. The last century saw several significant fundamental advances in our understanding of the process of inference, which make it clear that these are inferential theories. That is, rather than being a description of the behavior of the universe, these theories describe how observers can make optimal predictions about the universe. In such a picture, information plays a critical role. What is more is that little clues, such as the fact that black holes have entropy, continue to suggest that information is fundamental to physics in general. In the last decade, our fundamental understanding of probability theory has led to a Bayesian revolution. In addition, we have come to recognize that the foundations go far deeper and that Cox's approach of generalizing a Boolean algebra to a probability calculus is the first specific example of the more fundamental idea of assigning valuations to partially-ordered sets. By considering this as a natural way to introduce quantification to the more fundamental notion of ordering, one obtains an entirely new way of deriving physical laws. I will introduce this new way of thinking by demonstrating how one can quantify partially-ordered sets and, in the process, derive physical laws. The implication is that physical law does not reflect the order in the universe, instead it is derived from the order imposed by our description of the universe. Information physics, which is based on understanding the ways in which we both quantify and process information about the world around us, is a fundamentally new approach to science.
An interpretation and re-formulation of modern physics which removes the presumption of the space-time continuum, and bases physical theory on a small number of rational and empirical principles. After briefly describing the philosophical principles underlying the theory, we rigorously construct a discrete model of quantum mechanics. Kets are names, or labels which categorise states of matter but do not directly describe them. The principle of superposition is seen as a definitional truism in the categorisation of states. This resolves the measurement problem of quantum mechanics by attributing the collapse of the wave function to information. The probability interpretation has a natural meaning in which the configuration of interacting particles plays the role of a hidden variable. The model supports a form of relativistic quantum field theory which does not depend on quantisation or second quantisation from classical mechanics. Continuous laws of wave mechanics are found in a discrete metaphysic which does not involve waves. Classical law is the expected behaviour of many elementary particles. The constraints on the theory are sufficient to establish that particles are point-like entities with specific properties found in nature. Newton's first law and conservation of momentum are established from the principle of homogeneity. Maxwell's equations are derived from the simple interaction in which a Dirac particle emits or absorbs a photon. Feynman rules are calculated for the discrete theory and differ from the standard rules by the removal of the ultraviolet divergence and the use of proper loop integrals. They give finite and unambiguous results.
What is information? A short answer could be: Anything that changes our minds and states of thinking! Information is virtual. It can be carried by speech, an image, scratches on a stone, patterns in a photon field, the connections of neurons in our brains or even by the wavefunction of an electron. Information does not depend on the actual means of transmission nor does its identity change when changing the carrier. Information is physical. It needs to be sustained by a physical substrate and rearranging that substrate to contain the wanted information requires to spend energy or work. Without a physical world, information can not be stored, processed, or transmitted. Our world is permeated with different representations of information. Actually Information about the physical laws governing our Universe can be found everywhere and in everything. Could it then be that the real fundamental elements of this world are tiny bits of information? Could we get it from bit?1 A number of researcher investigate this possibility, and several of their attempts to recover physical laws from the principles governing information are reported about in this special issue on physics of information. The articles by Gregg Jaeger,2 by Ariel Caticha,3 and by John Skilling and Kevin Knuth4 address the reconstruction of quantum mechanics from information theory. Furthermore, Kevin Knuth and James Walsh5 show how to recover space-time from particles sending each others signals in an initially geometry-free setting. A central element of most works on information is the concept of a probability. Probabilities can be defined via various routes. Most familiar might be their definitions by the frequentists school as relative frequencies of events (in the limit of infinite amounts of such). A more general definition of probabilities is the Bayesians' one, which regards probabilities as quantifiers of statements, expressing how much those should be expected to be true. Although this definition looks subjective on the first glance, it rests on a solid mathematical theorem by Richard Cox6 that any extension of binary logic to a continuum of single valued truthfulness quantifier that respects a minimum of requirements must be isomorphic to probability theory. And this definition turns out to embrace the frequentists' definition in case that infinite numbers of comparable events are actually available and can be counted. The majority of authors of this special issues seem to favor the Bayesian perspective, which is probably a result of our own, the guest editors', preference and not necessarily a reflection of the prevailing position of contemporary scientists. The different views on probabilities of frequentists and Bayesians are addressed by the article of Allen Caldwell.7 He builds a bridge between these antagonistic camps by explaining frequentists' constructions in a Bayesian language. In particular, his argumentation why the omnipresent p-value in statistical and experimental literature might often capture a relevant signal of model misfit is reassuring, given that a large fraction of our medical system is based on decisions backed up by p-values. A Bayesian might add that it would be even more reassuring, if a system rigorously build on logic and its continuous extension would be used for decisions affecting live-or-death questions. The laws governing probabilities themselves seem to reflect that of more elementary mathematical structures. Kevin Knuth shows how these emerge from lattice theory, which deals with relations in partially ordered sets like that of collections of possibilities.8 The sum rule of probability theory is a direct consequence of the underlying lattice structure. Interestingly, a number of other theories are resting also on such lattices and therefore exhibit their own versions of sum rules: measures of sets, mutual information, the Euler characteristics of topological spaces, as well as quantum mechanical interactions that are quantified by pairs of numbers, the real and imaginary parts of complex probability amplitudes (see also4). The theory of information has very practical application in physics and elsewhere. Manfred Opper discusses how the solutions of stochastic differential equations can be inferred via variational methods, which have their origin in thermodynamics.9 Torsten Enßlin introduces information field theory (IFT), the information theory for fields, as a means to infer the configuration of a physical field from incomplete and noisy data.10 The mathematical apparatus for this is that of quantum field theory (QFT), just with ordinary probabilities instead of complex probability amplitudes as in the latter. Could it be that the difference between IFT and QFT is just be the same as between classical information theory and quantum mechanics, the transition from real valued to complex valued quantifier for statements? Anyhow, any practical inference machinery, be it for dynamical systems or for fields, requires sophisticated numerical methods. Michael Betancourt reviews Markov Chain Monte Carlo Methods to calculate expectation values over high dimensional probabilities.11 Theo Steininger and coworkers present the NIFTy3 software package for numerical information field theory calculations, which can handle very high dimensional field inference problems.12 Finally, in an article focused on the subtleties of the Monty Hall problem and mostly aiming at information theoretical entertainment, Torsten Enßlin and Margret Westercamp investigate the possibility that human intuition might be more rational than contemporary psychology thinks.13 A number of possible topics for an issue on physics of information are not addressed here. We would have loved to have articles on Landauer's principle, which states that erasing information has to cost energy, on QBism, a Bayesian version of quantum mechanics, on artificial and biological neural networks, as well as any other aspect of the information-soaked biological world. What has a reader of this special issue to be equipped with in order to follow the lines of thoughts? In addition to the usual physicist's education, the minimal additional equipment are the basics of probability theory and the concept of entropy. Most of the other concepts used in the articles rest on those and are introduced there. Let A, B, and C be propositions (or possibilities). The probability P ( A | B ) of A being true assuming that B is true is a number between 0 and 1, with P ( A | B ) = 1 expressing certainty of A under B ( B ⇒ A ) and P ( A | B ) = 0 expressing certainty of not A being the case under B ( B ⇒ ¬ A ).
Nihil est in intellectu quod non prius fuerit in sensu. The Physics of Information This book describes the limits for the communication of information with waves. How many ideas can we communicate by writing on a sheet of paper? How well can we hear a concert? How many details can we distinguish in an image? How much data can we get from our internet connection? These are all questions related to the transport of information by waves. Our sensing ability to capture the differences between distinct waveforms dictates the limits of the amount of information that is delivered by a propagating wave. The problem of quantifying this precisely requires a mathematical description and a physical understanding of both the propagation and the communication processes. We focus on the propagation of electromagnetic waves as described by Maxwell's theory of electromagnetism, and on communication as described by Shannon's theory of information. Although our treatment is mostly based on classical field theory, we also consider limiting regimes where the classical theory must give way to discrete quantum formulations. The old question of whether information is physics or mathematics resounds here. Information is certainly described mathematically, but we argue that it also has a definite physical structure. The central theme of this book is that Shannon's information-theoretic limits are natural. They are revealed by observing physical quantities at certain asymptotic scales where finite dimensionality emerges and observational uncertainties are averaged out. These limits are also rigorous, and obey the mathematical rules that govern the model of reality on which the physical theories are based. Shannon's Laws Originally introduced by Claude Elwood Shannon in 1948, and continuing up to its latest developments in multi-user communication networks, information theory describes in the language of mathematics the limits for the communication of information. These limits are operational, of real engineering significance, and independent of the semantic aspects associated with the communication process.
The experience from Quantum Information of the last twenty years has lead theorists to look at Quantum Theory and the whole of Physics from a different angle. A new information-theoretic paradigm is emerging, long time ago prophesied by John Archibald Wheeler with his popular coinage 'It from bit'. Theoretical groups are now addressing the problem of deriving Quantum Theory from informational principles, and similar lines are investigated in new approaches to Quantum Gravity. In my talk I will review some recent advances on these lines. The general idea synthesizing the new paradigm is that there is only Quantum Theory (without quantization rules): the whole Physics--including space-time and relativity--is emergent from quantum-information processing. And, since Quantum Theory itself is made with purely informational principles, the whole Physics must be reformulated in information-theoretical terms. The review is divided into the following parts: (a) The informational axiomatization of Quantum Theory; (b) How space-time and relativistic covariance emerge from the quantum computation; (c) What is the information-theoretical meaning of inertial mass and Planck constant, and how the quantum field emerges; (d) Observational consequences: mass-dependent refraction index of vacuum. I then conclude with some possible future research lines.
Information is physical. Today’s information processing and communication are classical: they are based upon the laws of Newton’s and Maxwell’s classical physics. This assertion holds all the way, from commercial computers and networks, up to their most abstract models, e.g. Turing machines. Research in quantum information was born some twenty five years ago, with the encounter of two major scientific achievements of the 20th century, namely quantum physics and information sciences. A technological motivation for that is an extrapolation of Moore’s law which seems to indicate that the amount of matter needed for one bit will be reduced to one particle sometimes before year 2020. A deeper, scientific driving force of this interdisciplinary research is that of looking for the consequences of having computation and communication based directly upon the laws of quantum physics, i.e. our current ultimate knowledge of the world of elementary particles, as described by quantum mechanics. Breakthroughs in cryptography, communications, information theory and algorithmics have shown that this transplantation from classical to quantum has far reaching consequences, both quantitative and qualitative, and opens new avenues for research within the foundations of computer science and physics. The principles 388 and most striking results and hot topics of this promising way of encoding, processing and communicating information are briefly introduced in this chapter from a mostly computer science point of view, with detailed examples in algorithmics and in cryptography. At the end of the chapter, a short list of some significant articles, textbooks and reports on quantum information are suggested for further reading.