Quantum mechanics as a noncommutative representation of classical conditional probabilities

TL;DR

This paper explores how quantum mechanics can be reconstructed from classical conditional probabilities using noncommutative representations, showing that empirical data constrains these representations to align with quantum theory.

Contribution

It demonstrates that increasing empirical data restricts noncommutative representations to uniquely match quantum mechanics, bridging classical probabilities and quantum formalism.

Findings

Empirical data narrows noncommutative representations to quantum mechanics.

Quantum representation emerges as the unique consistent model with sufficient data.

Classical conditional probabilities can be embedded within a noncommutative framework that converges to quantum theory.

Abstract

The aim of this paper is to analyze the reconstructability of quantum mechanics from classical conditional probabilities representing measurement outcomes conditioned on measurement choices. We will investigate how the quantum mechanical representation of classical conditional probabilities is situated within the broader frame of noncommutative representations. To this goal, we adopt some parts of the quantum formalism and ask whether empirical data can constrain the rest of the representation to conform to quantum mechanics. We will show that as the set of empirical data grows conventional elements in the representation gradually shrink and the noncommutative representations narrow down to the unique quantum mechanical representation.

Tables

Table 1. Table 1: Quantum mechanical representation

Operator assignment:		Born rule:
System $⟶$ $\mathscr{H}$: Hilbert space Measurements: $a_i$ $⟶$ $O_i$: self-adjoint operators Outcomes: $A_i^j$ $⟶$ $P_i^j$: spectral projections of $O_i$ States: $s$ $⟶$ $W_s$: density operators		$p_s\left(A_i^j|a_i\right)=\text{Tr}\left(W_s{P}_i^j\right)$

Table 2. Table 2: Noncommutative representation

Operator assignment:		Born rule:
System $⟶$ $\mathscr{H}$: Hilbert space Measurements: $a_i$ $⟶$ $O_i$: self-adjoint operators Outcomes: $A_i^j$ $⟶$ $P_i^j$: spectral projections of $O_i$ States: $s$ $⟶$ $W_s$: linear operators		$p_s\left(A_i^j|a_i\right)=\text{Tr}\left(W_s{P}_i^j\right)$

Table 3. Table 3: Quantum mechanical representation of three yes-no measurements in ℂ 2 subscript ℂ 2 {\mathbb{C}}_{2}

Operator assignment:		Born rule:
System: $⟶$ ${ℂ}_2$ Color: $a$ $⟶$ $O_a=𝐚𝝈$ Size: $b$ $⟶$ $O_b=𝐛𝝈$ Shape: $c$ $⟶$ $O_c=𝐜𝝈$ Black/White: $A^{\pm }$ $⟶$ $P_a^{\pm }=\frac{1}{2}\left(\mathrm{𝟏}\pm 𝐚𝝈\right)$ Large/Small: $B^{\pm }$ $⟶$ $P_b^{\pm }=\frac{1}{2}\left(\mathrm{𝟏}\pm 𝐛𝝈\right)$ Round/Oval: $C^{\pm }$ $⟶$ $P_c^{\pm }=\frac{1}{2}\left(\mathrm{𝟏}\pm 𝐜𝝈\right)$ State: $s$ $⟶$ $W_s=\frac{1}{2}\left(\mathrm{𝟏}+𝐬𝝈\right)$		$p_a^{\pm }=\text{Tr}\left(W_s{P}_a^{\pm }\right)=\frac{1}{2}\left(1\pm \mathrm{𝐬𝐚}\right)$ $p_b^{\pm }=\text{Tr}\left(W_s{P}_b^{\pm }\right)=\frac{1}{2}\left(1\pm \mathrm{𝐬𝐛}\right)$ $p_c^{\pm }=\text{Tr}\left(W_s{P}_c^{\pm }\right)=\frac{1}{2}\left(1\pm \mathrm{𝐬𝐜}\right)$

Table 4. Table 4: Quantum mechanical representation of the empirical conditional probabilities ( 11 )

Operator assignment:		Born rule:
Outcomes: $A^{\pm }$ $⟶$ $P_a^{\pm }=\frac{1}{2}\left(\mathrm{𝟏}\pm 𝐚𝝈\right)$ $𝐚\in {ℝ}^3,\left|𝐚\right|=1$ Pure state: $s$ $⟶$ $W_s=\frac{1}{2}\left(\mathrm{𝟏}+𝐬𝝈\right)$ $𝐬\in {ℝ}^3,\left|𝐬\right|=1$		$p_a^{\pm }=\text{Tr}\left(W_s{P}_a^{\pm }\right)$

Table 5. Table 5: A noncommutative but not quantum mechanical representation of the empirical conditional probabilities ( 11 )

Operator assignment:		Born rule:
Outcomes: $A^{\pm }$ $⟶$ $P_a^{\pm }=\frac{1}{2}\left(\mathrm{𝟏}\pm f\left(𝐚\right)𝝈\right)$ $f\left(𝐚\right)\in {ℝ}^3,\left|f\left(𝐚\right)\right|=1$ Pure state: $s$ $⟶$ $W_s=\frac{1}{2}\left(\mathrm{𝟏}+g\left(𝐬\right)𝝈\right)$ $g\left(𝐬\right)\in {ℝ}^3,\left|g\left(𝐬\right)\right|>1$		$p_a^{\pm }=\text{Tr}\left(W_s{P}_a^{\pm }\right)$

Equations

$$p_a^{\pm }$$

$$p_b^{\pm }$$

$$p_c^{\pm }$$

$$\{a,b,c\}\mapsto \{\mathbf{a},\mathbf{b},\mathbf{c}\}$$

$$p_a^{\pm },p_b^{\pm },p_c^{\pm }\&\mathbf{a},\mathbf{b},\mathbf{c}⟹W_s$$

$$W_s=s_{0}1+\mathbf{s}\mathbf{\sigma }+i(s_0^{\prime }1+{\mathbf{s}}^{\prime }\mathbf{\sigma })$$

$$p_a^{\pm }=\text{mbox}Tr(W_s{P}_a^{\pm })=s_0\pm \mathbf{sa}+i(s_0^{\prime }\pm {\mathbf{s}}^{\prime }\mathbf{a})$$

$$p_b^{\pm }=\text{mbox}Tr(W_s{P}_b^{\pm })=s_0\pm \mathbf{sb}+i(s_0^{\prime }\pm {\mathbf{s}}^{\prime }\mathbf{b})$$

$$p_c^{\pm }=\text{mbox}Tr(W_s{P}_c^{\pm })=s_0\pm \mathbf{sc}+i(s_0^{\prime }\pm {\mathbf{s}}^{\prime }\mathbf{c})$$

$$s_0=\frac{1}{2}{s}_0^{\prime }=0{\mathbf{s}}^{\prime }=0$$

$$p_a^{\pm }=\frac{1}{2}\pm \mathbf{sa}$$

$$p_b^{\pm }=\frac{1}{2}\pm \mathbf{sb}$$

$$p_c^{\pm }=\frac{1}{2}\pm \mathbf{sc}$$

$$\mathbf{s}=\frac{(p_a^{+}-\frac{1}{2})(\mathbf{b}\times \mathbf{c})+(p_b^{+}-\frac{1}{2})(\mathbf{c}\times \mathbf{a})+(p_c^{+}-\frac{1}{2})(\mathbf{a}\times \mathbf{b})}{\mathbf{a}\cdot (\mathbf{b}\times \mathbf{c})}$$

$$\mathbf{a}$$

$$\mathbf{b}$$

$$\mathbf{c}$$

$$p_a^{+}=p_b^{+}=p_c^{+}=:p$$

$$\mathbf{s}=(p-\frac{1}{2})\left(1,\frac{1+\cos \phi +\sin \phi }{1+\cos \phi -\sin \phi },1\right)$$

$$p\in [0.76,1]\text{mbox}and\phi \in [\pi /3,\pi /2)$$

$$p_i^j:=p(A_i^j|a_i)⩾0\text{mbox}with\sum _i{p}_i^j=1\text{mbox}foralli=1\dots k;j=1\dots n$$

$$p_i^j:=p(A_i^j|a_i)=\text{mbox}Tr(W_s{P}_i^j)$$

$$\{p_1^j\},\{p_2^j\}\dots \{p_k^j\}\&\{P_1^j\},\{P_2^j\}\dots \{P_k^j\}⟹W_s$$

$$C^{\ast }:=\{A\in V|\text{mbox}Tr(AB)⩾0\text{mbox}forallB\in C\}$$

$$\displaystyle\big{\{}p^{\pm}_{a}:=p_{s}(A^{\pm}|a);\quad s\,\,\mbox{is fixed}\big{\}}$$

$$\mathbf{a}$$

$$\mathbf{s}$$

$$f:S^2\to S^2;\mathbf{a}\mapsto f(\mathbf{a})$$

$$g:S^2\to {\mathbb{R}}^3;\mathbf{s}\mapsto g(\mathbf{s})$$

$$f(\mathbf{a})$$

$$g(\mathbf{s})$$

$${\vartheta }^{\prime }$$

$${\phi }^{\prime }$$

$$\text{mbox}Tr(W_s{P}_a^{\pm })=\text{mbox}Tr\left(\frac{1}{2}(1+\mathbf{s}\mathbf{\sigma })\frac{1}{2}(1\pm \mathbf{a}\mathbf{\sigma })\right)=\frac{1}{2}(1\pm \mathbf{sa})$$

$$f:S^2\to S^2;\mathbf{a}\mapsto f(\mathbf{a})$$

$$g:S^2\to {\mathbb{R}}^3;\mathbf{s}\mapsto g(\mathbf{s})$$

$$\mathbf{as}=f(\mathbf{a})g(\mathbf{s})$$

Full text

Keywords: noncommutativity, conditional probability, measurement-operator assignment

1 Introduction

In the quantum information theoretical paradigm one is usually looking for the reconstruction of quantum mechanics from information-theoretic first principles (Hardy, 2008; Chiribella, D’Ariano and Perinotti, 2017). This approach has produced many fascinating mathematical results and greatly contributed to a better understanding of the complex formal structure of quantum mechanics. As a top-down approach, however, its prime aim was to clarify the relation of the theory to higher-order (rationality, information-theoretic, etc.) principles and payed less attention to the “legs” of the theory connecting it to experience.

In this paper we take an opposite, bottom-up route and ask—in the spirit of the good old empiricist tradition—as to how the theory can be reconstructed not from first principles but from experience. More precisely, we will ask whether we can reconstruct the formalism of quantum mechanics from using simply classical conditional probabilities.

Why classical conditional probabilities?

Quantum mechanics as a probabilistic theory provides us quantum probabilities for certain observables. The question is how to connect these quantum probabilities to experience. The correct answer is that the probabilities provided by the Born rule should be interpreted as classical conditional probabilities. They are classical since they are nothing but the long-run relative frequency of certain measurement outcomes explicitly testable in the lab; and they are conditional on the fact that a certain measurement had been chosen and performed (E. Szabó, 2008). For example, the quantum probability of the outcome “spin-up” in direction $z$ is the relative frequency of the outcomes “up”—but not in the statistical ensemble of all measurement outcomes (which may also comprise spin measurements in other directions) but only in the subensemble when spin was measured in direction $z$.

What does it mean to reconstruct quantum mechanics from classical conditional probabilities?

First note that all we are empirically given are classical conditional probabilities. The question is how to represent these empirical data. As it was hypothesized in (E. Szabó 1995) and shown—under some specific conditions—in (Bana and Durt 1997), (E. Szabó 2001) and (Rédei 2010) classical conditional probabilities conforming to the probabilistic predictions of quantum mechanics need not necessarily be represented in the formalism of quantum mechanics. The so-called “Kolmogorovian Censorship Hypothesis” (or better, Proposition) states that there is always a Kolmogorovian representation of the quantum probabilities if the measurement conditions also make part of the representation. Thus, a stubborn classicist will always find a way to represent the empirical content of quantum mechanics in a purely classical framework.

On the other hand, quantum mechanics has proved to be an extremely elegant and economic representation of these empirical data. It provides a principled representation of an enormous collection of conditional probabilities together with their dynamical evolution.

Our paper is a kind of interpolation between the two sides. Our strategy will be to accept some parts of the quantum mechanical representation of classical conditional probabilities and ask whether the rest follows. More precisely, we accept the noncommutative probability theory (see Gudder 1988; Rédei and Summers, 2007) which in our case will boil down to representing observables and states by linear operators. We also adopt the Born rule connecting the quantum probabilities to real-world classical conditional probabilities; and the quantum mechanical representation of measurement settings and measurement outcomes. The only “free variable” will be the representation of the state of the system. Our main question will then be as to what empirical data ensure that the state of a system is represented by a density operator.

By this strategy we are going to analyze how quantum mechanics is situated within a noncommutative probability theory and to study whether the specific quantum mechanical representation of classical conditional probabilities within this broader frame can be traced back to purely empirical facts or is partly of conventional nature.

In the paper we will proceed as follows. In Section 2 we introduce the general scheme of a noncommutative representation of classical conditional probabilities. In the subsequent three sections we gradually enhance the set of empirical data that is the set of classical conditional probability of measurement outcomes. We ask whether by increasing the set of empirical data the noncommutative representation of these data necessarily narrows down to the quantum mechanical representation or some extra conventional elements are also needed. The empirical situation we are going to represent will be three yes-no measurements in Section 3, $k$ measurements each with $n$ outcomes in Section 4, and finally a continuum set of measurements with $n$ outcomes in Section 5. We will see how the conventional part gradually shrinks as experience grows until the representation finally zooms in on the quantum mechanical representation. We discuss our results in Section 6.

2 Quantum mechanical and noncommutative representation

Suppose there is a physical system in state $s$ and we perform a set $\{a_{i}\}$ $(i\in I)$ of measurements on the system. Denote the outcomes of measurement $a_{i}$ by $\{A^{j}_{i}\}$ $(j\in J)$. Suppose that by repeating the measurements many times we obtain a probability $p_{s}(A^{j}_{i}|a_{i})$ that is a stable long-run relative frequency for each outcome $A^{j}_{i}$ given measurement $a_{i}$ is performed. Now, quantum mechanics represents these conditional probabilities as it is summarized in the following table:

In the table the different concepts are presented. On the left hand side of the arrow/equation sign stand the empirical concepts to be represented; on the right hand side stand the mathematical representation of the empirical concepts. The two are not to be mixed. Although we do not use “hat” to denote operators, throughout the paper we carefully distinguish empirical terms (measurements, outcomes, states) from their representation (self-adjoint operators, projections, density operators). Thus, the physical system under investigation is associated to a Hilbert space ${\cal H}$; each measurement $a_{i}$ is represented by a self-adjoint operator $O_{i}$; the outcomes $A^{j}_{i}$ of $a_{i}$ are represented by the orthogonal spectral projections of $O_{i}$; and the state $s$ of the system is represented by a density operator $W_{s}$, a self-adjoint, positive semidefinite operator with trace equal to 1. In the second column the mathematical representation is connected to experience by the Born rule: the representation is correct only if the quantum mechanical trace formula $\mbox{Tr}(W_{s}P^{j}_{i})$ correctly yields the empirical conditional probability $p_{s}(A^{j}_{i}|a_{i})$ for any outcome $A^{j}_{i}$ of measurement $a_{i}$ and any state $s$.

Note the following two facts. First, the trace formula is associated to a conditional probability, not to a probability simpliciter. This means, among others, that in joint measurements one always needs to combine different measurement conditions. Second, the trace formula is “holistic” in the sense that the empirically testable conditional probabilities are associated to the trace of the product of two operators, one representing the state and the other representing the measurement. This leaves us with a lot of freedom to account for the same empirical content in terms of operators.

The main question of our paper is whether the above quantum mechanical representation of classical conditional probabilities is constrained upon us if the set of empirical data is large enough or whether we need some extra theoretical, aesthetic etc. considerations to arrive at it. In order to decide on this question, we consider first a wider class of representations which we will call noncommutative representations. We will then ask whether a noncommutative representation of a set of large enough data is necessarily a quantum mechanical representation.

What is a noncommutative representation?

Generally, a noncommutative representation is simply an association of measurements and states to linear operators acting on a Hilbert space such that some functional of the representants provides the correct empirical conditional probabilities. Obviously this association can be done in many different ways. In our paper we pick a special noncommutative representation which is very close to the quantum mechanical representation: We retain all the assignments (denoted by $\longrightarrow$) of the above table except the last one. That is we will represent the system by a Hilbert space, the measurements by self-adjoint operators, and the outcomes by the orthogonal spectral projections. We also retain the Born rule connecting the formalism to experience. The only part of the representation which we let vary will be the association of the state of the system to linear operators. That is we do not demand that states should necessarily be represented by density operators. We summarize this scheme in the following table:

Obviously, our noncommutative representation is only one special choice among many. One could well take different routes. For example one could demand that the state should be represented by density operators but abandon that the projections representing the outcomes should be orthogonal. Or one could replace the Born rule by another expression connecting the formalism to the world. As said above, the connection of the formalism of quantum mechanics and experience is of holistic nature; one can fix one part of the formalism and see how the rest may vary such that the resulting probabilities are in tune with experience. With respect to our aim which is to see how we are compelled to adopt the quantum mechanical representation by increasing the number of conditional probabilities to be represented, our above choice is just as good as any other.

What we will test in the subsequent sections is whether our noncommutative representation is necessarily a quantum mechanical representation. In other words, we will test whether for any choice of operators representing a certain set of measurements and the outcomes such that the Born rule yields the correct conditional probabilities, the state will necessarily be represented by a density operator. In Section 3 we start off as a warm-up with three measurements; in Section 4 we continue with $k$ measurements; and in Section 5 we end up by uncountably many measurements. It will turn out that the gap between noncommutative and quantum mechanical representation gradually shrinks as the set of empirical data grows.

3 Case 1: Three yes-no measurements

Consider a box filled with balls. Denote the preparation of the box by $s$. Suppose you can perform three different measurements on the system; you can measure the color, the size or the shape of the balls. Denote the three measurements as follows:

[TABLE]

Suppose that each measurement can have only two outcomes:

[TABLE]

Suppose you pick a measurement, perform it many times (putting the balls always back into the box), and count the probability, that is the long-run relative frequency, of the outcomes. What you obtain is the conditional probability of the outcomes given the measurement you picked is performed on the system prepared in state $s$:

[TABLE]

Now, suppose you are going to represent the above empirical facts not in the standard classical probability theory but in a quantum fashion. Since our model contains only two-valued (yes-no) measurements, it suffices to use only a minor fragment of quantum mechanics. Again, we summarize it in a table:

Here, the Hilbert space associated to the system is the two-dimensional complex space ${\mathbb{C}}_{2}$; and the operators associated to the measurements, outcomes and the state are all self-adjoint operators acting on ${\mathbb{C}}_{2}$. According to this representation, called the Bloch sphere representation, a self-adjoint operator $O_{a}$ associated to measurement $a$ can be represented by the inner product of a unit vector ${\bf a}=(a_{x},a_{y},a_{z})$ in ${\mathbb{R}}^{3}$ and the Pauli vector $\bm{\sigma}=(\sigma_{x},\sigma_{y},\sigma_{z})$. The two outcomes $A^{\pm}$ of measurement $a$ are associated to the spectral projections $P^{\pm}_{a}=\frac{1}{2}(\mathbf{1}\pm{\bf a}\bm{\sigma})$ of $O_{a}$, where $\mathbf{1}$ is the two-dimensional identity operator. Finally, the density operator $W_{s}$ associated to the state $s$ of the system is of the form $W=\frac{1}{2}(\mathbf{1}+{\bf s}\bm{\sigma})$, where ${\bf s}=(s_{x},s_{y},s_{z})$ is in the unit ball $B=\{{\bf r}\in{\mathbb{R}}^{3}:|{\bf r}|\leqslant 1\}$ of ${\mathbb{R}}^{3}$. If $|{\bf s}|=1$, then $s$ is said to be a pure state, otherwise a mixed state. Again, the empirical content of the representation is ensured by the Born rule which in this two-dimensional case boils down to the inner product: $p^{\pm}_{a}=\frac{1}{2}(1\pm{\bf sa})$. (Similarly for $b$ and $c$.)

Now, to give a quantum mechanical representation for the above situation we need to associate the three measurements to three Bloch vectors and the state of the system to a fourth Bloch vectors (either unit or smaller) such that the Born rule (the trace formula) yields the pre-given conditional probabilities (1)-(3). Thus, assign to each measurement a unit vector in ${\mathbb{R}}^{3}$:

[TABLE]

Suppose that the vectors ${\bf a}$, ${\bf b}$ and ${\bf c}$ are linearly independent. First, we show that given three pairs of empirical conditional probabilities $p^{\pm}_{a}$, $p^{\pm}_{b}$ and $p^{\pm}_{c}$ and also the assignment (4), the operator $W_{s}$ associated to the state $s$ gets uniquely fixed. Schematically,

[TABLE]

To see this, observe that any linear operator acting on ${\mathbb{C}}_{2}$ can be written as

[TABLE]

where $s_{0},s^{\prime}_{0}\in{\mathbb{R}}$ and ${\bf s},{\bf s^{\prime}}\in{\mathbb{R}}^{3}$. Now, applying the Born rule to the three measurements we get:

[TABLE]

which, assuming that $p^{\pm}_{a}$, $p^{\pm}_{b}$ and $p^{\pm}_{c}$ are real and ${\bf a}$, ${\bf b}$ and ${\bf c}$ are linearly independent, yield

[TABLE]

and hence

[TABLE]

the solution of which is $W_{s}=\frac{1}{2}(\mathbf{1}+{\bf s}\bm{\sigma})$ with

[TABLE]

where $\times$ is the cross product. (The linear independence of ${\bf a}$, ${\bf b}$ and ${\bf c}$ is needed for the triple product in the denominator not to be zero.)

This is a well-known result. Since the late 60s and early 70s there has begun an intensive research for the empirical determination of the state of a quantum system. In a series of papers Band and Park (1970, 1971) have extensively investigated how the expectation value of certain observables determine the state of a system. They investigated the minimal number of observables, called the quorum, needed for such state determinations; the structure and geometry of this set; and many other important features. The study of the quorum has become an eminent research project also in the new quantum informational paradigm. Quantum tomography, quantum state reconstruction, quantum state estimation etc. all follow the same path: they start from a set of observables and aim to end up with a more-or-less fixed state using empirical input (see for example (D’Ariano, Maccone and Paris, 2001)).

However, all these endeavors have a common pre-assumption, namely that the association of measurements to operators is already settled. They all start from a set of operators and (by means of a set of empirical probabilities) aim to reconstruct the quantum state of a system. But an operator is not a measurement but only a representation of a measurement. Calling operators observables overshadows the fact that the operators are already on the mathematical side of the project and without providing an association of measurements to operators the state determination cannot rightly be called “empirical”. This measurement-operator assignment is that which we are going to make explicit in what comes.

Consider the following measurement-operator assignment in the context of our above model: we associate the following three Bloch vectors to the measurements $a$, $b$ and $c$:

[TABLE]

and for the sake of simplicity we set the conditional probabilities as follow:

[TABLE]

The Bloch vector ${\bf s}$ for these special directions and empirical probabilities will then be the following:

[TABLE]

But the operator $W_{s}$ associated to the Bloch vector ${\bf s}$ will not necessarily be a density operator. For example for any

[TABLE]

the vector ${\bf s}$ will be longer than 1 and hence $W_{s}$ will not be positive semidefinite, that is, a density operator.

Thus, we have provided a noncommutative but not quantum mechanical representation of the above scenario. All the assignments of the table at the beginning of this section hold except the last one: the state of the system is represented by a linear operator but not a density operator.

This toy-example is, however, special in two senses: (i) the number of measurements is finite and (ii) the number of outcomes is two, that is, the scenario is represented in the two-dimensional Hilbert space which is always a special case. We tackle point (ii) in the next section and point (i) in the one after the next.

4 Case 2: $k$ measurements with $n$ outcomes

Let us then see whether a larger set of probabilities can also be given a noncommutative but not quantum mechanical representation. Suppose we perform $k$ measurements on a system such that each measurement can have $n$ outcomes. Suppose we obtain the following empirical conditional probabilities:

[TABLE]

Just as above we represent each measurement $a_{i}$ by a self-adjoint operator $O_{i}$ in the Hilbert space ${\cal H}_{n}$ and the measurement outcomes $\{A^{j}_{i}\}$ of $a_{i}$ by the orthogonal spectral projections $\{P^{j}_{i}\}$. The representation is connected to experience by the Born rule:

[TABLE]

where $W_{s}$ is a linear operator representing the state $s$ of the system. Again, we do not assume that $W_{s}$ is a density operator; our task is just to see whether it follows that $W_{s}$ is always a density operator.

Now, the empirically given probability distributions together with the conventionally chosen sets of minimal orthogonal projections provide constraints on $W_{s}$ via the Born rule. For a certain number of measurements $W_{s}$ gets completely fixed. Schematically,

[TABLE]

How many measurements are needed to uniquely fix $W_{s}$?

$W_{s}$ gets uniquely fixed if Tr$(W_{s}A)$ is given for $n^{2}$ linearly independent operators $A$. Our operators are minimal projections. The first set of minimal orthogonal projections provides $n$ linearly independent equations. Any further linearly independent set of orthogonal projection provides $n-1$ extra equations since in each set the projections sum up to the unity. That is $k$ linearly independent sets of minimal orthogonal projections provide $k(n-1)+1$ linearly independent equations which is equal to $n^{2}$ if $k=n+1$. Thus, performing $k=n+1$ measurements on our system (resulting in $k=n+1$ probability distributions) and representing all the outcomes by orthogonal projections in ${\cal H}_{n}$, the linear operator $W_{s}$ gets uniquely fixed.

But it will not necessarily be a density operator!

Our question is then: Do $k=n+1$ measurements constrain $W_{s}$ to be a density operator for all linearly independent sets of orthogonal projections representing the outcomes and all probability distributions generated from the projections by the Born rule? Again, what we test here is whether a noncommutative representation is necessarily a quantum mechanical representation.

Now, we show that the answer is: no.

As said above, a density operator is a self-adjoint, positive semidefinite operator with trace equal to 1. Self-adjoint operators in ${\cal H}_{n}$ form a vector space $V$ over the field of real numbers. This vector space can also be endowed with an inner product induced by the trace: $(A,B):=\mbox{Tr}(AB)$. The operators with trace equal to 1 form an affin subspace $E$ in $V$ and the positive semidefinite operators form a convex cone $C_{+}$. (A subset $C$ of a real vector space $V$ that linearly spans $V$ is a convex cone if for any $A_{1},A_{2}\in C$ and $r_{1},r_{2}\in{\mathbb{R}}_{+}$, $r_{1}A_{1}+r_{2}A_{2}\in C$ and $A,-A\in C\Rightarrow A=0$). The intersection of the two, ${\cal C}_{+}\cap E$, is a convex set in the affin subspace. The extremal elements of this set are the minimal projections in ${\cal H}_{n}$. Denote this set of minimal projections in ${\cal H}_{n}$ by ${\cal P}_{n}$.

Now, for any cone $C$ in $V$, the dual cone $C^{*}$ is defined as

[TABLE]

According to Fejér’s Trace Theorem the cone of the positive semidefinite operators is self-dual that is $C^{*}_{+}=C_{+}$.

Now, let us return to our example. Consider the $k=n+1$ linearly independent sets of orthogonal projections representing the measurement outcomes in ${\cal H}_{n}$. Let $D$ be the convex cone expanded by these projections in ${\cal P}_{n}$ as extremal elements. Obviously, $D\subset C_{+}$ and consequently $D^{*}\supset C^{*}_{+}=C_{+}$. Pick an element from $(D^{*}\setminus C_{+})\cap E$ and call it $W_{s}$. Lying outside $C_{+}$, $W_{s}$ will not be positive semidefinite but, lying in $E$, $W_{s}$ will be of trace 1. Hence for any set of orthogonal projections it generates a probability distribution by the Born rule.

Thus, we have found a counter-example (actually, continuously many counter-examples): $k=n+1$ linearly independent sets of orthogonal projections representing measurement outcomes and $k=n+1$ probability distributions such that the latter is generated from the former by the Born rule with an operator $W_{s}$ which is not a density operator (since is not positive semidefinite). Hence, we have provided a noncommutative but not quantum mechanical representation for a situation in which $k=n+1$ measurements with $n$ outcomes are performed on a system. This shows that our previous result is not a consequence of the fact that the Hilbert space is the special ${\cal H}_{2}$. Conditional probabilities of finitely many measurements with finitely many outcomes can always be given a noncommutative but not quantum mechanical representation.

But what is the situation if we are going to the continuum limit? Does our counter-example survive if the cardinality of the set of conditional probabilities to be represented is uncountable? To this we turn in the next section.

5 Case 3: A continuum set of measurements with $n$ outcomes

There is a theorem which immediately comes to one’s mind when going to the continuum limit, namely Gleason’s theorem.

Suppose we are given a continuum set of probability distributions of measurements with, say, $n$ outcomes. We are to represent this set in an $n$-dimensional Hilbert space ${\cal H}_{n}$. Now, suppose that we assign self-adjoint operators to the measurements such that the spectral projections of the various operators together cover the full set ${\cal P}_{n}$ of minimal projections in ${\cal H}_{n}$. In other words, there is no minimal projection in ${\cal P}_{n}$ which does not represent a measurement outcome. In this case we can invoke Gleason’s theorem to decide on the question as to whether there exist noncommutative representations which are not the quantum mechanical representation. Gleason’s theorem answers this question in the negative.

Gleason’s theorem namely claims that for every state $\phi$ in a Hilbert space with dimension greater than 2 there is a density operator $W$ (and vica versa) such that the Born rule $\phi(P)=\mbox{Tr}(PW)$ holds for all projections. In other words, if all projections are considered, then the state will uniquely be represented by a density operator. Translating it into our case, the theorem claims that if one represents the continuum set of measurement outcomes by the full set ${\cal P}_{n}$ of projections of a given Hilbert space, then one has no other choice to account for the whole set of conditional probabilities, than to represent the state by a density operator.

Note, however, that the previous sentence is a conditional: if we represent the measurement outcomes by the full set ${\cal P}_{n}$, then Gleason’s theorem tells us that the only representation is the quantum mechanical. This raises the following question: Are we compelled to represent a continuum set of measurement outcomes necessarily by the full set of minimal projections? Can we not “compress” somehow the set of projections representing the measurement outcomes such that (i) the outcome-projection assignment is injective (no two outcomes of different measurements are represented by the same projection), still (ii) the set of projections is only a proper subset of ${\cal P}_{n}$? As we saw in the previous section, in this case we can always represent the state of the system by a linear operator which is not a density operator. Or to put it briefly, can we avoid Gleason’s theorem by not making use of all minimal projections of ${\cal P}_{n}$?

As stressed in Section 2, it is of crucial importance to discern physical measurements from operators mathematically representing them. When we use Gleason’s theorem we intuitively assume that all projections in a Hilbert space represent a measurement outcome for a real-world physical measurement. The case of spin enforces this intuition since the Bloch sphere representation of spin-half particles nicely pairs the spatial orientations of the Stern-Gerlach apparatus with the projections of ${\cal P}_{2}$. In general, however, we have no a priori knowledge of the measurement-operator assignment. Particularly, we cannot assume that a set of measurements just because it is an uncountable set has to be represented by the full set of projections of a given Hilbert space. A priori it is perfectly conceivable that a set of real-world measurements, even if its cardinality is uncountable, can be represented by a proper subset of ${\cal P}_{n}$.

But now suppose that for a given Hilbert space ${\cal H}_{n}$ all the self-adjoint operators on ${\cal H}_{n}$ represent a real-world empirical measurement with $n$ outcomes and all states on ${\cal H}_{n}$ represent a real-world preparation of the system to be measured. In other words, take it at face value that the full formalism of an $n$-dimensional quantum mechanics has an empirical meaning. We coin the term full empirical content of the $n$-dimensional quantum mechanics for the full (continuum) set of conditional probabilities of measurement outcomes provided by the Born rule, that is by the trace of the product of the different spectral projections and density operators in ${\cal H}_{n}$. Now, our question is this: can the full empirical content of the $n$-dimensional quantum mechanics be represented in ${\cal H}_{n}$ in a noncommutative but not quantum mechanical way?

The previous three paragraphs amounts to two different questions, one concerning cardinality the other concerning full empirical content:

Is a noncommutative representation of a set of empirical probabilities necessarily a quantum mechanical representation if the cardinality of the set is continuum? 2. 2.

Is a noncommutative representation of the full empirical content of the $n$-dimensional quantum mechanics necessarily a quantum mechanical representation?

In what comes we will show that the answer to the first question is no and the answer to the second question is yes.

We start with the first question. Our task is to represent a continuum set of empirical probabilities in a noncommutative but not quantum mechanical way. The set we pick will be the set of probabilities of spin measurements in all the different spatial directions performed on an electron prepared in one given state. This set is obviously a continuum set but not yet the full empirical content of the two-dimensional quantum mechanics since we consider only one state. The continuum set of empirical conditional probabilities is the following:

[TABLE]

Here $a$ denotes the spin measurement in direction ${\bf a}$ and $A^{\pm}$ are the two spin outcomes. Now, in the Bloch sphere representation one associates two unit vectors

[TABLE]

to the spin measurement $a$ and state $s$ of the system, respectively, such that the Born rule yields the conditional probabilities (11):

As is well-known, the measurement outcomes in the Bloch sphere representation are associated to the full set of minimal projections ${\cal P}_{2}$, and hence $W_{s}$ must be represented by a density operator due to Gleason’s theorem. However, the Bloch sphere representation is not the only possible noncommutative representation of (11). Here is an alternative.

Consider the following two functions:

[TABLE]

and suppose that instead of $\bf a$ and $\bf s$ we associate

[TABLE]

to $a$ and $s$, respectively, where

[TABLE]

and $r>1$. Observe that $f$ is injective but not surjective: a spherical cap around the “North Pole” and “South Pole” is not in the image of $f$. It is easy to check that by these associations we obtain a noncommutative representation for the conditional probabilities (11):

The representation is a noncommutative but not a quantum mechanical representation since $W_{s}$ is not positive semidefinite and hence not a density operator. Note again that we have avoided Gleason’s theorem because we did not use the full Bloch sphere to represent measurement outcomes but only a “belt” defined by the angles (12)-(16). To sum up, even though the set of measurements is uncountable, the noncommutative representation is not necessarily quantum mechanical since the set of projections representing the outcomes is not the full set of projections ${\cal P}_{2}$ of the Hilbert space ${\cal H}_{2}$.

However, (11) contains only the conditional probabilities of the spin measurement for one state. Can we apply the above technique of “pecking a hole” in the surface of the Bloch sphere and “pushing out” $s$ such that $W_{s}$ will not be a density operator in the case when we take into consideration all states? In other words, can we provide a noncommutative but not quantum mechanical representation for the full empirical content of the two-dimensional quantum mechanics? This was our second question above.

This is point where the representation of the set of conditional probabilities gets rigid. It will turn out that if one is to represent the conditional probability of all measurement outcomes of all spin measurement in all states, then there is no other noncommutative representation but the quantum mechanical. We prove it by the following lemma.

Lemma 1**.**

Consider the Bloch sphere representation of spin. That is let ${\bf a}$ and ${\bf s}$ two unit vectors associated to the spin measurement $a$ and state $s$ of the system, respectively, such that the Born rule yields the conditional probabilities:

[TABLE]

Then, if there are two functions

[TABLE]

such that all the conditional probabilities (17) are preserved that is

[TABLE]

for all ${\bf a},{\bf s}\in S^{2}$, then

• (i)

$f$ and $g$ are the restrictions of the bijective linear maps

[TABLE]

to $S^{2}$, respectively;

• (ii)

$\hat{f}$ is the orthogonal transformation;

• (iii)

$\hat{g}=\hat{f}$.

For the proof of Lemma 1 see the Appendix.

Lemma 1 shows that there is no other transformation of the Bloch vectors which preserve all the empirical conditional probabilities encoded in the inner product but the orthogonal transformation. Consequently, one cannot avoid Gleason’s theorem and provide a counter-example of the above type in which the state is represented by a linear but not density operator.

In the rest of the section we prove that this result holds not only in ${\cal H}_{2}$ but in any $n$-dimensional Hilbert space. We show that one cannot preserve all the empirical conditional probabilities encoded in the inner product of the Hilbert space by other transformation than the unitary transformation. Thus, “compressing” the empirical content in a proper subset of ${\cal P}_{n}$ of a given Hilbert space is not a viable route to follow. If all the inner products of minimal projections have an empirical meaning, then the only way to represent them is via quantum mechanics.

Lemma 2**.**

Let ${\cal H}$ be an $n$-dimensional Hilbert space and let ${\cal P}_{n}$ be the set of minimal projections in ${\cal B}({\cal H})\simeq M_{n}({\mathbb{C}})$. If there are two functions

[TABLE]

such that

[TABLE]

for all $P,Q\in{\cal P}_{n}$, then

• (i)

$f$ and $g$ are the restrictions of the bijective linear maps

[TABLE]

to ${\cal P}_{n}$, respectively;

• (ii)

$\hat{f}$ is unitary with respect to the inner product on $M_{n}({\mathbb{C}})$ provided by the trace;

• (iii)

$\hat{g}=\hat{f}$.

For the proof of Lemma 2 see again the Appendix.111I thank Péter Vecsernyés for his help in proving both Lemma 1 and 2.

6 Discussion

Is quantum mechanics the only possible way to represent an empirically given set of classical conditional probabilities in a noncommutative way; or is this representation picked out from a broader set of representations by convention? Ultimately, this was the question we posed in this paper. To make this question precise, we specified a set of representations, called noncommutative representations, in which measurement choices and measurement outcomes were represented in the quantum fashion and the Born rule connecting the quantum probabilities to classical conditional probabilities was respected. We asked whether experience can ensure that this representation becomes not just partly but fully quantum mechanical, that is, the state will be represented by a density operator. Our answer was the following:

In case of finitely many measurements with finitely many outcomes the probability distribution of outcomes can always be given a noncommutative but not quantum mechanical representation. 2. 2.

In case of infinitely many measurements the probability distributions can be given a noncommutative but not quantum mechanical representation only if one can avoid Gleason’s theorem by not using all the projections of the Hilbert space in representing measurement outcomes. 3. 3.

If the physical situation is so complex that the inner product of any pair of minimal projections is of empirical meaning, then there exists no noncommutative representation which is not quantum mechanical. Thus, the quantum mechanical representation is not conventional.

The relation between point 2 and 3 is very subtle. It shows that simply the cardinality of the set of measurements does not decide on whether the situation can be given a noncommutative but not quantum mechanical representation. By “compressing” the projections representing measurement outcomes into a real subset of the full set of minimal projections of the given Hilbert space one can go beyond the quantum mechanical representation. The representation becomes rigid only if the inner product of any pair of minimal projections in a Hilbert space can be given an empirical content. This is the case for spin-half particles where projections can directly be associated to preparations and measurement directions. Whether one can provide a similar empirical account for the inner product of any pair of minimal projections in a Hilbert space of higher dimension, is a question which cannot be decided a priori.

Appendix

Proof of Lemma 1. (i) Let $\{{\bf e}_{1},{\bf e}_{2},{\bf e}_{3}\}\subset S^{2}$ be an orthonormal basis in ${\mathbb{R}}^{3}$. Then due to (18) the sets $\{f({\bf e}_{1}),f({\bf e}_{2}),f({\bf e}_{3})\}$ and $\{g({\bf e}_{1}),g({\bf e}_{2}),g({\bf e}_{3})\}$ are biorthogonal:

[TABLE]

Biorthogonal sets with cardinality $d$ in ${\mathbb{R}}^{d}$ form (in general two different) linear bases of ${\mathbb{R}}^{d}$. Hence, if ${\bf a}=\sum_{i}\alpha_{i}{\bf e}_{i}\in S^{2}$ and $f({\bf a})=\sum_{i}\alpha_{i}^{f}f({\bf e}_{i})\in{\mathbb{R}}^{3}$ with $\alpha_{i},\alpha_{i}^{f}\in{\mathbb{R}}$, then

[TABLE]

Hence, $f(\sum_{i}\alpha_{i}{\bf e}_{i})=\sum_{i}\alpha_{i}f({\bf e}_{i})$, that is $f$ is the restriction of the bijective linear map $\hat{f}$ characterized by the image linear basis $\{f({\bf e}_{1}),f({\bf e}_{2},f({\bf e}_{3})\}$ of the orthonormal basis $\{{\bf e}_{1},{\bf e}_{2},{\bf e}_{3}\}$. A similar argument shows that $g$ is the restriction of the bijective linear map $\hat{g}$ to $S^{2}$.

(ii) Using polarization identity

[TABLE]

it is enough to show that

[TABLE]

which, however, holds since

[TABLE]

and $\hat{f}$ is linear.

(iii) Using (18) and the orthogonality of $\hat{f}$ one has

[TABLE]

Hence, $\hat{g}=\hat{f}$ due to the uniqueness of the inverse map.

Proof of Lemma 2. (i) Since the trace is a faithful positive linear functional on $M_{n}({\mathbb{C}})$,

[TABLE]

defines an inner product on the $n^{2}$-dimensional complex linear space $M_{n}({\mathbb{C}})$. The real linear combinations of the projections in ${\cal P}_{n}$ span the real vector space of self-adjoint elements in $M_{n}({\mathbb{C}})$, and the complex linear combinations span the complex vector space $M_{n}({\mathbb{C}})$. Let $\{P_{i},i=1,\dots,n^{2}\}\subset{\cal P}_{n}$ be a linear basis in $M_{n}({\mathbb{C}})$. Then the inner product matrix $g\in M_{n^{2}}({\mathbb{C}})$ given by matrix elements $g_{ij}:=(P_{i},P_{j})\geq 0$ is an invertible matrix. Since $\mbox{Tr}(f(P_{i})g(P_{j}))=g_{ij}$ due to (19) $\{f(P_{i}),i=1,\dots,n^{2}\}\subset{\cal P}_{n}$ and $\{g(P_{i}),i=1,\dots,n^{2}\}\subset M_{n}({\mathbb{C}})$ are linear bases in $M_{n}({\mathbb{C}})$ due to invertibility of $g$. Defining the bijective linear maps $\hat{f},\hat{g}\colon M_{n}({\mathbb{C}})\to M_{n}({\mathbb{C}})$ by the linear extension of these image bases for $P=\sum_{i}\alpha_{i}P_{i}\in{\cal P}_{n}$ one has

[TABLE]

Hence, $f$ is the restriction of the bijective linear map $\hat{f}$ to ${\cal P}_{n}$, indeed. A similar argument shows that $g$ is the restriction of the bijective linear map $\hat{g}$ to ${\cal P}_{n}$.

(ii) Using polarization identity

[TABLE]

it is enough to show unitarity on ‘diagonal’ inner products:

[TABLE]

Since $f({\cal P}_{n})\subset{\cal P}_{n}$ by assumption, using the normalization $\mbox{Tr}(P)=1,P\in{\cal P}_{n}$ of the trace it follows that

[TABLE]

i.e. $\hat{f}$ is unitary on diagonals from ${\cal P}_{n}$. Using a spectral decomposition of self-adjoint elements by orthogonal minimal projections one concludes that $\hat{f}$ maps the real vector space of self-adjoint elements in $M_{n}({\mathbb{C}})$ into itself, moreover, it is unitary on diagonals from the space of self-adjoint elements. Since $A\in M_{n}({\mathbb{C}})$ can be written uniquely as a sum of self-adjoint elements: $A=R+iI$ with $R:=(A+A^{*})/2$ and $I:=(A-A^{*})/2i$ it follows that

[TABLE]

that is $\hat{f}$ is unitary on diagonals from $M_{n}({\mathbb{C}})$, which provides unitarity of $\hat{f}$.

(iii) Using (19) and unitarity of $\hat{f}$ one has

[TABLE]

Hence, $\hat{g}=\hat{f}$ due to the uniqueness of the inverse map.

Acknowledgements. I thank Zalán Gyenis, Sam Fletcher and especially Péter Vecsernyés for valuable discussions. This work has been supported by the Hungarian Scientific Research Fund, OTKA K-115593.