Diversity of observables in quantum mechanics

A question at the Physics Stack Exchange has reminded me about a philosophical preconception that "helps" to prevent many people – including people such as Brian Greene – from appreciating and understanding the foundations of quantum mechanics and the wonderful ways in which they radically differ from the foundations of classical physics.

In classical physics, there are some objective physical quantities – such as positions and velocities of all particles; or the values of several fields at each point and their time derivatives – and all observers may agree about their values, at least in principle. Moreover, all other things that can be in principle measured are simple real functions of these real, commuting variables.




In quantum mechanics, observables must be represented by Hermitian linear operators that usually refuse to commute with each other. (I fell in love with the verb "refuse" as a replacement for "fail" in similar situations because if one says "fail", it suggests – and leads many impressionable people to an incorrect belief – that there is something wrong that should be fixed so that the "failure" doesn't occur again. But the word "refusal" correctly indicates that if you don't like the result, it's just your problem.)

The fact that the operators don't commute with each other means that they can't simultaneously possess sharp, objective values, not even in principle; things that have objective real values do commute with each other because the real values do. That's of course nothing else than the Heisenberg uncertainty principle.

People who remain uncomfortable about quantum mechanics just can't live with the uncertainty principle so they invent various ways to "fake" quantum mechanics by a classical system – that may ultimately be described by some objective quantities with real, unambiguous values. They believe that a "trick" such as classical random generators, hidden variables, gadget to collapse a wave, or many universes is good enough to emulate the new features of quantum mechanics.

But it ain't the case and it can't ever be the case. Many additional pilot waves, classical random generators, or multiple worlds won't change the fact that the functions of the "classical" quantities that describe them will commute with each other. But the observables in the actual quantum world don't commute with each other. What this actually shows is that the observable quantities don't have objective values! Random generators, shrinking waves, pilot waves, or multiple worlds have nothing to do with the quantumness of Nature.

The first new insight of quantum mechanics is that the actual observables – whatever they are for a given physical system – don't commute with each other which directly means that they can't be functions of any objective, classical quantities: those would commute with each other. That's why all existing – and quite certainly all future – attempts to return and squeeze the world into the classical straitjacket have been proved incorrect. However, the classical straitjacket is just a religious dogma for certain people so they will never give up, regardless of any amount of evidence that their efforts are fundamentally misguided.

But there exists a more general assumption that these people are making and that quantum mechanics has refuted. It's the assumption that there exist "the right basic quantities" the world is all about; and "the right ways to measure them".

For example, in mechanics, one may talk about the positions \(\vec x_j\) and momenta \(\vec p_j\) of all particles. Derived quantities such as the angular momentum may be written as simple functions\[

\vec L_\text{total} = \sum_j \vec x_j \times \vec p_j

\] The classical framework encourages you to think that you may equip the Universe with lots of devices that measure the particles' positions \(\vec x_j\); and many apparatuses that analogously measure \(\vec p_j\). All the measured results are real numbers and all other observables – including the angular momentum – are functions of the measured positions and momenta. So all the other quantities are "redundant". You don't need extra apparatuses to measure them – to measure the angular momentum, for example.

This assumption is self-consistent. After all, classical physics has tried to described the world in this way for centuries. However, this assumption is incompatible with Nature, the real world around us. It is simply not true that you may reduce the measurement of a general observable to the measurement of a predetermined set of "basic observables" followed by a calculation of the appropriate function. Quantum mechanics has irreversibly demonstrated that there are no "basic observables" such that their measurements would be enough to determine the values of all observables.

In quantum mechanics, the positions \(\vec x\) don't commute with the momenta \(\vec p\). But you may still think about devices that measure \(\vec x\) and devices that measure \(\vec p\). And you may (incorrectly) assume that things such as the total angular momentum may be measured by using the \(\vec x\) and \(\vec p\) devices many times. But it ain't the case.

The person asking the question at the Physics Stack Exchange presented a simple example: the operator \(xp\) in one-dimensional quantum mechanics. First, let us get rid of a technicality. We often want the observables to be defined as Hermitian operators. However, \(xp\) isn't Hermitian. Its Hermitian part is Hermitian; it is equal to\[

\eq{
\frac{xp+(xp)^\dagger}2 &= \frac{xp+p^\dagger x^\dagger}{2} =\\
&= \frac{xp+px}2 = \frac{xp+xp+i\hbar}{2} = xp+\frac{i\hbar}{2}
}

\] In other words, \(xp\) is equal to \(i\hbar/2\) plus a Hermitian operator. The eigenvalues of \(xp\) are equal to \(i\hbar/2\) plus a real number. Without a loss of generality, we may allow quantities such as \(xp\) to be considered "observables". Let me propose a more general criterion for an observable. It is a normal linear operator \(M\) i.e. one that commutes with its Hermitian conjugate,\[

MM^\dagger = M^\dagger M

\] This definition allows anti-Hermitian operators, operators that differ from Hermitian or anti-Hermitian ones by a \(c\)-number such as \(i\hbar/2\) above, unitary operators (because both products above are equal to \({\bf 1}\) for them), and others.

So the anti-Hermitian part of \(xp\) isn't a real problem. It is a universal \(c\)-number, \(i\hbar/2\), and we may simply subtract it. Quantum mechanics implies that \(xp-i\hbar/2\) is a Hermitian observable. In principle, it may be measured. The measured values must belong among the eigenvalues. The spectrum of this operator is \(\RR\), of course; all real numbers are possible. The positive values appear for states for which \(x,p\) are positively correlated; negative values appear for states with a negative correlation between \(x,p\).

But let me return to the main point. The main point is that in principle, there exists a device that measures the observable \(xp-i\hbar/2\) exactly. However, this device cannot be constructed as one that measures \(x\), \(p\), and makes some classical calculations with the results. If you wanted such a combined device to be precise, you would need the subdevice measuring \(x\) or \(p\) to be precise as well, but a single precise measurement by these subdevices would already make the value of the other quantity, \(p\) or \(x\), completely undetermined and fuzzy, so no full result for \(xp-i\hbar/2\) could ever be precise.

Even though \(xp\) seems to be made out of the "basic" operators \(x\) and \(p\), it isn't made out of them in the usual classical sense. What I mean is that it cannot be measured by measuring the "basic" operators and by doing a calculation with the results for these "basic" operators. You need a completely new device if you want to measure \(xp\).

Is it legitimate to say that \(xp\) is a function of operators \(x\) and \(p\)? Well, it is a function because we can write it as a simple function, a product in a particular order,\[

f(x,p) = x\cdot p.

\] But it's important to realize that this function \(f\) isn't equivalent to a function of two real variables. A function of two real variables \(f(a,b)\) couldn't distinguish \(ab\) and \(ba\) simply because \(ab-ba=0\) for commuting real variables. However, it's not true for operators such as \(x,p\). So the product \(xp\) is an "operator function" which is more general and remembers the ordering and other things.

There are many operators in quantum mechanics. If the Hilbert space is \(n\)-dimensional, there exists an \(n^2\)-real-dimensional space of Hermitian matrices acting on this \(n\)-dimensional Hilbert space. If you pick a random matrix or operator \(M\) in this set, there will be some operators that commute with \(M\) but most of the operators will refuse to commute with \(M\). It will be rather unlikely that you will be able to write \(M\) as a function of simpler yet mutually commuting operators. In other words, you will probably not be able to reduce \(M\) to any simpler operators.

(Only if you measure the multiples of \(M\), it may be achieved by the same apparatus.)

To summarize, for each observable – each matrix on the Hilbert space – you need a completely new apparatus to measure it. The measurement can't be reduced to the measurement of some "basic" operators followed by a calculation reflecting the function by which the "composite" operator is defined!

These elementary facts contradict the prejudice held by many people – including all the anti-quantum zealots – that there exist "basic" operators and without a loss of generality, we may and we should talk about them only. I've discussed similar misconceptions in the case of the Bohmian models and their attempts to segregate observables into primitive and contextual ones. No such segregation is possible in Nature, of course. All observables – all normal linear operators – are as contextual as all others.

In the case of a particle on a line, it may sound particularly counterintuitive that we cannot reduce all observables to \(x,p\) in the classical sense because \(x,p\) "look" much more elementary than all of their functions. However, there exist examples in which you would find the very same claim less controversial. Consider the angular momentum \(\vec J\).

It has three components, \(J_x,J_y,J_z\). They don't commute with each other. For example, one of the commutators is\[

[J_x,J_y] = i\hbar J_z.

\] In fact, you may write the same commutator in a more suggestive form:\[

J_z = \frac{1}{i\hbar}\zav{J_x J_y - J_y J_x}.

\] In this form, you see that \(J_z\) is an "operator function" of the first two components! In classical physics, you would never think that \(J_z\) depends on \(J_x,J_y\): they are three independent components of an unconstrained vector, after all. But in quantum mechanics, if you allow the "operator functions", each of the three components may be written as a function of the remaining two (a multiple of their commutator).

But even though \(J_z\) is an "operator function" of the first two components, you would never think that the right way (or a possible way) to measure \(J_z\) is to measure \(J_x\) and \(J_y\) in various combinations and do a calculation involving the outcomes of these measurements. That would be stupid. \(J_z\) is as elementary as \(J_x\) and \(J_y\). After all, they are related to each other by rotations. You need a special gadget, e.g. a magnetic field in the \(z\)-direction, to measure \(J_z\).

Those people who are incapable of understanding why quantum mechanics is conceptually new and who are dreaming about squeezing physics into the 19th century classical straitjacket again are forced to admit that there exist some objective "preferred variables" in Nature and everything else that may be talked about are just "classical functions" of these "preferred variables". But the example of the angular momentum – much like pretty much any other example of operators in quantum mechanics – is enough to show that this viewpoint is indefensible.

At the end, your theory has to make predictions about the values of \(J_z\) that may be measured. To deny that the angular momentum may be measured and should be predictable isn't a path to make progress in physics. Because "preferred classical variables" is everything that your theory offers, all components of \(\vec J\) have to be some classical functions of these "preferred classical variables" (which may include corrections from "classical random generators"). But if they were classical functions of them, they would have to commute with each other. But they don't commute with each other. So they can't be "classical functions". Whether you are imagining that your would-be fundamental realist (i.e. classical) theory contains pilot waves, multiple universes, gadgets to shrink a wave function, or anything else, you won't be able to make coherent predictions for as simple observables as the components of the angular momentum.

And the angular momentum isn't an exception; it is a rule. A pair of random observables almost never commutes!

Quantum mechanics tells you that you can't reduce all knowledge to some elementary apparatuses that measure "basic" observables only. For the same reason, you should never think about "preferred bases" of the Hilbert space. Instead, quantum mechanics tells you that a physical system may have many features – described by arbitrary linear operators – and for each Hermitian operator, you may decompose the Hilbert space into a basis of its eigenstates. Every basis is as good as every other basis - but they are mutually incompatible.

One might speculate that this feature – the diversity of qualities that a physical system may have – is important for Nature's ability to produce complicated structures, living forms, intelligent life. A world where everything would boil down to classical functions of \(x,p\) could look "dull" and indeed it could be dull and sterile, too. Chemistry and biology depends on atoms' and molecules' discrete spectra – on the finite number of possible excitation energies that an atom or a molecule may have. It's important that the operator representing the energy, the Hamiltonian, is a "new object" that dynamically depends on the context. Even though the energy in simple quantum mechanical models is an "operator function" of \(x,p\), the value of the energy is independent of the values of \(x,p\) one could measure. That's obviously a necessary condition for the spectrum of the Hamiltonian to be discrete or mixed: the set of possible values of \(p^2/2m+V(x)\) with \(x,p\) eigenvalues substituted for \(x,p\) would obviously be continuous which would be bad.

Let me mention that I discussed two operators, \(xp\) and \(J_z\), that may be written as "operator functions" of some other operators, either \(x,p\) or \(J_x,J_y\), respectively. In the first case, it looked more natural to "reduce" \(xp\) to a "more basic" observable than in the second case. However, it's actually equally unnatural in both physical situations and one could construct various isomorphisms between the sets of operators in various limits.

So quantum mechanics has taught us that there aren't "preferred questions about Nature" and "preferred gadgets" that would make all other conceivable apparatuses redundant. For the same reason, there aren't preferred bases of the Hilbert space and preferred representations of the state vector. One may construct infinitely many observables. For each observable, we need a different device to measure it. Quantum mechanics is able to predict the probabilities that we get any result for any observable but these calculations cannot be reduced to any algorithm respecting a classical framework simply because Nature isn't classical, stupid.