Vacuum Catastrophe (script) (Patreon)

The most successful theory in all of physics is, arguably, quantum field theory. It has made the most precise predictions in all of science. And it has also made the worst. There’s a massive disagreement between the theoretical and the measured values of the energy of the vacuum. We call this theoretical fail the vacuum catastrophe.

The quantum vacuum is a seething ocean of activity. Virtual particles appear and vanish from nowhere, in seeming violation of our intuitions about the conservation of mass and energy. We saw in our recent episode that we can thank the Heisenberg uncertainty principle for this bizarre behavior. It tells us that there’s a quantum fuzziness in the amount of energy contained at every point in space; a non-zero zero-point energy in the quantum fields that can briefly manifest as particles. This suggests that space itself contains energy. But how much energy? Quantum field theory predicts that the energy of the vacuum should be up to 120 orders of magnitude greater than the measured value. It’s one of the greatest unsolved problems in physics.

Let’s talk about vacuum energy from a theoretical standpoint. From the perspective of quantum field theory, every point in space is represented by a quantum oscillator; one for each elementary particle type. Higher energy oscillations represent the presence of real particles. However even the lowest possible energy oscillation – the one corresponding to the absence of particles – has some energy. To satisfy the Heisenberg uncertainty principle the lowest energy state of any field oscillation must be half of the tiny Planck constant times the frequency of that oscillation. That’s a miniscule amount of energy; except that to properly describe a perfect vacuum, every single possible frequency mode must have that amount of energy. To calculate the energy of the vacuum we should add this tiny energy over an infinite range of frequency modes for all fields. Multiply a finite energy, no matter how small, by infinity and you get infinite energy.

We’ll get back to whether the idea of infinite vacuum energy is actually possible. It certainly sounds a bit excessive, so maybe we can avoid it. Let’s think in terms of the electromagnetic field. What if there’s a maximum possible frequency for virtual photons? Then our sum of frequency modes would be finite. It sounds like a hack, and it sort of  is, but we’re getting desperate, and anyway, there’s an almost sensible cutoff for photon frequency. It’s where photon energy is equal to the Planck energy – 10^19 GeV. This is where our understanding of physics breaks down. Until we develop a theory of quantum gravity, we can’t say whether photons above this energy are possible.

So if we add up the vacuum energy, including virtual photons all the way up to the Planck energy, we get a finite number. A very, very large finite number. The energy density would be 10^112 ergs/cm^3. This “back of the envelope” estimate was first made by John Wheeler and Richard Feynman, who noted that one teacup of space with this energy density would be enough to boil all of the oceans on the planet. Yet even if vacuum energy did have a value this high – in fact even if it were infinite – we wouldn’t notice. At least according to quantum theory. In fact in both quantum mechanics and classical mechanics, a particle’s equations of motion depend only on changes in energy; the zero-point of the energy scale is irrelevant. As long as the vacuum energy is the same everywhere then it’s inaccessible as an energy source. We’re going to come back to that point in an upcoming episode, so you can knowledgeably scoff at zero-point energy perpetual motion machines.

In general, a crazily high - even infinite - vacuum energy doesn’t affect the predictions of quantum field theory. Just redefine the zero point to whatever you want and you’re good to go. However you can’t be so cavalier with general relativity. Einstein’s theory tells us that any form of energy produces gravity, and what matters is the absolute amount of energy, not relative deviations. Vacuum energy should produce a gravitational effect, and a huge vacuum energy should produce a huge gravitational effect. That effect would be seen in two ways. An energy of space itself should cause exponential expansion, at least in the case of an already-expanding universe. It should also massively increase the spatial curvature of the universe. This is all extremely unintuitive, and we spent 5 episodes trying to give you a sense of why this is true. Feel free to jump down that rabbit hole if you haven’t already. But the upshot is that if vacuum energy really did have the enormous value predicted by theory then our gently expanding, geometrically flat universe would not exist.

The realization of this fact in the early days of quantum field theory was the beginning of what would become the vacuum catastrophe. It wasn’t a full-blown catastrophe yet; there was still a way out. If some fields have extremely high zero-point energies then perhaps others have negative zero-points that cancel them out. An extension to the standard model of particle physics called supersymmetry might partially allow this. It gives all particles a “supersymmetric” counterpart that may precisely cancel out their vacuum energy. “Basic” supersymmetry only allows us to cancel out photons down to the so-called electroweak energy, which brings the predicted vacuum energy down to a mere 10^47 erg/cm^3. But perhaps a yet deeper theory allows a perfect canceling down to zero. For a while, theorists assumed that something like this must be happening – meaning the vacuum energy was really zero. It was a glimmer of hope that catastrophe could be averted!

Until we discovered dark energy. In the late 90s astronomers discovered that the expansion of the universe is, in fact, accelerating in exactly the way we would expect from a non-zero vacuum energy. Unlike the Lamb shift or Casimir effect, the observation of accelerating expansion allowed us to measure the absolute density of vacuum energy. Assuming vacuum energy is to blame, dark energy weighs in at 10^-8 erg/cm^3. Compare that to the number predicted by quantum field theory. There’s a discrepancy of 120 orders of magnitude between the two, or 55 orders of magnitude if we use the electroweak energy cutoff instead of the Planck energy cutoff.

Now THAT is what scientists call a catastrophe. QFT can give us an extremely high vacuum energy, or a vacuum energy of exactly zero if we assume symmetry of positive and negative zero-points between different fields. But a very small non-zero vacuum energy? That requires what we call fine tuning. Gigantic positive and negative zero point energies would need to cancel each other out down to a very tiny non-zero value. That seems like a ridiculously unlikely bit of luck if we can’t invoke symmetries. A proposed solution is the anthropic one: We exist in an extremely rare universe whose fundamental fields canceled out their zero-point energy, at least enough of it to allow life and astronomers to evolve. That would imply countless other universes with different, less comfortable vacuum energies.

Of course it may also be that a theory of everything will resolve this theoretical catastrophe without requiring us to invoke the anthropic principle. The conundrum continues to perplex physicists, and will do so until we reach a deeper understanding of the true nature of space time.