It’s one of the biggest mysteries in science.
it’s the elephant in the room, that you can’t see.
It makes up the majority of the universe – 70%, the other 30% being ordinary and dark matter.
But we don’t really know what it is. It’s as if we are boats sailing on an invisible ocean, trying the figure out what the heck we are sailing on.
The burning questions we want to ask are:
1) What is dark energy?
2) What could be causing it?
3) Why is it there?
4) And finally, does it really exist, or could our measurements be completely wrong?
Ever since 1929, when Edwin Hubble had discovered that the universe was expanding, scientists had sought to find out what this expansion rate was.
Most scientists believed that while the universe was known to be expanding, this expansion rate had to be slowing down over time, because of the attractive effect of gravity pulling matter together. So scientists set out to measure this slowing down of the expansion rate.
This was not easy to do because in order to make these kinds of measurements, you have to be able to view the brightness and redshifts of galaxies that are hundreds of millions and billions of light years away.
These are object in the sky that we can not see with the naked eye.
There was no good way to measure this accurately for most of the 20th century, until very large telescopes and the Hubble telescopes were built.
In addition, some kind of measurement technique was needed to establish a standard luminosity that these galaxies could be compared to, because not all galaxies are equally bright at the same distance. This standard luminosity is called a standard candle.
In the late 80’s and early 90’s two groups of astronomers, one led by Saul Perlmutter and Adam Riess in the US, and Brian Schmidt in Australia, came up with a clever way to measure the true distances and redshifts of very distant galaxies by trying to find recent Supernova explosions – very special ones called type 1A supernovae.
These provide a standard candle because all type 1A Supernovae explode with about the same luminosity, because they grow to about the same size. So by measuring the brightness of these supernovae, they could ascertain the distance.
And my measuring the red shift of this same light, they could determine how much space had expanded from the time the light left the supernova.
When these two pieces of data were combined, from galaxies that were at various distances from earth, they could get an idea how much space had expanded over different time frames, and thus how fast the galaxies were moving away from earth.
What they found was shocking to most scientists because not only was the universe not slowing down, or remaining constant, it was speeding up. The expansion was accelerating. And no one could explain it.
Saul Perlmutter, who won a Nobel prize for this discovery, described this as throwing an apple in the air, waiting for it to come down, only to find that it took off like a rocket into outer space. So the question is if it took off like a rocket, where did the energy come from to fuel this rocket?
This apparent energy that is fueling the accelerating expansion of the universe is called “dark energy.”
So what could be causing it? There are three possibilities.
The first possibility is that this energy is a property of space itself. It comes from the vacuum of space. It comes from a non-perfect cancellation of quantum fluctuations. What do I mean by this?
According to Heisenberg’s uncertainty principle, quantum fields from which all particles come from cannot sit still. These fields are constantly creating virtual particles in empty space. So for example, electrons and positrons, protons and antiprotons, quarks and antiquarks, come in and out of existence in empty space.
This can be seen from this version of Heisenberg’s equation:
The change or uncertainty in energy, times the uncertainty in time is greater than or equal to planck’s constant divided by two times pi.
This equation also tells you that you can have virtual particles form out of nothing if the change in energy and or change in time is so small that it is less than Planck’s constant over two pi.
This is energy that you can’t really see. So is there any experimental evidence that this can energy can be positive? It turns out there is. And this is called the Casimir effect. I talked a little bit about this in my Warp Drive video, but essentially it is a physical effect that you can see and feel due to the fluctuation of quantum fields in empty space.
But the biggest problem with this theory is that when you use the knows equations of quantum mechanics, the theoretical energy is almost 120 orders of magnitude higher than what is actually measured from the type 1A supernovae data and cosmic microwave background.
This is the greatest mismatch between theory and observation in all of physics. The worst prediction of all time.
In other words, our equations indicate that this energy should be much larger than what it actually is. But obviously our equations are wrong because if the energy was as much as theorized, the universe would have expanded so fast that galaxies, solar systems, and we wouldn’t exist.
So the vacuum energy is close to zero, but not quite zero. Why is this the case, why is there just enough vacuum energy to cause the expansion rate to be just what we observe it to be.
One scenario could be that maybe the vacuum energy really is zero due to some overlying symmetry in the universe, which we don’t understand yet, but the slight amount of remaining dark energy is explained by some other mechanism.
The most extensively investigated alternative mechanism to quantum fluctuations is a set of theories generically called quintessence. This is similar to Aristotle’s fifth element “essense” – the other four being fire, earth, water, and air.
Quintessence would be like an extra energy field the pervades the entire universe which happens to be repulsive. This field would be almost same throughout space and throughout time, but not exactly the same. It is something that can change slightly in time, and can fluctuate from place to place. Unlike lambda or vacuum energy which is the same throughout the universe, It would be a dynamical dark energy. In other words, it can change slightly from place to place, and in time.
The interesting thing about dynamical dark energy, or quintessence is that it can be measured. How? because the differences in dark energy in different parts of space would be measurable. And if it is found to be true, it would eliminate the idea of the dark energy emanating from quantum fluctuations.
It can be thought of as a fifth fundamental force, the other four being gravity, electromagnetism, strong nuclear force, and weak nuclear force. This would be a force mediated by a new boson particle..
There is a fine tuning problem that I want to mention here, which is called the coincidence scandal. If you look at the current measured energy density of the universe, you see that dark energy consists of 70%, and matter is the other 30%. These are really almost the same in the grand scheme of things. 30% is almost 70%. The difference between the two is only about 2.3 times.
The reason this is considered coincidental is because the dark energy density of the universe is staying constant from moment to moment. But the energy density of matter is going down dramatically because as the universe expands, the energy density of matter goes down, matter becomes less dense.
If you go back to the time of the cosmic microwave background, 380,000 years after the big bang, the matter density was about a billion times more than the energy density of dark energy. So the ratio between the two was about one billion. In the future the difference between the two densities will also be dramatic. Why do we happen to live in a moment in the universe where the difference is only about 2.3?
Why should it be that way? This is called the coincidence scandal.
If we go with the dark energy model explained by vacuum fluctuations, which predict that that dark energy stays exactly the same throughout time and throughout space, then you have to concede that there is something really special about now. And it is indeed a huge coincidence.
But if you believe quintessence is true, then you can say, hey this is not really a coincidence because dark energy is dynamical, and there is some underlying mechanism that has made it so that the current energy density ratio is only about 2, because quintessence can change over time.
What would we see if we could detect this quintessence field?
For one thing, if the field exists it would be slowly going down in value over time. We would see this as a decrease in dark energy density over time.
But here’s another way that quintessence could be detected. The excitations of the quintessence field would manifest itself as a particle, probably a boson, but unlike every other boson, it would have negative energy. So in empty space, you could create some positive energy particles and some negative energy particles without violating conservation of energy laws. This would mean that lighter particles could decay into heavier particles by emitting quintessence particles with negative energy.
Of course this has never been seen. But if we ever did see it, and were able to harness it, we could realize Zefram Cochrane’s dream, and the dream of Star Trek fans everywhere, because we could finally build that warp engine and start our 5 year mission to other worlds.
But so far attempts to explain or detect quintessence have not worked. The models that fit the current data indicate that dark energy is not dynamical, but stays constant throughout space and time. It is smoothly distributed in space and does not clump in the presence of gravity. If it did, we could detect this in gravitational lensing.
Could dark energy be photons or radiation? No because the energy density of radiation and photons changes as space expands. Dark energy however stays the same. This is the reason many scientists are convinced that dark energy is a property of space itself. So as more space is created, more dark energy is created. And this is the reason it could be expanding.
Why is it that if the density doesn’t go down, it makes the universe accelerate? Because what it means is that every cubic centimeter of space is expanding at a constant rate. So over very large distances, the cumulative effect is an acceleration of the universe. As more space is added, more energy is added, and the result is accelerating expansion
But if more energy is being created, doesn’t this violate energy conservation? It turns out that this doesn’t because the positive energy is perfectly balanced by an increase in negative gravitational energy of the universe.
This can be thought like lifting a ball off the floor. The energy that you are adding to the ball is balanced by the negative gravitational potential energy of the ball.
The third possibility is that maybe we are wrong about dark energy, and maybe it doesn’t exist at all. There is a small minority of scientists who believe this. Most notably among them is Subir Sarkar, Theoretical physicist at the University of Oxford in the UK. His group examined the Type 1A supernova data after the raw data was made available to the public in 2014.
He argues first that the original supernova data were not compelling because they had a correlation of only 3 sigma, instead of the typical standard for cosmology, which is 5 sigma.
In addition, he argues that the original calculations did not take into account the uneven distribution of matter in the universe. If the Milky Way itself is being pulled in one direction, then this would cause an error in the measurements.
In other words, maybe we are the ones decelerating instead of the rest of the universe accelerating. Given these potential errors, he says the original data correlation is likely to be even lower than 3 sigma, more like 2 sigma. He therefore believes dark energy may not exist.
Dr. Sakar is firmly in the minority on his views, for several reasons. Scientists knew from the very beginning that it was 3 Sigma data. This is nothing new. By the way 3 sigma data still correlate at a reasonably high 99.7%. 2 sigma is 95%. Cosmologists prefer 5 sigma which is 99.99997%.
But even at a 2 sigma correlation of 95%, there is significant correlation.
Part of the reason cosmologists readily accepted this data back in 1998 is because prior to this, cosmological models predicted that the universe had something called a critical energy density, which means that it was not curved in 4 dimensions. It was flat in 4 dimensions. But the matter only accounted for 30% of this critical density. So the remaining 70% had remained a mystery.
So when the Supernova data indicated that the amount of dark energy needed to make the universe accelerate is 70% of the critical density, the mystery of the missing 70% energy was solved. It was a Eureka moment, and everything fell in place, and physicists quickly adopted the dark energy model.
The other fact is that the 1998 supernova data is not the only data that provides evidence of dark energy. Subsequent independent studies have confirmed this, in particular the 2000 cosmic microwave backround experiments and the sloan digital sky survey in 2003 showed strong evidence that the matter density of the universe is about 30% of critical density, meaning 70% dark energy is needed to justify what we observe. So dark energy makes many of the puzzle pieces fit together to tell a story that is consistent. This is why attempts to discredit dark matter are not generally accepted.
You could still say what if all this is wrong? Maybe the universe is not flat. Or Maybe there is something wrong with Einstein’s theory of gravity, or a refinement of it needs to be made to include a field that creates this cosmic acceleration. But when scientists attempt to modify general relativity, it causes problems and the whole equation breaks down.
For now, what is generally accepted is that dark energy exists, and fits with both our observations and the best models of cosmology. So unlike what Aristotle believed, we are not only not the center of the universe, we are not even made mostly of the same stuff. We are made of only the miniscule 4% that the universe is made of.
