By 1994 the Supernova Cosmology Project had proved that they could deliver distant Type Ia supernovae "on demand." A postdoctoral fellow at the Harvard-Smithsonian Center for Astrophysics that year, Brian Schmidt, had been aware of the SCP's previous work but skeptical that supernovae could ever be used to do cosmology.

Type Ia supernovae are thought to result when a white dwarf star in a binary system accumulates enough matter from its larger companion. When the white dwarf reaches the critical Chandrasekhar mass, about 1.4 times the mass of our Sun, high internal density and temperature ignite a thermonuclear explosion. Because the masses of Type Ia supernovae are similar, their brightnesses are similar.

Then Schmidt, who would soon move to the Australian National University's Mount Stromlo and Siding Spring Observatories, heard visiting astronomer Mario Hamuy from Cerro Tololo show "that type Ia supernovae could be used to measure very accurate distances, and soon after Saul's group at Berkeley demonstrated that it was possible to find distant supernovae in large numbers. These two advances were the spark that led to Nick Suntzeff and I forming the High-Z SN search team in August 1994. We decided to compete against Saul's team...." The High-Z Team included a number of astronomers who were supernova veterans.

In June, 1995, at a conference in Spain on "thermonuclear supernovae" — Type Ia supernovae, that is — Perlmutter presented a paper from the Supernova Cosmology Project team that was the first to use a sample of high-redshift supernovae to measure the mass density of the universe. In this paper, published in 1996 as "Scheduled discoveries of 7+ high­redshift supernovae," the authors noted that "SNe Ia at high redshift are difficult to work with for at least three reasons: they are rare, they are rapid, and they are random." Much of the paper was devoted to how the systematic uncertainties associated with Type Ia supernovae could be addressed. 

The same issues, based on analysis of the same first seven supernova, were discussed in much greater detail in a paper appearing early in 1997, titled "Measurements of the cosmological parameters Ω and Λ from the first seven supernovae at z ≥ 0.35." Among the uncertainties analyzed in this "first-seven" paper:

• Extinction — the absorption and scattering of light, usually by intervening dust;

• K-correction — converting observed magnitudes to rest frame (non-redshifted) magnitudes in order to compare the brightness of distant supernovae at different redshifts;

• Malmquist bias — the increasing proportion of visible bright objects with increasing distance;

• Gravitational lensing — in which intervening mass might disperse a supernova's light;

• Evolution, or "metallicity" — the likelihood that objects formed early in the history of the universe contain fewer heavy elements, with possible effects on brightness.

The "first-seven" paper discussed ways to identify and reduce error from these sources and others, plus other means of insuring accurate measurements. As a first step toward proving the techniques, the paper demonstrated what would be possible when they were applied to the much larger supernova dataset the SCP was then in the process of collecting.

The large dataset was in fact crucial to the discovery of a cosmological constant and confidence in the result. While the error bars on some of the first seven measured supernovae were large enough to allow low-mass-density cosmologies, the results also appeared consistent with the presumed decelerating universe, and a decelerating universe appeared to fit the data from the small sample better than an accelerating one. (Indeed, as it later developed, the sample did include a single, atypical supernova.)

The SCP regarded these results as only the first rung of a ladder that had to be climbed — a ladder that was being climbed as the team accumulated a large enough dataset of dependable distant Type Ia supernovae. The methods they used to eliminate systematic errors were subsequently employed by both teams, as was the SCP's technique for finding supernovae on demand. Soon discoveries of reliably identified, ever-higher-redshift supernovae began pouring in.

The work of both groups now depended on the next generation of wide-field cameras. Gary Bernstein of the University of Michigan and Tony Tyson of Bell Laboratories and their colleagues built a camera of this kind in 1996, which was supported at the CTIO 4‑meter telescope. It became available to the supernova hunters of both teams beginning early in 1997.

	Starting in 1997 a new wide-field camera built by Gary Bernstein and Tony Tyson, mounted on the 4-meter telescope at the Cerro Tololo Inter-American Observatory in Chile (housed in the dome pictured here), was used by both competing teams of supernova hunters.

In March of 1997, during a two-night search at the CTIO 4-meter telescope that yielded 16 new supernovae, the SCP team discovered SN 1997ap, a Type Ia supernova at z = 0.83, the highest redshift yet. Their paper on the discovery, "Discovery of a supernova at half the age of the universe and its cosmological implications," was the first to show a high-redshift spectrum taken near maximum brightness. By comparing it to a series of low-redshift spectra, the SCP showed for the first time that a Type Ia supernova at this great age and distance looked just like a Type Ia supernova today — a strong qualitative argument against large evolutionary effects.

The new data reduced uncertainties, zeroing in on a value of the mass density of the universe substantially lower than that suggested by the "first-seven" paper. It demonstrated, as Goobar and Perlmutter had proposed in 1994, that a large range of redshifts could be used to separate measurements of omega-mass, the density of mass in the universe, from omega-lambda, the density of the cosmological constant.

Moreover, for anyone who assumed that the universe was geometrically flat, the lower-mass-density value from the SCP dataset including SN 1997ap was itself enough to point to a positive cosmological constant. Many astronomers and astrophysicists favored a flat universe, which was predicted by inflation theory, but it was yet to be proved.

SN 1997ap proved to be a turning point in cosmology. The supernova was found in March; "Discovery of a supernova at half the age of the universe and its cosmological implications" appeared in Nature on January 1, 1998. By then, the events of 1997 had accelerated so quickly that the rate at which papers could be published no longer kept up.

Measuring a positive cosmological constant

By the fall of 1997, the SCP team had analyzed 40 of their high-redshift supernovae and found that they were significantly fainter than seemed possible in a high-mass-density universe.

In fact, these distant supernovae were so faint that no models except those with a positive cosmological constant would fit the data. Perlmutter presented the results at a Physics Department colloquium at UC Berkeley on December 1, and on December 11 to the UC Santa Cruz Physics Department (where astrophysicist Joel Primack called the results "earthshaking"). Meanwhile SCP team member Gerson Goldhaber addressed a colloquium at the University of California at Santa Barbara's Institute for Theoretical Physics (now the Kavli Institute) on December 11, where he discussed the SCP results with members of the competing team.

The SCP team went public with their result in January 1998 at the American Astronomical Society meeting in Washington, D.C., in a press conference and a series of oral and poster presentations. The most profound scientific impression may have been made by the data-packed poster, as indicated by the reminiscences of cosmologist Michael Turner of the University of Chicago:

I was wandering aimlessly around a room filled with poster papers. Saul Perlmutter, the leader of the Supernova Cosmology Project at Berkeley, grabbed me and asked if I wanted to see something interesting.
     What he showed me change the direction of my research and recharged my science batteries. It also changed the course of astronomy and physics....
     After many ups and more downs, including a chorus of "you'll never succeed" sung by many astronomers, in 1998 they [the SCP team] announced their amazing result: The universe is speeding up, not slowing down.... Not wanting to be left out of the fun, I turned the focus of my research to this new mystery.

The poster was noteworthy for demonstrating for the first time how a large statistical sample size of supernovae made it possible to check for the list of systematic uncertainties discussed in the "first-seven" paper — and made it clear that the surprising results indicating a cosmological constant could be relied upon. The SCP team soon made the poster widely available on the web, and it was frequently cited during the following year.

The Supernova Cosmology Project's poster presentation at the 1998 annual meeting of the American Astronomical Association "changed the course of astronomy and physics." It gave evidence based on 40 distant Type Ia supernovae that indicated a positive value of omega-lambda (energy density of the universe), and thus for a cosmological constant. To see an online version that allows each section of the poster to be greatly enlarged, click on the image or go here.

In February 1998, UC Berkeley astronomer Alex Filippenko (a member of the Supernova Cosmology Project during 1994/1995 who switched to the newly formed High-Z team in 1996) presented the competing team's first evidence for a positive cosmological constant at the Dark Matter meeting in Marina Del Rey. This evidence was based on 14 of the High-Z team's own discoveries plus two from the SCP team, including the highest-redshift confirmed Type Ia in the SCP sample.

The two teams' results, presented in back-to-back talks at this meeting, were in close agreement. The High-Z Team's findings were based on a full analysis of 10 high-redshift supernova light curves, plus an additional four high-redshift supernovae for which the peak magnitudes were estimated based on a single photometry date together with a spectrum, following a technique developed by the two competing teams together, plus the two (including the highest-redshift supernova) found by the Supernova Cosmology Project.

At the Fermilab conference early in May, where two-thirds of those in attendance expressed overwhelming confidence in the results, the effect on theoreticians was particularly electrifying. A rush of theoretical preprints and papers followed, trying to explain the new results with physics models such as dynamical scalar fields and "quintessence." Michael Turner, who'd first encountered the evidence for a cosmological constant less than four months earlier when he saw the SCP poster at the AAS meeting, decided that the unknown energy component of the universe needed a broader descriptive term. By June of that year he'd come up with the one that stuck: dark energy.

The excitement surrounding the discovery of dark energy rapidly spread throughout the scientific community and beyond. Science magazine chose "the accelerating universe" as the Breakthrough of the Year in December 1998. Books, articles, and television and radio shows about dark energy have proliferated since. With the passage of the years, the profound mystery of dark energy has engaged a world-wide, science-interested public.

	Some of the members of the Supernova Cosmology project in 2007: background (photo by Roy Kaltschmidt) from left; front row, David Rubin, Saul Perlmutter, Josh Meyers, Hannah Swift; standing: Tony Spadafora, Kyle Dawson, Rahman Amanullah, Nao Suzuki, and Kyle Barbary. Inset (photo by Rosemary Nocera) from left; front row, Kyle Barbary, Serena Nobili, Rahman Amanullah, Gerson Goldhaber, Tomonori Totani, Yutaka Ihara, Saul Perlmutter, Chris Lidman, Don Groom; middle row, David Schlegel, Josh Meyers, Hannah Swift, Kouichi Tokita, Nao Suzuki, Tomoki Morokuma, Ariel Goobar, Mamoru Doi, Tony Spadafora; back row, David Rubin, Greg Aldering, Alex Gude, Pilar Ruiz-Lapuente, Naohiro Takanashi, Takeshi Oda, Naoki Yasuda, Xiaosheng Huang, and Kohki Konishi.

The competing teams and their leaders have been awarded numerous major international prizes, including Perlmutter's 2006 International Antonio Feltrinelli Prize in the Physical and Mathematical Sciences, awarded once every five years by the Lyncei Academy in Rome; the 2006 Shaw Prize in Astronomy to Perlmutter of the SCP and to Adam Riess and Brian Schmidt of the High-Z team, awarded in Hong Kong; and the 2007 Gruber Cosmology Prize, awarded in Cambridge, England and divided among Perlmutter, Schmidt, and the two teams these researchers led.

The sequence of scientific advances that culminated in the discovery of the accelerating expansion of the universe was summed up in a Supernova Cosmology Project study of 42 high-redshift supernovae, presented in December of 1998 and published in the June, 1999 issue of Astrophysical Journal. Going beyond the results presented at the January 1998 AAS meeting, this was a detailed, tabulated accounting of the sources of uncertainties in the supernova data, both statistical and systematic. It showed that the analysis was robust whether the entire sample of supernovae was used or only the best-measured, "clean" sample. Because of its size, the larger data set incidentally made it quite clear which of the first seven supernovae was the outlier from the larger population, the one that had initially masked the evidence for a cosmological constant.  

Members of the Supernova Cosmology Project who contributed to the 1999 paper were Perlmutter, Greg Aldering, Gerson Goldhaber, Robert Knop, Peter Nugent, Patricia Castro, Susana Deustua, Sebastien Fabbro, Ariel Goobar, Donald Groom, Isobel Hook, Alex Kim, Matthew Kim, Julia Lee, Nelson Nunes, Reynald Pain, Carl Pennypacker, and Robert Quimby of Lawrence Berkeley National Laboratory; Chris Lidman of the European Southern Observatory; Richard Ellis, Mike Irwin, and Richard McMahon of Cambridge University; M. Pilar Ruiz-Lapuente of the University of Barcelona; Nicholas Walton of the Isaac Newton Group, La Palma; Brad Schaefer of Yale University; Brian Boyle of the Anglo-Australian Observatory, Sydney; Alexei Filippenko and Thomas Matheson of the University of California at Berkeley; Andrew Fruchter and Nino Panagia of the Space Telescope Science Institute in Baltimore; Heidi Newberg of Fermi National Laboratory; and Warrick Couch of the University of New South Wales. While many remain at these institutions, many others have moved to astrophysics and astronomy centers around the world. Today the Supernova Cosmology Project includes members from eight countries in Europe, North and South America, and Asia.

It was early in 1998 that both the Supernova Cosmology Project and the High‑Z Supernova Search Team went public with their evidence for a positive cosmological constant. Their data and analyses were substantially independent; the papers that detailed their conclusions, one based on 16 supernovae, the other based on 42 and including estimates of systematic uncertainties, were quite similar. The scientific community could be confident that something — call it dark energy — was accelerating the expansion of the universe.

Reconceiving the cosmos

Prior to 1998 only a few theorists had been willing to consider that the universe might include anything like Einstein's long-abandoned cosmological constant. Perlmutter and his colleagues in the Supernova Cosmology Project worked tenaciously for years, first to establish methods for studying distant Type Ia supernovae and then to convince the astronomical community that many fundamental questions about the nature of the cosmos could be answered using these methods.

Their success inspired competition, which in turn supported the revolutionary finding that mass and ordinary energy account for only a fraction of the density of the universe. The rest is the cosmological constant — or some other form of dark energy.

Theorist Frank Wilczek has called dark energy "the most fundamentally mysterious thing in science." Discovering its nature is one of the most pressing questions facing astronomers and physicists in the 21st century: the next big challenge for the Supernova Cosmology Project.

In the next issue of Science@Berkeley Lab, Part III of Dark Energy's 10th Anniversary will examine the aftermath of the discovery:

• Additional evidence from independent measures of the cosmic microwave background and other observations;

• The search for Type Ia supernovae with z greater than 1, dating from before the epoch of acceleration;

• A better understanding of the physics of Type Ia supernovae;

• The SuperNova/Acceleration Probe, SNAP, a satellite to trace the expansion history of the universe by collecting thousands of distant supernovae and determining the distribution of mass by measuring the weak gravitational lensing of distant galaxies.

Return to Dark Energy's 10th Anniversary, Part I

Additional information

• For a chronology of the events leading to the discovery of dark energy see "Critical point: Dark energy," by science historian and philosopher Robert P. Crease, published in the December, 2007 issue of Physics World.
• "Scheduled discoveries of 7+ high-redshift supernovae: First cosmology results and bounds on q0," by Perlmutter et al, appeared in the Proceedings of the NATO Advanced Study Institute on Thermonuclear Supernovae held in Begur, Girona, Spain, June 20-30, 1995 (published February 23, 1996)
• "Measurements of the cosmological parameters omega and lambda from the first 7 supernovae at z >= 0.35," by Perlmutter et al, appeared in the January 27, 1997 issue of Astrophysics Journal.
• "Discovery of a supernova explosion at half the age of the universe and its cosmological implications," by Perlmutter et al, appeared in the January 1, 1998 issue of Nature.
• "Cosmologists Ponder 'Missing Energy' of the Universe," by John Noble Wilford, appeared in the May 5, 1998 issue of the New York Times.
• "Prospects for probing the dark energy via supernova distance measurements," by Dragan Huterer and Michael S. Turner, appeared in Physical Review D Rapids, August 30, 1999.
• "Measurements of Ω and Λ from 42 high-redshift supernovae," by Perlmutter et al, appeared in the Astrophysical Journal, June 1999.
• "Observational evidence from supernovae for an accelerating universe and a cosmological constant," by Riess et al, appeared in the Astronomical Journal, September 1998
• "Supernovae, dark energy, and the accelerating universe," by Saul Perlmutter, appeared in Physics Today, April 2003
• More about the 2006 Shaw Prize in Astronomy
• More about the Lyncei Academy's 2006 Antonio Feltrinelli International Prize
• More about the 2007 Gruber Cosmology Prize
• Supernovae: The stellar route to understanding dark energy, Part I
• Supernovae: The stellar route to understanding dark energy, Part II
• Supernovae: The stellar route to understanding dark energy, Part III
• The website of the Supernova Cosmology Project