Rise of the machine cosmologists

Scientists such as Shirley Ho and Cora Dvorkin are using artificial intelligence and machine learning to unlock the universe’s best kept secrets.

Shirley Ho, Senior Research Scientist, Center for Computational Astrophysics, Flatiron Institute

At Manhattan’s Flatiron Institute, Shirley Ho is teaching computers an ancient human activity: gazing at the night sky and trying to make sense of it all.

She leads the institute’s Machine Learning X Astrophysics group, which uses artificial intelligence (AI) to simulate cosmic phenomena. They create detailed models of how dark matter—the invisible form of matter theorized to comprise about 85% of all matter in the universe—influences the formation of galaxies.

Machine learning also allows scientists to shift from traditional low-resolution models to what Ho describes as “high-resolution, full-HD simulations of the universe.”

“Before machine learning, we would compare simplified, blurry models of the universe to observational data, like a Monet painting of New York versus the real city,” she tells FirstPrinciples on the phone from that very city. “Now, with AI, we can simulate the universe in stunning detail, like comparing a full-HD video of New York to the actual city.”

Maximizing experimental value with AI simulations

The Flatiron scientists utilized the Simulation-Based Inference of Galaxies (SimBIG) framework, an AI-driven method that makes cosmological measurements much more precise.

SimBIG is a framework that uses advanced simulations and neural inference to extract cosmological information from galaxy distributions, enabling precise constraints on the universe's growth and expansion. (Credit: Shirley Ho/Liam Parker)

The AI extracts hidden information from the distribution of galaxies to estimate cosmological parameters—such as the amount of dark matter and the nature of dark energy—with impressive accuracy. To put this in perspective, SimBIG halves the uncertainty in determining the “clumpiness” of matter in the universe, which is like suddenly doubling telescope power without building new hardware.

Such advancements translate to a much greater return-on-investment, particularly for large, expensive experiments.

“Each of these surveys costs hundreds of millions to billions of dollars,” Ho notes. “We want the best analysis possible to extract as much knowledge as we can and push the boundaries of our understanding of the universe.”

Ho envisions a future where AI doesn’t just analyze existing data but actively contributes to the generation of new insights.

“With AI, we can create high-resolution simulations of the universe much cheaper and faster,” she says. “This allows us to test theories and compare them to observational data at an unprecedented level of detail. If we only have a few expensive simulations, AI can extrapolate from those to create a broader understanding, reducing the need for costly experiments.”

She sees potential applications for AI across cosmology, physics, and science more broadly. Ho and her colleagues are currently developing a polymathic AI—“the ChatGPT of science,” as she describes it.

“By teaching machines concepts from multiple sciences, we can create systems that recognize patterns across fields,” Ho explains. “This could lead to discoveries we might never have made ourselves.”

From childhood curiosity to the forefront of dark matter research

Cora Dvorkin, Professor of Physics, Harvard University (Credit: Diana Martinez Llaser)

When Cora Dvorkin was a child, her father would take her to visit his friend Manuel Sadosky, an Argentine computer pioneer who had brought the country its first scientific computer.

In Sadosky’s cluttered apartment, surrounded by books, the girl was captivated by the questions Sadosky posed. He asked if she knew the origin of zero, or how mathematics explained the universe.

“I left those afternoons feeling more curious,” Dvorkin recalled in her 2020 TEDx Talk in Buenos Aires. She explained how her first encounters with a dazzling human intelligence set her on a path to the forefront of artificial intelligence.

Now at Harvard, Dvorkin and colleagues are combining human and artificial intelligence in search of answers to the cosmic mystery of dark matter. Because it cannot be observed directly, dark matter’s existence is inferred through its gravitational effects on visible matter.

“We know it’s there, but we can’t see it,” says Dvorkin. “That’s what makes this problem so fascinating, and so challenging.”

Dvorkin leads a Harvard team that uses AI to analyze gravitational lensing, a phenomenon where dark matter bends the path of light from distant galaxies. Traditional methods for identifying dark matter clumps are time-consuming and imperfect, requiring manual analysis that can take months. With machine learning added to the mix, Dvorkin and colleagues can analyze enormous datasets from galaxy surveys or the cosmic microwave background in a matter of seconds.

A universe of AI applications

Ho and Dvorkin are among a fast-growing tide of scientists worldwide applying AI to the puzzle of dark matter and other cosmic phenomena.

Among those efforts, the “Learning Dark Matter” project at CERN’s Large Hadron Collider (LHC) uses machine learning to sift through particle collision data looking for indirect signs of dark matter interactions.

AI is aiding dark matter simulations as well. The Argonne National Laboratory’s Aurora Supercomputer leverages exascale computing and AI to perform high-resolution simulations of dark matter dynamics, for example. The CAMELS (Cosmology and Astrophysics with Machine Learning Simulations) project combines machine learning with state-of-the-art simulations to model the interplay of dark matter, dark energy, and baryonic matter (ordinary matter like stars and gas).

The Vera C. Rubin Observatory (Credit: Rubin Observatory/NSF/AURA/B. Quint)

Researchers at the Rubin Observatory Legacy Survey of Space and Time (LSST) are using AI to analyze enormous catalogs of galaxies, detect anomalies, process imagery, and identify targets. The Canadian Hydrogen Intensity Mapping Experiment (CHIME), meanwhile, uses an automated real-time data analysis pipeline employing AI algorithms to sift through terabytes of data every second. This helps CHIME researchers identify brief, intense radio-wave pulses from deep space, known as fast radio bursts (FRBs), whose origins are unknown.

A golden era of AI and cosmology

Dvorkin says she feels lucky to be a scientist at a time when our technology is catching up to our theoretical ambitions, helping to answer some of humanity’s most fundamental questions about the universe.

“I sometimes wonder what Manuel would say if he could see what we’re achieving with these intelligent machines,” she said in her TEDx Talk.

With more data pouring in from observational efforts worldwide, AI is poised to play an ever-growing role in solving some of our universe’s greatest puzzles for the foreseeable future. These advances aren’t just academic—by understanding dark matter and the fundamental nature of our universe, we gain insight into our cosmic origins. As AI and astronomy continue to evolve together, they’re helping us answer questions that have fascinated humanity since we first looked up at the stars.