Researchers from the International Center for Radio Astronomy Research (ICRAR) in Perth, Oak Ridge National Laboratory in the United States and Shanghai Astronomical Observatory in China processed 400 gigabytes of data a second as they tested data pipelines for the Square Kilometre Array (SKA) telescope, according to an ICRAR statement released on Tuesday.
"Completing this test successfully tells us we'll be able to deal with the data deluge of the SKA when it comes online in the next decade," said Andreas Wicenec, the director of Data Intensive Astronomy at ICRAR.
Wicenec said it was the first time radio astronomy data has been processed on this scale.
The SKA is one of the world's largest science projects, with the low frequency part of the telescope set to have more than 130,000 antennas in the project's initial phase, generating around 550 gigabytes of data every second.
"The fact that we need the world's biggest supercomputer to run this test successfully shows the SKA's needs exist at the very edge of what today's supercomputers are capable of delivering," said Wicenec.
Summit, located at the U.S. Department of Energy's Oak Ridge National Laboratory, is the world's most powerful scientific supercomputer, with a peak performance of 200,000 trillion calculations per second.
The test run used a cosmological simulation of the early universe at a time known as the Epoch of Reionization, when the first stars and galaxies formed and became visible.
They used the Adaptable IO System developed at the Oak Ridge National Laboratory to resolve a bottleneck caused by trying to process so much data at the same time, according to Oak Ridge lab.
An Tao with the Shanghai Astronomical Observatory said the data was first averaged down to a size 36 times smaller before being used to produce an image cube of a kind that can be analysed by astronomers.
"Finally, the image cube was sent to Perth, simulating the complete data flow from the telescope to the end-users," said An.
Construction of the SKA is expected to begin in 2021. The SKA project is supported by 13 national members, including Australia, Canada, China, France, Germany, India, Italy, the Netherlands, New Zealand, South Africa, Spain, Sweden and Britain.