Data mining the sky

Stuart Weston collects data observed by the Warkworth radio telescope and shares it with scientists who use it to run their own experiments.

We meet Stuart Weston at the lab on a crisp autumn afternoon, just shy of winter. Stuart is a PhD candidate at AUT and has been responsible for collecting and distributing the data collected by AUT's two radio astronomy telescopes in Warkworth. He's been working in radio astronomy for over five years, after working in commercial IT in the UK and New Zealand.

Stuart Weston at the radio telescope

We couldn’t be happier that he made the switch from the commercial world to the academic world — he is one of our biggest consumers of bandwidth in the country and has a unique passion and enthusiasm for getting the most he can out of REANNZ. And it doesn’t stop there: he’s challenging us to push further boundaries, making it even easier for radio astronomers to collaborate internationally.

Stuart’s PhD is titled “Data Mining for Statistical Analysis of the Faint Radio Sky: the Pathway to EMU.” In short, he is collecting data about the universe in multiple frequencies (optical, radio and infrared). Different telescopes are used to observe different frequencies. So when researchers find something of interest in radio, they traditionally embark on a manual and time consuming process to see if the same object has been observed in different frequencies by previous observations. Stuart is developing a statistical method so computers can algorithmically determine matches across different frequency catalogues, and place the results into a database. This database will make it easier for scientists to process and interact with the data. The result is scientists spending more time analysing their results to further science, and allowing computers to do what they are good at: finding matches in large data sets.

The difference between a week and a day

"The difference between a week and a day is huge. It makes a big difference to my work and what I’m capable of achieving."

Stuart Weston, Radio Astronomist at the Warkworth Radio Telescope

The work is understandably data-intensive, with a specific requirement to be able to give access to this data to local and international colleagues. It wouldn’t be possible on a traditional ISP network. This level of data transfer requires an ultra high speed fibre connection. Up until recently, it would still take weeks to move the data observed by the telescopes. However, since the 10 Gbs link from Warkworth was installed, Stuart has been able to transfer these same experiment data sets in under a day. “The difference between a week and a day is huge. It makes a big difference to my work and what I’m capable of achieving,” he says.

Part of Stuart's role at Warkworth is to process the raw data collected by the telescope and perform experiments with the data for organisations and institutions outside AUT. The geodetic experiments are of particular importance, as they feed directly into the GPS network that we have become accustomed to using everyday, whether we are aware of it or not.

Stuart Weston is justifiably enthusiastic about his work

The GPS in your car or on your smartphone relies on the data that is coming out of AUT's telescope, and it requires very quick turn around. An experiment conducted on Thursday needs to be sent to the data correlator on Friday. Once processed the data is fed into the GPS system, ensuring the most up to date solutions are available the following Monday. "Having a good high speed network means we can meet and now even work to improve those turn-around times," Stuart says. 

Looking to the future

"REANNZ is ahead of the ball-game and ahead of the curve. It’s now up to us to come up with ways of using what REANNZ is providing in bigger and better ways."

Stuart Weston, Radio Astronomist at the Warkworth Radio Telescope

Looking into the future, Stuart says he’d like to break some of the laws of physics. “No matter how fast the network is, we still face latency issues. If we could re-write relativity that would be nice, but that’s probably not going to happen. The REANNZ network is now at a point where I’m not worried about whether it can keep up with the work I’m doing. REANNZ is ahead of the ball-game and ahead of the curve. It’s now up to us to come up with ways of using what REANNZ is providing in bigger and better ways.”

That doesn’t stop Stuart from thinking bigger though. He wants to see a shift in network usage across New Zealand. He wants to move his work away from being a ‘data collector’ to a ‘data processor’ and ‘data user’, and he’d like to see the same happening in other areas of data-intensive science. He would like to see the REANNZ link to Australia and the United States ‘running red’ coming back into New Zealand, rather than only going out. “I’d like New Zealand to be contributing at the science level, not just the data level. That’s my ambition and that’s my goal.”

We finish up our Warkworth visit with lunch at the local vineyard and a sampling of the Ransom Vineyard’s 2009 “Cosmos” Chardonnay. It seems that Stuart is not the only one in the region enamoured by space and all of the possibilities that sit right at our fingertips.

REANNZ services in use at the Warkworth radio telescope

Stuart and his team worked with REANNZ to design the network connection to the telescope, and rely on the ultra-fast REANNZ 10 Gbs connection to quickly and reliably turn around data to colleagues in the sciences and GPS data correlators.

High capacity, high-speed network and internet.

Advanced networking security for REANNZ members.

Case Studies
Find more examples of how REANNZ members use our network and services.


Find anything about our products, services, and more. Enter a query in the search input above.