Science: Net Struggles with Data Overload

Society | September 19, 2003, Friday // 00:00

Most home net users would love to have a link that can handle gigabits of data every second. But for some folks such high speeds are simply too slow. Some science experiments can generate so much data that it still takes hours to send the information across the net so it can start to be analysed.

Technologies like the Grid look set to help get the data to where it needs to be and fast net providers are looking at better ways to get data moving.

One of the recent developments in radio astronomy involves linking up a distributed network of telescopes to form a giant, virtual instrument as big as a continent.

The good thing about this technique, which goes by the formidable name of Very Long Baseline Interferometry (VLBI), is that it produces very high quality radio images of astronomical phenomena such as exploding supernovae, pulsars and black holes. The downside is the vast amounts of data that it generates.

The European VLBI project links 16 telescopes that each generate one gigabit of data per second over the 25 days of an observation session.
The end result is a pile of data tens of terabytes deep recorded on to huge numbers of magnetic tapes.

The problem with delivering data by road is the time it takes for the tapes to travel from where they are collected to where they are analysed. Sometimes it arrives weeks after an observation session has finished.

At very high speeds the net's Transmission Control Protocol (TCP), that helps ensure data reaches its intended destination, slows things down because it checks to see if packets have arrived.

Cardiff University is currently installing a gigabit network as part of the Welsh e-science centre being created at the institution.

Tom Wiersma, manager of the network at Cardiff University, said the fast network will help some of its scientists do more with the huge amount of data they gather.

As an example, Cardiff researcher Adam Schultz is using huge numbers of seabed sensors to monitor geological activity and refine theories about the workings of the Earth's interior. Mr Wiersma said that the high-speed network would make it much easier for scientists like Dr Schultz to move information around and added that broader work on packaging data for analysis would be useful too. "Scientists need to be able to connect to the Grid, take their data and find the computing resource they need to crunch it without having to worry about preparing the data, splitting it up and so on," he said.

We need your support so Novinite.com can keep delivering news and information about Bulgaria! Thank you!

Society » Be a reporter: Write and send your article

Advertisement
Advertisement
Bulgaria news Novinite.com (Sofia News Agency - www.sofianewsagency.com) is unique with being a real time news provider in English that informs its readers about the latest Bulgarian news. The editorial staff also publishes a daily online newspaper "Sofia Morning News." Novinite.com (Sofia News Agency - www.sofianewsagency.com) and Sofia Morning News publish the latest economic, political and cultural news that take place in Bulgaria. Foreign media analysis on Bulgaria and World News in Brief are also part of the web site and the online newspaper. News Bulgaria