/ MEDIA STATEMENT / The content on this page is not written by Polity.org.za, but is supplied by third parties. This content does not constitute news reporting by Polity.org.za.

Increased investment in research and development by both the public and the private sectors will enable South Africa to secure a greater share of global markets in the big data industry.

Speaking at an information and communication technology conference at the Gert Sibande Technical Vocational Education and Training College in Secunda, Mpumalanga, this morning, Minister Pandor said that, with the right interventions, there was great potential for South Africa to compete in the big data industry.

Mobile devices, online activity, commercial transactions, traffic cameras and remote sensing instruments, among others, are continuously collecting data at rates far larger than can be processed. The datasets collected often are semi-structured and new innovative techniques are needed to collect, curate, transmit and analyse these resources to make sense of historical patterns.

With astronomy, it has become increasingly clear that the old ways of working with data no longer apply. Precursors to the Square Kilometre Array (SKA) radio telescope have ushered in an era of data-intensive astronomy.

One such pathfinder – LOFAR, the Low Frequency Array built by the Netherlands Institute for Radio Astronomy ‑ has a data collection exceeding 20 petabytes. To illustrate the immensity of this figure, it would take about 2 000 years to play only one petabyte of average-length MP3-encoded songs.

With South Africa's MeerKAT and the Netherlands' Apertif (APERture Tile in Focus) telescopes both expected to come online this year, the scale of data collection is poised to increase significantly. Data at these scales present unique challenges not just for managing the collection, but also for how researchers extract their science.

The two countries last year signed an agreement towards unlock the hidden secrets in the immense amount of data generated by SKA. The agreement on data science establishes data centres for astronomers around the world to access large-scale data infrastructures and associated high-performance computing needed to make sense of the data.

Minister Pandor said it was clear that the current bigdatachallenge required a wide range of new skills, policies and practices, technologies and legal frameworks. In addition, fast broadband networks, trusted repositories and nationally accepted practices were required.

"We have a lot of science programmes to work with ‑ the SKA, bioinformatics programmes, Earth observation data initiatives, the SA Research Infrastructure Roadmap and others," said Minister Pandor.

A big data Africa programme is also being proposed for the African countries involved in the SKA and African Very Long Baseline Interferometry Network (AVN). The focus is to develop high-performance computing capabilities in these countries, through training researchers, students and scientists.

Issued by Department of Science and Technology