South Africa’s education department has reacted angrily to a World Economic Forum (WEF) report that ranked the country last out of 148 countries for the quality of its maths and science education.

The WEF’s 2014 Global Information Technology Report also ranked South Africa 146th for the overall quality of its education, below a host of other African countries including Swaziland, Zambia, Zimbabwe, Mozambique, Kenya, Malawi, Uganda, Rwanda, Senegal, Nigeria, Tanzania, Liberia, Sierra Leone and Mali.

Only Yemen – at 147th position – and Libya – at 148th – fared worse.

In response, South Africa’s Department of Basic Education issued a press release dismissing the report as “not a credible or accurate reflection of the state of education in South Africa”.

Is the WEF ranking accurate?

There are two key problems  – both of which we touched on in a previous report that looked at the WEF’s research methodology.

Firstly, no standardised tests were conducted to assess the quality of maths and science education in the countries surveyed.

Secondly, the ranking was derived from an annual “Executive Opinion Survey” carried out by the WEF.  This opinion survey draws on interviews with various unidentified “business leaders”.

In this instance, around 50 South African “business leaders” were asked to assess the quality of maths and science education in the country and score it from poor to excellent. Individual responses were then arbitrarily weighted according to the sector each “business leader” worked in. The weighting was based on the “estimated contributions to a country’s GDP of each of the four main economic sectors: agriculture, manufacturing industry, non-manufacturing industry, and services”.

Martin Gustafsson, a researcher in the economics department at Stellenbosch University, told Africa Check that the report offered no valuable insight into the quality of maths and science education in South Africa.

“There is valuable data in the report. For things like business confidence it is useful. But you can’t apply opinions to things like education. It is like asking business experts what they think the HIV rate is,” he said.

The education rankings are not, in fact, an assessment of the quality of maths and science education in South Africa or any of the other 147 countries covered. Instead they are simply a reflection of the personal opinions of a small group of unidentified “business leaders” about a topic in which they are not expert.

Where is South Africa ranked?

In order to accurately rank an education system you need to do a lot more than ask unnamed “business leaders” for their opinions.

To compare educational performance across a number of different countries you need a standardised test and a representative sample of students to take the test in each country.

The most comprehensive and most current data on educational performance across a number of African countries was compiled by the Southern and Eastern Africa Consortium for Monitoring Educational Quality (SACMEQ). Countries represented in the consortium include Botswana, Kenya, Lesotho, Malawi, Mauritius, Mozambique, Namibia, Seychelles, South Africa, Swaziland, Tanzania, Uganda, Zambia and Zimbabwe.

SACMEQ conducted three major education policy research projects between 1995-1998, 1998-2004 and 2005-2010.  Data for the most recent research project was collected during the last quarter of 2007 from 61 396 grade six students and 8 026 grade six teachers in 2 779 schools.  Students were required to answer multiple-choice questions on reading, mathematics and health.

South Africa’s average student maths score placed it eighth out of the fifteen countries.  Mozambique, Uganda, Lesotho, Namibia, Malawi and Zambia all scored lower than South Africa in the SACMEQ test, yet they were all ranked higher in the WEF’s report.

Zambia, for instance, came in last in the SACMEQ test. But in the WEF report, it  was ranked in 76th position, higher than Lesotho (105), Malawi (113), Uganda (119), Namibia (128), Mozambique (137) and South Africa (148).

Major problems do exist

Of course this does not mean that all is well in South Africa. The performance of South Africa’s education system has been subject to severe criticism in recent years.

A 2012 study published by the University of Stellenbosch found that while 71% of children in grade six were functionally literate, only 58.6% could be considered functionally numerate.

The study noted that “at least a quarter of children are enrolled but have learnt so little in six years of formal full-time schooling that they have not even mastered functional literacy or numeracy”.

The basic education department’s own academic assessments revealed last year that just 3% of school pupils in grade nine had achieved more than 50% in mathematics.

Concerns have also been expressed about South Africa’s high dropout rate. For example, when the 2013 matric class started grade one in 2002, there were 1 261 827 pupils. By the time they sat their final exams, those numbers had more than halved to 562 112.

Conclusion – SA school system in trouble but WEF ranking gives little insight

South Africa’s basic education department is correct. The WEF’s education rankings are not an accurate reflection of the state of education in South Africa. At best they are a reflection of the opinions of around 50 “business leaders”.

An accurate ranking system would require a system of standardised tests conducted by a representative and accurately weighted sample of pupils across  a number of different countries.

There are still very real concerns about the state of education, literacy and numeracy in South Africa.

Gustafsson summed it up: “The bottom line is that test-based data suggests that indeed South Africa’s quality of education requires a lot of fixing, and is well below where it should be. Yet the catchy slogan that we are ‘at the bottom of the world’ is not supported by the evidence.”

Click here for the original article with click throughs

Researched by Kate Wilkinson, Edited by Julian Rademeyer

Additional reading:

Is South Africa’s education system the worst in Africa?

Why the matric pass rate is not a reliable benchmark of education quality

Sourced from Africa Check, a non-profit organisation which promotes accuracy in public debate and the media, to be found on Twitter at @AfricaCheck and at www.africacheck.org