Umalusi approves the 2017 national examinations results

29 December 2017

Umalusi, the Council for Quality Assurance in General and Further Education and Training, has once again fulfilled its legislative mandate in relation to the quality assurance of the management and administration of exit point examinations in the following qualifications per assessment body:

Independent Examinations Board (IEB) – NSC AND GETC:ABET

South African Comprehensive Assessment Institute (SACAI) – NSC

Benchmark Assessment Agency (BAA) – GETC: ABET

Department of Basic Education (DBE) – NSC

Department of Higher Education And Training (DHET) – GETC:ABET; NATED N2-N3 Programmes; NC(V)

Umalusi conducted the external moderation of question papers and a sample of site- based assessment (SBA) marks across all assessment bodies. The Council also monitored the conduct and marking of exams as well as selection of markers in all the nine Provincial Departments of Education (PDEs) including all private assessment bodies.

This year the South African Comprehensive Institute (SACAI), which is an independent assessment body provisionally accredited by Umalusi to administer the National Senior Certificate examinations, administered its NSC exams for the fourth time. Another new independent assessment body, Benchmark Assessment Agency (BAA), which is also in a process of obtaining accreditation from Umalusi, ran a pilot of its third GETC examinations in November. Benchmark’s results were also subjected to Umalusi’s standardisation process.

The class of 2017: The class of 2017 was the fourth Grade 12 cohort to write final examinations under the Curriculum Assessment Policy Statements (CAPS). The learner performance of the cohort of 2017 has moved closer to the average historical performance profile. It is pleasing to see that there is a steady upward trend in the marks of subjects such as Mathematics, Mathematical Literacy, Physical Science and Life Science. However, some of the subjects that do not fall under the category of gateway subjects such as Visual Arts, Dramatic Arts and Agricultural Science need to be given more attention and be afforded the status that they deserve in terms of providing them with the necessary resources, for example, Learning and Teaching Support Material.

There were 104 001(16%) progressed learners in the Class of 2017. This is slightly down from the 108 742 of 2016. Umalusi has taken note of the evidence-based reports on interventions and improvement strategies implemented by both national and provincial education authorities for learners in general and for progressed learners in particular and the positive impact of these on the overall assessment system as well as on teaching and learning. It is important to note that while the class of 2017 is the 10th cohort of learners to sit for the NSC, it is only the 4th cohort to write the CAPS aligned NSC Examination.

So while the 2017 cohort has benefited from the maturity of the NSC system over the past 10 years, the CAPS aligned system introduced in 2014 is slowly beginning to strengthen after only four years. As a result, Umalusi has put in place rigorous and robust procedures that provide assurance that all learners receive appropriate recognition for their performance in line with agreed national standards so that no learner should feel that s/he was born in the wrong year.

Improved quality of question papers: The quality of question papers is a very important aspect of the integrity and credibility of the examination. It is therefore pleasing to see a marked improvement in the overall quality of question papers across the assessment bodies as more papers are approved after first or second submission to external moderators.

Irregularities: Umalusi conducts not only the process of standardisation, which is required for national examinations, but it also, through rigorous procedures, assures the quality and integrity of the entire examination process.

Gaining the approval of Umalusi Council for the release of the results is determined by the examinations’ level of compliance with policies, directives and guidelines issued by both Umalusi and each of the assessment bodies. Before such an approval is granted, Umalusi Council has to satisfy itself that no systemic irregularities have occurred to undermine the integrity and the credibility of the examination process. To this end, Umalusi requires that each assessment body provides a report on irregularities.

Umalusi is pleased to report that based on the evidence presented to Umalusi by the Department of Basic Education, the 2017 National Senior Certificate examinations were largely incident-free with a few minor disruptions experienced in some parts of the country. This attests to the success of the rigid measures put in place by DBE to ensure fool-proof and incident-free examinations.

Umalusi commends the Department of Basic Education and Provincial Departments of Education for working tirelessly to achieve this goal. While irregularities occur in all examinations/assessments, the South African public can rest assured that the examination system in South Africa has mechanisms to manage irregularities, to avoid lapses in the credibility of examinations. However whether an irregularity becomes a threat to the system depends on the nature and extent of the irregularity.

Standardisation of results: In order to mitigate the sources of variability that impact on learner performance from one year to another. (eg. cognitive demand and difficulty levels of questions, marking, curriculum changes, interventions etc.) Standardisation is employed as an important ‘quality assurance process used the world-over.’

Standardisation aims (in the main) to achieve equivalence of the standard of the examination across years, subjects and assessment bodies and to deliver a relatively constant product to the market: universities, colleges and employers.

We can expect that when standards of examinations are equivalent certain statistical mark distributions should correspond. It is this principle of correspondence that forms the basis for comparing distributions with norms/historical averages that are developed over 4-5 years. This comparison includes qualitative data, medians, means, pass/failure and distinction rates and pairs analyses which play a valuable role in the absence of historical data.

The adjustments (decided by the Assessment Standards Committee of Umalusi) consistently follow certain guiding principles. The ASC consists of academics with extensive experience and expertise in statistical moderation, statistics, assessment, curriculum and education.

While the final stages of the process (viz. standardisation) may seem highly statistical, this process of adjustment is the culmination of an eighteen month process of receiving and reflecting on qualitative and quantitative inputs. This starts with setting of papers, then moderation, writing of exams, marking of exams, verification and only then finally adjustment of mark distributions.

Given the complex nature of the stages and processes followed, it can lead to misinterpretations especially if one observes any one of the stages in isolation or just the final one. The whole process of standardization is the basis for Umalusi to declare exams fair, valid and credible; thereby building public trust and confidence.

STANDARDISATION DECISIONS AND APPROVAL STATEMENTS – NSC EXAMINATIONS

INDEPENDENT EXAMINATIONS BOARD (IEB)

Having studied all the evidence at hand on the management and conduct of the National Senior Certificate examinations administered by the Independent Examinations Board (IEB), Umalusi is satisfied that, apart from a few instances of technical irregularities reported, nothing has compromised the integrity or credibility of the November 2016 NSC examinations as a whole. We are satisfied that the examinations were fair, valid and credible.

We commend the IEB for running a successful and credible examinations process. Accordingly, we hereby approve the release of the results of the National Senior Certificate Examinations administered by the Independent Examinations Board.

SOUTH AFRICAN COMPREHENSIVE ASSESSMENT INSTITUTE (SACAI)

Having studied all the evidence presented and having noted that there were no systemic irregularities reported, that might have compromised the credibility and integrity of the November 2017 NSC examinations administered by the South African Comprehensive Assessment Institute, the Executive Committee of Council approves the release of SACAI results for the November 2017 NSC examinations.

The Executive Committee of Council commends the South African Comprehensive Assessment Institute for conducting a successful and credible examination.

DEPARTMENT OF BASIC EDUCATION (DBE)

Having studied all the evidence presented and having noted that there were no systemic irregularities reported, that might have compromised the credibility and integrity of the November 2017 NSC examinations administered by the Department of Basic Education, the Executive Committee of Council approves the release of the DBE results for the November 2017 NSC examinations with the proviso that:

- the results of the candidates implicated in the examination irregularities be blocked, and be investigated.

- the DBE reports the outcome of the investigations to Umalusi.

The Executive Committee of Council commends the Department of Basic Education for conducting a successful and credible examination.

STANDARDISATION DECISIONS AND APPROVAL STATEMENTS – NATED and NC(V) EXAMINATIONS

NATED N2 – N3

BUSINESS LANGUAGES N3

Having studied all the evidence presented and having noted that apart from the alleged leakage of five question papers, as well as challenges of non-submission of ICASS evidence by some centres, there were no systemic irregularities reported that might have compromised the credibility and integrity of the November 2017 Report 190/191 Engineering Studies N2-N3 examinations administered by the Department of Higher Education and Training. Therefore, the Executive Committee of Council approves the release of the DHET results of the November 2017 Report 190/191 Engineering Studies N2-N3 examinations based on the following provisos:

The DHET is required to block the results of candidates/ centres and subjects implicated in irregularities and the results of the implicated centres that did not submit evidence of ICASS pending the outcome of further DHET investigations.

That said, Umalusi remains concerned about the need to improve the management of assessments and examinations conducted at private colleges. The Executive Committee of Council commends the DHET for conducting a successful and credible examination.

NC (V) LEVEL 2-4

Having studied all the evidence presented and having noted that apart from the challenges of non-submission of ICASS evidence by some centres, there were no systemic irregularities reported that might have compromised the credibility and integrity of the November 2017 NC (V) Level 2-4 examinations administered by the Department of Higher Education and Training. Therefore, the Executive Committee of Council approves the release of the DHET results of the November 2017 NC(V) Level 2-4 examinations based on the following provisos:

Pending the outcome of further DHET investigations, the DHET is required to:

Block the results of candidates implicated in irregularities and

Block the subject results of the centres that did not submit evidence of ICASS. The Executive Committee of Council commends the DHET for conducting a successful and credible examination.