Reviews and Evaluations (RE)

RIISA Directorate

About Reviews and Evaluations (RE)

The Reviews and Evaluations (RE) directorate enhances the contribution of the NRF as a strategic partner in knowledge production, human capital development and infrastructure provision.

The directorate ensures the operation of a world–class review and evaluation system that conforms to the principles of confidentiality, access to information, transparency, ethical considerations and commitment to excellence. RE ensures that the NRF is able to carry out its mandate by:

Review of research proposals submitted for various funding instruments.
Evaluation and rating of researchers according to the NRF Rating System.
Evaluation of programmes operated by the NRF, as well as those commissioned by external agencies.
Evaluation of performance of research-funding and/or research-performing institutions.

Funding Reviews

The purpose of the funding reviews is:

To assess research proposals fairly and based on merit.
To review proposals submitted by Higher Education Institutions (HEIs), National Research Facilities (NFs), Science Councils and other NRF accredited research entities against pre-defined review criteria aligned to the funding instrument.
To provide a quantitative (scoring) as well as detailed qualitative written assessment of the proposal.
To provide a recommendation on whether or not the proposed applicant should be funded.
To provide comprehensive scientific feedback to the applicants.

One-stage and 2-stage reviews are used selectively. In the case of one stage reviews, the panel is constituted of specialists qualified to actually review the individual proposals. In the case of two-stage reviews, the proposals are reviewed remotely by specialists who submit review reports, which are the primary inputs for the panel to consider in assessing applications.

These reviews are conducted by national and international subject experts of high academic and professional credibility and who are recognised internationally for their research contributions. Remote reviewers review proposals and the candidate against set objectives, providing expert opinion on scientific merit, feasibility, capability, capacity, etc.

Review panel meetings include pre-meeting briefing sessions and success of the work of the panel is, among others, dependent on the integrity, expertise and experience of panel members for the assigned roles. Panels are generally constituted on directed and thematic areas. There is always effort to diversify a panel in terms race, gender and institutional representation. A panel is generally comprised of the reviewers, an external chairperson, an independent assessor, RE providing the secretariat and a scribe.

Rating

The NRF rating system is a key driver in its objective to build a globally competitive science system in South Africa. It is a voluntary tool for benchmarking the quality of researchers in South Africa against the best in the world. NRF ratings are allocated based on a researcher’s recent research outputs and impact as perceived by national and international peer reviewers.

It encourages researchers to publish high quality outputs in high impact journals or through other vehicles. Rated researchers as supervisors will impart cutting-edge skills to the next generation of researchers. The NRF Rating System awards the following rating categories:

A – Researchers who are unequivocally recognised by their peers as leading international scholars in their field for the high quality and impact of their recent research outputs.

B – Researchers who enjoy considerable international recognition by their peers for the high quality and impact of their recent research outputs.

C – Established researchers with a sustained recent record of productivity in the field, who are recognised by their peers as having:

  • Produced a body of quality work, the core of which has coherence and attests to ongoing engagement with the field; and
  • Demonstrated the ability to conceptualise problems and apply research methods to investigating them.
  •  

P – Young researchers (normally younger than 35 years of age), who have held the doctorate or equivalent qualification for less than five years at the time of application and who, on the basis of exceptional potential demonstrated in their published doctoral work and/or their research outputs in their early post-doctoral careers, are considered likely to become future international leaders in their field.

Y – Young researchers (40 years or younger), who have held the doctorate or equivalent qualification for less than five years at the time of application, and who are recognised as having the potential to establish themselves as researchers within a five-year period after evaluation, based on their performance and productivity of quality research outputs during their doctoral studies and/or early post-doctoral careers..

Programme and Institution Evaluations

Programmes and institutions are evaluated through a comprehensive evaluation process. This is typically done on a cyclic basis, typically every 5 years. Generally, the objective of conducting such evaluations is to assess the performance of the instrument/institution, against its mandate, objectives and strategy, so as to decide its fate going forward. In practice, any evaluation (retrospective) includes a degree of review (prospective). The process will typically include the following, tailored for each case:

(i) A trigger letter/memo which formalises the go-ahead;
(ii) An implementation plan, which details the project, including role-players;
(iii) In certain cases, a reference group, which provides oversight and advice;
(iv) A panel, who conduct the actual evaluation; and
(v) Terms of Reference for the reference group and panel.

The process includes:

(i) Engagements with the reference group and with the panel;
(ii) Reading material for the panel;
(iii) Stakeholder survey;
(iv) Interviews/meetings of the panel, with stakeholders;
(v) Site visits; and International benchmarking.

The entire project can take anything from 2 months to 8 months.

Management of Projects

RIISA has adopted project management as a method of managing its core activities. The processing of submissions for funding from the NRF, as well as various evaluations, are packaged into projects with clear delivery milestones scheduled through the year. The delivery of the projects, against these milestones is monitored, intervened as may be necessary and reported on, to the RIISA Executive, on a monthly basis.