How does one measure research output? Research output is often measured using quantitative methods such as citation counts, the h-index, and journal impact factors. In South Africa, research output can be measured and rewarded using the NRF rating system.
When did it all start?
The NRF was established in 1999. From 1984 to 2001, the evaluation and rating system was applied only to scientists in the natural sciences, engineering and technology, but the NRF Board approved the extension of this system to researchers in the social sciences and humanities in 2002 (Pouris, 2007).
What is the NRF rating system all about?
One of the main activities of the NRF is to award ratings to South African researchers (Breetzke, 2020). The rating application process involves the submission by researchers to the NRF of research outputs generated over the past eight years. This includes peer-reviewed publications, books, conference presentations, patents, policy or technical reports, and publications in the public press (Breetzke, 2020). It should be noted that ratings are not based solely on the volume of outputs or citation rates but rather involves the peer-review of applicants by several nominated and appointed local and international reviewers (Breetzke, 2020). NRF ratings are allocated based on research outputs and the impact as perceived by international peer reviewers. The rating system encourages researchers to publish in high impact journals/outlets (Breetzke, 2020).
How does it work?
When a researcher applies for a NRF rating, a panel considers a number of reviewers’ reports on the applicant’s research standing and output during the eight years prior to the review, and thereafter assigns a rating to the applicant (Breetzke, 2020). Where the panel cannot reach consensus, the application and the reviewers’ reports are referred to a committee for further consideration and final decision (Breetzke, 2020). Applicants who are dissatisfied with the outcome of a rating decision are able to have the matter reviewed (National Research Foundation, 2023).
What are the categories of ratings?
NRF ratings are awarded according to the following categories:
• A – Leading international researchers
• B – Internationally acclaimed researchers
• C – Established researchers
• P – Prestigious awards
• Y – Promising young researchers
Subjectivity and bias
The NRF rating system has been the subject of much debate. The NRF rating system works through reviewers chosen by the person being rated (Callaghan, 2018). Callaghan (2018) states that those listing their lifelong friends as reviewers of their work can achieve extraordinarily high rankings. Furthermore, the researcher applying for rating is not anonymous, so one is exposed to bias that is not experienced in the blind peer review method (Callaghan, 2018).
Pros of the NRF rating system according to Breetzke (2020)
Firstly, one can attribute the vast growth in the number of research publications produced by South African universities to the rating system and its associated incentives. Secondly, the prestige of being acknowledged by one’s peers as being a leading international scholar in a respective discipline cannot be under estimated. Thirdly, a number of universities and research-performing organisations in the country use rating status as one of the criteria for purposes related to personal promotion, resource allocation, ‘performance’ awards, and employment retention. Finally, and perhaps the most important, the NRF provides funding to rated researchers who hold a valid NRF rating.
Where to go for more information about the NRF ratings, etc.?
UKZN Research Office http://research.ukzn.ac.za/Homepage.aspx
National Research Foundation https://www.nrf.ac.za/
Breetzke, G. D., & Hedding, D. W. (2020). The changing and challenging research landscape in South Africa. Studies in Higher Education, 45(11), 2245-2259. doi:10.1080/03075079.2019.1602758
Callaghan, C. (2018). A review of South Africa's National Research Foundation's ratings methodology from a social science perspective. South African Journal of Science, 114(3-4), 1-7. doi:10.17159/sajs.2018/20170344
National Research Foundation. (2023). NRF. [Online] Available: https://www.nrf.ac.za/
Pouris, A. (2007). The National Research Foundation's rating system: why scientists let their ratings lapse. South African Journal of Science, 103(11/12), 439-488.