Democracy has been called the least worst system of government. Peer review is the least worst system for assessing the merit of scientific work.
Peer review is the written evaluation of a paper by other experts in the field. Though this sounds like assessment by equals, the power imbalance created by the roles of reviewer and reviewed distorts the relationship and affects the tone of the review. Reviews can be patronizing, demanding and unkind.
It is painful to read harshly worded criticism of work that has taken a team hundreds or thousands of hours and been submitted hopefully and in good faith. From our experience, we know that reviews can be accurate, robust and make every scientific point while using language and tone that is helpful and supportive.
Supportive review
We are a team of editors of an open-access Canadian kidney journal, the Canadian Journal of Kidney Health and Disease. When we founded our journal in 2014, supportive review was the first of our guiding principles. Since then, we have written supportively as editors, selected reviewers who write supportively and participated in training the next generation of Canadian kidney scientists to conduct reviews that are complete, rigorous and kind.
Supported by a larger group of like-minded people from multiple disciplines, we recently published an editorial outlining these principles. A dozen other kidney journals expressed their support for the idea, with Nature Reviews Nephrology, NDT and Pediatric Nephrology publishing co-ordinated editorials recommitting to principles of constructive criticism.
The long process of research
Scientific papers condense a large amount of work into a structured format, usually no longer than four to eight times the length of this article. The work of a paper starts with an idea that may be developed by the team for a year or more before it crystallizes into an application for funding, which may go through rounds of revisions.
Once funded, people and budgets are assigned to the project and the work proceeds. The work can involve the time of multiple team members for months and even years.
When the work is complete, they write a paper, detailing what they did, how and why, what they found and what they think it means. This paper itself is often the product of hundreds of hours of work, with multiple authors contributing their specific expertise and working on the messaging of the whole.
The journal receives the manuscript and assigns an editor, who assigns peer reviewers. Peer reviewers are other scientists working on similar topics. They must be totally unconnected with the people writing the paper. With notable exceptions, most journals employ single-masked peer review: the reviewer sees the authorship of the paper but the authors of the paper will not see who wrote the review.
Peer reviewers are not paid or rewarded for their review of the manuscript — they take it on as part of the work of academic life. Essentially, it is an unrewarded activity performed by people who are themselves authors. It varies by discipline, but in biomedicine, they may spend three to six hours on a review.
Harsh reviews
How does this altruistic activity, undertaken by a reviewer who is very familiar with the author role, lead to such pain and frustration for other authors?
We think that scientists sometimes confuse harshness with intellectual rigour and that a reviewer’s experience of harshness in reviews of their own work, amplified by the power imbalance between reviewer and reviewed, leads to perpetuation of harsh and unhelpful review. Other reviewers and editors avoid these pitfalls entirely.
“It looks to me like one of your first attempts at scientific publishing, and I can understand that you are also writing in a non-native language” wrote one anonymous reviewer to a mid-career woman scientist with 13 first-author peer-reviewed publications. “I just want to give up today,” she wrote.
But she won’t. Scientists are prepared to receive this kind of feedback and be hurt over and over in the name of science. As editors, we believe there is a better way — that feedback should be rigorous, but will be more readily incorporated if kindly given, to the advancement of science.
These are not new ideas. In 2006, Prof. Mohan Dutta suggested 10 commandments for reviewers, all of which focus on the collaborative nature of relationship between reviewer and reviewed. Advice for reviewers often includes a recommendation to write constructively, though sometimes this is phrased as something like “write constructively, and then turn to criticism,” as if those are mutually exclusive.
We can take this principal further and — thanks to our community of reviewers in kidney medicine — we and other kidney journals make a commitment to kindness in review. Dutta’s 10th commandment is “do unto others as you would have them do unto you.” Every branch of science would be improved by implementing this idea.
Catherine Clase has received consultation, advisory board membership or research funding from the Ontario Ministry of Health, Sanofi, Pfizer, Leo Pharma, Astellas, Janssen, Amgen, Boehringer-Ingelheim and Baxter. In 2018 she co-chaired a KDIGO potassium controversies conference sponsored at arm's length by Fresenius Medical Care, AstraZeneca, Vifor Fresenius Medical Care, Relypsa, Bayer HealthCare and Boehringer Ingelheim. Catherine is a member of the Cloth Mask Knowledge Exchange, a research and knowledge translation group that includes industry stakeholders. Industry stakeholders contribute to the Cloth Mask Knowledge Exchange by contributing to grant funding, and through in-kind contributions of time and expertise. Industry stakeholders make masks and distribute polypropylene and other fabrics. She is a member of McMaster's Centre of Excellence in Protective Equipment and Materials, and editor-in-chief of clothmasks.org. Catherine Clase receives funding from CIHR, and is a member of the Green Party, the American Society of Nephrology, the Canadian Society of Nephrology, the American Association of Textile Chemists and Colorists and ASTM International.
Josee Bouchard receives funding from CIHR, Kidney Foundation of Canada and CDTRP. She is affiliated with the Hopital Sacré-Coeur de Montréal, Université de Montréal. She is a member of the Canadian Society of Nephrology and American Society of Nephrology.
Manish M Sood receives funding from CIHR, the Kidney Foundation of Canada, the Canadian Medical Association and the Heart and stroke foundation. He is supported by the Jindal Research Chair. He has received speaker fees from Astrazeneca. He is affiliated with the Ottawa Hospital Research Institute, uOttawa and the Ottawa Hospital.
Rachel Holden receives research funding from CIHR, the South Eastern Ontario Medical Organization, and the Translational Institute of Medicine at Queen's University, Kingston, Ontario. She has received investigator initiated research funding from OPKO Renal. She has received consultation or advisory board funding from Sanofi, Bayer and Oksuka. She is a member of the Canadian Society of Nephrology.
Sunny Hartwig does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.
This article was originally published on The Conversation. Read the original article.
