Once a month, the J&J Professional Development Committee chooses an industry article to discuss with the greater J&J team. The topic can range from industry-spanning threads such as Open Access to specific matters such as the careful use of inclusive language in reviews. In this space, J&J employees—from new hires to executive leaders—can pick each other’s brains and discuss the evolving industry from many points of view. Here are some of the highlights from this December’s discussion of Open Reviewer Identities: Full Steam Ahead or Proceed with Caution?
Open Access has been the hot topic of almost everything scholarly publishing since its introduction to the industry through the Budapest Open Access Initiative in 2001. While the focus of the initiative lands primarily on societies, publishing houses, and authors to make their work more accessible to the greater scientific community, the allure of “openness” in research has stretched the meaning of the word.
Now “open” has taken on new meanings, as anything related to Open Access, transparency, or free content is given the label “open ___.” In December of last year, the professional development team at J&J spoke about one of the most crucial practices in scholarly publishing: peer review and the idea of open reviewer identities.
The premise is simple; the identities of reviewer(s) and/or author(s) are made available to one another pre-publication, or, as many put it, the feedback and comments are “signed” by the reviewer. In some cases, the reviewer(s) is also made public once the literature is published. The two problems that this peer review system is trying to alleviate are the lack of peer reviewers and integrity of reviews.
It’s no surprise to anyone reading that the number of peer reviewers is heavily outnumbered by the weight of papers yearning to be published. We know that scholars and reviewers are overworked, having their own papers to write, labs to manage, patients to care for, and classes to teach. In addition, little to no compensation is given for the massive responsibility that is peer review, which disincentivizes experts from the reviewing and pushes them towards authorship as they face the dreaded “publish or perish” ecosystem.
In our current system of keeping reviewers private, many authors suggest reviewers or ask for certain experts to refrain from being a reviewer. This has a number of poor outcomes, such as reviewer cliques, in which authors review one another’s works with little to no constructive feedback in an attempt to release their papers as soon as possible. This also leaves a large number of underrepresented reviewers who are outside of the mainstream networks of the global north or lack the name recognition of older experts. These practices can create a staleness in peer review; a few good reviewers are burdened with all of the work, whereas less constructive reviewers can flourish behind hidden identities in the name of privacy.
So how do open reviewer identities help solve these peer review predicaments? The most obvious to our colleagues at J&J was the potential for…
Recognition:
“…it could allow editors to form a more holistic picture of various scholars. Histories as peer reviewers, which could aid them in the screening process and give them a better sense of which scholars are more likely to offer constructive and in-depth reviews”—Christopher Hill
“Another pro could be if you see a Reviewer’s record- like completed 5 reviews in the last year, you know that person is current on up-to-date science in their field. With how fast science moves and evolves, I think this is a pro for people to know in the community.”—Jennifer MacBain-Stephens
Giving scholars recognition for their work is critical; not only will it naturally highlight the best reviewers available, it could also drive more experts to give their reviews, which led the conversation towards…
Training and Networking:
“When I was in school, grad advisors encouraged their grad students to review articles they or other colleagues had authored. This helped students learn the process and establish themselves as valuable critical resources”—Kathleen Hines
“Open peer review could enable reviewers that are seeking to build their portfolio and CV as well as build their network with other researchers and reviewers.”—Andrew Torres
Reviewing papers shouldn’t be reserved for only a limited number of experts, as being critical of work and helping others in writing can improve your own writing skills and jumpstart a young academic’s networking process. Speaking of the underutilized academics of the world…
Diversity and Inclusion:
“Bias and the impact on those more vulnerable were listed under the risks but having open identities in peer review can eventually turn these into “positives” by revealing biases/underrepresentation and allowing readers, authors, etc. to hold journals/societies/etc. accountable for who they are selecting to review.”—SJ Griffin
“I love the idea of exposing underrepresentation in academic fields and also working hard to give women and people of color in all science fields more of a platform and opportunity…”—Jennifer MacBain-Stephens
We will never know the extent of exclusion in publishing if we do not keep track of who is and isn’t writing and reviewing. Bringing identities to light can help us create a more diverse reviewer pool and give underrepresented academics their launching point.
One Size Does Not Fit All
While many pros were celebrated, the team knew that one solution could not remedy all ills, and further concerns were discussed, devil’s advocate was played, and questions of implementation were raised.
“…one con that I’m not sure was considered is that of size. Though it seems quite large, the academic world is still fairly small. In many circumstances, publications ask authors to suggest reviewers and, for a particularly abstruse topic (a niche topic if you will), authors often choose those with specialties minutely similar to theirs. Anonymity is, thus, not always possible…”
“Often, authors want to choose reviewers (if they can) who will positively review their work. Anonymity can help mitigate this concern and potential bias that may come with the selections (although it’s difficult to lessen bias either direction).”—Kathleen Hines
While the potential to grow reviewer pools is a perceived advantage of open reviewer identities Kathleen does correctly point out that some subjects are so niche that there are very few experts to go around, which makes suggested reviewers (by the author) the only option. It also doesn’t totally alleviate the clique problem mentioned above, as institutions, publishers, or societies would have to get involved to identify, report, and remove certain reviewers.
“Though it seems quite large, the academic world is still fairly small. In many circumstances, publications ask authors to suggest reviewers and, for a particularly abstruse topic…”
Another interesting and rarely mentioned point was given by one of our editorial staff in this hypothetical, “I also think of safety. In some journals, there may be very emotional responses, (even aggressive/inappropriate e-mails, rebuttals, etc,) and if one of those people knew that I didn’t value their work, maybe I’d look over my shoulder a bit?”
While the responsibility falls on authors to not take reviews as insults and reviewers to keep their comments purely constructive, there would be questions of reputation and job security if an influential author or reviewer was very unhappy with their counterpart.
The final questions raised pertained to the applications of open identity in the peer review process.
“I do think it’s important that both authors and reviewers’ consent.” And a great scenario was given as an example: “one author of an article actively asks not to select certain reviewers. The researcher sent a note, stating that the article written was, to some degree, a rebuttal to another article whose points were good, but not always correct, according to the researcher. Thus, open identities could prove difficult in this scenario.” Kathleen Hines’ hypothetical is important because science is constantly proving other science wrong or partially correct. There are times to exclude reviewers in order to get the most ethical feedback.
Adding to the point, Christopher Hill speaks to the creation and application of policy, stating, “I also found myself wondering whether there might be potential pros and cons to allowing open identities on a case-by-case basis (i.e. allowing peer reviewers to choose whether their identity is revealed to contributors) or adopting a consistent policy regarding reviewer identities. Obviously, consent is paramount in any situation that could potentially compromise someone’s privacy–but in my experience, consistency is a key part of making any new policy effective.”
“…consistency is a key part of making any new policy effective.”
Open reviewer identity initiatives seem to create as many questions as they try to solve and definitely aren’t a one-size-fits-all solution to the challenges facing the peer review backlog. Yet the J&J Professional Development Committee sees the advantages such an initiative could play, especially for underrepresented scholars who need a more accessible platform to reach the science mainstream. They also agree with the usage of this peer review system in order to create a greater number of experienced critics and writers for the future.
Follow us on social media, @jjeditorial on Twitter and follow J&J Editorial on LinkedIn and let us know what you think about a potential move towards open identities and what other systems could do to help build on the integrity and speed of peer review. Stay tuned for another edition of J&J Professional Development Knowledge Exchange!