Once a month, the J&J Professional Development Committee chooses an industry article to discuss with the greater J&J team. The topic can range from industry-spanning threads such as Open Access to specific matters such as the careful use of inclusive language in review. In this space, J&J employees—from new hires to executive leaders—can pick each other’s brains and discuss the evolving industry from many points of view. Here are some of the highlights from this December’s discussion of What’s Wrong with Paying for Peer Review?
Academic professionals have always tried to stay an arm’s length away from a purely capitalistic approach when it comes to how they work. Many feel their motives should be entirely altruistic, advancing science for the greater good. The work, though, must be financed, services paid for, and profit made for the industry to stay afloat (at least in the west). The delicate balance between the altruistic nature of the global scientific community and the economic forces of the world leads to difficult ethical dilemmas and one of the most unique incentive programs in economics.
Although there are monetary incentives for much of the work done in the academic sector, there are also numerous non-monetary rewards that fit better into the altruistic environment of academia. This system of pats on the back has been community-created as authors, researchers, and librarians recognize one another’s hard work and increase each other’s reputations. In the end, this system results in a better job and more reputable individuals getting more recognition.
The academic system is more or less set up like so: Do great work, gain recognition, be paid for said work (by way of a better job/position). That middle step is a very important distinction because it is controlled by the community and allows it to make some rules of its own. Today, the J&J Professional Development team will be discussing a different one of these new “rules” in the form of paid review.
As it stands, peer-review is mostly on a volunteer basis, free, and unrecognized. (Speaking of peer review recognition, check out our last piece on reviewer identities to get a greater grasp on that topic!) While there are many suggestions to alleviate the issue, we are going to focus on paid peer review. The idea: a monetary incentive will increase the number of available reviewers, help the financial situation of struggling researchers, and add a new level of healthy criticism to existing reviews.
The J&J Professional Development team began by asking themselves about the article at hand first. The great piece by Tim Vines and Alison Mudditt carefully points out many reasons the paid review system would introduce more issues to the peer-review process than answers. The Prof Dev team took their time outlining which issues raised the most concern and detailing many of the possible negative effects.
“I think the ethical concerns are the most persuasive for sure,” said Kaitlin Butler. “The author even makes the point that reviewers may submit short reviews or reviews for articles out of their scope for the potential profit, for a process that’s generally known to be a service to the academic community.”
This ethical concern was shared by her colleague Ivy Sudweeks. “I most strongly resonate with the altruism point. While not the most persuasive argument, paying for reviews seems like a breakdown of the major tenant of science: the open flow of information and ideas. The minute you raise costs in any way, certain researchers are unable to submit their papers because of the cost. The idea of ‘pay for speedy reviews’ already makes me uncomfortable, but then paying reviewers for the speedy review would be even more limiting.”
Ivy began to hint at the next largest issue, the many financial strains and exploits a paid review system could introduce. “I think I was most persuaded by the subjectivity in determining fee.” Jacob Holstein commented, “In addition to the concerns stated in the article, it occurs to me that not all journals have access to the same resources, and many use different financial structures, so it would be difficult to standardize a fee across multiple journals. This would, I think, lead to a concentration of publishing to a few journals with the financial capacity to pay reviewers at a rate the reviewers would probably want.”
This opened the floodgates to a litany of financial issues and inequities that authors are already facing: high APC costs in OA, existing costs to publish, and the financial disparity between the global North and South.
“I thought it was a strong way to start this piece, outlining the costs. It was a reality check to see numbers so high ($6000 for one review???), so reading the rest of the article felt like ‘yeah, I agree with these points too, but it doesn’t even matter because those costs!‘” exclaimed Emily Hammond as she reviewed the possible costs to journals/authors.
“I think it would be highly unfair if authors whose research is just as or more valuable than a wealthier author were phased out because they couldn’t afford higher prices,” said Kaitlyn Butler, echoing the many inequity issues introduced by APCs since the beginning of OA.
Adding to this, Ashley Owens points out, “I liked the comment on how those high payments to reviewers would also lead publishers to raise prices, which is a concern because that just leads to the materials published being less accessible. I think that’s a really valid point.”
The financial woes did seem like the most relatable topic, as although paid review might keep some reviewers above water, the real financial burden would fall to authors once again. Large publishing houses could easily absorb the costs of reviews but would leverage their market dominance to increase costs, and smaller society publishers would have to transfer that burden to the author to maintain their financial health.
Once the theoretical issues were hashed out, the Professional Development Team was asked how a paid reviewer system might look in practice. How would fees be determined, how could reviews and reviewers be evaluated, and what steps might be taken by both good- and bad-faith players?
“To me, the most basic argument in favor of paying reviewers is anti-exploitation. Expecting scholars to take on more work and responsibility for the direct financial benefit of academic journals is difficult to stomach IMO,” mentions Jacob Holstein. “I do think that paying reviewers would be good, even if it isn’t feasible to do so. Thus, I don’t think we can or should discount anti-exploitation arguments out-of-hand.”
Adding to Jacob’s point, Allegra Torres mentions, “I think there is also the question of who can afford to do high-quality work for free? Are reviewers really able to do their best and most thoughtful work when they have to slot unpaid work into a busy schedule?”
When asked about exactly how reviewer fees were going to be handled, Craig Lincoln points out the many possible routes in compensation.
“It is a tricky subject, I do think offering flat fees will see bad actors exploit the system, which could compromise integrity. Could be credits to submit without publication charges or free open access, as compensation, but that also has issues.”
While it is true that some bad actors would take advantage of the new situation, Ivy tunes in again to make a very pertinent observation.
“This might be a bit of a reach. But just because people do work for free doesn’t necessarily ensure that they’ll give an unbiased, quality review. In academia, there is so much prestige and posturing about being asked to review anyway. People might accept a review that they’re not exactly qualified to do just to say they did it. Paying reviewers might make that problem worse, but it’s not like it would invent the problem.”
This point put many of the ethical concerns of paid review to rest, as in any system, there will be bad actors, but in a paid review system, the journal, society, or publisher would have some control over the quality of the review. So finally, a system for review evaluation was discussed in brief before the hour-long meeting was up.
The favorite system of the group was one we’ve seen in a few OA journals and preprint servers already: a system of published reviews that allows the review to be viewed by any reader and lets the reader determine the value of the review and paper.
“I feel something like an open peer review system, where reviews are published, and some sort of karma/review the reviewer situation would be better for judging reviewer quality,” Craig noted.
“That goes back to the point made earlier about how to make a rubric that would evaluate the quality of a review? That would then determine what kind of/how much compensation they receive,” asked Amelia Bryant, which led to the question of who is responsible for making said review and handing the check to reviewers.
This was quickly answered by Craig, and the opinion went unchallenged for the rest of the discussion. “I feel it needs to be the journal, otherwise the institution could create a conflict of interest in funding reviewers.”
The discussion ended at a crossroads much like our industry is facing today regarding paid review. While our team fully supports a fair wage for labor and have firsthand experience with tracking down reviewers, the current system is like an old highway in desperate need of a revamp. Adding new avenues piecemeal will only cause greater issues as authors, journals, reviewers, and publishers scramble to adapt to new traffic patterns. As Craig says, “The real solution is we probably need to rebuild these things from the ground up. What does that look like? I have no clue, but that is probably the way forward.”
Well, we may have a little more of a clue as COVID-19 brings about an interest in scholarly articles and review from the general public. Change does appear to be on the horizon as the old highway struggles to keep up with the increased flow of articles and interest. We’ve shared our ideas, hesitations, and eagerness to see change, and we’d love to hear yours! Follow us on social media, @jjeditorial on Twitter and J&J Editorial on LinkedIn. Stay tuned for another edition of J&J Professional Development Knowledge Exchange!