Each paper under consideration for publication in PS is usually reviewed by at least two external reviewers. In certain cases, only one reviewer may be invited (e.g., for short Comments or for revisions). The usual turn-around time to provide a review is 3 weeks, though individual invitations to review may deviate from this general guideline.

The default setting in the journal management system where Pre-PPRs are performed is “double-anonymous”: neither authors know the identities of the reviewers nor do the reviewers know the identities of the authors. However, authors may choose to disclose their identities. Similarly, reviewers in the Pre-PPR phase also have choices available to them. Specifically, they can choose to (a) disclose their names confidentially to the authors, (b) have their name appear on the published paper as a reviewer, (c) have their review published along the paper (if accepted), and (d) have their name attached to the published Pre-PPR. These choices are independent of each other, and reviewers can indicate their choices for each review round. In certain cases, an editorial assistant gets in touch with the reviewers to finalize their choices once the paper is accepted. The journal encourages open peer-review, but each reviewer always has the choice of which information will be openly shared or not.

Apart from making the three choices outlined above, all Pre-PPR reviewers are asked to indicate whether (a) someone has helped them with their review, (b) they would be willing to re-review the paper (if a revision is invited), (c) they suspected plagiarism, (d) ethical concerns may be an issue, and (e) any actual or perceived conflicts of interests may bias the review. Reviewers may elaborate on these points in confidential comments to the handling editor.

Of course, reviewers also provide comments to the authors which should be concise, firm, polite, helpful, and written in a constructive spirit. Next, the paper can be rated on a scale from 0 (worst possible paper) to 100 (best possible, perfect paper). Subsequently, reviewers are asked on rating scales to judge the level of contribution, language, readability, novelty, importance, impact, utility, and readership reach. In addition to these, each category and type of paper (see Paper Formats) comes with a slightly different set of specific questions (e.g., for empirical papers: questions about methodology and transparency, amongst others). For empirical papers with associated supplements, reviewers are asked to also review these additional files. Lastly, reviewers give an overall recommendation along six predefined choices (no recommendation / cannot assess – reject – major revision – moderate revision – minor revision – accept).

At the launch of the journal, the feature of Post-PPR is not installed yet, but we are intending to roll out this feature. This means that each paper published in PS may be “reviewed” even after publication. Such Post-PPR would always be openly available (perhaps with a DOI of its own), linked to the paper, usually non-anonymous (there may be the option to anonymize the reviewer’s identity in the Post-PPR comment, but the handling editor will need to verify the identity of the reviewer), and moderated by the handling editor of the paper or another editorial team member.