It is not often that we get to write in a free, unrestricted manner in academia as for example in a blog. Usually, everything and anything we write, whether it be a paper for a journal, a conference presentation, or a departmental seminar talk, is weighted, measured, and, as it is sometimes the case, it is found wanting. When these moments occur, we go back to the computer to re-address our work based on the feedback we got from colleagues, the reviewers, or our mentor.
The peer-review process in academia, regardless of discipline, is absolutely necessary in order to have some sort of verifiable qualitative control over our own work, heightening our sense of integrity and our belief that the system works. The most common critical points for review we receive usually deal with the adoption of a different method of research analysis; reading some essential bit of background information which we may have been unaware of, and has been helpfully pointed out for us. We could be asked to re-address the discussion section in a paper, or review the broader conclusions of our work, perhaps even to include a ground-breaking observation which may have eluded us in the first place (though this is just wishful thinking – it’s often about the exact opposite: taking our conclusions down a notch or two).
It could be argued that the procedure of having one’s own work evaluated by one’s peers, alongside of replicating existing research, should be seen as essential for any study course in which research plays an integral part – and this is why Durham University’s Department of Music offers both of these options to its students in the Music & Science pathway. At the same time, being a reviewer and having the opportunity to read other researchers’ work and help it take shape, also helps the development of critical thinking of one’s own work, assessing the quality and impact of our research output. I’ve had the good fortune of been a reviewer for two journals in my discipline, and could only say that it has been a most insightful experience and most definitely beneficial for my own writing: the majority of journals provide their reviewers with guidelines as to how to deliver an effective and in-depth review. By knowing which points reviewers focus on while carrying out a review, authors who are also reviewers can enhance their critical awareness and improve their writing skills as a result.
I’ve spent the winter holidays of 2018-2019 reworking two papers which I had submitted at two different journals. (There is a common misconception to outsiders that people in academia have long, relaxing holidays; they are forgiven). Both papers were around 8,000 words, they were submitted to different journals at around the same time (winter of 2018), and both were returned to me after about a period of seven to eight months from the time I submitted them. Oh, yes – and both papers had been rejected.
For the first paper, the two reviewers and the editor provided me with around 3,000 words of detailed feedback over 8 pages relating to practically every aspect of my submission, focusing primarily on the discussion and conclusion section, how to improve my analysis, but also not failing to provide feedback on my writing style as well. The three of them provided a total of 22 items of suggested references for me to read, with one reviewer even proposing two alternative journals which may be more specialized to deal with the topic of my publication, because s/he felt my paper may be too specific to be included in their journal. To date, this remains the best and most detailed feedback I have ever received on anything I have submitted over the years – including accepted journal or conference papers and successful grant proposals. My paper being rejected was of little matter, as the feedback I received was of such quality that left me certain that, following the reviewers’ and the editor’s advice, I was on the right track of not only addressing the points they raised, but, after doing so, I would be submitting something of excellent quality and possibly maybe even something substantial in my respective discipline (apologies to those in quantum physics who are about to prove interstellar travel is possible, for me daring to think that a paper in cognitive ethnomusicology may be a substantial contribution).
I wish I could say the same for the reviews I received for the second paper. The first reviewer’s comments for an 8,000 word paper, measured 84 words: “This article presented some interesting points, and intriguing questions about the interaction between musicians and notated music. Although it seems like the paper is leading up to an argument, the final conclusion is more of a summary than a strong synthesis. The exploration of non-Western approaches to notation is developed using interviews. Although this is standard practice for ethnomusicology and endorses the ethnomusicological idea that music should be studied culturally, this approach seems unbalanced with the discussion of Western music using figures like Taruskin.”
The second reviewer was a bit less verbose, managing 79 words: “Initially, the thesis of the project was mostly clear. I enjoyed the multinational scope of the project, however when the author(s) tried to give context and greater significance to the thesis, I started having trouble understanding. The confusion was due to the phrasing, and some of it came from conclusions drawn/arguments made by the author(s) along the way. Indeed, it felt like the author(s) leapt to some conclusions, and often found myself wondering, “where did he/she/they get this from?”
That’s it. That’s all of it. No attachments, no lists, nothing extra. I can readily accept that my submitted paper may not have been to the standards of the journal (and thus be rejected), but I cannot accept that for an 8,000 word submission, the feedback I received was 163 words in total. From both reviewers. According to the first reviewer, if I removed “figures like Taruskin” from my dialectic, and provided a strong synthesis in my conclusion, then my paper would be improved (that’s just an assumption, by the way). The second reviewer fixated on the fact that they had trouble understanding my phrasing (where? Let me know the points and I’ll change it) and my drawn arguments along the way (which ones? I’ve made several arguments, even though the first reviewer didn’t manage to find any), not to mention that s/he felt I was leaping to conclusions, making them wonder as to “where did he/she get this from” (pray tell me which one of my conclusions required a leap of faith, so I may ground it to earth with orthodox fervor).
Speaking of feelings, the reviewers will allow me to feel that they may have been in a rush to get it over with, perhaps because of a pressuring deadline or other obligations; if that were so, they could have contacted the editor and asked for more time. I did not get the feeling that they took the time to review my work as fairly and carefully, as I assume they would expect me to review their work under submission. I will not comment on the ethics aspect of the peer-review process (integrity, conflict of interest, duplication, plagiarism, double submissions, and collection of data, among other issues); this is a topic which requires its own space. Let us focus on the reviewing process itself instead. As a reviewer, you are supposed to read a paper very carefully. I read it twice if not three times at the very least, just to get the gist of it. Then, depending on the guidelines from the journal’s editor, reviewers are usually asked to provide a written assessment upwards of 300 words – it’s not that anybody is counting, but this is actually a low estimate considering the amount of issues that need to be covered. A thorough review aims to cover i) the clarity, overall structure and adequate presentation of the topic under discussion ii) whether the references are adequate and specific to the topic iii) the appropriateness of the research methodology and technical accuracy adopted to address to topic iv) how thoroughly the data have been analysed v) the resulting argumentation in the discussion section, and whether the author(s) covered all possible aspects vi) the overall novelty, significance and importance of the paper vii) its relevance to the audience of the specific journal viii) general comments on the language and clarity of the author’s writing style, with a particular interest on ix) the introduction and conclusion, and if they fulfil their purpose; and last and most importantly x) suggestions for improving the submission. This list is not exhaustive. Then, the reviewer is often required to rate the submission on a 1-5 or 1-7 scale, where the manuscript is evaluated on a variety of issues such as its subject originality, its importance, its approach, its structure and argumentation, among others. It is imperative in this assessment to be very objective, very specific, and ensure that all comments and argumentations provided to the author(s) are relevant and helpful. Finally, as a reviewer, I always aim to finish on a high: in case the paper is accepted for publication, I congratulate the author(s) on a job well-done, drawing their attention to the corrections that need to be done in order for the submission to be accepted and sent to the manuscript editors for formulation. If the paper is not ready to see the light of day, then I encourage the author(s) to carry out the suggested reviews, and motivate them to re-submit once more either to the same, or to a different journal. I’ve reviewed around twenty papers in my short academic career so far, not including conference submissions in which I’ve been in the organizing committee. I’ve never spent less than a working day on each paper sent to me by a journal (but do take into account that English is not my first language, and I work with a Thesaurus). I’ve been told experienced reviewers work three times as fast (ie three papers in one day). Therefore, it is essential that before I even agree to carry out a review, I make sure I have the time for it.
As authors, we evaluate our options using rationality and critical thinking: (i)who, is saying (ii)what, and (iii)why? And finally, (iv) how is this helpful? In the case of my second submission, the (i) reviewers (ii) rejected the submission, because of (iii) not very clear (at least to me) reasons. The (iv) helpfulness of such reviews may only become apparent when we are aware of what to do next: we may contact the editor and express our concerns and asking additional feedback, drawing their attention to lack of transparent commentary, aiming for a re-review, and perhaps even identifying a weak reviewer to the journal. Alternatively, we may simply thank the editor and the reviewers for their comments as they were, keep them in mind whatever these may be, make (any) corrections we believe are necessary, and re-submit to a different journal (and perhaps avoid the other journal in the future). Whatever the outcome, our belief that the peer-review process works should not be undermined. If, by contacting the editor we aim to draw their attention to a potentially weak reviewer, then by all means we should do all we can in order to improve the system. It has to be pointed out that academics are often not really “trained” in the review process, but just thrown into it, with sometimes little guidance of how to do it in the first place, let alone well. Re-reviews and re-submissions are part of the learning process, and it would be well to keep in mind not to take it personally, because in academia we are constantly exposed to review: in our daily work by our mentors and colleagues, in conference presentations by our peers, in journals and grant applications by specialists in our field. The peer-review process, particularly in journals, is usually based on anonymity, and therefore should be seen as an evaluative judgement on the work submitted, and never on the person. In conferences or in short departmental presentations and seminars, where feedback is more direct and may feel more personal, it is imperative that we take it all in, thanking those who took the time to provide it. We should always appreciate critical judgement of our work, and strive to make our peers aware that not only do we welcome it, but we actively seek it, as it may yet give us clarity of thought into what we do and how. And if we find ourselves in the fortunate position to provide feedback, let us try to take the time to provide it to the best of our abilities, and with the utmost respect; after all, it falls to us to maintain the integrity of the peer-review system.
In case you wish to enhance your skills on how to deliver good peer-reviews, consider checking out the following links:
Music & Science Lab 