I got an email today to tell me that a paper I submitted to a journal had been ‘rejected’. This is a major blow and I’m sure any student I’ve ever failed will take great delight in my feeling of utter, well, rejection when I read the word.
I wondered why the paper had been dealt a terminal knock and I reluctantly logged in to read the reviewers’ comments. I thought maybe one had rejected it, but the other might have accepted it (it’s the worst one which always counts).
I was stunned to see that both had ‘rejected’ it. Several strong cups of tea later I got hold of the anonymous feedback and saw that, in fact, my paper had scored quite well in all the important areas, but quite low in ones that didn’t matter (‘theory’ for example – it isn’t a theoretical paper, but a descriptive one on design education policy). The problem is, the reviewers had scored me low on that and factored it in to the overall percentage (which you’re not supposed to do!) leading to the equivalent of a ‘C minus’.
And they hadn’t actually rejected the paper but given it a ‘resubmit with revisions’ which is pretty normal, so I was a lot happier than the original email had made me (cue short reply suggesting a more tactful way of telling me the news in future!).
But then I had the pleasure of reading the actual remarks. These were interesting.
One said the paper was too general, the other said it was too specific.
One said it lacked references to other literature, the other said it relied too much on it.
One said it was of no interest to an international audience (but had obviously not been in the presentation where it was deemed to be of great interest given that similar issues are happening everywhere).
One said it had no conclusion, the other said it had too strong a conclusion.
So you see the problem…
Reviewing a paper is difficult. I did one the other week and rejected it outright but spent about an hour providing a lot of very detailed feedback to the author on why. These two reviewers didn’t go beyond short statements that occasionally made sense but often made me wonder if they’d actually read the paper at all.
But I have to be honest – I hated the damn thing. It was good in its first form but was about three times as long as was allowed. After the editing process it read like a lump of crap and I wasn’t happy. So in a way I’m glad I have to rewrite it, which is what I was going to do anyway until the deadline hit me. In a way it’s like I’ve got a two-week extension on my essay!
One bit of feedback was useful in that it confirmed what I knew: the structure was all over the place and that’s another consequence of the editing process – I normally write without much editing and the structure develops itself, but here I’d move things around so much the argument got lost.
So a rewrite beckons, though I don’t plan on spending my Easter break on it as I really need a work-free holiday. I’m okay about it because, as I said, I thought it needed a reboot. But the experience has raised a lot of questions for me about the refereeing process, especially as the journal (like many) has a policy of not reviewing reviewers’ comments. There are some that I really could go to task on, and a couple that are just silly.
Given the amount of checks and balances that go in to marking undergraduate dissertations it’s odd we don’t do the same to ourselves. One of the reviewers appears very knowledgeable about design but ignorant of British policy towards design education, the other doesn’t seem to know some of the key pedagogical theorists; consequently both made comments that struck me as extremely odd. I’m not impressed by the seeming randomness of the process – if I were asked to review a paper in a field I didn’t feel expert enough, I’d send it back.
If I’m wrong, and the reviewers do know their stuff, it’s not clear in their comments – I once read a review a colleague received and the reviewer had made it very clear they were an expert in the area, and the criticisms were well-taken. In this case I got nothing from the reviews at all. I hope when I review work, whether it’s a peer’s research or a student’s essay, that my critique is informative and constructive. I think that should be a rule.