Away with Page Limits on Submissions (II)

As discussed here, the FOCS 2013 PC did not impose page limit on submissions. As this was an experiment, it is important to try and assess its success. Clearly, no page limit in itself makes it easier on the authors, but we tried to make the CFP more demanding on authors in terms of presentation. A major concern was that the PC work (and that of sub-reviewers) would become much harder. I have asked PC members to give me feedback about their experience. The overall message is – this experiment deserves to be continued.

On the first question “Was it more work than usual? By how much?” the replies were:

On the second question “Would you recommend a future PC to continue with no page limit?” the replies were very similar (one PC member thought that it was a bit harder but still worth it):

I will not repeat arguments for this change that I already discussed. Instead, let me discuss some replies from the PC. The clearest suggestion raised by the PC members is that more education of and instructions to authors and reviewers is needed. I think that this will partly happen by itself if this will become the default, but the CFP may need to be revised as well. In particular, we phrased some recommendations to the authors and future PCs may consider turning them into instructions.

Some quotes on the reviewers’ dilemmas and education:

• “It was no more work than usual, since I didn’t let it be: I regarded reading the right ten pages as doing my duty by the submission.”
• “There were a few papers which misused this and I spent more time on them because of it. Of course, I was harsh on them in the ratings. In hindsight, I should have been harsh on them by not wasting too much time on them.”
• “I would recommend it, but have a little more guidance to people about what constitutes doing a “good enough job” of reviewing”
• Instruction needed to make it easier to “to prune the efforts spent on badly written papers.”

One of the PC members that were least happy, was mainly upset with two very long (50+ pages) badly written paper out of the close to 40 assigned. Usually one would just reject these papers for presentation but:

• “The papers claim amazing results so it’s hard to reject offhand. At the same time, it’s hard to say with confidence that they are correct, especially since the editorial didn’t succeed in succinctly explaining the main ideas.”

I would say that I doubt if such papers would be any easier to review with the page limit (as the 10 pages would still not explain the main ideas and one would have to look into the very long appendix).

Regarding the PC experience there were many different sentiments:

• “Most papers really thought of it as a soft page limit of 10 pages. Hence, some went to 12-15 pages to explain all the ideas rather than cramp everything in 10 pages by reducing margins etc. In most cases, couple of extra pages really helped in the exposition and I found it useful.”
• “I think having no page limit gives the authors too much rope to hang themselves with.”
• “I didn’t think it was more work at all. Most papers that were well-written were well-readable. Badly written papers took time.”

Two other points that were raised:

• There is an environmental costs for people who want to review printed papers and with no page limit cannot just print the first 10 pages.
• “The submissions are very close to the corresponding arxiv versions in most cases. I think this is of great value to the community.”