by Boaz Barak and Omer Reingold
————————————————————————————
The debate about the future of FOCS/STOC has been long and heated. A wide range of criticism (containing at times contradicting complaints) was answered with one simple truth: FOCS/STOC have played and still plays an invaluable role for the TOC community. Indeed, the authors of this proposal have a deep connection to FOCS/STOC. Nevertheless, though often exaggerated, we do acknowledge the validity of many of the concerns regarding FOCS/STOC. As the community evolves, we feel the need to evolve its central meeting place. So while not broken, and not in an urgent need to be fixed, we put forth a proposal to improve (perhaps revive) FOCS/STOC.
FOCS/STOC play a dual role in our community: as both publication venues and as meeting places. In their former role, FOCS/STOC has been incredibly successful, every year many of the best papers in TOC appear in these conference, and FOCS/STOC papers (including recent ones) have led to major awards including ACM dissertation awards, the Grace Murray Hopper award, the Rolf Nevanlinna prize, the MacArthur fellowship and the Turing award.
Thus, while undoubtedly FOCS/STOC are not perfect publication venues, our thesis is that their main shortcomings are as meeting places. Indeed, attendance has been flat over the last decade or so, even as the field has seen significant growth and specialized workshops (such as those at the Simons Institute) often draw an audience half the size of FOCS/STOC. In particular we feel that while FOCS/STOC provides an opportunity for social meetings and small-group collaborations, it falls short in terms of the wider-range exchange of ideas (specifically, the ideas in the papers published in these conferences). We believe it is possible to revise STOC/FOCS to make them a significantly more attractive event (in a sense, a “must-attend” item on every theoretician’s schedule), and a better forum for exchanging ideas across subfields of TOC, while preserving their nature as publication venues (in particular, no dramatic changes in the number of accepted papers, nor in the selection process).
The crux of our proposal is a single combined FOCS/STOC meeting that will be longer (and scheduled appropriately with respect to the academic year) and that will be specifically designed to allow the spread of ideas of appeal to the general community (thus countering the fragmentation of the community) as well as forums for sub-communities to exchange more specialized ideas. While many details can be open to tweaking, in a nutshell we suggest to have an annual weeklong “Theory Festival”. This theory festival would contain presentations of the STOC and FOCS papers, as well as many other activities, including invited talks, tutorials, mini-courses, workshops, and more. The organizers of the theory festival, which would be logically separate from the FOCS/STOC PC’s, would take as input the paper selection by the PCs, but would have considerable latitude in using this input to assemble an attractive program, including a mix of plenary and highly parallel sessions, or any other way they see fit.
Our Proposal in More Detail
The core of our proposal is to collocate FOCS and STOC (and possibly additional events) into a single somewhat longer event at an appropriate time of the year (for example, after the end of the academic year). At least at first, the two PCs of FOCS and STOC will operate similarly to their current operation. In particular, a list of accepted papers (including links to online versions) will be made public in a timely fashion. In addition, a separate organizing committee will be responsible to the selection and scheduling of the joint event. The committee will have representation from the two PCs but will have a separate agenda: to create the most effective program, optimizing for the TOC audience rather than the authors. In particular, it is natural to expect that part of the program will be in a plenary session whereas the rest will be organized as a collection of sub-conferences/workshops in multiple parallel sessions.
Attendees of the joint conference should get an opportunity to catch up on the most exciting developments in TOC (research trends, results and techniques) that are ready for general TOC audience as well as more complete perspective in their specialized area of research. For this purpose, in either the plenary session or the parallel sessions, the organizing committee will not be limited to talks by authors of FOCS/STOC accepted papers. Important results that appeared elsewhere should be represented. In addition, surveys of collections of papers may be at times more effective than talks on individual results.
Let us emphasize that at this point, we are suggesting merely to change the event and make no changes to the paper selection process. That is, there will be two separate FOCS and STOC PC’s that will work on a similar schedule as they do today, where at the end of each PC’s process, the list of accepted papers and the electronic proceedings will be published. The only difference would be that the paper presentations would be deferred to the annual “Theory Festival” that is organized by a third committee. Of course, we are not ruling out making changes to the selection process as well. In fact we believe that decoupling to some extent the event from the selection might open some possibilities for improving the latter that would not be otherwise possible.
Advantages, Concerns and Possible Future Extensions
- As mentioned, the only change necessitated by this proposal is to the meetings, but FOCS/STOC can keep their character as publication venues (both for the authors as well as for external committees that evaluate TOC researchers). On the other hand, the organizing committee will be free to optimize the meetings for the audience experience and for the exchange of ideas. The meetings could also evolve and reflect developments in TOC as a growing research field.
- FOCS/STOC PCs will not need to select papers in multiple tiers. In addition, the organizing committee will also be free from choosing the “strongest” papers. The plenary session (while hopefully a prestigious talk opportunity) will not be intended as an award for papers (as again, the focus is on the audience not the authors).
- The scheduling choices will be intended to be “ephemeral”. The organizing committee will be free to use non-scientific considerations, including diversity of areas or speakers, in making these choices. It can be conveyed to the speakers that all FOCS/STOC papers were equally selected by the PC, and it would be “poor form” to list in the publication list on your CV or webpage the fact that the talk was presented in one session rather than the other. (Of course one can worry that people will still do that, but the risk in alienating potential evaluators will probably outweigh any benefit, and in any case we believe we should not make our events unattractive just to protect against the possibility of abuse.)
- The quantity and high quality of papers accepted in FOCS and STOC together, go a long way towards the effect desired by a federated theory conference (which may not be easy to obtain otherwise; also FOCS/STOC together may provide enough “critical mass” to encourage other conferences to colocate).
- A single event could significantly increase attendance. In particular, it could be easier for researchers with limited travel budget or other travel constraints (e.g. young children) to keep part of the community. Moreover, by “network effects”, with people knowing that this is the place they will meet most theorists, it may well be that the number of attendees in this event would be larger than the union of STOC and FOCS.
- Sub-areas that have grown distant from FOCS/STOC could be welcomed back. As a first step, they could be incorporated as invited talks without asking authors to give up on their more specialized venues. With time one may hope that more papers from these sub areas will be submitted to FOCS/STOC. Similarly, papers submitted to venues with inconsistent publication rules (e.g., some ECON journals) could be easily incorporated in the major meeting of the TOC community.
- A major concern is of increased fragmentation of the community due to additional parallel sessions in the non-plenary part of the program. We argue that the effect of a substantial part of the program (say half) being in a single session more than compensate for this effect.
- The organization committee will also have the flexibility of having plenary survey talks to expose attendees to ideas from outside their area. Furthermore, attendees that used to focus on a few areas of interest (which characterize in our opinions most of the attendees) are more likely to be exposed to talks outside of their area, given the more restrictive filtering offered by the plenary session.
- Some areas (for example Cryptography and Quantum Computing) are more likely to see increased attendance in talks, as at least some of the papers will be in the plenary session, and in any case, we believe there will be increased attendance over the current state. An important concern is the attention to papers that are in more isolated areas, and are not of wide enough appeal to appear in the plenary session. Care should be given to such papers in the program design. It is important to note that these kinds of papers suffer from lack of audience in the current system as well.
- One could worry that by moving to an annual publication cycle, papers presented will be more “stale” than the current model. We agree that this is a concern. However, we posit that FOCS and STOC are primarily meant to educate researchers about progress outside their immediate area. While even few months could be too long a wait to hear about the latest improvement on the problem you’re working on (which is one more reason to be grateful to the arXiv), waiting 6 months to a year to hear a (perhaps more mature and well digested) talk about exciting results in another area may well be acceptable (note that even specialized workshops find value in presentations of papers that are one or two-year old). The organizing committee will have considerable latitude in selecting the program and in particular, if the conferences contained a sequence of papers that improved on one another, it may decide to schedule a single talk that surveys all these papers.
On changes to FOCS/STOC as a publishing venue
We acknowledge that, despite their success, many of the critiques of FOCS/STOC are as a publication venue, including suggestions that they have become too selective, or not selective enough, papers are too specialized, or too shallow, that the deadline-driven process yield “half-baked” papers, and more. These issues deserve discussion, but we note that our proposal is largely independent of any modifications to the selection process to address those, and we believe would yield a more attractive event regardless. Moreover, as we mentioned, decoupling the selection from the event naturally allows some modifications such as selecting more papers, or having more deadlines, that may be infeasible in the current model.
Windows On Theory 
STOC and FOCS haven’t changed in 25 years in spite of growth in the field, so we should definitely welcome proposals such as the one above. The place that STOC/FOCS occupy as meeting places has fallen dramatically in this same time period. This is a fact I’ve confirmed over numerous conversations with colleagues. This is in sharp contrast to SODA and ALGO which continue to grow in attendance during the same time period.
In terms of content, you adduce to an important flaw “sub-areas that have grown distant from FOCS/STOC could be welcomed back.” This is a direct consequence of limited number of slots. Quality of acceptance can be maintained if STOC/FOCS were to welcome once again papers in Crypto, Data Structures, Electronic Commerce, Computational Geometry, Computational Logic, Languages and Automata Theory, Computational Topology, Compression and so on. Accepting the top three or four papers in each one of these areas would allow FOCS to grow by 30-40% while maintaining quality and increasing its appeal as a meeting place for all theoreticians. Same applies to STOC.
We shouldn’t fear change. Indeed change is happening regardless: the field is growing, STOC/FOCS are becoming specialized, attendance as percentage of the field continues to drop. This is an opportunity for us to drive this change instead of letting STOC/FOCS slowly devolve by doing nothing.
Thank you. As we said, we do not rule out other changes. Often top papers in some areas don’t appear in STOC/FOCS because they weren’t submitted there in the first place. The event we envision would include also talks on progress in such fields. Also we feel that one of the ways to get people in more areas to submit their best work to STOC/FOCS would be to make it a more attractive event.
” Often top papers in some areas don’t appear in STOC/FOCS because they weren’t submitted there in the first place. ”
I agree. It becomes a chicken-and-egg thing. If no one submits papers in a given area, then researchers in said area do not become PC members and they don’t attend the conference since there are no papers there and why would you submit a paper to a conference where the intended audience is notable for their absence?
To break this cycle, aside from speakers, you need to send a strong signal by having at least two PC members from each of those topics. This brings us back to another long overdue evolution: increasing the PC size, as currently there isn’t enough distributed expertise for all areas of relevance to have a natural champion.
p.s one possible way to grow is via a secondary PC which meets electronically and finished off with a smaller senior PC with a physical meeting.
I think this was the intent of the STOC experiment a year or so ago (when Joan was the chair). It’s a pity that this experiment was not continued.
I was on the “senior PC” of that STOC. I think this method has its advantages and disadvantages (one of the latter is that very few people in our community have the management abilities of Joan to run a two layer 60+ people PC). However, to my knowledge we didnt get a more diverse submission pool due to this. If you’re part of 60+ person PC then your ability to champion a paper is quite limited.
But again, I think any changes in PC structure can be considered independently of out suggestion.
I think it’s a great idea for discussion. And as someone who will usually forgo STOC/FOCS for SODA/SoCG, I’d love to have a reason to come regularly. The paper acceptance and the Theory Week are not as decoupled as all that though: separating the paper acceptance from presentation removes some of the artificial capacity limitations and allows the conferences to accept more papers.
This is very much in line with how large venues like NIPS and KDD do things (especially NIPS, with their multiple levels of presentation and every paper getting at least a poster)
sounds like many great ideas that are all about adapting/ evolving to changes in the field itself. thx for keeping us all in the loop. it truly is amazing the important behind-the-scenes stuff that now shows up in blogs. aka what in politics is called “trial balloons”. like it!
“It can be conveyed to the speakers that all FOCS/STOC papers were equally selected by the PC, and it would be “poor form” to list in the publication list on your CV or webpage the fact that the talk was presented in one session rather than the other.”
it is hard to imagine why anyone would do that unless someone perceives that either one of STOC/ FOCS is more prestigious than the other (which mystifies me a bit). maybe just simple citation guidelines will cover this. also, as articulated above, there is the case where someone might want to list their talk on a cv at the joint conference when they had no particular accepted paper at either conference.
This is an interesting proposal. I have a few questions/comments on the logistical details though:
– You mention that the conference would be held “at an appropriate time of the year”. Presumably, this means summer. How would this affect the conference deadlines ? Presumably, the STOC deadline could remain unchanged, but the current FOCS deadline (April) would need to happen a few months earlier. Is this what you envision?
– Several CS conferences (VLDB,ICML, SIGMOD, PODS) have experimented or are experimenting with multiple submission deadlines for papers presented at the same meeting. It might be worthwhile to learn from their experiences, especially since some of them (ICML) apparently decided to abandon this approach.
– Keeping the conference nominally single-tiered but de-facto multi-tiered strikes me as somewhat contradictory and, to large extent, unrealistic. Any reasonable scheduling will invariably attempt to maximize the audience for the most “interesting” papers, which is an implied endorsement. Not that there is anything wrong with the multi-tiered conference system (I have been a fan of posters for a long time), but I think it is better to be explicit about it.
Hi Piotr,
First about your question regarding FOCS, I think that the current schedule with a huge lag between notification and publication is largely a legacy of the printed proceedings era. This year I notified people of the decisions on June 25th and we could have easily given people 2-3 weeks to come up with the camera-ready version and put the proceedings up by the end of July. So I think having the conference sometime in August is feasible without major modifications in the deadline, though we might want to take this opportunity to try to balance the deadline of STOC and FOCS so we get roughly the same number of submissions.
It is a great suggestion to talk to people involved in these other conferences, though note that what we are proposing is not exactly the same as multiple deadlines, since the selection process will stay as it is now.
One of the things that makes the task of a program committee hard is that it has to think of different constituencies: the authors, the community at large, and the people attending the conference.
I would like to allow the organizing committee to think only of the latter group. That is, not to think what is “fair” to the authors or even what are the best paper scientifically, but simply what would be the most intellectually stimulating program. For example, if a paper is great but they’re certain that the talk would be incomprehensible, then they can decide to schedule it in a parallel session, and perhaps have an invited talk on this general area. I would also like them to be free to experiment with various formats, including talks with different lengths, single talk for two or more papers, plenary/parallel, posters, etc.. I think it’s best if these experiments are “ephemeral” and whether they succeed or fail is only reflected in the event and doesn’t leave a permanent record on people’s CV. (It might also be a little ridiculous if people add “Scheduled for 30 minute talk” on their CV’s.)
Perhaps if after time we will stabilize on a single format people think is best (e.g., plenary vs parallel) then one could move to a formal multi-tier system,
Adding a little to Boaz’ comments:
First, a small sample of colleagues that I asked find the multiple submission deadlines very successful. As Boaz wrote, this is not what we are suggesting here. But I think it could be a good future development (with our suggestion it will be easy to implement).
I find it important that the paper selection is in a single tier. Also, the organizing committee will not be optimizing for best papers but rather for best program. Of course, presenting in the plenary session should be prestigious (and I expect CV’s will reflect it in “list of talks” section), and will give some signal about the paper. But it is a more complicated signal than “best papers”.
Couldn’t you achieve something very similar simply by co-locating, say, STOC, SODA, CCC (or CRYPTO or SoCG or…), making each of them longer, and coordinating their schedules?
I am not sure I know all the reasons, but co-locating several conferences together, even on a one-time basis, seem to be difficult to achieve in practice. Also, I think if three or more different PC’s are scheduling the talks, it can end up being somewhat of a disjointed event.
Indeed, the federated theory conference was suggested by many of us and some efforts in this direction are still in play. But it is indeed hard both to convince other conferences to come together and also to integrate them successfully into one program. One can view our suggestion as a development of that idea, and we are certainly not precluding additional conferences collocating.
I did not intend my comments to be a topic for a long discussion, as they focus mostly on logistics, not the essence of the proposal. Still, many interesting proposals in the past tripped over their execution, not the essence. So I am posting my answers below just for completeness.
1) The 6 months lag from the meeting to the deadline is not just a remnant of the printed proceedings era. You need a couple of months to review the papers and make the decisions. Then you need to give the authors a few weeks, up to 1 month, to implement the comments (which in most cases are easy to do, but occasionally require a substantial paper rewrite). I do not see how you can finish the program before that – even the accepted papers are occasionally withdrawn, and scheduling a week-long meeting is a pretty complex undertaking. Then the question is how much time to give the prospective participants to review the program and decide whether to attend and register. I would be hesitant to allocate less than 1 month, especially for international travellers. Then there is the usual 1 month gap between the hotel discount reservation deadline/early registration deadline and the event itself. In total, about 5 months, allowing virtually no slack (which carries risk for a conference of large size).
Altogether, I do not see an April deadline for an August conference as realistic. Of course, opinions might vary.
2) Regarding the multiple deadlines issue: I think the risk here is the one you pointed out in your post: a paper submitted in November will be 10 months old if presented in August. I am not sure if this is what prompted ICML to switch back to the single submission format, but it would be good to ask.
3) We can agree to disagree regarding the pros and cons of multi-tiered formats. But I think it is a good idea to acknowledge the reality of the talk selection process, and I am glad that Omer’s comments go in this direction.
While ICML switched back (and it would be good to know why), VLDB didn’t. and appears to be quite content with its rolling monthly deadlines. In fact it’s possible for a paper to be accepted in May of year 1 and only be presented in August of year 2 under their scheme.
I view the “delay” issue as a bit of a red herring. Let us consider the two important components of paper acceptance: one is the stamp of quality and the other is the presentation of the result. The first is not affected by the delay, if the reviewing process is conducted as normal. As for the second, there is a longer delay, but we give talks on work that’s as old as 9 months all the time (in colloquia, workshops, etc). The 9 month delay only r becomes an issue for really fast-moving topics, like when the TSP results were coming quickly. But even then there will be many people who will enjoy hearing the result, and (again) if the publication is decoupled from the presentation, the presenters now have the chance to present a larger overview.
Piotr, you make some good points and it may indeed turn out that implementing this requires changing the deadline timing. However, I think the first question is whether this is something the community is interested in the first place. Personally I would find a meeting like this to be much more attractive than the current format, but if most people feel differently then, as you point out, the issues of logistics are a moot point.
“Personally I would find a meeting like this to be much more attractive than the current format.”
Current attendance figures are unequivocal. The present format is not working. A simple way to get more attendees is to increase the number of papers as well as coverage. Merging the two meetings achieves the former but not the latter. Invited speakers from areas that have drifted away from STOC/FOCS is a good first step but we need to increase the slots if we are expecting those areas that have left to come back and routinely submit. This is simple math.
I appreciate the sentiment, but I think that it is an oversimplification to assume that increasing the slots will bring back authors or audiences. Some of the specialized forums simply provide better exposure to works in specific areas. I am definitely not opposed to increasing the slots (and our proposal will make it easy to do), as long as it is done in a controlled and smooth manner (governed by the quality of submissions rather than by any number). But I think that making the event more attractive to the audience is much more likely to have the desired impact. In any case, whether or not we manage to get submissions in some topics, I want to be able to hear the most exciting work in all areas of TOC.
” to assume that increasing the slots will bring back authors or audiences. ”
Oh I agree, it is a necessary but not sufficient condition.
“as long as it is done in a controlled and smooth manner (governed by the quality of submissions rather than by any number).”
Good, what I’m suggesting is announcing that a certain number of extra slots have been set aside for work in novel areas. Then state that those slots will not necessarily be filled to capacity. This way we encourage new areas to submit but we do not sacrifice quality.
” I want to be able to hear the most exciting work in all areas of TOC.”
Same here. The goal is to keep STOC/FOCS covering all areas of TOC and serving as the flagship conferences in the field they once were.
Ironically, SODA and ALGO which by inception were conferences only meant to cover a subset of TOC are are closer to that goal.
SODA and ALGO are important conferences, but I do not agree that they have a more complete coverage of TOC. In any case, we are drifting away from the topic of the post.
I see the confusion by “that goal” I mean “being flagship conferences” by virtue of their large attendance. I did not mean “covering all areas of TOC” which I hope they never do since that is not their mandate. They shouldn’t be doing what STOC and FOCS are mandated to do.
Thanks for the clarification!
I think that this is an interesting proposal to discuss. Disentangling the publication from the meeting makes changes easier, and I agree that the current publication model works well.
At least for me, the main new goal of such a “theory festival” that I would like to see, will be the ability to learn (in detail) about the last year innovations in TCS. I think that the current STOC/FOCS format fails to achieve that, with too many short talks which are many times aimed at experts in the sub field. I would be much happier with fewer, longer talks combined with more time for networking. Indeed, the focus should be the audience. However, authors should have the ability to present their works, so probably some combination of long talks with parallel short talks is required. Still, I would argue that we should have >50% of the time dedicated to long talks.
I also want to chime in to support the idea of fewer/longer single-session talks which would be aimed at a broader audience. 20 minutes is often not enough to understand technical details, and it’s too long to spend if I don’t get much out of it beyond the statement of the result.
Still, as others noted, I suspect that giving a plenary talk would naturally become a new stamp of approval, and people will put it on their CV no matter what, if it helps. It looks like a new form of endorsement, even if the intent is to think only of the audience. A third round of decision making, in addition to conference and journal refereeing. Perhaps conferences should be run this way and the rest should be just journals (I guess that’s the way they do it in math…)
This link twitted by Lance Fortnow is relevant to the discussion: “Rise of the Rest: The Growing Impact of Non-Elite Journals” (**), I think one can say the same about TOC. Once upon a time nearly every outstanding result appeared in STOC/FOCS. Today this is far from the case. They are now so restrictive that a substantial number of important papers routinely get rejected.
Why people think this makes STOC/FOCS in any way better is a mystery to me. As the article above points out, it really only helps alternate venues publishing those outstanding papers become better.
(**) http://googlescholar.blogspot.ca/2014/10/rise-of-rest-growing-impact-of-non.html
I have to say that I am very concerned. At STOC/FOCS I am mostly coming for the regular 20 minutes talk, and I would hate for them to get any less attention. Yes, I like going to few general talks, but otherwise I go very much by topic based on my personal research interests. Often the papers that are most interesting to me are not those that make it to the plenary sessions. Going to the talks helps me decide which papers to read, and having been to the talk often makes it much easier to read the paper. It is not that I like the presentations to take 20 minutes (sometimes too much, sometimes too little), but we cannot give all papers extra time, and the more time we spend on featured talks, the less time there will be for the rest.
If regular STOC/FOCS papers were not properly presented, would people submit?
Imagine you have a top result in your area, e.g., in my case data structures, and you submit to STOC/FOCS. They accept, but then, much later, the new audience friendly PC decides that your result is too specialized for a broad audience and offer you a poster presentation !
I would totally regret having submitted to STOC/FOCS. At SODA the same result would probably have been highly rated with a good audience for the talk.
In other communities, they at least tell you if your paper is only accepted for a poster presentation, so you can decide to withdraw and find a conference more interested.
Personally, I am pretty beat after 3-4 days of conferencing. I am not sure I could take a 6-10 day Theory Fest. It is fine with me if there are some extra events for those interested before and after. I am curious who would pay all those invited speakers.
All that being said, there is always room for improvements.
Best, Mikkel
If you have a top result in data structures then either it is appealing enough for the plenary session, or it will be highly attended in the relevant specialized session. Hopefully with the higher attendance in this theory festival, your paper will receive more attention than in a standard FOCS/STOC. Details need to work out for this proposal to make sense and be fair to everyone, but it is important to keep the audience as an important part of this equation.