This blog post seeks to solicit input from the theoretical CS community on possible changes to STOC starting 2017. This planning was set in motion as a result of a long discussion session at FOCS 2014 (see these two earlier posts) when strong support was expressed for a longer “theory festival” that would include STOC but have have other events, and broader appeal.
The SIGACT executive committee has asked us (Sanjeev, Boaz, Piotr, Omer and Tim) to prepare suggestions for the scope and shape of such an event, which we will present in the STOC 2015 business meeting. We welcome your input, either as comments below or as email to the address theoryfestival at gmail.com. We are also studying conference structures in other research communities, and soliciting opinions directly from many individuals.
Main consideration: What role should STOC play these days, now that papers are electronically available many months before the conference? Perhaps the focus should be on promoting network effects. Exchange of ideas between sub-communities is an important feature of theoretical CS, and one of the main reasons for holding a “big-tent” conference in the first place. An event with a bigger menu of activities to pick and choose, as well as a strong plenary component, could serve a variety of audiences and bring together people from different subfields, potentially increasing attendance. This means papers get more exposure; and attendees have a higher probability of connecting with colleagues, all of which make the conference even more attractive, thus creating a virtuous cycle.
What kind of activities might such a “theory festival” contain above and beyond a typical STOC? Some options are:
- Plenary talks: In recent STOC/FOCS’s there has been very little room for them; which is a waste of a rare opportunity when broad swaths of the theory community get together. Such plenary talks could contain:a) “Outward looking talks” by leading researchers in adjacent fields, whether it’s other parts of computer science or other sciences such as mathematics, physics, biology or economics.
b) “Inward looking talks” by leading theoretical computer scientists that can survey for the audience recent advances in some area of TCS. (Recent STOC/FOCS’s have sometimes contained such talks, but often very few of them.)
c) Presentation of recent technical work, whether it is from the current conference (where typically these days only the best paper awardees are presented in a plenary session) or top papers from other theory conferences.
- Workshops/tutorials: Recent FOCS/STOC have had a day of these, but it would be good include more “mini workshops”, ranging from a couple hours to a full day. Researchers often have many time and travel budget constraints which sometimes preclude them from going to a broad conference such as STOC/FOCS where only a tiny fraction of the talks will be in their immediate area. Having a specialized workshop as part of it might make it more appealing, promote information exchange within subcommunities and allow subcommunities to present their most important work in a polished form to the STOC attendees.
- Social activities: These bring the community together and help researchers of different sub-fields, geographical areas, and seniority levels get to know one another. Currently, STOC/FOCS have little room for these.
- Poster session, rump session, etc.: Poster sessions (consisting of papers selected by the PC) are an important part of conferences such as NIPS. They present information in a way that makes it possible to interactively browse through a vast menu, and ability to determine the quantum of time and attention (whether 2, 5 or 10 minutes) to devote to each. Poster sessions have had limited impact on theory conferences so far but this could easily change. If poster sessions were selected by the PC, they would attract better papers, which in turn would draw a bigger audience, which in turn would attract even better papers. Poster sessions could also give smaller sub-communities within theoretical CS a more permanent place at STOC, instead of being represented by a couple of papers.
We’ve been tossing around some of the above ideas, but we would love to hear more suggestions for other activities and content that could make the conference more attractive for you.
Alas, even if STOC expands to 5 or 6 days, it may not be possible to fit everything without some tradeoffs. So it is important to us to learn which tradeoffs are acceptable to you. Possible approaches to get more flexibility in scheduling include:
- More parallelism – moving from 2 parallel sessions to 3 parallel sessions for those talks that are not scheduled in the plenary session. (Note however that if there is 50% growth in attendance then each talk could still get the same visibility. In addition, accepted papers could be presented in evening poster sessions.)
- Paper presentations of varying lengths. Some papers presented at plenary sessions; others presented in shorter talks (possibly accompanied by poster presentation), as is done at NIPS. One can also have a “fast forward session” (as at SIGGRAPH) where authors give pointers to look for in the parallel sessions. Experimenting with different talk for a brief presentation of their paper (aka “advertisement talk”) so that people know which formats might be a good idea regardless of schedule constraints — we have become very used to the standard format of a 20 minute talk in 2 parallel sessions, but there may be other ways to facilitate an exchange of ideas, and enable participants to get samples from areas of research outside their own.
- The default mode presentation mode for most STOC papers could be as a poster presentation together with a 3-5 minute “advertisement talk”. A subset of papers would be selected for the plenary session (which would, as mentioned above, feature also TCS papers from other conferences). Other papers might be presented as part of the specialized workshops and tutorials.
What do you think? What would make you more likely to go to STOC? What changes could turn your off? Please do comment below.
Two requests: First, we prefer signed comments, but if you must comment anonymously, please include some information on your research area, country where you work, and your current academic status (e.g., student, faculty, postdoc etc..) and any other pertinent information. One of our goals is to understand whether STOC/FOCS serve some populations better than others. Second, please make no comments about open access and copyright issues, or issues such as conferences vs journals, as those are beyond the purview of this working group that is focused on making STOC a better event for the people that attend it.
We will monitor these comments until the STOC 2015 business meeting where we hope to hear from you in person. We are looking forward to your input!
Sanjeev Arora, Boaz Barak, Piotr Indyk, Omer Reingold and Tim Roughgarden
Windows On Theory 
Having more plenary talks seems like a good idea, even at the expense of parallel sessions and/or posters. For example, I think having plenary talks from best paper winners of the most recent CCC/SODA/FOCS/SOCG/ICALP/etc. is a topic worth discussing. A best paper winner has generated some level of serious enthusiasm within that community, and seeing more research of this kind profiled at STOC (coupled with an introduction about *why* they are enthusiastic) would be nice for everybody.
Or, in a completely different direction, perhaps we could have some mechanism for the “crowd” to nominate and vote on plenary talks from recent work that they’d like to see.
Not sure it’s a priority suggestion at all, but there is one thing I really enjoyed from ITCS’15 and wish were more spread. That is, there was a 5mn-presentation at the beginning of each session, by the chair of that particular session, where she or he presented the papers of the session and describes the big picture and connection between them. I felt it was a big plus, and enabled one to get a better sense of the session and its results before getting to the specificities of each talk.
(I’m a 3rd-year PhD student at Columbia University, focusing on property testing and computational learning)
I think Ryan’s suggestion is excellent.
On the other hand, I dislike the third scheduling suggestion about the default presentation mode being a poster presentation with a 3-5 minute ‘elevator-pitch’ talk. While this may work in some other more ‘visual’ fields, I think it is not the right format for a majority of our papers. Also, perhaps one of the reasons many of us submit papers to STOC (besides it of course being one of the two premier venues) is that it actually gives us a decent chance to communicate the salient points of the work to others. Twenty minutes is short but not too bad; five minutes seems neither here nor there.
I agree with Raghu.
Twenty minute talks are already short. But it is still possible to tell something about the problem and solution in 20 minutes. Today, when I go to a complexity or crypto talk, I can understand the problem and learn something about the solution. If we make talks shorter, then speakers will have to give talks only for experts. I don’t think it is a good idea.
I feel that minority is given less and less attention on their work (from 20 minutes to 5 minutes), while the “top experts / leaders” are given more and more attention to talk about their philosophy and visions and highlight the research done in their groups.
In the current system, it is a fair competition based on scientific merits. In the new system, it depends more on power/connections/popularity and it is harder to imagine a researcher from a small school or from a “non-mainstream” area will be invited to give a plenary talk.
I share this fear. (Professor in Europe.)
Hi there, I’m 22 years old and have just started attending research conferences in theoretical computer science. I figured I’d comment to provide a student’s perspective.
For me, the main benefits are meeting researchers, discussing the problems I’m working on, and learning about the problems other people are working on. Although, it feels like there is an obligation for me to try and submit research work in hopes of getting accepted because (as a potentially naive youngster) it seems that people are rated by how many papers they write and which conferences they get accepted into.
I’ve participated in Rump Sessions, attended poster sessions, and attended a larger multi-track conference as well.
I really enjoyed participating in the rump session because I had the opportunity to talk about whatever I wanted in a slightly less formal setting. However, I was still up in front of an audience of 50-100 people and I didn’t get much audience interaction. I think smaller groups for multiple simultaneous rump sessions would be really cool and promote more interaction within the smaller group to discuss certain topics. Also, I think young researchers would be more likely to present there thoughts in a smaller group.
I barely even noticed the posters in the poster session at the conference I attended because everyone was more focused on the talks. Even if more people were focused on the posters, I felt like only one person could look at a poster at a time.
I really enjoyed the multi-track conference that I attended. It allowed me to meet people who work on all sorts of different areas in TCS. Also, it forced me to get up and move around and switch from one conference room to another. This led to more interaction with others and got the blood flowing.
When I attended a conference where all the talks were in one room, I felt like I was forced to attend all the talks and just stay put. I felt it was better when I was up moving around and it encouraged me to skip various talks which was good so that I didn’t get too overwhelmed and could really focus on the fewer talks that I attended and ask meaningful questions.
I also had one other bias comment to make. I’m really passionate about the subject “Automata Theory”. However, in the recent major US TCS conferences (based on paper acceptances and my personal experiences) it seems that this topic isn’t included in the programs. Is there a reason for why this is? The subject appears to still be fundamental to the subject of TCS. It’s taught in introductory courses in the US, internationally there is a lot of ongoing research work, and automata related problems are often discussed in TCS blogs and on stackexchange.
I guess I mention this because I like the idea of having mini-sessions to discuss topics that may or may not be properly accounted for by the general conference program. If I were able to submit a proposal to organize a mini-session on recent problems in automata theory, I would.
Thank you for giving us the opportunity to provide our input and thank you for your careful consideration. Have a nice day! 🙂
Looking at this year’s program (http://acm-stoc.org/stoc2015/program.html), I see that sessions are not divided by topic, apparently. I would prefer to see topic-specific sessions.
Also, to increase the number of student attendants, I would like to see “student sessions”, where students present their research (and a paper) and the audience (including more experienced researchers) interact with them to ask questions and suggest improvements.
Peter, I think usually people overestimate the influence of the “elites” on STOC/FOCS acceptances (I say this from experience over the years). What is sometimes worrisome is the trendiness of areas, which is not decided by elites, but by the (often very young) PC members. It is this trendiness which is sometimes criticized.
It is conceivable that opening multiple channels within STOC such as multiple workshops, poster session etc. would have the opposite effect of what you are afraid of: they would reduce the ability of a smaller group to set the agenda of the field.
We won’t know until after we try for a few years.
💡 yes thx for asking for open feedback. brainstorming/ free associating here.. you mention online papers which has changed scientific practices significantly. maybe theres a way to integrate cyber/ online elements into the proceedings more. blogs, social networks, etc…. plz make it so the electronic refs are easy to find, powerpoint presentations are archived, videos are saved, tags on the content, searchable, etc….!
here is an example of a TCS cyber outreach effort from a few yrs ago.
this is a neat article by klarreich/ simons institute on mathoverflow, global mathematics commons, hope something similar might be written for TCS some day.
(ps hi MW!)
Hi VZN. 🙂
I’ve stuck on “Main consideration: What role should STOC play these days, now that papers are electronically available many months before the conference?” In my eyes, this is THE issue. Why then not ask the core question: need we now conferences as such at all? These “kaleidoscopes” of presented results, most of which are already known from other sources? Plenary talks, focusing on breakthroughs would be still in order (here I agree with Ryan).
Should we not just turn back to the original role/aim of conferences: meeting peers, friends colleagues? Without trying to give “quality stamps”. Without trying to replace a normal publishing procedure. Instead of sinking in “save what possible” details.
Thanks to everyone who commented – please keep those coming!
I understand Raghu and Kostya’s point about the dangers of shorter talks. There are many good speakers in our community that manage to give great 20 mimute talks, where even someone outside the field can understand what was the result, why we should care, and get some hint of the ideas used in the solution.
However, I think a marathon of 16 or so such talks (as would be the case in a typical day in the next STOC, with the morning session being 6 consecutive ones) can be less than fully pleasurable. For this reason, I often find myself enjoying workshops more, since they typically have fewer and longer talks, and a more relaxed schedule with ample breaks.
Now of course a conference is different than a workshop in the sense of having much more content in often a shorter amount of time. Still to me a good schedule will involve a mix of several different types of activities beyond just talks, and the talks themseleves should be varied in content and format to keep things interesting (e.g., different lengths, different topics, survey vs report on recent result, plenary/parallel, etc.. )
I think mixing different types of activities and formats is a good idea as well. However, making the *default* be a 5 min talk sounds too drastic to me.
I believe that our community is mainly driven by open problems, and for us to stay as one community we should aware of the current challenges in other subareas (perhaps to the point that we have some idea which paper should win the best paper award when we see a FOCS/STOC accept list). For this reason, I would love to attend an “open problems” session at FOCS/STOC if there is one, where top experts (or perhaps anyone) explain open problems that they view as the current major open problems to people in other subareas. Presentations in this session should focus on the importance of these open problems (thus, not a tutorial-style presentation). Perhaps, it will help if these problems are clearly labeled, e.g. as a hard problem to solve, or as a new problem that will impact the industry.
Sanjeev, I feel like you missed some of the point of Peter’s comment. He is giving credit to FOCS/STOC for being a “fair competition based on scientific merit.” The concern is that a focus on invited talks would skew toward the “in crowd” in a manner that is not proportional to merit. I think it’s hard to argue with that position (look around at any similar circumstance in any walk of life).
One of the big advantages we have as a scientific field–and especially one that is formal and mathematical–is that our means of evaluation have a strong objective component. I think it it’s legitimate to ask how much we are willing to sacrifice on that front for the sake of other (admirable) goals.
I believe I correctly understood Peter’s comment. If you look at other conferences like NIPS which have multiple presentation formats, the focus is on participation by all (big poster sessions, 10 parallel workshops, etc.). The effect is to level the field more than to elevate the interests of a select coterie (Who rotate yearly anyway.)
Many of the “invited” talks we envisage would be proposed by various theory subcommunities (crypto, data structures, etc.). Some talks would be from outside theory or even outside CS. For people who don’t have resources to travel frequently (to say Simons workshops or other venues), all this would be a good opportunity to learn and catch up a bit on major new ideas.
BTW I have found that many people —including very prominent theorists—-disagree with the the characterization of STOC/FOCS or any conferences as “fair competition based on scientific merit.” Remember the infamous NIPS experiment last year about PC bias.
>> BTW have found that many people —including very prominent theorists—-disagree with the the characterization of STOC/FOCS or any conferences as “fair competition based on scientific merit.”
As other people have pointed out before, a consequence of the lack of growth in size for STOC and FOCS is that they now discriminate papers on second order signals. There are more than enough good papers every year to fill up two STOC and two FOCS sized conferences. So the game is no longer about who has an interesting result solving a long standing question or introducing a new area. It is now about who has the most complicated proof or talks about the hot fashionable topic or has a champion in the PC or a funny title or you name it, anything but scientific merit, which is cleared by a much larger number than the final tally of accepted papers.
Another consequence of the lack of growth is that people do not attend. Back when TCS was mostly complexity classes, half the program was of interest to all attendees. Now that the field has branched out in so many directions (a healthy sign to be sure), the number of papers in area X for any value of X is likely to be less than five, i.e. not enough to justify attendance.
Double the size of STOC/FOCS so (1) we go back to measuring mostly first order signals and (2) assuming most people have at least two areas of interest, we are now talking about 10×2=20 papers of interest.
another quick idea: was thinking that TED is a highly successful model for an international conference in the cyber-age. obviously its much different than a theoretical scientific conference but worth examining for ideas/ even possible “crosspollination”. very slick stuff, highly polished, grown rapidly over not that many years, some lectures get massive views on youtube, basically going viral, very high “production values”. the content is varied but quite a bit of it is STEM related. not saying its all appropriate but some analysis of the style/ presentation seems worthwhile. also the ICM, international conference of mathematicians is virtually like a 4-yr olympics event in the field and worth noticing also. re RWs idea of “best paper awards” think its great esp when there are categories that are more varied & not so strict eg “best student/ graduate paper” etc… coincidence/ cybersynchronicity, actually partly found MWs groundbreaking/ outstanding recent paper that way!
I think that the suggestions are fair. However, I didn’t fully understand a crucial aspect of the new conference, i.e., the proceedings, and the status of publishing in STOC.
Specifically, if a paper receives a plenary talk, while another one a five minutes talk, is there any difference vis a vis their status of publication? Or both are considered as published at STOC? And what about talks for papers already published in other conferences, are they considered as published at STOC?
So my question is basically this: STOC (as any CS conference) has two functions, the first as a venue to hear and discuss science, and second as a venue to officially publish in, which is a necessary requirement in most universities and research institutes.
It seems that you change only the first function, while the second function is kept unchanged. Is this indeed the case?
Another suggestion is to make the choice between plenary and short talks random. So that over several years you get a nice coverage of many areas in both plenary and short talks, as well as maintain fairness and avoid the establishment of first and second tier STOC publications.
Hi Stewart,
Our group is focused solely on the *event* and not the publication process, and so as far as we’re concerned the latter can be left intact. (Whether there are improvements or changes that can be made to the publication process is a topic for a different discussion and not within our purview.)
Nothing is decided, but at least the way I think of it, there will be two separate committees. The program committee will do its job as usual, selecting the papers that will appear in the proceedings. Then a separate committee will be in charge of organizing the event, and their job will include scheduling the paper presentations, inviting plenary and other talks, scheduling workshops, social events, poster sessions etc.. etc…
The second committee is focused solely on making a great event for the people that attend it, and has no input to the proceedings. So, whether a paper appears in the morning or the afternoon, or is scheduled for 15 minutes vs 25 minutes, will not matter to its publication status. Similarly, if the event organizing committee invited someone to speak about their recent CRYPTO paper, it does not make that work doubly published also in STOC.
Hi Boaz,
This is quite disturbing the way you describe it.
Like many others who commented here, I am against making some of the talks shorter. I am also against varying the length of the STOC talks, this adds another round of selection to the process. The fact that there will be a separate committee to decide which papers get longer/shorter talks is disturbing to me. Are they going to read all of the accepted papers before deciding? Are they going to read the reviews as well and get a summary of the PC discussion? Are their decisions going to be based on their selective and subjective knowledge of who they think is a good speaker? This means that a timely talk by someone at say MIT can have a big impact on their talk length. I would be very angry if I got a paper into STOC and then later found out it was only given a 5 minute talk slot.
You made the point before that there would be too many talks. So? Why do you need to go to all of the talks? I’d rather give a 20-minute talk to 20 interested people then give a 5-minute talk to a bigger audience.
Finally, why is the discussion here focused on STOC? Why not try to co-locate with as many theory conferences as possible, but co-locate in the true sense that they are at the same location and the talks are interspersed. There are many theory conferences that happen in the summer — STOC, Complexity, SoCG, Crypto, PODC, Random, Approx. There’s no reason it has to be centered around STOC. Why not try to get as many of them as possible to co-locate, and then have a week-long event with all of these conference talks scattered throughout the week (parallel sessions for different conferences seems OK to me). If the point is to get as much of the TCS community as possible together at once then that seems like a good way.
Eric Vigoda
Hi Eric,
Thank you for commenting, indeed this is exactly why we have this thread, as well as the in-person discussion in the STOC business meeting. Varying the lengths of talks is simply one idea that has been suggested, and clearly the reception to this idea will impact whether it is adopted or not. BTW it’s not clear that the choice is between a 20 minute talk to 20 people and a 5 minute talk to a larger audience: perhaps one can have both, whereas people give a short talk in the plenary session that advertises their longer talk in the parallel sessions / workshop.
The point I was making is not that one must go to all the talks, but rather that I think a typical day of the conference should consist of some other activities except a marathon of 20 minute talks one after another. When you schedule 32 talks in two parallel session (as is a typical STOC day) then there’s simply not much room for anything else.
I believe co locating many theory conferences has been attempted and there are significant obstacles to this, which is the reason SIGACT has asked us to focus on suggestions for planning STOC 2017 – more knowledgable people can comment on this more.
But this discussion is pertinent for such a theory “mega conference” as well – if anything such a conference would only contain more talks, and so if we want to include in it other activities we would need to find some scheduling solutions.
FWIW my idea was that the STOC PC itself would decide which talks to propose for plenary sessions (or longer presentations or whatever else the format is). It would indeed not make sense for a second committee to make that decision. I would imagine that the overall festival committee would
decide upon workshops, and the overall shape of the event, not micromanage STOC.
The shape of STOC is being considered as part of this process by our committee only because it affects the amount of time available for other events and the overall feel of the event.
Eric, sorry if it wasn’t clear from our writeup, but other theory conferences would indeed be free to colocate or to locate nearby. (e.g. this year many ML meetings are being held during July in Western Europe and Britain(.
Sanjeev
I think that we should just get rid of the proceedings at conferences. There are a few reasons for this:
– no papers that contain half of the material and are not falsifiable;
– authors have an incentive to make their full paper electronically available, as otherwise it does not exists until accepted to journal;
– no problem with submitting the same material to journals (now there should be something new);
– keep the selection process for _presentation_ slots, which is the actual point of the conference.
If conferences are a place to gather and hear talks, let that be the purpose and let journals take the part of publishing printed work. Each time more I think the only reason we keep having proceedings is that they can be sold and we have some bean counting. Keep the bean counting, if wished (count acceptances to conference), but force the authors to make the papers available, if they want to get proper credit.
As Boaz mentioned, the short vs. long talk selection processes described above (i.e., via a separate committee) is just one of the options discussed. There are other options. In particular, in many applied CS conferences (e.g., NIPS), the oral presentation vs. spotlight presentation selection is done by the program committee itself. So this is another possibility.
Hi Boaz,
As I said before, I think the suggestions of variable-length talks is interesting and will make the event more appealing in itself. However, in order to maintain (a perceived) fairness, for which many commenters before expressed their concern about, I would raise the following original suggestion: make the decision between plenary and short talks *random*.
This way:
1) We achieve a good event, because we have variable length talks, and all the benefits you described above.
2) The event is fair, and the publication status at STOC doesn’t change at all (there are no two tiers of publications).
3) There is no need to an additional committee to decide the division between short and long talks. The original PC does this.
I don’t think a random schedule is the best way to maximize the benefit of the conference for the people that spend the time and money to attend it.
Again, variation of talk lengths is simply one of the ideas that have been suggested. If it turns out that there is a lot of resistance to this idea, then one can keep all papers presented by a 20 minute talk, as usual. I don’t think fairness has much to do with it – there’s nothing more inherently unfair about talk length variations than there is in selecting which papers to accept in the first place. In both cases the organizers are given a finite resource — the conference attendee’s time and attention — and try to find the best way to utilize it.
The usual “formula” I hear of for a 20-minute STOC/FOCS talk is to spend the first 10 minutes explaining the background, motivation and main result of the paper. The second 10 minutes attempts to give some details of the proof of this result. These days (as a postdoc), for those talks I actually want to listen to, I already know most of the background of their presentation. I’ve probably even briefly looked at the pre-print. As such, the first 10 minutes is mostly just a reminder. What I come to STOC/FOCS for is the second 10 minutes, where I hope to get a small nugget of insight that I can actually understand and “take home”. Few papers are written with such insights highlighted in the introduction (This mirrors Sanjeev’s view that 10-page conference versions can force authors to better highlight their contribution. While I agree with the sentiment, I feel that no one reads these conference versions and so authors place no value on them.). Often, these insights are just proofs of a toy-version of the problem, and the actual proof just generalizes it (often in non-trivial ways). Unfortunately, I feel that most speakers attempt to explain too much detail and do so in a way geared toward those working in their sub-field.
To me, this is the “real” problem with the conferences as is. I feel that as a community we need to have higher standards for presentations. I realize there are few concrete changes one can enact at STOC/FOCS to make this happen, but this is a similar complaint to those who becry the poor quality of the reviews they receive.
—
More concretely though, I don’t think shortening these presentations is a good idea. For those talks I’ve already decided I want to attend, nothing new can be said in 5 minutes. For the rest of the talks, 5 minutes is too short for a description to entice me.
Indeed, I feel that any change to the presentation time /must/ be coupled with a discussion of the conference-versus-journal discussion. Why would someone submit to STOC/FOCS to only get 5 minutes of talk time, with potentially a poster presentation? While I realize there are some who highly value poster presentations, I cannot help but feel that they are too burdensome for the audience. That is, while a poster-presenter can give their talk at the poster, this format is much more driven by audience questions. Based on the relatively few questions we ever receive at any given STOC/FOCS presentation, I can’t see poster presentations working well without more effort to persuade the community that when asking questions you can get answers you actually understand.
I was talking with a prominent theorist (won’t name him simply because I haven’t asked his permission to quote him) and he made the following interesting point.
Every system –whether STOC/FOCS, or NIPS, or Science/Nature—is designed with good intentions, but after some decades people figure out how to game it. They optimize the kind of work to do, and how to write it, to optimize chances of acceptance. (I must say I tend to agree with this observation. For instance, it is well-known that some people game the Science/Nature process, and “overfit” their research to it.)
So his point was: change will be good for STOC. It will get around this overfitting process for a while and freshen up things. When a new equilibrium/overfitting appears in a decade or two, it will be time to change again.
Obviously, any changes must not be random and still be carefully designed, hence this process. I just thought this was an interesting viewpoint.
Sanjeev
Personally, I find most STOC talks to be very dry and I don’t get much out of them. The main reason I go to conferences is to meet people and make connections, not to listen to talks. Of course, talks have value, but I think there would be considerable benefit to having STOC be more than a few days of back-to-back talks.
A poster session would be an excellent idea. Sure, posters aren’t the best way to communicate proofs, but they really are just conversation starters. They allow more interaction and flexibility than talks and hopefully allow people to connect.
I recall a poster session at STOC’11. What was the feedback on that?
One thing to add: I think the bar for posters being accepted should be set quite low. (In particular, it should not be an achievement that you list on your CV.) Allowing people who would ordinarily not be able to present their work at STOC to have a poster would make the event more inclusive, particularly for students, and it would encourage more people to come even if they don’t have a paper.
Let me make a few comments as an outsider (but a long-time fan). STOC/FOCS (and other conferences) are extremely successful scientific activities with crucial influence on the way TOC/TCS is advancing. The crucial role of the event is the selection of the (accepted) participating papers. The other events (the one day devoted to workshops and very few longer talks) are of secondary importance.
I also think that 20 minutes is already short and shortening the time for accepted papers is a bad idea. With the best papers decoration notwithstanding, having all accepted papers treated equally is a very good merit of the event and has a major role in its success.
Perhaps the most important question to discuss is the idea to make the event one day longer.
If NO then I would suggest to leave the format roughly as it is.
If YES, then I I would suggest to make the program little less dense, add perhaps a round-table discussion or a rump session of some sort and add no more than 3 longer lectures (50 minutes, say). Adding a day can also accommodate a very modest
increase of 5-8% in the number of accepted papers – but such an increase should be decided on its own.
(On the more theoretical side of making changes: the very general point regarding changes vs overfitting/gaming, while very interesting, does not have, in my opinion, much merit, and, in this generality, it is rather irrelevant to this case.)
For me personally, any cutback in the current presentation format (20 minutes, 2 parallel sessions) would be a turn-off. I may be able to live with a third parallel session, but other cutbacks would cause me to reconsider submitting to this venue.
Starting in 2012 in Chapel Hill, the Computational Geometry community has experimented with a CGWeek: the next will be in Eindhoven (http://www.win.tue.nl/SoCG2015/). The Symposium on Computational Geometry changed from three days (9-5) to four mornings (8-1:30), allowing afternoon parallel sessions 2-4 topic-based workshops, a young researcher forum, and a “multimedia exposition.” This has helped broaden participation and boost attendance. The variety within a day mitigates the “technical session overload” and allows people to vote with their feet about attending longer or shorter talks offered by the workshops (or doing research in the lobby.)
It is a challenge for local arrangements chairs, who need to provide a range of room sizes (either at a university or by partitioning rooms for the afternoon) and make sure the workshops stay somewhat coordinated (fix the break times) to prepare a unified program. The ’12-’14 offerings gained important organizational flexibility by having only SoCG sponsored by ACM and subject to their TMRF process.
Come to TU Eindhoven, the Netherlands, June 22-25, 2015, to check it out!
Jack Snoeyink, currently rotating at NSF CISE CCF Algorithmic Foundations (AF)
I was also going to bring up the SoCG model. I attended the last conference in Kyoto, and have slightly mixed but mostly positive feelings. The plenary talks, workshops and tutorials are indeed very nice, and spanned a range of topics. Also a less crammed program made it possible to enjoy the events with less fatigue. On the other hand I felt that because of the three parallel sessions the average talk was sparsely attended. I guess that’s an inevitable trade-off. But it does hurt papers from under-represented areas more.
My feeling is that the “problems” you are trying to solve are mainly due conferences being both a place where one goes to listen to talks, chat with colleagues, etc., and a publication venue. It is hard to change the format of the presentations since some people would (correctly I guess) think that having only a 5-minute talk or a poster would minder their STOC publication. If this change occurs, I am pretty sure we’ll see in some publication lists the mention “full 20-minute talk” or something like that as note on some papers. In some sense, with such a solution, STOC proceedings would become a somewhat classical journal, but with one issue per year and a number of papers fixed in advance (with some flexibility though). Why not simply abandon STOC proceedings and publish in journals then? (Note that this latter solution has my preference.)
[Assistant prof. in Europe]
Thanks to everyone who commented! Again, please keep those comments coming – we’ll be monitoring this thread until the STOC 2015 business meeting on Monday June 15. Everyone is also very much invited to come to the meeting and present their opinions, as well as vote on them, in person.
Lately, the comments seem to have centered on the issue of variable length talks. This is merely one tradeoff, that is used in other conferences, but is by no means an indispensable part of this proposal – if the majority of the STOC-attending population strongly dislikes it then we won’t have it.
Also, as mentioned, our focus is on making some improvements to STOC as an event, and so do not want to wade into the whole journal vs. conference debate. Perhaps that can be the topic of a separate, future, post. We are trying to make evolutionary, rather than revolutionary, changes to STOC which we view ultimately as a very successful conference, even if its format can use a little updating.
Aside from discussions of tradeoffs, I would like to hear more about what people would like to actually see in an expanded STOC.
First, do you feel the need for any changes? Maybe you’re happy with the conference as is?
Do you find the content we suggested appealing? (e.g., plenary talks including from prominents scientists in other fields, highlights of theory works that did not appear in STOC/FOCS, etc..)
Are there other activities or content that would make it more likely for you to attend STOC?
If those are implemented, would it be worth your time to spend an extra day or two at the conference?
Having variable-length talks sounds like a minefield for the PC and an annoyance (and potential insult) to authors.
If 5-minute talks were implemented, then authors should be able to choose whether or not they want that. i.e. there should be a separate category of submissions that correspond to shorter talks.
I like the idea of having a separate category for “short papers” which are given shorter talks. This category would have a lower bar for technical wizardry and a higher bar for presentation and brevity, with the ideal paper being one that gives a 3-page proof of a 40-page long paper that appeared last year.
Note that all the changes that we floated (including variable length talks) are ones that are already implemented and used, arguably successfully, in other conferences (such as NIPS).
However, I feel that it would be good to discuss other aspects of this proposals, and in particular its positive ones.
There’s no need to talk about making tradeoffs on presentation length if people are happy with the event as is, and don’t see the need for additional content such as the options we suggested above.
What’s the growth rate of NIPS over the last twenty years? How about other conferences such as SIGIR, INFOCOM, AAAI and SIGGRAPH? How about STOC and FOCS?
I don’t think five minute talks is either the problem nor the solution. I don’t think there is anyone out there that is thinking: I’d go to STOC if only it had five minute talks.
People will go if the have papers to present and interesting talks to attend. Ergo we need more of those: not just a couple more invited talks but dozens more papers, thus increasing participation and coverage. The rest will follow.
p.s. I’m not against poster sessions, it’s just that I don’t think they go to the core of the issues here, but that’s just my opinion.
I have never tried to make a serious study of conferences in other areas but Suresh Vankat once collected some (partial) data on top tier conferences in other areas of CS, see here http://blog.geomblog.org/2010/06/on-acceptance-rates-and-flagship.html
As far as I recall, the main way that STOC and FOCS are outliers are in having a relatively high acceptance rate. Also, many (but not all) of the top tier conferences have a single session and so they accept a much smaller number of papers.
Let me quote Suresh’s takeaways from his data:
“To me, there are two things that stand out from this.
1) The number of papers accepted does not appear to make a difference to the attendance. SOSP happens once every two years, and accepts 23-25 papers, and gets 500 attendees !! ICML gets a similar number of attendees with 150 papers accepted each year.
2) There are a TON of activities at these conferences. Indeed, I think ICALP and ESA match them in terms of level of activity, but certainly not STOC. I’ve been a proponent of satellite events around a conference to increase attendance, and the STOC/EC/CCC colocation does seem to have helped. I’m also intrigued by the idea of colocating SoCG with STOC. “
Thanks for the link to Suresh’s statistics. I would like to add that
(1) all the smaller venues than STOC/FOCS have shown substantial growth over the last twenty years, e.g.
SOSP 1995: 22 papers
SOSP 2013: 30 papers
Growth rate: 36%
SIGCOMM 1995: 25 papers
SIGCOMM 2015: 45 papers
Growth rate: 80%
STOC 2014: 80 papers
STOC 1995: 80 papers
Growth rate: 0%
(2) all the other events were much bigger in terms of papers presented overall. Either they had more regular track papers to begin (some with more than 10x the count of STOC/FOCS) with or a lot more workshops/poster sessions.
So it seems that STOC/FOCS are unique in sticking to their small numbers and in their low attendance.
Lastly Suresh writes “it would seem that attendance correlates more with the perception of being ‘flagship’ than the actual number of papers accepted.” It seems to me that if we follow this to its logical conclusion TCS no longer considers STOC/FOCS flagship conferences. They are still highly prestigious and perhaps the highest notch in one’s belt in terms of publishing, but they are long removed from their flagship, must-attend conference status past incarnations.
The fastest, least controversial way to get back on track is to push that 80 accepted papers to 150 or 160, and continue with the satellite workshop/tutorial efforts.
You are suggesting to double the number of accepted papers to accepting 300-320 papers per year, and moving STOC/FOCS to 50%-60% acceptance rate. I am not sure what this would achieve, but I disagree with your claim that it is not controversial.
My own viewpoint is that STOC (and FOCS) is an extremely successful conference. STOC papers, including recent ones, have had major impact not just in theoretical CS but also in other areas of CS and even other sciences. So, I don’t think STOC needs a revolution but only a gentle evolution.
STOC/FOCS currently enjoy significant prestige despite them having a somewhat high acceptance ratio compared to flagship conferences in other areas. I think the reason is that the theory community has a reputation for high standards and self selectivity, which helps us in other situations as well, including hiring and NSF funding. We should take care to maintain that reputation.
One should note that there are many successful applied conferences with acceptance rates similar to those of theory conferences. E.g., the acceptance rate at NIPS’14 and ICML’14 was 25%. This is on par with FOCS’14 (26%) and SODA’15 (27%); I am guessing the rates for STOC’15 are similar.
(Note that this counts both oral presentations and spotlight/posters, which are considered equivalent from the proceedings point of view.)
I feel that comparisons to other conferences, especially those outside theoretical computer science, needs to be done carefully. Papers in theoretical computer science (these days) are often increasingly narrow and deep. That is, the focus is narrow and one needs a deep background to even appreciate the result. For myself, while I can /hope/ to glean something from talks in this area, I often have low expectations in terms of what I understand (and sometimes even these are not met).
In contrast, I feel that papers in computer systems (though I am no expert) are comparatively more broad and shallow. That is, it seems much easier to have a shallow background and still understand a broad swath of papers. As an example, after having looked at the SIGCOMM14 program, I feel that I probably would understand a random SIGCOMM paper better than a random STOC paper.
I think the reason STOC/FOCS don’t get such a large attendance as SIGCOMM is because of this issue: the audience simply isn’t getting enough from the talks. To some extent, this seems inherent based on our field.
The current STOC/FOCS best paper sessions do a decent job at addressing this problem. They are (sometimes) slightly longer, so authors have a better chance to explain their ideas, and they are single-track, so that they can attract the attention of a broader audience (and ideally, the authors position their talk so this broader audience can understand their work).
I think we should enlarge the number of “best papers” to perhaps 5, or even 15. This way attendees would all see what the “really good” papers are, and hopefully actually understand them. I think this would improve on the experience of the audience. The rest of the papers could be the typical STOC/FOCS format, but perhaps being triple-track to accommodate logistics. This way the overall publication prestige would not change dramatically.
“and moving STOC/FOCS to 50%-60% acceptance rate.”
Huh? SODA accepts twice as many papers as STOC/FOCS yet its acceptance rate is nowhere 50-60%. How so? if we look at the data most theory conferences have acceptance rates between 25-35% due to self-selection. If STOC/FOCS were to increase their acceptances we would have more submissions thus keeping the acceptance rate more or less the same.
“My own viewpoint is that STOC (and FOCS) is an extremely successful conference. STOC papers, including recent ones, have had major impact not just in theoretical CS but also in other areas of CS and even other sciences. ”
You just argued there that STOC/FOCS have a very high selectivity threshold, which is undoubtedly true. Nowhere in there though, is there an argument that they are good conferences.
This goes to the crux of why STOC/FOCS are falling behind. People in charge keep confusing selectivity with the true goals of a conference, namely dissemination, interaction, cross pollination and yes, selectivity as well. In my opinion STOC/FOCS aren’t doing too well in any of those four parameters.
You see, you seem to argue that higher selectivity ==> better conference. This is true only to a degree. A conference with exactly one paper a year would be highly selective but a sure contender for the title of worst conference out there. It is clearly possible to overdo it with selectivity. How can we tell if a conference is too selective? I dunno, maybe because people stop attending because they don’t find enough papers of their interest?
“So, I don’t think STOC needs a revolution but only a gentle evolution.”
Doubling the size of the program over four years, in a controlled fashion is far from a revolution. We have plenty of examples of flagship conferences in other areas which have grown by this much or even more while remaining essentially the same.
I do agree with you sentiment that we don’t need a revolution. However the threshold for change seems to have been set rather low on your part when even an increase in the number of papers is considered a “revolution”.
A compromise option is to keep the number of “long” (20 min) talks at the current level, and use spotlights/poster combination as a room for growth. This should alleviate concerns about “demoting” long talks to posters, while also expanding the reach of the conference.
And yes, if posters are implemented along these lines, then the PC would need authors’ approval to classify them as such. Perhaps the simplest option is for the authors to check an appropriate box at the submission time if they are OK if the paper is accepted as a poster.
Having two different categories of submissions (long and short) is, alas, an approach that has a long track record of not working. It was tried at SODA and SoCG, and for whatever reasons did not pan out at either.
Here is a suggestion:
1) To leave FOCS and STOC more or less as they are
2) To transform ITCS into a long event taking place once every two years or even once every *three* years, with, at the center, invited lectures with longer term perspectives (a few from neighboring sciences, perhaps) and perhaps a couple of workshops/tutorial round tables and *also* papers selected from submissions (with an attempt to genuinely emphasize conceptual papers). For those, various things could be implemented regarding presentation. (Although I would probably vote for a traditional 20 minutes two-section format which will occupy some of the time.)
Gil, there is an intuitive appeal to your suggestion which is essentially an “ICM for TCS”, though if it was up to me, I would not associate it with any particular conference and have the program consist exclusively of invited talks, as it is supposed to cover the best of the last X years, rather than the papers that happened to be submitted in the last 6 months.
However, the problem with this approach is that many (maybe most?) people feel that there are already too many demands on their travel time and so would find it hard to justify to attend it. Indeed, this is why Omer’s and my original suggestion ( https://windowsontheory.org/2014/10/08/focsstoc-protect-the-venue-reform-the-meeting/ ) was to merge STOC and FOCS into a single annual conference. However, there didn’t seem to be a lot of enthusiasm for this in the FOCS business meeting.
Hi Boaz,
ICM is not a good example since it is a huge event with hundreds of invited and plenary talks. What I am suggesting is a conference with 10-15 invited long talks, a couple of workshops and tutorials, some events like round table discussions and rump sessions, and, in addition, contributed talks, with the hope that it will turn into a true festival.
As a matter of fact my proposal is quite similar to what you guys are proposing with two differences
1) In addition to make the conference longer and add activities, it will take place only every 2 or 3 years and not every year
2) The changes will be implemented on ITCS and not on STOC
There are several reasons to implement such a change on ITCS and not on STOC. (One reason is that ITCS already have some of these proposed activities.) This may also help having the submitted and selected ITCS papers of different characteristic compared to FOCS/STOC, representing more longer-term and conceptual efforts, as was initially planned. (But this is left to be seen.)
The burden on the society is a serious concern. Both in terms of travel time and in term of committee and reviewing time. My proposal will actually reduce this burden (especially if we do it once every three years.)
One last remark:
“it is supposed to cover the best of the last X years”
I suppose that we see it eye in eye, but it is important to emphasize that for the invited lectures by the “best of X years” we need to balance the best ideas, best new directions, best results, best **speakers**, best balancing between, areas of research, geographic areas, gender, etc.
I agree with Stasys, and some other commenters that STOC/FOCS would be better if not used as a publication venue. I also agree with Michael Forbes that today, many of the talks are not as beneficial as they could be, and I believe this issue also stems from the use of STOC/FOCS as a publication venue.
Hence, ideally, I would like STOC/FOCS to consist only of plenary talks, workshops, and talks about works that were already published in some other venue. Something like a bigger Oberwolfach.
I understand that this kind of change is out-of-scope for the current discussion, so for now, I support having as many of the activities suggested in the post as possible, and as few published papers as possible.
I am also against shorter presentations, and agree with the commenters who said that 20 minutes is already too short. If possible, I would prefer presentations of 25-30 minutes.
Or, I sense a contradiction among your two views though it possibly you don’t see it. The proposal for shorter talks or posters is one way to implement your vision in the first para (i.e., to create enough time to have more plenary events). Another way would be to have 3 or more parallel sessions.
But you and several commenters have rightly picked out the nub of the problem: conferences are serving two goals, as a kind of journal and as a kind of meeting/learning place. We are trying to explore different points in that design space than the status quo.
I’m aligned with the opinions of Alex Lopez-Ortiz and Eric Vigoda. Alex suggests accepting more papers. This is the main way I know of to get a conference most people within a community will attend. When I look at all the conferences that I plan to attend in a year (because I have a paper) or the more specialized workshops/other venues I plan to go to, there’s unfortunately generally not room left over for travel to STOC or FOCS; as Alex points out, its rare that there would be enough papers in areas I’m interested in to move me from that inertia. (I recognize this is not true for everyone, but being selective in travel is an issue that arises for people with families, or other regular obligations.) Eric’s suggestion regarding colocating to bring more of the the theoretical computer science community together also make sense. I think we could take what we’ve learned from FCRC to make a more compelling version with a reasonable number of theory conferences.
I understand that there are other paths to conferences that are very compelling for people to attend (even if they don’t have a paper). There are multiple systems conferences (e.g., SIGCOMM) that accept a small number of papers but have many hundred attendees. Unfortunately, I’m not clear on how to reproduce that kind of buy-in from the theory community. I don’t think a host of plenary talks will do it. (Though I think a larger and longer conference, with more papers accepted, and a daily plenary talk is a workable system.)
That’s my opinion; it’s not clear to me how to get a representative idea of what people want, though. My take is that business meetings do not consist always consist of representative samples, so getting more opinions here is very worthwhile.
Since my blog post from ages ago was quoted, I think it’s time for me to chime in :). Boaz argues that STOC and FOCS are very successful in their intellectual impact on the field, and don’t need major changes. But of course the whole reason to have this discussion is that there may be places where STOC/FOCS could do better, most importantly in drawing a bigger crowd and (re)-taking their place as the “must attend” venues in theory.
First of all, some comments on growth in other venues. I’m most familiar with the data mining/learning venues (and just ran the PC for the SIAM Data Mining conference). NIPS, ICML and KDD have exploded in attendance over the past few years, fueled largely by the general interest in data science and the more specific interest in deep learning. NIPS and KDD are struggling to find venues big enough for the attendance, which is well over 1500 at this point. But this is with essentially the same format as the conferences have always had: namely, plenary talks, some spotlights, LOTS and LOTS of posters, and workshops, tutorials, demo sessions, panels and more.
So it’s not at all clear to me that format changes will automatically draw in more people. As I think Alex points out, drawing in more people requires more interest from people, not a changed format per se.
But format changes are beneficial in their own right. I fully support what Jack S said about the SoCG format changes. Having a diverse set of events really helped break up the monotony of talk after talk after talk. This is also true at the data mining venues, where the poster sessions and workshops/tutorials allow one to escape the talks as needed.
Also, having published often in data mining/ml venues, I really don’t understand the trepidation about posters/short talks/long talks, and suspect it’s a matter merely of getting used to a format change (if we decide to go that route). People get their papers into (say) NIPS and list it as such on their CV. If they are selected to give a spotlight presentation they mention that as well. But the marginal benefit of a spotlight presentation vs a regular presentation is not major except possibly when on the job market. Many many high impact NIPS papers were – well – just papers.
I do feel that there might be something to the idea that theory papers don’t lend themselves as well as to a “visual” metaphor. In a data mining poster there’s always some experiments to display. Having said that, there’s no shortage of hard core math in ML posters, and I think we underestimate how intuitive some of our papers could be if we just tried to render them on a poster: graphs ! points ! interactive proofs !
Apropos of that, here’s my post on the STOC 2012 poster session that someone had asked about: http://blog.geomblog.org/2012/06/extracurricular-events-at-stoc.html
While I was biased (being one of the organizers) I thought it was quite the success, in terms of the number of discussions at each stand. It’s exhausting, but fun.
Finally, a meta-request. If we do decide to make some format changes, it’s important that we do two things:
1. Promise to persist with the change for a few years. Any kind of change takes time to get used to, and every change feels weird and crazy till you get used to it, after which point it’s quite natural.
Case in point: STOC experimented one year with a two-tier committee, but there was no commitment to stick to the change for a few years, and I’m not sure what we learned at all from one data point (insert joke about theorists not knowing how to run experiments).
Another case in point: I’m really happy about the continued persistence with workshops/tutorials. It’s slowly becoming a standard part of STOC/FOCS, and that’s great.
2. Make a concerted effort to collect data about the changes. Generate surveys, and get people to answer them (not as hard as one might think). Collect data over a few years, and then put it all together to see how the community feels. In any discussion (including this one right here), there are always a few people with strong opinions who speak up, and the vast silent majority doesn’t really chip in. But surveys will reach a larger crowd, especially people who might be uncomfortable engaging in public.
In all the discussion, I neglected to answer Boaz’s original question: What would I LIKE to see at STOC 2017 in this new format ? That’s a good question. Here are some items, as well as some thoughts on things that have been suggested or appear in other communities.
* More tutorials and workshops on new and emerging areas. Especially tutorials on areas outside of STOC/FOCS theory.
* A “fast-forward” session to allow authors to highlight their papers (or posters) in a 1-2 minute slide. I ran such a session at SIAM Data Mining this year, and you’d be surprised how much you can say in 2 minutes. I’d do this day-by-day or even half-day by half-day, and would include tutorials/workshops as well as part of the fast forward.
* I’m ambivalent about open problem sessions, although I know some people like them. I’d rather that there be open times in the sessions where people can self-organize such events.
* More breaks in the day, and more diverse scheduling. So rather than X days of talks followed by Y days of tutorials/workshops, have (on each day) a half day of talks and a half day of other events. It really helps break the grind into easily digestible bits.
* While I personally am in favor of having a large majority of papers (i.e the default) being posters, and having carefully selected spotlight papers that reflect diversity and broad audience appeal, I can see that there’s a vocal group of people opposed to it. What I don’t know is if this vocal group is actually a significant fraction.
* while I don’t think this has ever been tried, I wonder if a panel discussion might be interesting. Something like “log shaving is for yaks” or “data structures are dead” or something equally controversial :).
Suresh, your first two points are a major design goal for the committee.
Let me just add a small remark from a young researcher with a family, working overseas (Europe).
A trip to the US for a conference is for me a big investment: in terms of own time and tiredness (jetlag, flight), in terms of toll taken by the family (already mentioned that family people are selective in travel), and also in terms of the fraction of group’s travel budget it consumes (it was a serious issue more than once).
This made last FOCS (2014) my first ever US conference, with the previous my “regular” conference being ICALP 2012. In the meantime, I attended multiple workshops and meetings (Dagstuhl, Bertinoro, and others) – generally trying to avoid every regular conference (by sending there coauthors) and bending the reality to attend every workshop I’m invited to. I just found the latter much more fruitful, and thus preferable in the presence of travel limitations.
If I were to treat STOC or FOCS an “attend-if-possible” meeting, I think it would need to significantly expand in terms of “other activities” (workshops, tutorials, rump sessions – all sound great) to balance against the big effort of travelling. I’d be very happy to have such a yearly event to meet people and see new things – the name “Theory Festival” appeals to me – but let me express a bold opinion that the current format makes it for me closer to an “avoid-if-possible” meeting, unfortunately. (Avoid attendance, not avoid sending papers there!)
I agree with Michael Forbes and others that there’s a reason STOC/FOCS talks are longer than 5 minutes. If they were shorter, they’d most likely be not understandable to anyone but the coauthors. So, I also strongly disagree with making the default talks shorter.
My suggestions:
1. Add a third parallel track and increase number of accepted papers by 30%. As Mike Mitzenmacher and others have said, the most compelling reason for attending is having an accepted paper.
2. Instead of having one day for tutorials and workshops, reserve that day for longer workshops (with series of talks) and have a tutorial session (maybe a couple of parallel hour-long talks) every day.
3. Definitely, more plenary talks and social activities. I like Suresh’s idea of having some open time for people to host self-organized events. For instance, Gil Kalai and Aram Harrow’s debate about quantum computing, or a behind-the-curtains look at the faculty hiring process for graduating students.
4. Raise more funds to support student travel.
I agree a lot with Arnab’s proposal’s. Here is mine.
***Do not decrease talk lengths***
Instead have 3 parallel sessions instead of 2 if necessary.
***Plenary sessions of invited talks/tutorials (like point 2 in Arnab’s proposal)***
– I would like to have like two invited talks/turorials every day (one in the morning and one after lunch). It is important that they are mixed with the normal program so that there is some variation.
– It is better if those talks are invited because if someone is going to speak for 50 minutes to broad TCS audience then it is preferable if the talk covers more than the very latest result.
– I still think it makes sense to have the best paper awards as a plenary session but maintain the length of those talks to 20-30 min.
***Keep the miniworkshops before the conference***
– I like them :).
***Social activity***
– Honestly, it is quite tiring to spend 3-4 days in a conference room (which often equals the basement without windows). A fun event one evening would also stimulate more interaction. If possible, it would therefore be good to have some conference dinner or event. It doesn’t need to be fancy… For example, last FOCS there was some inofficial house party which was great…
Thank you Suresh and Arnab. Am looking forward to see more comments from everyone on what kind of activities would induce YOU to attend the conference.
Would also be interested in hearing people’s views on the length of the conference – would you be willing/able to attend a 5 day conference if it contained these activities?
For me a 5 day event works if it’s not 5 days of talks. I.e a day of tutorials, 3 days of a mix of talks and other events, and then another day of other activities. For an example, look at the format for this year’s ICML (http://icml.cc/2015/). By day three or four of a talk, my brain is shut down anyway.
The people advocating increasing the number of accepted papers (which I support) hopefully also realize that if we do this, we need to go to either posters OR a much longer program OR more parallel sessions. Given the antipathy towards all three options, I’m not sure how this will work.
Again, some data. NIPS accepts upwards of 200 papers (and gets over 1100 submissions) and put the vast majority in poster sessions (partitioned over a few days). ICML accepts 150+ papers as well, but goes a different route with massively parallel sessions (last year had 6 parallel sessions). KDD accepts 150+ papers (159 this year out of 819) and does the ICML-style parallel tracks (3) but also does posters for all accepted papers.
These are all 5 day events.
Returning to the point about workshops and tutorials bringing people in, this is definitely true from the other venues I attend. I bump into MANY people who are just there for a workshop or something. BUT
these venues have a LOT of workshops and tutorials. Again, looking at the data
* NIPS 2014: 6 tutorials and >18 workshops
* ICML 2014: 6 tutorials and 17 workshops
* KDD 2014: 12 tutorials and 25 workshops
My point is that T/W can bring in a larger audience, but not unless there are MANY of them. So if we wish to go this route, we may want to commit to a multi-year experiment with growing the number of such events and carefully tracking attendance at these events (via registration: all these conferences allow for a la carte workshop/tutorial registration and these numbers are used to decide what topics are viable for future years when new proposals come in – I’ve been tutorials chair before)
Other conferences I am used to (ISIT, Allerton, INFOCOM) have a larger number of parallel sessions.
As others have suggested, an issue with theoretical work is it is often “narrow” and “deep” and hence perhaps most papers at a conference are of interest to a limited group of people. More parallel sessions (with sessions organized by topic) allow a greater chance that people will be interested in something. Ostensibly, if attendance growth is commensurate with the increased number of papers, average talk attendance will stay the same even with parallel sessions.
So I’m OK with more parallel sessions (and a slightly longer conference with more of various activities that have been discussed). I realize these are implications of accepting more papers.
I do think that when you move to a 5-day conference you have to accept that many attendees will not stay the whole conference; people often only stay 2-3 days. I think that’s OK as well but something to keep in mind for logistics and implications.
Michael: are these conferences (ISIT, Allerton, INFOCOM) for communities where these are also the top-tier publication venues?
Boaz —
No, but these conference has 6 parallel tracks (Allerton), 9 parallel tracks (ISIT last year), and I don’t know how many for INFOCOM these days.
I don’t think going to 3 parallel tracks will damage FOCS-STOC. And as some others have pointed out in there comments, there’s varying levels of opinion as to the nature of FOCS-STOC being “top-tier” publication venues for theory at large already (arguably, because of the limits on the number of accepted papers).
I also don’t see a problem with more parallel sessions. It will even be exciting to have several great sessions from which you have to choose only one!
An apology. I only read the initial post; reading all comments feels
way too imposing and I assume this will only be done by the committee.
I like some of the proposals, but find it very weird to discuss
quite significant changes to a venue without offering, as a basis,
an analysis as to the problems with the current way the venue operartes.
Indeed, how can one say that a specific suggestion is good or bad
without an analysis of what is currently bad and where one wants to go.
In my opinion, what is wrong about the current STOC/FOCS is
* too little interest in the actual contents of the venue; and
* too much preoccupation with the effect of the venue on one’s CV.
Note that I wrote “too little/much” rather than “decreasing/increasing”
because my point is not comparison to a “glorious” past but rather a
comparison to what is desirable. Indeed, unfortunately, currently,
any selection (of a program) has a side-effect of being used as
a measure by evaluation committees, but the question is what is
the balance between the actual interest in the program and the use
of the program for evaluation purposes. My claim is that currently
the balance is too much in the direction of the evaluation purposes,
which I view as a bad side-effect of the program selection
(“bad” — for reasons outlined at other places…).
Assuming that I am right, the question is what can be done about it.
One solution is to give up and cancel the venue altogether.
Advocates of this solution may say that although this venue served us
well in the past, it seizes to do so now, and there is no way to fix it.
I am not so pessimistic. Furthermore, I believe that one should try
to fix things before disposing of them…
My view is that changes in the format and operation of the venue
may effect a change in its nature. So I would value any change
that is directed towards increasing the actual interest in the program
and decreasing the preoccupation in its side-effect (i.e., measuring).
The obvious way of doing this is to smoothen the current accept/reject bit
into a spectrum of forms of presentation. This suggestion is based on
the observation that there is no quantum gap between the quality of most
of the works that are included in the program and the quality of many works
that are not included in it. (This point will be elaborated below.)
Hence, using a hard threshold that accepts some of these works while
not accepting works that are almost of the same quality, introduces
a discrepancy that does not exist in reality. Such an action does
a bad service to the audience (i.e., it misleads the audience),
and it has a distorting effect on the aforementioned evaluation activities.
The issue, of course, is with the non-accepted submissions that are almost
eliminated from the view of the audience, since future decisions of researchers
as to which papers to look at are biased by the inclusion bit.
What I suggest is to replace this single bit by many bits,
which allow to better qualify the opinion regarding the various works.
Let me elaborate on the spectrum of works that exists in reality.
When you sit on a PC, you see that, with the exception of
few papers that “must be included in the program no matter what”
and some papers that “should not be included in the program no matter what”,
the bulk of the submissions are quite close to the acceptance threshold
that is being used. Specifically, if a scale of 1-10 is used for scores,
where 5.5 is used as a threshold, then most submission (actually
at least 2/3 of the submission) will have a score between 4.5 and 6.5,
and many of them will have scores that are spread in the interval [5,6].
Let me stress that this claim refers to an ideal evaluation of the works;
that is, I am not referring here to the fact that the actual scores
deviate from the ideal value (“as random variables with mean equal to
the ideal score of the paper and a standard deviation determined by
the quality of the evaluator” [Baruch Awerbuch, circa 1985]).
Hence, accepting all papers with score above 5.5 and rejecting the others
introduces a huge distortion in the quality metric.
This distortion effects both the audience (which is given access to
the former works but not to the latter), future readers who may use
the program in order to learn what was highlighted by the PC,
and evaluation committees of the aforementioned type.
Note: Although the conferences are no longer used for dissemination
of the works themselves, they are used as advice towards what is
worthy of attention. Indeed, this advice is the actual contents
of the PC’s decision, which is widely assumed to be based on good
judgement that is well informed of all relevant information.
As hinted above, I suggest that the PC uses a “soft decision” rather
than a hard threshold. Rather than deciding whether or not to accept
a submission, the PC will decide whether to accept it, how much time
to allocate it, and at what forum. These decisions need not be based
solely on the quality of the work; they may be based on an estimate of
who may gain from hearing about the work and how much time is most
cost-effective for the communication of the work’s contents.
Options may include:
* an X-minute presentation in a plenary session,
where X is, say, in {30, 20, 10, 5};
* an X-minute presentation in one of Y parallel specialized sessions,
where X is as above and Y may be in {2, 3, 4, 5}.
There is no clear ranking between many of these 4 times 5 configurations;
the PC should just select what it thinks bests suits the submission.
In my opinion, such a format will serve the audience better than the
current one. In particular, it greatly reduces the aforementioned distortion,
allowing the audience access to a wider array of presentations
while providing a more clear advice about the contents of the talk
and offering a more cost-effective way of attending the conference.
An interesting suggestion, though note that changing the PC selection process is not part of our committee’s mandate.
1. While I like in theory the idea of posters, and it works very well in some ML/NLP conferences, I am yet to see this being successful in theory conferences. My impression was that in some of these conferences, the choice whether a paper is a poster/oral presentation is based on how much the audience would be interested and not on quality. In particular, the oral/poster partition is done after all the papers that get accepted are decided, and is done somewhat independently.
2. If STOC wants more people to participate, then STOC needs to accept more papers. This can be to some extent by having other venues, but some of the additional papers would have to be accepted to STOC itself. Accepting less papers has a positive impact on selectivity, but really hurt the community service purpose of a conference.
In the long run, nobody cares if a paper appeared in STOC or SODA or ESA, but rather about the result itself. Furthermore, PCs have a bad track record in identifying what are the important results. I think that being over selective is a greedy short term strategy – accepting more papers is somewhat hard intellectually (putting a strong result and a weak result on the same footing), but is the right strategy in the long run – see SODA and SoCG.
3. To some extent, this is too late – STOC/FOCS are very broad conferences, which means that they can accept very few papers in specific topics. Many people would prefer to go to their home conference (in my case, SoCG) than going to STOC, independent of what STOC would do.
4. One idea that I liked in the past, is having one day (or several sessions) with no parallel sessions (i.e., the “best” results in the conference), and the rest with a few (or many) parallel sessions.
All good points, especially (1).
As for (3), the hope is that it is not too late. I suppose we won’t find out until we try.
About people asking to increase the number of accepted papers. There is a solution for this problem that does not even entail increasing the acceptance rate:
Accept papers that are regarded as very good, based on their own merit, without comparing them to other submissions. This way you don’t really lower the standards of STOC at all nor its selectivity. You might even end up, in extreme cases, accepting less papers than usual.
Now, if we contend that STOC PC’s have done up to this point a good job in selecting the program, we must accept also that PC’s have the ability to sort out the good papers, even without comparison to other submissions (indeed, sub-reviewers are done independent of other submissions).
A similar change was done quite successfully a couple of years ago in IEEE LICS.
As someone working at the more experimental end of algorithms, I always feel a bit uneasy when people talk about TCS and the theory community. Maybe it’s because I don’t really know what theory is, or maybe people seem to be talking about theory as a category instead of a tag.
When I go to conferences, I prefer the small specialized ones. There I get to meet friends, a good fraction of the talks are both interesting and relevant, and I’m quite likely to come back with new ideas. The big algorithms conferences (ALGO and SODA) come next. They’re not as useful or relevant as the specialized conferences, but they’re interesting for their breadth.
On the third place come some bioinformatics conferences. Many good algorithmic results appear in them. Often those results would not be good theory results, because they concentrate on techniques that work well on real data, and because theoretical analysis is better at proving performance bounds than predicting actual performance.
Theory conferences come only after those bioinformatics conferences. While the topics are often interesting, the presentations are too technical, and relevant results rarely appear in them. My main bibliography has one FOCS paper, one STOC paper, and no ICALP papers from the last five years.
Therefore, speaking as a relative outsider to the theory community, my main wish is to make it explicit what is the theory community the new STOC is trying to attract. Is it the more inclusive theory-as-category community, where papers concentrating more on experiments and implementation techniques than proofs and bounds are welcome to STOC. Or is it the more exclusive theory-as-tag community, where theory and algorithms are considered separate though largely overlapping fields of research.
The tag one. “Theory-as-tag” is the audience here.
You raise good questions but changing the PC selection process is beyond the purview of this working group. It is possible that having a more inclusive event will change the PC’s perceptions over time of what a good theory paper is.
I support shorter talks. I strongly support adding a third track, to allow more talks and open up more time in the schedule.
I also support putting in place incentives for giving better talks, e.g., giving out 5 “best talk” awards. (Would these just go to the best technical results, or the most trendy areas? If that is a worry, then give out 10 best talk awards, so everyone is in the running.)
I don’t think more plenary talks is a priority. In practice, I don’t think these have worked so well. (I don’t know why. Giving a good plenary talk is extremely difficult.)
More social events are a very good idea. (A poster session counts a little bit.) Compared to the conferences I go to in other fields, the STOC/FOCS conferences are much less friendly and sociable. It is a real turnoff, a negative advertisement against the whole field.
Be careful about paying attention to this feedback. A lot of younger researchers seem to be happy to sit in a room for eight hours straight listening to a stream of (often poorly presented) 20-minute talks. But what is the value added there? You can read all the paper introductions yourself just as quickly. And you can often find longer, more in-depth presentations on YouTube. Making and renewing social connections is a much bigger added value for a conference, and right now the conference organizers seem to ignore this aspect completely. (And the tight scheduling in fact works against it.)
Workshops are also nice.
John (faculty)
For me personally, the best predictor of whether I attend STOC/FOCS is if I have a paper (and from the comments it seems like I’m not the only one). Given this to increase participation, I would support accepting more papers (SODA accepts more papers and gets a larger attendance). Personally, I’m fine with having a 3rd parallel session (again see SODA): I tend to not attend a lot of talks so having another session that I might skip talks in will not diminish my STOC experience.
Once I decide to attend STOC, having a varied 5 day program should convince me to stay longer: I really like the SoCG model!
Few other comments (some of which repeat things that have been mentioned in the comments above):
(1) I think having talks/workshop etc. on topics that are not well represented at STOC would be nice. Having more talks about open theory problem in more applied areas would be great. We need to invite folks who can speak both the theory and applied languages but finding such folks should not be an issue.
(2) I really like the ITCS model of having a PC member give an overview of why the papers in a given session were selected: I prefer this model to a succession of short presentation on their paper by the authors. In fact, I will find the following very useful in the 3 parallel session model– each session will start with the session chair giving their view on why the PC liked the papers in their session and then folks decide which talks to attend in that session. (In an ideal world, all the sessions chairs would give this 5 min presentation in the same location one after the other so everyone has a chance to get an overview of all the talks in all the parallel sessions but I suspect this will be a logistical nightmare.)
(3) Maybe once the STOC festival idea really takes off, folks will come to attend the conference irrespective of whether they have a paper or not. However, at least to get to that state, accepting more papers seems to be a good spark.
— Atri Rudra
I think a discussion of the number of papers to accept, and whether it should be higher or lower makes sense, but to some extent should be decoupled from the current discussion on organizing the event. Even if the number of papers is increased by 20%, which is quite significant, I am doubtful that this on its own would have a radical effect on attendance. And it could have a detrimental effect on both the cohesiveness of the program (in the sense of splitting into more parallel and specialized sessions) and the information in the signal that the selection provides to the community.
A more radical increase in the number of papers would likely necessitate making most of them posters or very short talks, which most people here seem to resist.
Note that the most sensible comparison points to STOC are other CS conferences that serve as the top tier publication venues of their respective subfields such as SOSP/OSDI, SIGCOMM, NIPS/ICML, SIGGRAPH, etc…
Some of these accept much fewer papers than STOC and/or have lower acceptance ratios. Others accept most papers as posters or very short presentations. I believe that for all a good fraction of the program is *not* just a sequence of 20 minute talks.
Hi Boaz,
I think Prasad and Mark B. expressed my opinion better on how we can get folks more opportunity to present at STOC. I have not been to any of SOSP/OSDI, SIGCOMM, NIPS/ICML, SIGGRAPH but I have been to SIGMOD/PODS a couple of times and they do have a tons of affiliated workshops (where unlike the the current STOC/FOCS workshops) that accept papers: thus, increasing the number of opportunities were folks can present their new research to attendees.
I also want to very strongly second Suresh’s suggestion on making the decisions data driven. In particular, I think surveys should be used to get a better sample of what things would encourage more folks to attend (instead of relying on feedback on blogs and/or the business meeting).
Finally, is there any data out there on the reasons why folks attend the conferences that have much larger attendance? That might help to design the survey to solicit things that might encourage more folks to attend.
People outside North America (like me) need >12 hours flights to get to most of STOC, Complexity, SoCG, Crypto, PODC, IPCO, EC etc. They essentially take place within a month and majority of them are in North Ametica, so most of people outside North America have to pick up only ONE (or two at best) conference for summer (cannot afford to go to more than two for many reasons).
In this case, people tend to pick up the “safe” choice (i.e., the best fit), and for many people, this would be a more specialized conference, but not STOC.
But if most of theory conferences in summer are co-located, this is different. Probably we would not mind staying for a week.
I was a local organizer of SODA 2012, and I was really proud to see one fact:
http://blog.geomblog.org/2012/01/soda-review-ii-business-meeting.html
“the attendance was quite high, I think around 350 people. And the splits were almost exactly 1/3 NA, 1/3 Europe, 1/3 Asia”.
This suggests that STOC could attract 200-250 or more people outside North America.
How many people outside North America show up STOC? 100 or less?
Indeed, the goal of the theory festival is to attract more theorists, which will necessarily mean more theorists from abroad.
Venue (esp. proximity to major international airport) is indeed an important consideration for that group.
I think that the only reliable approach to increase attendance at STOC/FOCS, is to give more people a chance to present their work in some form. Invited lectures and tutorials alone will not change people’s decisions whether to attend the conference.
Assuming the above fact is true, two approaches seem viable:
1) Co-locating with CCC/etc: While STOC/FOCS have low attendance, I get the feeling that conferences such as APPROX/RANDOM/CCC are doing much worse. By truly colocating these conferences, we might solve both issues. By truly colocating, I mean that the entire conference should only last 4 days, with several sessions in parallel. it should not be too twice as expensive to attend both the conferences.
While the above suggestion appears to be consolidating the audiences of various theory conferences together, it will also have the effect of making the event broader and more attractive thereby attracting more attendees.
2) Increase the number of acceptances at STOC:
As pan-theory conferences, STOC & FOCS should act as spotlights on the most important developments in each of the subareas of theory. Having a single longer list of accepted papers does not help this goal. For example, there would be no way for me to know which among the 6-7 accepted papers in (say game theory/crypto) are important, I might end up skipping all of them.
It would be much better to have 15-20 papers accepted for a 30 min plenary session.
The rest of the papers could be accepted for 20 min talks distributed over 3 or 4 parallel tracks.
I am not in favor of shorter talks or poster sessions, I feel theory might not lend itself well to these formats.
I share the opinion of Eric VIgoda.
Regarding the number of accepted papers I think that it would be good to significantly increase the number of accepted papers. This could be done gradually over a few years and I wouldn’t mind to have twice the number of papers. While this might decrease the average quality of the papers slightly, I don’t think the impact on the quality will be too big. The benefit of having more papers in a conference as broad as STOC will be to have more paper in one’s area of specialization.
I think that ESA/ALGO has a good model regarding co-locating conferences. One interesting idea is that one can submit rejected papers to more specialized co-located conferences (i.e. ESA -> WAOA), which will also get a good attendance.
I second the opinion of Ken-Ichi that there is a potential to attract more researchers from outside the US. For this purpose, a longer and bigger event would also be helpful. Travelling to the US for 5-10 interesting talks is not very attractive. Finally, I think the choice of conference locations also has an impact: I would probably attend the upcoming STOC if it was in NY or Boston, I might have attended, if it was in SF or LA and I will most likely not come to Portland. There is no direct flight, which adds another 3-4 hours of travelling due to the time one has to plan for immigration in addition to the connecting flight. Travelling 18+ hours to a three days event is not very attractive.
Christian Sohler
Christian (and Ken-ichi) makes a good point about location. Again, referring to related conferences (SIGMOD/VLDB/NIPS/ICML/KDD), they tend to be in large centrally located venues. NIPS has a SIGGRAPH-like “let’s stay in one place for a long time” format unlike the others, but that also works to create a “rhythm” and sense of regularity (much like with ITA, Allerton and other un-conferences that generate lots of attendance).
It’s not necessary to have a fixed location to achieve this, but keeping foreign visitors in mind when deciding where to host the conference is a good idea, rather than merely relying on whoever happens to bid. In particular, it would make sense to limit this event to large metro regions on either coast to make it easy for attendees from Europe/Asia.
I am not sure if how important is the attendance issue compared to other aspects regarding FOCS/STOC. From my view the main role of FOCS/STOC is as a platform to advance TOC/TCS research. There are many issues that can be discussed (and are intensively discussed over decades) regarding it, but the attendance issue does not look central. Still, some points about it:
1) A question: what was the number of participants in recent FOCS/STOC conference?
What is the reason of thinking there is a problem regarding attendance.
2) Lowering the registration costs (and other costs) is an obvious way to enhance attendance.
3) Allowing people to register for 1-2 days and pay registration proportional to the number of days they participate can make a difference in attendance of researchers especially (but not only) from the geographic area.
4) Since recently the lectures are videotaped anyway, participation via steaming (even for a modest registration fee) can be good for many people. Well, this is not equivalent to “real” participation but it can still be valuable.
Gil, I don’t have the data offhand but believe registration has been fairly flat recently. There isn’t any crisis in attendance, nor any other crisis. STOC would also be fine if it continues as is, but we want to think if it can be made even better. Increasing attendance is obviously a good thing but isn’t the only goal.
The conference is not designed to make a profit, and so decreasing registration fees will have to come at the expense of something. Note that for non local participants, registration is typically a small fraction of the overall cost to attend, and that well connected cities that are easy to get to are often also more expensive.
I think exploring ways to use technology such as streaming is very interesting, and would be happy to hear about people’s experience with conferences that had a virtual component. It may make sense for smaller theory conferences to try this first, maybe even trying a fully virtual conference.
Gil, to clarify: the stated goal of this process is to design a pan-theory festival that appeals to more of the (couple thousand) theorists who currently don’t attend STOC. This is not in response to any “crisis”, just a different goal for SIGACT (which is the organization of all theorists worldwide).
It is possible some people question whether this is a worthwhile goal.
Our working group has no official position on how many papers should be accepted for publication in the STOC proceedings. It may make sense to visit that issue as part of this planning, but we are not charged with that.
Our group is perfectly OK with the possibility that the community decides to not tinker with STOC at all. But we all concluded that to accomodate 90-100 talks of 20-min each we would need to switch to 3 parallel sessions instead of 2. There seems no other way to fit the pan-theory events in 5 days.
A few comments:
1) I tend to agree with Gil Kalai and a few others that advancing theory research through selection of papers has been the primary long-term benefit of STOC/FOCS. For this reason, one should be careful about changes aimed at making the event itself better that would interfere with the signaling function of the conference.
Even if one is unhappy about STOC/FOCS bearing this function, it is not productive to complain about it without suggesting a viable alternative. One could argue that this function should be offloaded to a journal. There is a subtlety here with respect to an ever-changing committee making the decisions vs a more-or-less fixed editorial board. In any case, until this happens, STOC is “stuck” with this selection/steering function and should respect it.
2) Creating variable-length talks, or any other distinctions, may have unintended consequences one should think about. For example, people do note things like “NIPS oral presentation, 3% acceptance rate” on their CVs. So more of the (noisy) ranking by the PC will get exposed this way. In addition, currently, your paper is either accepted or rejected. Sometimes very good papers get rejected, but you always have the opportunity to submit elsewhere or to a different (more appropriate?) venue. What are your options if your paper gets accepted but doesn’t get the “tier” you think it deserves?
3) The only sustainable way to get more attendees is to give many people the opportunity to present in some form. The AMS-MAA joint meeting is a great example of an event with 1000s of attendees and dozens of parallel contributed sessions. Some people suggested co-locating with smaller conferences, which sounds like a very good idea. Another idea is to have a mix of “archival” talks (the current STOC) running in 2 or 3 parallel sessions, and a more specialized “non-archival” track with very basic acceptance standards (or where sessions are organized by a person who invites the speakers like at the AMS or INFORMS events) running in 10+ parallel sessions (if need be).
A few comments:
1) I tend to agree with Gil Kalai and a few others that advancing theory research through selection of papers has been the primary long-term benefit of STOC/FOCS. For this reason, one should be careful about changes aimed at making the event itself better that would interfere with the signaling function of the conference.
Even if one is unhappy about STOC/FOCS bearing this function, it is not productive to complain about it without suggesting a viable alternative. One could argue that this function should be offloaded to a journal. There is a subtlety here with respect to an ever-changing committee making the decisions vs a more-or-less fixed editorial board. In any case, until this happens, STOC is “stuck” with this selection/steering function and should respect it.
2) Creating variable-length talks, or any other distinctions, may have unintended consequences one should think about. For example, people do note things like “NIPS oral presentation, 3% acceptance rate” on their CVs. So more of the (noisy) ranking by the PC will get exposed this way. In addition, currently, your paper is either accepted or rejected. Sometimes very good papers get rejected, but you always have the opportunity to submit elsewhere or to a different (more appropriate?) venue. What are your options if your paper gets accepted but doesn’t get the “tier” you think it deserves?
3) The only sustainable way to get more attendees is to give many people the opportunity to present in some form. The AMS-MAA joint meeting is a great example of an event with 1000s of attendees and dozens of parallel contributed sessions. Some people suggested co-locating with smaller conferences, which sounds like a very good idea. Another idea is to have a mix of “archival” talks (the current STOC) running in 2 or 3 parallel sessions, and a more specialized “non-archival” track with very basic acceptance standards (or where sessions are organized by a person who invites the speakers like at the AMS or INFORMS events) running in 10+ parallel sessions (if need be).
-Mark Braverman
Yes, it is a fair assumption that people will report all sorts of things on the CV (they currently do so too, eg invitations to the conference special issue) .
Conferences with no selectivity are uniformly uninteresting in my experience.
Some people asked about gathering data. In addition to the blog posts and business meeting discussion and votes, we do plan to run some surveys on theorists’ opinions about STOC as well as gather some information on nontheory conferences, including surveying our colleagues on what works and doesn’t in their conferences. (Our plan is to start small but we might possibly do larger scale surveys later.)
However, if anyone knows of any other data collection efforts on CS conferences, please do point them out here or by email to us.
I would suggest the following tentative scheme:
1. Make variable-length talks: say, 25 and 15 minutes;
2. Parallel sessions with longer talks will have fewer talks, and thus will supposedly attract on average less people. Accordingly, place these sessions in smaller rooms;
3. Allow the *authors* to select whether they would like a short talk (larger audience) or a long talk (smaller audience, but more time). (This might be based on a first-come, first-served scheme.)
To a certain extent this was done quite successfully in LICS ’12.
Let me add my thoughts.
*The event, no matter how extravagant, should not last more than 5 days (and very preferably no more than four). People just do not have the attention span and they get worn out. Also, 3 days of technical talks is an upper bound for me. For this same reason collocating conferences is better in theory than practice. Last time STOC and CCC were together in Cambridge, I was unable to get anything from the CCC talks after attending 3 days of STOC. I think I had to look at my driver’s license just to recall my name at the airport on the way home.
*I do not see the problem with adding more tracks to fit everything into four days (okay, I see the problem, but it seems superior to the alternatives). Conferences could also be collocated and mostly run simultaneously (thought this reduces many of the benefits, and few people really likes FCRC as far as I can tell.) Also, I would prefer less time scheduled each day rather than packing the schedule. Of course you can also NOT go to something. However, there is a coordination game/problem of when to find others not attending things, as well as some social pressure to attend certain events.
*The mixed model of plenary/tutorials/workshops/research paper talks on the same day sounds intriguing, though I have not experienced it myself.
*As far as WHAT to offer:
1) My favorite are tutorials that are taught by more than one person, who exposit a new area with tons of open problems waiting to be worked on. These programs should be as long as necessary so that by the time people leave they could actually work on the new problems right away. For some areas, this means one hour, but for many, realistically four hours of presentations. (This could be spread across 2-3 days as well). Ideally there are related breaks/social for the people attending so that they are not completely passive and people can get clued in to parts they did not understand.
2) I liked the poster sessions from the past. It is a real pain for the presenters, but I can actually spend the time to understand things (or leave as soon as I realize I am no longer interested/following). However, people are not going to come for a poster session.
I am not a big fan of traditional open problem sessions. People usually keep the best ones for themselves (and I do not blame them).
If a workshop (which is often just a series of technical results) is competing with a STOC technical track, I would usually want to go to the papers that actually got into conference. However, it is nice if you know that a critical mass of researchers on a particular topic will be at the conference (due to the workshop). Also, as people point out, this is a way of increasing the number of people with papers at STOC without increasing the number of papers accepted at STOC.
—
I agree with a lot that has been said. STOC/FOCS seem to be very successful publication venues, but only a moderately successful conference qua conferencing. There may be a bit of a trade-off here due to all the issues people have raised. In this case, the question is: are we at the right equilibrium (say in number of papers)? Of course you are giving something up by accepting more, but do you get more in return, or should these venues become even more selective—only the big major results that everyone really needs to know about?
I think it is telling that few people have offered suggestions on what would make them attend STOC besides have a paper there or at least having more papers in the area there. It seems to suggest either 1) a lack of imagination or 2) that people don’t really want these add-ons, or at least do not see it affecting their decision to come.
Thanks to everybody for their comments. Pls keep them coming!
Though the following is not something the committee is studying, I’d still welcome thoughts on the following.
Is it true that STOC/FOCS are doing a great job in identifying “the best” work in theory, whatever that means?
Young folks are increasingly reporting that even these days when CS is expanding everywhere, a CV with N STOC/FOCS papers is no longer sufficient (nor necessary) to get a job, or even to get many interviews.
PS. The reason I bring this up is because when I talk to random theorists around the world, many feel estranged from STOC/FOCS. Part of the goal of the proposed festival is to bring some of these theorists back into a single event. A flagship event for SIGACT, which arguably is a world organization devoted to theoretical CS.
pps This should also clarify to those who were earlier asking “Is there a crisis in attendance?” I believe that is not the point here.
The goal of this process is to have a more inclusive pan-theory event. Feel free to write about whether you think that is not a worthy goal.
I was intrigued by the earlier suggestion to have ITCS be the vehicle for this experiment, but I didn’t see the logic behind it. Currently that has an attendance of 100 or so; STOC seems a more reasonable starting point.
Dear Sunjeev, the main logic for experimenting on ITCS is that both the risk of failure and resulting damages are higher for STOC compared to ITCS. Also given the amount of competing activities of various nature in TOC and related fields, a festival once every two or three years seems like a smaller burden on the community and has better chance to be successful and perhaps even enjoyable.
Another possibility, however, will be to turn STOC into a longer festival but not every year but rather every three years and to leave the format as it is in the other years.
“Is it true that STOC/FOCS are doing a great job in identifying “the best” work in theory, whatever that means? ”
Since they presently are not covering all of theory, the answer is clearly not. To start with papers in some areas aren’t even submitted, and even within the areas that traditionally appear in STOC/FOCS there is a strong emphasis on technical difficulty over all other parameters.
There isn’t a crisis in STOC/FOCS at all. What we have is a slow erosion of their role as flagship conferences and selector of worthwhile papers. They have long ago stopped being the must-go conferences they were in the 80s and early 90s. In terms of selection they are now splitting hairs and leaving behind equally good papers on non-quality components.
JACM went through the same process. Many well meaning editors tried to stem this flow over the last two decades but they were all hampered by their timidity. Recently JACM was officially stripped of the title of flagship journal.
Given the cautiousness and reticence to change I fear this is exactly where STOC/FOCS will find themselves ten years from now.
Many people have raised good points. I try to make it shore and to the point:
1- I think a very short talk (shorter than 15-20 minutes) will be useless to almost everybody. I strongly prefer to have more parallel sessions than short talks. I
2- I support the idea of more papers accepted. That directly will increase the number of people attending. There is less randomness IMO in terms of quality of a paper submitted vs a non-theory conference. Hence there is a bigger self-screening done by the authors, that’s why the number of submissions to these conferences (STOC/FOCS) is lower vs big non-theory conferences. I believe if the number of accepted papers is increased we will see a proportionally larger number of submissions (like SODA). Hence the acceptance rate won’t increase (at least significantly).
3- I think we should try to engage/interact with other areas perhaps by inviting (more often) people who are not directly doing theory as invited speakers.
I think one problem here is that some of the youngsters here have no idea how much has already been lost. They also lack the historical examples to know how things could have been otherwise.
For example, those of us who are old enough, academically speaking, to have attended STOC/FOCS in the late 80s/early 90s know how much ground has been lost in the “must attend” status of STOC/FOCS.
We are also familiar with, for example, the continued rise of SODA in quality and number of participants all while steadily increasing the number of accepted papers.
On the flip side, we’ve seen the fall in JACM where tepid reforms proved too little too late.
The STOC/FOCS I and others envision are still the elite, exacting conferences in the field, impossibly hard to get a paper into, but large enough to cover *all* areas of TCS with a reasonable number of papers in each and hence once again the must-attend conferences in the field.
Some researchers may lack the imagination to envision a quality conference with more accepted papers, in spite of no shortage of examples of larger conferences with rather exacting acceptance thresholds, both within and without TCS.
In fact, just by doing simple math we can see that if we were to slowly bring back logic, geometry, topology, data structures, experimental algorithms, machine learning, distributed computing, formal languages, and crypto with six papers (give or take) each we need 50 or so new slots. Adding another 20 slots to traditional areas seems also rather reasonable.
And bingo!, we just doubled the size of STOC/FOCS without giving up *an* *inch* in quality.
In addition, allow me to quote a Harvard management guru to say that when a restaurant is failing you don’t ask the current customers what is wrong with it. If they are *in* the restaurant clearly there is nothing wrong with it as far as they are concerned. You need to ask the people *outside* the restaurant why they are not coming in.
Along the same vein, top notch researchers working in well represented areas in STOC/FOCS (thus routinely publishing in them) will have a hard time understanding how broken they are for the people who are not submitting/attending the conference today. I hope they keep this in mind when comparing their judgments with those of others here and elsewhere
Amen.
Dear Alex, “There is a strong emphasis on technical difficulty over all other parameters”
I dont think it is just technical difficulty but also mathematical depth and quality. Having (frequently) cutting-edge mathematical level is quite unique for STOC/FOCS and surrounding subcommunities.
“If we were to slowly bring back logic, geometry, topology, data structures, experimental algorithms, machine learning, distributed computing, formal languages, and crypto with six papers (give or take) each we need 50 or so new slots.”
This proposal is problematic. The way STOC/FOCS emphasized the interest on some major directions while quickly abandoning (fully or partially) others is one of the strength of these events and this community.
“The way STOC/FOCS emphasized the interest on some major directions while quickly abandoning (fully or partially) others is one of the strength of these events and this community.”
Your proposal is interesting. Make STOC/FOCS the home for mathematically sophisticated arguments, as opposed to the home of quality theoretical computer science.
Alex, I did not propose anything like that. In fact, I talked in the same comment about two different things. One comment was about technical difficulty, I think what you referred to as “technical difficulty” refers also to mathematical depth and quality. ToC is to a large extent a mathematical subject and the mathematical quality is a criterion. And mathematical quality does not refer only to sophistication. Often you do need to build a large mathematical infrastructure to push the theory forward.
The tendency to move quickly from one front to the other is a different issue and it has a lot to do with quality/relevance for theoretical computer science (in the eyes of the community/committees). There are mathematically deep and sophisticated areas within TCS which were also (to a large extent) abandoned.
Then I don’t get your point. If you are trying to argue that technical difficulty can be valuable then this is obviously so. The issue is that it seems to have become a necessary condition to get into STOC/FOCS, rather than merely a sufficient condition.
As it stands, if someone were to find a simple arithmetic based proof to a well known problem it would likely be rejected from STOC/FOCS.
For example, judging from recent instances, I’m pretty sure the typical STOC/FOCS reviewer would comment on the simple methods used in the proof of Goedel’s incompleteness theorem while completely glossing over the fact that it resolves a long standing open question which had been addressed by many.
Alex:
I don’t think that mathematical difficulty is a necessary condition for a paper to be accepted into STOC/FOCS. In fact, there are many STOC/FOCS papers whose main claim to fame is that they give a simpler proof for a known result.
More generally I don’t agree with your claim that there are so many indistinguishable submissions that the decisions rely on second order effects such as “who has the most complicated proof or talks about the hot fashionable topic … or has a funny title”. I don’t think this is the experience of most people that served on a STOC/FOCS program committee.
Again, I believe an argument might be made to mildly increase the number of accepted papers. I just don’t think this is the argument.
Personally, I think the program can be made more diverse by using, for example, 10 talks slots to invite presentations from other theory conferences than by using those slots to accept more STOC submissions. The problem is that, as you mentioned, there are some areas of theory where many people don’t submit their paper to STOC/FOCS and the program committee cannot accept papers that were not submitted. I don’t see a reason why increasing the number of accepted papers will automatically change the distribution of research areas represented in the conferences.
” In fact, there are many STOC/FOCS papers whose main claim to fame is that they give a simpler proof for a known result. ”
Many? Merely a handful and only in areas of intense coverage in STOC/FOCS.
“More generally I don’t agree with your claim that there are so many indistinguishable submissions that the decisions rely on second order effects such as “who has the most complicated proof or talks about the hot fashionable topic … or has a funny title”. I don’t think this is the experience of most people that served on a STOC/FOCS program committee. ”
This is not what one hears in the street. Let me take this opportunity to repeat a comment I made earlier:
Top notch researchers working in well represented areas in STOC/FOCS (thus routinely publishing in them) will have a hard time understanding how broken they are for the people who are not submitting/attending the conference today. I hope they keep this in mind when comparing their judgments with those of others here and elsewhere
“I don’t see a reason why increasing the number of accepted papers will automatically change the distribution of research areas represented in the conferences.”
Oh I agree. I’ve suggested elsewhere that we reserve up-to-five slots for each of certain areas for a few years, to attract areas that are not currently submitting.
Notice the “up-to-five” part: if people do not submit good papers we are under no obligation to accept. What this means only is that excellent papers in those areas wouldn’t be fighting for a slot with excellent papers in other areas, for a limited time.
Lastly let me remind you that by not changing the number of accepted papers you *are* changing STOC/FOCS. They are already very different beasts than what they used to be 25 years ago. They’ve lost their flagship status, they’ve lost coverage of many areas that used to be covered, they gave way for SODA/ALGO to emerge as the go-to conferences in the field. So if sitting on your hands sounds like the safe an non-revolutionary thing, this is merely an illusion.
The steady, non-revolutionary thing would be for STOC/FOCS to grow along with the field, i.e. a small percentage increase in papers every year for each of the last 25 years i.e. roughly doubling the number of papers today to make up for years of neglect.
No single action will bring STOC/FOCS back to flagship status. We need more workshops, we need collocation, we need more plenary talks and we need more papers accepted. Unless we do all four we’ll end up with a tepid JACM-style reform which looked radical to timid souls at the time but had no effect in stemming the decline of that journal (I’m sorry to pick at JACM, but in a way we are fortunate to have such a relevant, recent example of what happens when an academic venue fails to evolve in consonance with the times).
Alex:
FWIW, from my (admittedly, limited) experience on STOC/FOCS PCs, something along the lines of the “designed slot” policy you are talking about is being implemented.
Specifically, in the PCs I was on, the chair put a lot of emphasis on identifying submissions from less “mainstream” subareas and ensuring that:
(a) we find subreviewers from that exact subarea, so as they are able to appreciate and understand the contributions.
(b) the paper does not fall through the cracks during the final discussions and avoids getting discarded just to make room for another “mainstream” submission.
In fact, my experience was that the main reason such less “mainstream” submissions get rejected is simply because the subreviewers from the relevant area judge them to be “below STOC/FOCS quality” themselves.
So, either these subcommunities are too harsh on themselves (and there will always be a problem for them to get appreciation they deserve in any broader context), or indeed, as Boaz said, the best results from that subcommunities do not even get submitted to STOC/FOCS.
Clearly, there is not much we can do about the former, but we certainly can (and should) try to remedy the latter. In particular, making the “designed slot” policy you mention more official (as opposed to communicating it only internally to the PC) would probably be a good idea.
Still, I would prefer if you stopped reiterating so vehemently the picture of mythical STOC/FOCS “elites” that closed themselves off in an ivory tower, worshiping the Goddess of “technical sophistication”.
Whatever your personal feelings about the whole issue are, you can’t deny that there is a genuine effort here to improve the way STOC/FOCS serves our *whole* community. Even if the measures that were implemented so far might indeed have not been optimal or sufficient.
Clearly, you deeply care about this issue, which is something I very much respect and appreciate. Let us focus, however, on being as scientific and constructive as possible here and avoid voicing our more emotional opinions.
Looking forward to continuing the (mostly) constructive discussion.
Maybe those comments came across the wrong way. I do not see successful STOC/FOCS researchers as an “elite”.
All I’m pointing out is that when going over a bump in a car it feels different to a passenger than to the driver, and that this is something we should keep in mind.
Similarly for the case of STOC/FOCS the experience feels different for areas that are well covered than they do for areas that are not covered at all (e.g. logic). There is no insularity, dark conspiracy, or anything else implied in that rather factual observation.
Coming partly from Computer Science and partly from another field (Physics), I feel that part of the problem with FOCS/STOC is the assumption that there is a one-to-one correspondence between good talks and good papers. At other conferences I attend, a good talk can be a review of several recent papers, explaining a new technique; at the other end, a talk could describe a half-baked idea or open question which hasn’t yet turned into a paper, but which the organizers believe is worthy of pursuit.
One approach would be for each submission to be a proposal for a talk: a short abstract describing what the talk will cover, with an argument for its timeliness and relevance, and its relation with current, recent, and future work. I see no need to assume that ideas only deserve a talk when they have become publishable results (or that all publishable results deserve a talk).
Generalizing the definition of STOC talks in this way could make it the best possible kind of conference: namely, one where you learn something from every talk, and where each talk presents new ideas and directions for the community to go.
Distinguishing paper vs talks is indeed something that several of the proposals are trying to do.
STOC plays dual roles of “journal of record” and “place to socialize and learn about interesting new research.” Most people seem to want to retain the former, but maybe it is time to think of ways to improve the 2nd function.
I am quite happy with my amended proposal following the exchange with Sunjeev above, so let me repeat, elaborate, and try to justify.
1) STOC2017 is going to be transformed into a large pan-theory festival. With the intention (if successful) to transform every third STOC into such a large-scale event, leaving the other STOCs in their current format (including the two parallel session format).
2) The event will be a six day event Saturday-Thursday, where, like now, Saturday will be devoted to workshops and the full scale activities will start on Sunday.
3) There will be 90 or a little more 20-minute lectures of accepted papers in three parallel sessions. (Like today some best papers will have a presentation to the entire audience.)
(A small increase of the number of paper accepted for these three-years SUPER-STOCS and perhaps an informal more-openness-policy may be considered,)
4) There will be additional workshops, tutorials, discussion-sessions, along the days as well as 10 or so plenary lectures. There will be an effort to bring some major development in ToC of the last 5-10 years, with careful efforts about the scientific quality, the quality of presentation, the balance and openness within TCS and towards neighboring sciences.
5) Attempts will be made to make this flag-ship 2017 ToC super-STOC event (or events if continued) accessibly financially and subsidized.
The are many reasons for having such event only every three years and leaving the other STOCS as they are. A yearly or bi-yearly such event are excessive and unjustified from the scientific perspective, they are excessive from the point of view of burden, on travel-time, and organizational burden. With this format I see higher chances to increase interest and attendance, not to harm traditional STOC role, get perhaps financial support, and leave organizational effort tolerable. (Still organizing such a superSTOC will be very difficult.)
It is an interesting idea to hold the festival only every second or third year and we’ll consider it.
However, is cost such a big factor? Hotel costs for a 5-day conference should only be 25% more than the current 4-day version. Airfare is the same.
(The organizational burden is indeed higher, but many communities seem to find it worthwhile.)
The added cost of an extra day is indeed not so large. My suggestion was to make efforts to lower costs and assist participants financially in a dramatic way through external sources (as done in many, more localized, ToC events). This can make a large difference.
First of all, I’d like to state the (for me) obvious: I think it’s great that this discussion is happening, that there is a willingness to take a risk and experiment so as to make our conferences better and more relevant. This is needed but it takes effort to make the plunge and take the risk. As someone pointed out, a nudge, whatever form and direction it takes, may be the only thing needed to jump-start a productive cycle for finding a new local optimum.
There has been much discussion of acceptance rate and talk format, mostly directed to points 1. and 2. in the original blog post. We should not forget 3. and 4. The main conferene in my area, QIP, has about ~30 accepted talks, and ~400 attendance. It rotates around the world, and most in the community try to attend. It last 5 days, has a single track, a few invited talks, and social activities. Why does every grad student dream of attending QIP every year, when the best they could hope for is often just to present a poster?
The obvious answer, I think, is that because this is the major event of the year, and “everyone will be there”. This is already one problem with STOC/FOCS: well, it’s “STOC/FOCS”; there are two of them. Which one to attend? So the community is split, there is a choice to be made, and that choice might resolve itself into not attending any of the two conferences and going for a more specialized venue instead. This is especially so for people coming from abroad who might not have a paper in: if there was a single event they could attend and be guaranteed that, not only the “top results”, but also most of their colleagues, would be, they would go, but this is currently not the case. In this respect the idea of differentiating STOC and FOCS, singling out one major event, is already a step in the good direction.
Many comments have pointed out the importance of conferences for networking. I think STOC and FOCS could do a much better job at this. Currently there is simply not enough time: too many talks, too little space. Whether we admit it or not an important reason for going to a conference, especially for the younger crowd, is simply that it’ll be “fun”. We really ought to put much more thought into the social aspects of conferences: conference dinner, rump session, cocktail reception, excursion, etc, I believe all of these are much more important than they seem. This is when talks get digested, ideas exchanged, contacts formed. In my opinion a good conference should not only expose me to great results and ideas, but also, and more importantly, give me the opportunity to exchange on those results and ideas with the community. At QIP I get plenty of time to do this; at STOC and FOCS I spend most of my time running around and gathering small packets of information that unfortunately tend to stay there, and not have time enough to mature.
I don’t have any specific suggestions; I think we should do just what is planned: experiment. But I wanted to emphasize the importance of social/networking activities, and perhaps others will have more concrete suggestions.
I agree with Thomas that increasing social/networking activities would make the conferences much more welcome. I’m not sure if it is just me getting old but even in the past 7 years I think there’s a decline in this aspect.
For instance, I really liked the Invited Tutorial/Talks organized in STOC 2008 (my first STOC/FOCS) which were somehow more informal and welcoming for a beginning graduate student (happening late at night for one helped).
Hi Boaz and everyone,
I totally agree. A change is needed – more social, more risky, perhaps once a year rather than twice, by making it a longer conference…
Main point is that it should be far more interactive and try to bridge the gaps between subfields, and avoid the specialization and the highly technical focus of papers, to allow people from different fields to interact.
I personally do not mind several sessions for more specialized talks, but I think there should be a blend of talks, with some good portion of the talks longer (half an hour?) and a significantly greater number of perspective/inspirational/general talks; and there should be more time to digest and discuss and interact socially.
One could also think of focused sessions (half days?) oriented at creating connections between two or more subareas, which might seem disconnected at least in terms of their communities (SoS and entanglement comes to mind as an example…)
The ICM in india in which there were many parallel sessions but also great public talks and many social events is a conference I would try to get inspiration from – as a conference that managed to beautifully bypass the difficulties of a huge crowd with many different areas of interest, and create a sense of a community with thousands (!) of people.
Good luck! I think this is of true importance to our field.
Dorit
First of all, thank you for starting this whole discussion. Our conference ecosystem is indeed needing a major overhaul. In particular, I feel that without a single “must go” conference we will slowly fall apart as a community (we can see it happening already) and by doing so we will lose our greatest strength: ease to make unexpected connections between seemingly unrelated concepts from different fields.
However, looking at the whole discussion, I can’t help but think that by asking explicitly about changing STOC, you unnecessarily narrowed the whole issue we need to tackle here. In particular, this prompted many people to discuss the issue of STOC accepting more papers and how good (or bad) job it is doing in representing the “best” work of TCS community. This issue is, of course, extremely important and very related. But, despite what some people seem to suggest, accepting more papers alone will not fix it and clearly there are many opposing views here.
This does not mean that we should not discuss these issues. To the contrary. However, it might be worthwhile to devote part of our attention to some other possible measures that we could take. Hopefully, ones that will seem less controversial and easier to implement.
In particular, in my opinion, one of the major reasons why STOC is becoming less and less of “must go” conference is FOCS. (And vice versa.) After all, SIGCOMM, SOSP, or SIGGRAPH are the “most go” conferences they are partly because it is clear to everyone that they are *the* conferences in their field. (And this is despite them accepting relatively few papers.) This is obviously not the case for us and having two *the* conferences is simply not working out for us, at least from the point of view of logistics.
Personally, I made a point of going to all STOC/FOCS (and US-based SODA) conferences since I started grad school. Even when I was in Europe and it meant having to fly to such “well-connected” cities like Portland or New Orleans. (This is another thing that should change if we want to have more European participants.) However, each time I did that it was a major commitment of time and effort (not to mention, money) and, have I stayed in Europe, I don’t know how long I would be able to keep it going. (So, certainly I can’t blame anyone who is not able to do that.)
However, in the age of arxiv and Internet, why do we even need to still have two “flagship” conferences a year? I remember that some time ago there was an idea of “merging” STOC and FOCS floated. Namely, as far as I remember, the idea was to keep both “brands” with their own PCs, submission deadline and such, as is. But hold only one physical meeting per year where all the accepted papers would be presented.
Yes, this would require some logistical concessions, like making the meeting last five-six days, introducing (more) parallel sessions, etc. But, I would say that the total logistic effort for organizing *two* sperate meetings in two different cities is much more of a burden. Both for the organizer and the attendees.
Also, it would have an added benefit that this change is orthogonal to the contentious issue of “should STOC/FOCS accept more papers/different papers”. So, we could implement it already now, while having time to work out the latter issue with the care and attention it deserves.
I do not remember now why this idea of merging STOC and FOCS did not get traction back then, but would be glad to revisit it now. It certainly would be a good starting point for a “theory festival”. Don’t you think?
Hi Aleks,
SIGCOMM and SOSP are indeed good examples of “the” conferences in the field (SIGGRAPH is perhaps more complicated, as SIGGRAPH Asia is now gaining in prominence). This said, there are also very successful communities with multiple top conferences, such as machine learning (NIPS and ICML). I think this all depends on the size and preferences of the community. We will have to figure out what works best for us.
Piotr:
Yes, ML community seems to be successful in its maintaining of two flagship conferences and we should try to learn from them.
However, is it really the case that most ML people attend *both* ICML and NIPS each year (even if they do not have papers in either one of them)?
I am checking the facts. After all, ML community is very big, so even if people tended to choose only one of the meetings each year, both meetings would end up being very successful.
Also, as far as I know, there are slight but noticeable differences in flavor of NIPS and ICML, which is not the case for STOC/FOCS. So, maybe this slight differences is another thing allowing these two conferences to thrive alongside each other.
Thanks again to everyone that commented:
Thomas/Aleksander: Indeed a “soft merge” of STOC and FOCS was Omer’s and my original suggestion, but there seemed to be a lot of resistance to it in the FOCS business meeting. Perhaps it makes sense to first experiment with a larger meeting and perhaps if it’s successful then people would eventually be more open to merging the conferences.
Thomas: I agree that the “non talk” portions of the conference are very important, though they are often the first to go when one needs to schedule a lot of talks in a short amount of time. I do hope that we’ll be able to keep this from happening.
Dorit: I completely agree that one of the main advantages of a broad theory conference such as STOC is to connect different areas. A person doing quantum computing would always find more talks in their area at QIP, and similarly for complexity theorist at CCC, an algorithms person at SODA, a cryptographer at CRYPTO etc. etc. So, you come to STOC mostly to connect with theorists in *other* areas, but this is of crucial importance: much of the progress in our field has come by making connections and porting ideas between different areas.
Thanks, Boaz. I found now the relevant blog entry (https://windowsontheory.org/2014/10/08/focsstoc-protect-the-venue-reform-the-meeting/) that discusses this proposal.
Frankly, skimming the comments there, the sentiments expressed seem very similar to what we can see here. So, it is not clear to me that you gained much by making your proposal (even) less radical 🙂
In fact, one important point that Alex made is that we really *need* change. One way or the other. By not doing anything we are only losing ground.
The other proposal was made more than a half year ago (and certainly I remember other similar discussions happening much earlier than that). However, after everyone voiced they opinions, there was no follow up.
I must admit that I find this a bit disappointing. After all, given how important this issue is to our community, I would expect SIGACT (or a similar body) to take concrete actions by now. For example, by appointing a committee that examines this issue in a careful and systematic manner (including making a community-wide poll) and then recommends concrete measures. I believe that a great example of how such an important community-wide change should be performed is the SoCG community’s handling of leaving ACM sponsorship.
Needless to say, as much as I appreciate the initiative of Boaz et al., having everyone vent out in blog comments, throwing anecdotical evidence at each other and voicing their personal – obviously, very subjective – opinions, will not get us anywhere. (In particular, that’s why I initially tried to stay away from this discussion.)
Hi Aleksandr,
In case it wasn’t clear, Sanjeev, Piotr, Omer, Tim and I were appointed to do exactly what you ask for. Study the issue, including the opinions of theorists and what happens in other conferences, and come up with a concrete suggestion for turning STOC 2017 into a theory festival.
I hope that if this is successful then in the future other conferences, including FOCS and others, would also join this festival.
Oops, my fault. In the flurry of comments that appeared since the original post I somehow filtered out this important information. Happy to hear that my disappointment was unwarranted though 🙂
Still, beyond this blog post and the discussion during the STOC 2015 business meeting, do you plan to use some other channels for reaching out to the community?
Of course, having these is very useful but, as Alex pointed out, soliciting input during a STOC business meeting carries an inherent bias. The same is true (for different reasons) about asking for input via blog post comments.
I am wondering: maybe we could try to solicit input during business meetings of SoCG/SODA/LICS/PODS/CRYPTO (i.e., all the conferences we feel are moving away from STOC/FOCS)?
In addition to this (and possibly future) blog post and the business meeting discussion, we are also reaching out to a small number of both theory and non theory colleagues to get some qualitative responses. We are also considering doing a larger scale survey, potentially of all SIGACT members, which include many theory researchers who are by no means STOC/FOCS regulars.
Since Joan Feigenbaum doesn’t generally look at blogs, on her behalf let me point you to her writeup of her experience as STOC 13 Chair, http://www.cs.yale.edu/homes/jf/STOC13FinalThoughts.pdf , which discusses several of the issues that have arisen here (as well as others) from her point of view as chair.
Thanks Michael, this is a great link. Indeed several of Joan’s proposed changes on section 3 in the document you linked to are along the lines of the things we’ve been considering.
Since input from the broader community was explicitly solicited, I’ll contribute my current view.
As has been observed by others, the current form of STOC/FOCS is a holdover from a time when sixty-odd papers really represented the work of note in the past six months in theoretical computer science, and when twenty-minute talks were enough to explain results to a general audience. Neither of these conditions has held for many years now, and in particular, I cannot recall anyone else I have talked to actually liking the twenty-minute format. STOC/FOCS currently serves as an awkward compromise between this older model and a spotlight on the best* work in the last six months. As has also been observed, it is not serving either of these purposes particularly well, and different models would serve these different purposes much better.
For the spotlight role, I mostly concur with an earlier suggestion by Prasad Raghavendra: (in my words) a committee should pick a reasonable number of papers, say 10-20, that the committee feels that the entire community should hear about, and give these authors the necessary time to explain their results to a general audience. This roughly corresponds to a 30-45 minute plenary session.
In order for the conference to publish all of the work of note within a reasonable length conference, we would either need many tracks, or else we would need to turn to posters, possibly backed by short (say, five-minute) “advertisement” talks. I politely disagree with the many voices who said that five minute talks would not be understandable. This is like saying that the abstract of a FOCS/STOC paper could not be understandable because we already
have trouble understanding the ten page conference versions.
I recognize that some authors are concerned about “devaluing” the unit of a STOC/FOCS paper. Continuing to provide the standard unit of a “STOC/FOCS paper” is a third role, distinct from the two above. The selection of an arbitrary subset of the submissions of about the same size could continue; a set of twenty-minute talks could even be provided for the masochistic. (It’s none of my business what other consenting adults do amongst themselves.) But, I agree with the sentiment that we currently do not have venues that serve the first two roles above, and it would be nice to see some part of this theory festival play one or both of these roles, whatever happens to the third role.
“Continuing to provide the standard unit of a “STOC/FOCS paper” is a third role, distinct from the two above. ”
Just to point out that the “standard unit of STOC/FOCS paper” is actually rapidly changing in value. It is at least an order of magnitude more difficult to publish in STOC/FOCS today than it was in 1985. I recall “back in the good ol’ days” (TM) where all you needed was a rather good result and you knew it would make it into STOC/FOCS for sure. Then we started running out of room and good papers started being rejected en masse in areas that didn’t have enough champions in the PC committee. Being good was no longer enough. You needed extra sauce such as topicality and technical difficulty.
COLT, SoCG and SODA came out as part of this realization: to create fora for perfectly fine papers that were no longer being accepted in STOC/FOCS.
Around this time we had the switch from “it solves P=NP *and* the proof is simple!” (i.e. a good thing) to “it solves P=NP *but* the proof is simple” (i.e. a flaw).
I am in agreement with most of what Prasad Raghavendra said. More thoughts below.
STOC/FOCS represent many areas with a limited budget. No single topic has too many papers and the bar is naturally rather high (though the variance can often be large). People need to publish papers to stay in the game so they send to other conferences and there is less incentive to attend STOC. Same goes with students. Do I want a beginning algorithms student to go to SODA or to STOC/FOCS? It seems more useful to send them to SODA in terms of value for money/time. On the other hand, having a rather selective conference with breakthrough results has also been beneficial for visibility, and as an incentive for people to push themselves to do good work. One can imagine combining the best of both worlds with the following model.
* Accept many more papers with more parallel sessions. If SODA can have three parallel sessions I can imagine STOC being fine with four or even five.
* Select a smaller set of papers that are clearly worth recognizing for breakthroughs and/or connections and give them spotlight status. There could also be spotlight papers
in sub-areas that are in parallel sessions.
* Have workshops/tutorials/plenaries to summarize, condense and highlight directions of research
* Improve social and networking opportunities
The above changes are very likely to increase participation but the “new” STOC will be quite different from the current STOC. Perhaps such a drastic move is unlikely to happen but I don’t think minor tweaks will increase participation dramatically. Also, the new model will require more resources in terms of much larger and hierarchical PC, ability of PC members to submit etc. It is also not clear that we can have such a model without changes to the rest of the theory conferences that are currently in play. I don’t think colocating a few small conferences will really make STOC an attractive event. ISIT (the annual information theory conference) has many papers in large # of parallel sessions and it is not hard to get a paper accepted and has high participation.They have started spotlight papers from this year.
I have had colleagues from inside and outside CS complain strongly about STOC/FOCS not accepting their work while they see some other accepted work at STOC/FOCS on the same or related topics as being about the same quality. Insiders view acceptance/rejection as a repeated game while outsiders walk away after a few iterations. Insiders also can become outsiders over time as topical interests change. Some of this is inevitable but things could be set up better than where we are right now.
One more option is to keep STOC as it is but instead have a separate biennial or triennial theory fest (such as ISMP for mathematical programming) which is a big gathering with many parallel sessions and plenary and semi-plenary talks and no proceedings. Most people in math programming make an effort to go to ISMP since it is only once in three years. There is a wide variance in quality but it acts more as a get-together and networking event.
I would think that to avoid the danger of “premature specialization” it is important to send beginning grad students to STOC/FOCS. But it’s also important for STOC/FOCS to offer such a student something more than just a marathon of 20 minute talks.
Thanks to all the commenters – I hope to see many of you in the STOC business meeting.
Over the next week we plan to post several blog posts (starting with this one https://windowsontheory.org/2015/06/06/imagining-a-theory-festival-a-personal-post/ ) on our vision for the theory festival, some of the lessons we learned from the comments and the many other discussions and mini-surveys we had, and some of our more concrete suggestions.
I think any idea of a 2-tier conference is going to drive down
attendance, alienating rather than including researchers from the less
dominant areas. Let me stipulate that I am fine with some more plenary
invited talks, e.g., tutorials. I am talking about the suggestion of
increasing the number X of talks promoted to plenary sessions while
demoting the remaining talks to either triple parallel sessions, or,
even worse, to < 10 minutes talks. In either case, we are talking about
giving 3 times as much attention to the top X talks (I know it is not
called top but somehow it is selected and will end up on CVs).
I think there are many problems associated with increasing X.
The bigger you make X, the more second rate it will feel to not be in
the top X. One thing is to have a few best papers celebrated, but as X
grows, it becomes more like two classes. I, myself, much prefer the
feeling of being at a conference where all accepted papers/talks are
equal, except perhaps a few best papers (normally SICOMP selected
papers are not known/public at the conference).
The bigger you make X, the smaller a difference you expect between
paper X and paper X+1, and the less reasonable it becomes to give the
top X triple attention. On top of that, as X grows and the epsilon
difference decreases, most people will have very different opinions
about what are the top X papers. Then it gets more annoying that a lot
of the conference time is spent on plenary talks with no choice, while
the talks you personally find interesting get downgraded. I can see
the point in saying that no talk wants to compete for audience with
some official amazing best papers, but as X grows, this point is muted.
The above effects will become far worse for people from the less
popular areas. Often what makes people from an area not come to
STOC/FOCS is that there are too few papers of interest to them, and if
those papers are now demoted to underclass talks, then it makes far
more sense to go to a specialized conference where people care about the
same stuff as you, with good attendance for the talks etc.
As an example, let me take FOCS'14 that did an experiment with many
plenary talks (X=13). It had two parallel session, so a plenary talk
amounted to double (not triple) attention. One area I personally like
is dynamic graph algorithms, which did have some great results, e.g.,
"Decremental Single-Source Shortest Paths on Undirected Graphs in
Near-Linear Total Update Time" and "Online bipartite matching in
offline time". These were only regular talks, but to me, personally,
they were far more interesting than most of the plenary talks.
For contrast, at the same conference, there was a plenary talk
"LP-Based Algorithms for Capacitated Facility Location" with an
LP-based approximation algorithm for something where we already have a
much better and faster local search approximation. This may be of
great interest to people with a special technical interest in
LP-approximation (I also liked it), but I do not get why such a result
has to be made plenary with no alternative talk to go to. The
dominant areas that have power to select papers for the plenary top
that are really very specialized from an outside perspective.
My general prediction is that dominant areas will be even more
dominant in the top X, for multiple reasons: (1) It is often hard to
deem outstanding a paper from an area you don't really know, e.g.,
often I see PC members downgrade scores from overly enthusiastic
expert subreviewers, but for big areas, the PC members are themselves
the enthusiastic experts. (2) When the top incomparable contenders have
been selected, it is more natural to vote for a paper in an area you
find interesting, and since most people work in the
area they like best, a paper from their own area has a bigger chance
of getting their vote. Combined, these effects mean that a strong
contender from an area like approximation, complexity, or crypto is
very likely to get picked, while papers from other areas typically
depend on a strong champion getting involved in the process,
preferably as a PC member.
All in all, I am pretty sure that increasing the number of
contributions promoted to plenary talks will have the effect of making
STOC/FOCS much less desirable for the smaller areas.
Mikkel, I think the relevant metric is comparing how much attention are most papers getting now vs how much they will get in a new format. I believe that with increased overall attendance and other scheduling changes, it is quite possible that the average number of listeners per talk will grow even if we move to a 3-parallel sessions.