Why I will not submit to NeurIPS any more

I am not sending my next piece of work to NeurIPS. Not because the reviewing is bad. Because of what the format does to an argument before anyone gets to have it.

Six months is the whole objection

You write the sentence in May. Reviews come back in the autumn. If it is accepted you stand next to a poster in December. Somewhere in there the result stopped being news, and you spent the gap not allowed to talk about it properly.

Six months is a long time in a field where the thing you are optimizing changes underneath you. Half the value of a result is that it arrives while the question is still open. A conference schedule is indifferent to whether the question is still open.

The argument happens in private

Three anonymous readers, one round, and then it is over. Those readers are usually good and often right. But nobody else sees the exchange, the strongest objection never reaches the people who would have made it better, and I cannot answer it in public.

A post gets argued with the day it goes up, by whoever actually knows, in a thread anyone can read afterwards. That is a better instrument. It is worse at conferring status and better at finding out whether I am wrong, and I would rather find out.

The page limit decides what gets explained

Eight pages is eight pages. What survives is the result and the benchmark table. What gets compressed into an appendix, or cut, is the part that took longest to get right: why the estimator is the one to reach for, what breaks if you weaken the assumption, the derivation that makes the whole thing obvious in hindsight.

That is the part I want to write. It is also the part with no natural length, which is exactly what a deadline and a page budget cannot accommodate. Here there is no deadline, so a piece of mathematics can take the space it needs and stop when it is finished rather than when the template runs out.

What this is not

It is not an argument against peer review. I still review, and the careful anonymous reader is a real thing that a comment section does not replace.

It is also not permission to be sloppy. Dropping the venue does not drop the bar: a claim still needs a proof or an experiment, an ablation still has to isolate the thing it says it isolates, and a number without a denominator is still worthless. If anything the bar goes up, because nobody is checking it for me before it goes out. What changes is where the argument happens and how long you have to wait for it, not how carefully it is made.

If you want the long version

Most of what I publish here will be the argument in full: the proofs, the statistical evidence, the assumptions written down where you can attack them. If that is what you came for rather than the summary, subscribe below and it comes to you.