Systematising the obvious

Ok, so this is one of those that comes about because I found myself saying the same thing to different people a couple of times within a week or so. Which I usually think means I should probably write it down.

I had been talking with Ph.D. students about selecting research topics, doing research, and what is interesting subsequently to report in papers. In doing so I recalled some advice that was given to me during my Ph.D. — so I’m not taking credit for this, though I can’t recall who specifically said it to me like this first; possibly Ralph Becket. What follows is of course scoped to computer systems research as far as I’m concerned, though it may well apply more broadly.

Anyway, to the point. I have sometimes found it useful to think about research topics and their results as placed along two axes, each of which I split into two categories. The first axis might be labelled obviousness with the two categories labelled “obviously…” and “non-obviously…”.¹ The second axis might be labelled success with the two categories “…works” and “…doesn’t work”. Finally, I put an indicator in each quadrant as to the desirability of work in that quadrant – shown in the picture as ticks (the more ticks, the more desirable), or crosses (undesirable).

Considering each of the quadrants then, we have in order of desirability:

• non-obviously works: Bingo! This is when you’ve done something that people didn’t think would work but it turns out you had sufficient insight, skill, luck, resources, or whatever to make it work. Reporting this sort of work is happy times – fame, glory, riches, the prize. No, wait, that’s not right. Oh yes — you get a good published paper. Nice.
• obviously works: The bulk of published research in my experience. Certainly not bad, and definitely contributing to knowledge. But not really surprising — unlikely to create that elusive “wow! that’s cool!” feeling.
• non-obviously doesn’t work: Far too little of this sort of thing is published in my opinion. I feel quite pleased when I manage work like this because I don’t need to care quite so much as I used to about getting papers accepted and I find the process and outcome interesting. But it is sometimes hard to get this work out there as it often seems too easy for reviewers to focus on the fact the thing didn’t work rather than considering the contribution you’ve made by showing that everyone else’s intuitions are wrong. Sad but mildly self-righteous times ensue.
• obviously doesn’t work: No-one cares. Tends to get a big “yeah? and? idiot” from reviewers. I wouldn’t usually bother except as a personal learning exercise.