Stress in Research. Part I: The Tournament and the Axe

By Charles Sutton on March 3, 2018

Perhaps the first post in a series.

From one point of view, research is the best job in the world. You get to be creative and follow your curiosity. You are always learning; if you ever stop, you have stopped doing research. You can collaborate with brilliant people, and help young smart people to find the frontiers of knowledge.

And for all that, I think research will never stop causing me heartache and stress. If you have any capacity in your psyche for self-doubt, you will find that capacity nurtured and magnified during your career in research. But why should this be? The requirements of the job are blessedly open-ended, and we are granted substantial flexibility in how we manage our work. I mean, come on, how stressful can a job be if you’re not expected to roll in to work until 10 o’clock in the morning?

In this series, I will try to isolate and understand the sources of stress and in research, and how we might cope. I say “isolate”, “cope”, and not “eliminate” — I suspect that top-quality work is not possible without accepting some stress. But maybe there’s a way to get by accepting a little bit less.

The first triggers of stress that I want to talk about arise as a consequence of how people get jobs in research, and how the research community is structured. These are the Tournament and the Axe.

First, the Tournament. The research community is structured around an series of increasingly more selective, and more difficult, international tournaments. Most undergraduate students choose to focus their work elsewhere, but of those that choose to do a PhD, most — including your humble author — are not admitted to their first-choice school. Of those who start a PhD, many do not complete it. Of those who complete their PhD, many do not go on to another research position. Of those who become assistant professors in the US, some do not receive tenure. Finally, even of the top researchers, those who have a large group and regularly publish well-received papers, only a small number of them can honestly say that they have ever in their career achieved a groundbreaking new result.

And now let’s consider your competition in the Tournament, your international colleagues in the research community. When you first join a research community, usually by attending your first workshop or conference, it’s overwhelming to meet so many new interesting people, who are fascinated by the same arcane technology as you, and who all seem smarter than you. This is partly an illusion: all these people have been around the community longer than you, so they simply have more background and experience, not more raw power.

But there’s truth in the illusion too: There really are lots of people smarter than you. That is no insult. I’ll explain. For the sake of argument, let’s suppose that you are a unique genius, a special talent, one in a million. Even if you are one in a million, that means there are 7600 people in the world who are smarter than you. And the organizations of the research community are specifically designed to help you to meet them.

What’s more, the outcome of the Tournament is public. The papers we publish, the prestigious jobs that we accept, the awards that we win, these are all public. The people stronger than you, or luckier, you can see the evidence of their success. And you suspect that others, your colleagues and friends, can see where you stand as well.

And then, the Axe. Many levels of the Tournament are up or out. You either succeed in a competition to move the next level, or you move to a career outside of research. The tenure system for assistant professors in the United States is famously up or out, but all earlier levels of your career path are, too. For example, I thought being a postdoctoral researcher was the best job in the world. I loved coming in to work every day. But I also knew there was the Axe over my head. If I hadn’t made enough progress in the two years that I had, I might not be able to remain in a research active position.

All this together causes a lot of stress, or at least it has for me. I like to think that there are attitudes and habits of mind that can reduce the stress of the Tournament, at least a little bit:

  • Understand that every Tournament is also a game. Success in the Tournament is not solely a reflection of your intelligence or your skill, but also your ability to play. It’s so easy to conflate your success in your career with your self-worth, but to do so too strongly is destructive. Disconnect your self-worth and your regard for others from performance in the Tournament.

  • Appreciate that winning the Tournament is not always the right move. The trap of the Tournament is that because it is a game, we naturally feel that winning a game is better than losing it. Not so. Choosing to leave a game is better than winning, when the game is not worth playing — when you enjoy other careers more than research, or if another career provides a better path for you to contribute professionally to the world. It would be almost a cliche to say “there is no shame” in leaving research, but it would also be a grave disservice, the language already buying into the bogus narrative that the competitions are there to be won. So I say instead: To start down one career path, recognize that it is not the best way for you to contribute, and choose another — I salute and admire the strength and self-knowledge that is needed to do this.

  • The Tournament is not the core of research. I’ve been describing the research community as a competitive, perhaps even unfriendly, place. And it is competitive. But I’ve not found it to be unfriendly. I’ve met so many colleagues who have helped me by collaborating with me, explaining their work, graciously helping me to understand the field better. This creates the cognitive dissonance, how can the community be both competitive and helpful at the same time? I reconcile these views by viewing competition as a type of collaboration. We have, together, decided to build a competition for attention, in the hope that it will help all of us to reach our full potential and teach each other as much as we can. So it is not the competition that is central to research, but learning.

  • To paraphrase the philosopher Marsellus Wallace, jealously only hurts. It never helps. You meet someone smarter than you, someone who has accomplished more than you despite being ten years younger, it’s all right to be impressed or amazed, but not jealous. You need to train yourself to think instead: hey, what a great person to learn from!

  • Accept the existence of the Axe. Perhaps not helpful advice, because it is too easily said, but still important. There is nothing to be gained by spending too much time staring at the Axe above you. You know it’s there and you know it’s sharp. You still need to get on with what you have to do, if you intend to stay in the game.

Travel a lot? Here's how to stay organized, like professors do

By Charles Sutton on December 2, 2017

We had a fun time on the professors’ Facebook the other day, swapping stories of the dumbest travel mistakes that we’ve made. You know, booking flights to the wrong country, forgetting to book a hotel, registering for the conference twice, and other such hilarity.

Let’s face it, professors travel a lot. We are also very bad at remembering things. This is a combination that makes for comedy, unless you’re the poor sod who has to live through it. (That’s Professor Sod, to you.)

What this means is, from long and painful experience, I’ve learned how to not to forget things as often when I travel. And now I will tell you my secrets, if I can remember them.

I keep a packing list (shocker!) that I reuse every time I pack my bags. I have a master list that I keep electronically and make a copy of for each trip. The important part is: Every time I arrive somewhere and forget something (toothpaste, underwear, etc), I add it to the master packing list for next time. After many years of this, I am now convinced that this list now contains everything I could possibly want to bring, and now I will never forget to pack something ever again. Well, hardly ever.

For toiletries I have an additional system, as they are too easy to forget. I keep a separate complete set of toiletries in a drawer, for travelling only. When I pack, I just unload the drawer into my luggage, and I know I have everything. When I return, I return the toiletries to the drawer, checking to see if anything needs to be replenished. This includes medicines; I always travel with paracetamol, just in case, except when I travel to the US, in which case I bring acetaminophen instead.

But the packing list can’t keep track of everything. For example, if you try to list “make sure hotel is booked” or “tell wife that I’m travelling” on your list of what to pack, it turns out that you see that a little bit too late to be useful. Or that’s what my wife says, anyway.

So now I have a “travel prep list” as well, which lists the things I need to do weeks in advance, or after I return. I have a document that keeps track of the lists for all of my upcoming trips: book flights, book hotel, register for conference, submit travel reimbursement, etc. This dramatically reduces the chance that I will book two hotel rooms for the same conference.

And finally, there is TripIt. TripIt is an amazing service that I have used for over a decade. Every time you get a confirmation email from an airline, hotel, travel agent, etc, you forward it to a special email address TripIt parses all these emails and collects them into a single itinerary, using the dates to figure out which emails belong to the same trip. You never have to root around looking for a confirmation number again. It’s great! Especially clever: to sign up, just forward your first confirmation email to

Now this leaves one more question. Why are professors always so forgetful in the first place? There’s a very good reason for that, actually. But that’s a different post…

To PhD applicants: A word about department rankings

By Charles Sutton on November 12, 2017

When I was applying for my PhD, I used rankings of computer science departments to help me decide where to apply. Rankings are never perfect, but I didn’t have access to detailed knowledge of the research landscape, and the rankings helped to steer me in the right direction. In retrospect, I can see that the rankings were not perfect, and I made one or two silly mistakes about where to apply, but I would have made even more silly mistakes without them.

I’m saying this to give you this context: I’m not anti-ranking. Very possibly rankings have had a negative effect on higher education overall, but they can be useful if done right, and if you read them in the right way.

The Computing Research Association has just released a statement urging everyone to ignore the new rankings of global computer science departments from US News and World Report. I’m sorry to see this, because I found the US News rankings helpful when I was an undergraduate. But I’ve read the new US News rankings, and I have to agree with the CRA.

These US News rankings are absurd. They are garbage. No one should read them, and I won’t even link to them. You can find them easily via a search engine. Please don’t. The ranking methodology is flawed, for a simple reason that any computer science researcher could tell them immediately. And we did. Influential researchers in computer science pointed out the flaws directly to editors at US News; they were ignored. I don’t know why the editors of US News would ignore this feedback, unless they cared a lot about creating a controversy that would generate page views, and not at all about helping students who are applying for their PhD.

I’ll repeat: Please do not read these rankings at all, not even if you intend take my advice and ignore them. If you click on them, even to laugh at them, you are spending advertisers’ money to support this magazine in misleading PhD applicants who are not as well informed as you.

My advice: If you need rankings, instead go to This is a fully open ranking from Prof Emery Berger at UMass Amherst that ranks global computer science departments directly by the amount of research they produce. You can filter the rankings by geographic area and research area. No ranking is perfect, but this is defensible and open.

I mentioned that rankings are only useful if you read them correctly. Here are some thoughts about how to do that:

  • Overall ranking is not the same as subject specific ranking. The department ranked #50 isn’t ranked that way because its research is #50 in every area of CS. Instead, it will have some research areas that are #50 — which is still pretty damn good — but a few groups that are in the top five. If you are in one of those top groups, then you are in a top group, with all the same excitement and opportunity as the top groups at a bigger name school.

  • Disregard small differences in ranking. Ranking is an ill-defined problem, so you can’t take small differences seriously. As far as overall strength goes, the school ranked #1 is exactly the same as the school ranked #5. Exactly the same. But #1 is going to be overall stronger than #18.

  • For your PhD, what matters most is your supervisor and their group, rather than the department overall. This relates to what I said above, and is probably worth a blog post of its own.

  • Rankings are not life. The distinctions that we are talking about here are small distinctions at the very top. The school ranked #100 — I haven’t looked up what it is — is a fine university with brilliant researchers where you will learn a lot. Here’s an analogy. The weakest football player in the English Premier League, who spends most of his time on the bench, is still a prodigiously talented football player who would run circles around anyone who you and I have ever met. The difference between Lionel Messi and that guy — that’s the level of difference we’re talking about here.

The key point: Use rankings as a way to discover departments you didn’t know about that are strong in your area. Don’t use them as a way to decide between departments: For that, you should be reading the work of potential supervisors that interest you. Doing a PhD is about learning to do research. What types of papers do you want to write?

How to read a research paper

By Charles Sutton on November 4, 2017

There’s lots of advice you can read about how to read a research paper. There’s some good advice in this paper:

S. Keshav. How to read a paper. ACM SIGCOMM Computer Communication Review, 37(3):83–84, 2007.

But there’s one tip that I can offer you to organise your reading of a paper that I can’t remember seeing elsewhere. Ask yourself:

What is the 5 minute summary that you would give to a Very Smart Friend?

I don’t understand a paper until I can explain the paper to a smart person who hasn’t read it. I need to be able to explain enough to the VSF so that she understands: what problem the paper is trying to solve, what sort of methods does it use, and how does it relate to the literature, i.e., what does it add.

But there’s two rules.

Rule 1: You have to use your own words, summarising the paper without looking at it. If you find yourself repeating sentences from the paper, the you haven’t internalised the paper’s message.

Rule 2: You cannot take anything the paper says at face value. You can assume that the authors won’t lie to you. But they might oversell a bit, and if you are a independent expert, you might not agree with everything they claim, or with how they interpret the new evidence that they have provided. Or you might be able to describe what’s going on a little bit better than they managed. What do you think that they have shown?

Another way of saying this: I know that my imaginary Very Smart Friend will jump on me if I say something inaccurate. So I don’t want to make a claim to my iVSF unless I can argue for it, based on what I have learned from the theory and experiments in the paper. If I just say something like “well, the authors claim X,” but X is controversial, or even dubious, then my iVSF will immediately want to know why they say that, do they really have evidence, and I had better have a answer.

It can also be good to try this exercise even before you are done. After reading the introduction, how well can you guess what the methods will be, even before you read them? Then read to see if you were right.

To sum up, I hope that I’ve convinced you that having an imaginary friend can help you in your research. You might not want to tell everyone on the internet that you have an imaginary friend, as I have just done, because it might not improve their respect for you. But hey, if it’s good for your research, then where are your priorities?

My special-est email folder

By Charles Sutton on October 7, 2017

I’ve tried a lot of different methods to organize my email. None of them work.

After many years of trying, I’ve come to the conclusion that the Gmail solution of “put all the email in one bucket and search” is the way to go. Over the years I have gotten very, very good at searching my email. But that’s a topic for another day.

Today, I wanted to mention a special email folder that I have. Maybe it’s the most important folder I have, even though I might only look at it once a year, if that.

What could be so special? Emails to pick me up when I’m feeling down. You know which ones I mean. The emails that contained job offers, promotions, grant awards, or even just kind words from trusted senior colleagues. I archive emails like this, emails with special professional successes, in their own folder. Then every once in a while, if I’ve been dealing with a difficult situation or am having a particularly bad bout of impostor syndrome, I can open up the folder and look at it just for a minute, and then I feel a little bit better.

You might ask why these are professional emails. Hopefully I’ve had happy things happen in my personal life as well? Don’t be too worried. It’s just that my personal life doesn’t usually happen over email. That’s what my wedding photos are for!

I got this idea from a blog post somewhere by some other professor, but I don’t remember who or where. If you do, please let me know in the comments!

Update: Rob Zinkov has kindly reminded me that I learned about this idea from the excellent 7 year postdoc blog post by Radhika Nagpal about life for new faculty members. Thanks!

Tags: advice, email