Programmer’s Block: Analysis Paralysis

Saturday, October 22nd, 2011

I’m in the process of writing a tiny tool. What it does is not important, only that it’s nothing big. I’ve almost finished writing it, and it currently weighs in at about 260 lines of code. The bizarre thing is that it took me three days to write. By my estimates, it shouldn’t have taken me longer than perhaps a day. So why did it take so long?

Like always, whenever I start writing a program, I begin with the absolute heart of the problem I want to solve. I then write the rest of the program around that. I never really design anything up-front, especially when it’s such as small project. As normally happens, I run into some potential problems. The solution is almost always obvious. Part of the program might block on a non-responsive network resource? Fine, build some threading around it. External tools may need to access internals of the program? Alright, let’s build in a simple RPC server to communicate with the process. Functionality should be extensible by system administrators or other programmers? Cool, just write some plugin architecture with a tightly defined API. In short, everything just flows.

“Flow” is, to me at least, exceptionally important. Problems and solutions present themselves almost instantaneously when I’m writing code. More often than not I need to refactor almost constantly during the development process. Some might see that as a problem; I don’t. It’s just the way I work. It’s part of my flow. I can’t design an entire program up front on paper or in diagrams ^[1]. I’m one of those “Solve the problem at hand and nothing else” kind of developers. I always gruesomely over-design things, scared of ending up with a design that is not extensible or flexible enough, which leads to Feature Creep. Usually, halfway through the implementation I will encounter something non-optimal in the design and feel bad for either following or not following the design. Following design breaks my flow, and it demotivates me. So I don’t fully design up-front, although I usually have a rough outline in my head.

This time however, my flow was interrupted by something completely different. It was such a small and ridiculous thing that I’m almost ashamed of talking about it. You see, I needed to return some results from a method call, and I couldn’t see the right way to proceed. My normal minimalist approach would have been to simply return the results as a list/array of values. My “Upcoming Problem” alarm bells started ringing however, as I anticipated that a simple list wouldn’t do. The calling code would need to do various things with the results, which really weren’t the responsibility of the calling code at all. So should I create a new object to hold the results, accompanied with some methods to perform operations on those results? Again it didn’t feel like the optimum solution. It would mean tightly coupling that object with other code. I considered some other options, but it was beginning to feel obvious: my flow was broken.

So I did what I usually do when my flow is broken: I go and do something else. Let my subconscious think about it a bit. A solution is bound to jump to the top of my mind, and I can get back to work, right? Well, it didn’t. Then I did what I almost never do: I let the problem fester in my mind. In the next two days I intermittently returned to my (still open) editor, looked at the code a bit, and went to do something else again.

After two days I suddenly realized I was having Programmer’s Block over something completely idiotic! This was such a minor problem that I almost couldn’t believe I wasted two days on it. My lack of progress wasn’t because the problem was too hard, or that the solution was sub-optimal. That happens all the time, and I either keep on tinkering until I’ve solved the problem, even if it means the solution is sub-optimal. The problem was that I had multiple possible solutions in my head and all were equally sub-optimal, and I couldn’t choose between them. I was over-thinking the implications of my choice; suffering from Analysis Paralysis. The very thing I try to avoid by not designing too much up-front.

In the end I just went with a random sub-optimal solution to the problem. I still don’t know of a better one. Perhaps one will come around later on in the development, or during my final refactoring round. It’s really not important, which is why it bugs me that I got stuck on such a minor problem in the first place. In any case, my flow is back and the program is almost finished ^[2].

So, the lessons learned?

• Sometimes you just have to put your perfectionism aside.
• Sometimes, all you have are bad choices. Making a bad choice is better than making no choice.
• “Go and do something else” can sometimes actually be more harmful then just persevering in solving the problem, even if it is sub-optimal. Remember, it’s just code. You can always throw it away if it doesn’t live up to expectations. I was afraid of wasting time by making the wrong choice, but in the meantime I wasn’t working on the problem at all, so the time was wasted anyway.

^[1] Actually, I can design up-front. I just have to keep it very minimal. Some diagrams on the architecture of multi-tier projects. A couple of doodles of how the major components will interact. A small list of data that needs to be stored. But if I actually start making class diagrams, the flood-gates open and I end up with a beast of a design.

^[2] Experience tells me the program isn’t nearly finished. The “programming” bit of a project usually only accounts for about 40% of the total project I have to do. The rest goes to the final refactoring round, writing documentation, testing, packaging, release management, etc