Rewriting Things

In the life of every nontrivial project, there is a point where you look back and ask yourself: "How did we get here? What a tangled web we have weaved! Can we ever recover? If we push ahead, are we digging ourselves into a deeper hole, or is there a light at the other end?"

I'd like to offer some personal advice.

If you look at your code base and say "There is so much that's right, even if it's really messy and has some serious bugs." then you should keep going, especially if other people are involved in the project. If you look at your code and say "Actually, no one is really using this yet, and if I switch things around before people have to deal with it, they'll thank me later." then you should head back. Start over. Get it right from the beginning while you have a chance, or it's really going to hurt later.

You'll know when it's ok to start over, because you won't be too worried about it. You won't deliberate very long, the choice will be pretty obvious.

But if there is some turmoil in your heart, you need to step back and get some perspective. If it's taking you weeks or months to figure out what to do, don't give up. The answer is that you need to work through it. You need to spend that time reflecting on exactly what is wrong, and what you can do about it. If you get abandon all your work at that point, you're going to regret it later. Especially if you divide the community that is contributing to it, they'll end up duplicating their effort just to learn the same lessons.

Take OpenCV as an example. The C interface was a solid backbone for thousands of developers, but the community got to a point where it needed to work past the limitations of that language. C++ looked incredibly appealing, but it would required some significant changes to how OpenCV works in order to support those features. What did they do? They didn't rewrite OpenCV from scratch, but they started building a new foundation by reflecting on what worked and what didn't work in the original interface. They took the whole community along with them and now OpenCV 2 is strong and better than ever. One of the most important things is that a lot of the same people are involved. If there was a fork, or someone decided to start from scratch, then nothing learned during the development of OpenCV 1 would be completely applied to OpenCV 2. Some of the same paradigms might be there, but a complete, continuous understanding would be missing.

On the other hand, look at pd and Max. They could have combined efforts, and learned from each other in a more direct way. But now we have two slightly different environments that are each missing features of the other.

This might all sound unnecessarily verbose for such a simple message. But that's because this is actually a metaphor for something completely unrelated to software development.