Throwing away working code

Lean startups work by systematically eradicating waste. This builds on a lot of great thinking that has come before, like the agile movement's insistence that only the creation of working code counts as progress for a software development team. But lean startups can't afford to be satisfied with just that definition, because there are situations where working code is itself a form of waste.

I used to think that investments in metrics were a form of waste. Customers don't care if you have good metrics, only if you have a good product. And while it's true that metrics sometimes can lead to a better product, in my experience just as often they had led to no insight whatsoever, like fancy reports that nobody reads or after-the-fact rationalizations (with graphs!) that justify decisions already made.

The only reason we learned the art of metrics-based decision making at IMVU was out of necessity. We set sales targets from day one, $300 the first month. I think it was $350 the next, and so on. We picked those numbers by counting up the numbers of friends, family, and relatives we thought we could cajole into buying our early product. Pretty soon we ran out of friends.

I've told the story of how this desperation turned us on to Google AdWords in SEM on five dollars a day. We used those five dollars to buy 100 clicks, every single day, in the hopes that some of those people would become paying customers. When none did, we worried that we'd miss our numbers, pathetically small though they were. We started making changes to the product: making it easier to pay, then making it more obvious that you should pay, then making it easier to download, get started, and so on. In retrospect, we were moving backwards through the funnel, in the most optimistic order possible. We certainly didn't want to believe that our product was fundamentally flawed - we preferred to think it was a mechanical, usability, or technical problem.

As the experiments progressed, day-in-day-out we weren't making sales. We started to put in simple data probes at each of these key stages in our customers' usage of the product. Well, lack of usage, really. The probes always came back zero, or close to it. The bare fact of our failing to make sales forced us to peel back the onion, step by step, until we realized just how big a problem we had. It's true that our product was of low quality, and lacked many features we thought of as essential. But none of that mattered: no customers were making it past the first few screens!

We eventually uncovered the problem, that a core aspect of our product concept was deeply flawed. And along the way we learned our first profound lesson in the power of metrics, what I would later come to realize was a key part of the fundamental startup feedback loop. None of it would have happened if we had plenty of cash, or were content to count our progress by traffic or product development milestones.

The question of why the original concept, an IM add-on for 3D chat, doesn't make sense is a topic for another post (hint: it has something to do with Bowling Alone). Suffice to say, we concluded those features weren't getting the job done.

I had personally written the code that made the IMVU client interoperable with every other piece of IM software I could find, including all the majors and plenty of obscure players. We had invested countless hours in polishing and testing those thousands of lines of code, and had other people maintaining and improving on it, too.

When it came time for us to change company directions, sad to say, I had to be dragged kicking and screaming. I did not want to let go of all that code. I was constantly coming up with arguments to save it: that we couldn't be sure that the feature wouldn't be useful later, that maybe some customers liked it even though most didn't, that it wasn't consistent with our company values to become a proprietary IM network. Most of all, these arguments stemmed from a simple misconception: that the code we had written was an asset, and that it would cost us to throw it away. But code written that doesn't help the company succeed is a sunk cost.

Luckily for me, my co-founders eventually prevailed and we went on to build a product that customers actually wanted. This episode contained important lessons for me: about the power of metrics, the pain of letting good work go, and the complexity of understanding waste.

In order to build a great company, you have to be absolutely crystal-clear about the goal you are trying to accomplish. For startups, that goal must be to create a rapid-iteration feedback loop that enables you to learn what customers want and will eventually pay for. Everything else, including anything that optimizes any other goal, is waste.