Continuous deployment for mission-critical applications

Having evangelized the concept of continuous deployment for the past few years, I've come into contact with almost every conceivable question, objection, or concern that people have about it. The most common reaction I get is something like, "that sounds great - for your business - but that could never work for my application." Or, phrased more hopefully, "I see how you can use continuous deployment to run an online consumer service, but how can it be used for B2B software?" Or variations thereof.

I understand why people would think that a consumer internet service like IMVU isn't really mission critical. I would posit that those same people have never been on the receiving end of a phone call from a sixteen-year-old girl complaining that your new release ruined their birthday party. That's where I learned a whole new appreciation for the idea that mission critical is in the eye of the beholder. But, even so, there are key concerns that lead people to conclude that continuous deployment can't be used in mission critical situations.

Implicit in these concerns are two beliefs:

1. That mission critical customers won't accept new releases on a continuous basis.
2. That continuous deployment leads to lower quality software than software built in large batches.

These beliefs are rooted in fears that make sense. But, as is often the case, the right thing to do is to address the underlying cause of the fear instead of avoiding improving the process. Let's take each in turn.

Another release? Do I have to?
Most customers of most products hate new releases. That's a perfectly reasonable reaction, given that most releases of most products are bad news. It's likely that the new release will contain new bugs. Even worse, the sad state of product development generally means that the new "features" are as likely to be ones that make the product worse, not better. So asking customers if they'd like to receive new releases more often usually leads to a consistent answer: "No, thank you." On the other hand, you'll get a very different reaction if you ask customers "next time you report an urgent bug, would you prefer to have it fixed immediately or to wait for a future arbitrary release milestone?"

Most enterprise customers of mission critical software mitigate these problems by insisting on releases on a regular, slow schedule. This gives them plenty of time to do stress testing, training, and their own internal deployment. Smaller customers and regular consumers rely on their vendors to do this for them, and are otherwise at their mercy. Switching these customers directly to continuous deployment sounds harder than it really is. That's because of the anatomy of a release. A typical "new feature" release is, in my experience, about 80% changes to underlying API's or architecture. That is, the vast majority of the release is not actually visible to the end-user. Most of these changes are supposed to be "side effect free" although few traditional development teams actually achieve that level of quality. So the first shift in mindset required for continuous deployment is this: if a change is supposedly "side effect free," release it immediately. Don't wait to bundle it up with a bunch of other related changes. If you do that, it will be much harder to figure out which change caused the unexpected side effects.

The second shift in mindset required is to separate the concept of a marketing release from the concept of an engineering release. Just because a feature is built, tested, integrated and deployed doesn't mean that any customers should necessarily see it. When deploying end-user-visible changes, most continuous deployment teams keep them hidden behind "flags" that allow for a gradual roll-out of the feature when it's ready. (See this blog post from Flickr for how they do this.) This allows the concept of "ready" to be much more all-encompassing than the traditional "developers threw it over the wall to QA, and QA approved of it." You might have the interaction designer who designed it take a look to see if it really conforms to their design. You might have the marketing folks who are going to promote it double-check that it does what they expect. You can train your operations or customer service staff on how it works - all live in the production environment. Although this sounds similar to a staging server, it's actually much more powerful. Because the feature is live in the real production environment, all kinds of integration risks are mitigated. For example, many features have decent performance themselves, but interact badly when sharing resources with other features. Those kinds of features can be immediately detected and reverted by continuous deployment. Most importantly, the feature will look, feel, and behave exactly like it does in production. Bugs that are found in production are real, not staging artifacts.

Plus, you want to get good at selectively hiding features from customers. That skill set is essential for gradual roll-outs and, most importantly, A/B split-testing. In traditional large batch deployment systems, split-testing a new feature seems like considerably more work than just throwing it over the wall. Continuous deployment changes that calculus, making split-tests nearly free. As a result, the amount of validated learning a continuous deployment team achieves per unit time is much higher.

The QA dilemma
A traditional QA process works through a checklist of key features, making sure that each feature works as specified before allowing the release to go forward. This makes sense, especially given how many bugs in software involve "action at a distance" or unexpected side-effects. Thus, even if a release is focused on changing Feature X, there's every reason to be concerned that it will accidentally break Feature Y. Over time, the overhead of this approach to QA becomes very expensive. As the product grows, the checklist has to grow proportionally. Thus, in order to get the same level of coverage for each release, the QA team has to grow (or, equivalently, the amount of time the product spends in QA has to grow). Unfortunately, it gets worse. In a successful startup, the development team is also growing. That means that there are more changes being implemented per unit time as well. Which means that either the number of releases per unit time is growing or, more likely, the number of changes in each release is growing. So for a growing team working on a growing product, the QA overhead is growing polynomially, even if the team is only growing linearly.

For organizations that have the highest standards for mission critical, and the budget to do it, full coverage can work. In fact, that's just what happens for organizations like the US Army, who have to do a massive amount of integration testing of products built by their vendors. Having those products fail in the field would be unacceptable. In order to achieve full coverage, the Army has a process for certifying these products. The whole process takes a massive amount of manpower, and requires a cycle time that would be lethal for most startups (the major certifications take approximately two years). And even the Army recognizes that improving this cycle time would have major benefits.

Very few startups can afford this overhead, and so they simply accept a reduction in coverage instead. That solves the problem in the short term, but not in the long term - because the extra bugs that get through the QA process wind up slowing the team down over time, imposing extra "firefighting" overhead, too.

I want to directly challenge the belief that continuous deployment leads to lower quality software. I just don't believe it. Continuous deployment offers three significant advantages over large batch development systems. Some of these benefits are shared by agile systems which have continuous integration but large batch releases, but others are unique to continuous deployment.

1. Faster (and better) feedback. Engineers working in a continuous deployment environment are much more likely to get individually tailored feedback about their work. When they introduce a bug, performance problem, or scalability bottleneck, they are likely to know about it immediately. They'll be much less likely to hide behind the work of others, as happens with large batch releases - when a release has a bug, it tends to be attributed to the major contributor to that release, even if that association is untrue. 
2. More automation. Continuous deployment requires living the mantra: "have every problem only once." This requires a commitment to realistic prevention and learning from past mistakes. That necessarily means an awful lot of automation. That's good for QA and for engineers. QA's job gets a lot more interesting when we use machines for what machines are good for: routine repetitive detailed work, like finding bug regressions. 
3. Monitoring of real-world metrics. In order to make continuous deployment work, teams have to get good at automated monitoring and reacting to business and customer-centric metrics, not just technical metrics. That's a simple consequence of the automation principle above. There are huge classes of bugs that "work as designed" but cause catastrophic changes in customer behavior. My favorite: changing the checkout button in an e-commerce flow to appear white on a white background. No automated test is going to catch that, but it still will drive revenue to zero. Continuous deployment teams will get burned by that class of bug only once.
   
4. Better handling of intermittent bugs. Most QA teams are organized around finding reproduction paths for bugs that affect customers. This made sense in eras where successful products tended to be used by a small number of customers. These days, even niche products - or even big enterprise products - tend to have a lot of man-hours logged by end-users. And that, in turn, means that rare bugs are actually quite exasperating. For example, consider a bug that happens only one-time-in-a-million uses. Traditional QA teams are never going to find a reproduction path for that bug. It will never show up in the lab. But for a product with millions of customers, it's happening (and being reported to customer service) multiple times a day! Continuous deployment teams are much better able to find and fix these bugs.
5. Smaller batches. Continuous deployment tends to drive the batch size of work down to an optimal level, whereas traditional deployment systems tend to drive it up. For more details on this phenomena see Work in small batches and the section on the "batch size death spiral" in Product Development Flow. 
   

For those of you who are new to continuous deployment, these benefits may not sound realistic. In order to make sense of them, you have to understand the mechanics of continuous deployment. To get started, I recommend these three posts: Continuous deployment in 5 easy steps, Timothy Fitz's excellent Continuous Deployment at IMVU: Doing the impossible fifty times a day, and Why Continuous Deployment?

Let me close with a question. Imagine with me for a moment that continuous deployment doesn't prevent us from doing staged releases for customers, and it actually leads to higher quality software. What's preventing you from using it for your mission-critical application today? I hope you'll share your thoughts in a comment.

Why vanity metrics are dangerous

In a previous post, I defined two kinds of metrics: vanity metrics and actionable metrics. In that post, I took it for granted that vanity metrics are bad for you, and focused on techniques for creating and learning from actionable metrics. In this post, I'd like to talk about the perils of vanity metrics.

My personal favorite vanity metrics is "hits." It has all the worst features of a vanity metric. It violates the "metrics are people, too" rule: it counts a technical process, not a number of human beings. It's a gross number, not per-customer: one hit each from a million people is a very different thing than a million hits from just one person. Most normal people don't understand it: what counts as a hit, anyway (an image, a page, a JS file...)? And it has no built-in measure of causality: if we get a million hits this month, what caused them? How could we generate more hits? Are those hits all of equal value? Since each of these questions requires diving into a different sub-metric, why not just use those metrics instead? That's the essence of actionable metrics.

But actionable metrics are more work. So it's reasonable to ask: what's wrong with vanity metrics, at least as a proxy for customer behavior? If hits are bigger this month than last month, that's progress. Why do we need to ask hard questions about the metric, if it's at least directionally correct?

When we rely on vanity metrics, a funny thing happens. When the numbers go up, I've personally witnessed everyone in the company naturally attributing that rise to whatever they were working on at the time. That's not too bad, except for this correlate: when the numbers go down, we invariably blame someone else. Over time, this allows each person in the company to live in their own private reality. As these realities diverge, it becomes increasingly difficult for teams to reach consensus on what to do next. If those teams are functionally organized, the problem is amplified. If all the engineers work on the same thing at the same time, and all the marketers do the same, and QA, and ops, all the way down the line, then each department develops its own team-based private reality. Now picture product prioritization meetings in such a company. Each team can't believe those idiots down the hall want to try yet another foo project when it's so evident that foo projects always fail.

Have you ever built one of those charts that shows a metric over time, annotated with "key events" that explain what happened to the numbers at key inflection points? If you never have, you can create your own using Google Finance. Go ahead and try it, then come back. You've just experienced vanity metrics hell. Everyone knows those charts are totally unpersuasive. At best, they can only show correlation, not causation. Sure I can build a stock chart, like this one, that shows that eBay's stock price went into a four-year decline immediately after "eBay Inc Acquires Dutch Company Marktplaats.nl." But do you really believe that's what caused eBay's problems? Of course not. At worst, these kinds of vanity metrics can be easily used for gross distortions. And that potentiality can cripple companies at just those key moments when they need to understand their data the most.

Let me take an example from a company that was going through a tough "crossing the chasm" moment. They had just experienced two down quarters after many quarters of steady growth. Naturally, they had just raised money, and their new investors were understandably pissed. The company struggled mightily with how to explain this bad news to their board. They were accustomed to measuring their progress primarily by gross revenue compared to their targets. When the numbers started to go down, they started to investigate. It turned out that, during the course of the decline, one customer segment was losing customers while another was gaining customers. It's just that the declining segment's customers were more valuable. In retrospect, I can see the irony of the situation perfectly. This decline was actually the result of the company successfully executing its strategy. The customers on their way out were more valuable in the short-term, but the new customers coming in were where the real growth was going to happen in the long-term. Unfortunately, the magnitude of the shift, and the relative values of the two segments, took the company by surprise.

Guess how well those board meetings went. All of the sudden, now that the company was struggling, new metrics were being used to judge success. The vanity charts appeared, showing the changes the company had made to its strategy and the subsequent changes in customer behavior broken down by segment. All very reasonable, well designed, well argued. In other words, a total disaster. This board had no way to know if they were hearing real insight or just well-crafted excuses. The insight turned out to be correct (it's always clear in hindsight). Too bad several of the executives making that presentation weren't around to be vindicated.

The whole situation could have been avoided if the company had used actionable metrics to set and evaluate goals from the start. The strategy changes could have been rolled out gradually, segment by segment, in controlled trials. The data from those trials could have been used to predict the future effects, and allowed the company to make smarter decisions. Actionable metrics don't guarantee you'll make good decisions. But at least you can have facts in the room at the time.

And that's my metrics heuristic. Consider a team's last few critical decisions. Not the ones with carefully choreographed discussion and a formal agenda. Those are the easy meetings. I'm talking about the adhoc crisis decisions, the periodic product prioritization meetings, and the failure post-mortems. How much actionable data was in the room at the time? With data, teams have an opportunity to improve their decision making over time, systematically training their intuition to conform to reality. Without it, they're just rolling the dice.

And that's why vanity metrics are dangerous.

What is Lean about the Lean Startup?

The first step in a lean transformation is learning to tell the difference between value-added activities and waste. That foundational idea, so clearly articulated in books like Lean Thinking, is what originally led me to start using the term lean startup. I admit that I haven't always done such a good job emphasizing this connection; after all, there's an awful lot to the lean startup theory, and I'm always struggling with how best to explain it fully. Luckily, I've had some excellent backup.


The following is a guest post for Startup Lessons Learned by the legendary Kent Beck. One of the most amazing things about the past year has been the opportunity to meet many legends and personal heroes. And yet, I have a confession to make. Many of these heroes have proved disappointing: some have been defensive, stand-offish, and downright mean. Not so with Kent Beck. 


Longtime readers will recall how I first met him. I was giving my first-ever webcast on the lean startup. For those who've heard it, it contains a length discourse on the subject of agile software development and extreme programming, including its weaknesses when applied to startups. Now, this webcast was packed, hundreds of people were logged in. The chat stream was flying by in my peripheral vision, a constant distraction, hard to focus on. As I'm pontificating about agile, I see the name Kent Beck in my peripheral vision. I was truly terrified, and I almost completely lost my train of thought. Was that really the Kent Beck? I assumed he was there to refute my critique of extreme programming, but nothing could be further from the truth. In fact, of all the gurus and leaders I've had the chance to meet, he has been by far the most open-minded. He instantly understood what I was saying, and since that first encounter, our exchanges have made me a lot smarter. 


So when he weighed into a recent thread on the Lean Startup Circle mailing list on this very subject, I asked if he would expand his comments into a guest post. The following is the result. - Eric

Names matter. By pulling in a web of associations, names help people quickly assess ideas. Chosen well, they draw attention from people likely to appreciate the ideas they identify.

There is a dark side to naming. When a name is misused, as with some of the claims to "agility" extant, the initial interest is followed by disappointment when customers discover there is no corned beef between the slices of rye. It's tempting to ride the coattails of a popular idea by using a word with momentum, but in the end it backfires for the idea and the word.

The naming question has been raised about the "lean" in Lean Startups. Are lean startups really lean or was the word chosen because it is widely recognized and popular?

I had a background in lean manufacturing (book knowledge, anyway) and lean software development (hands on) before encountering Lean Startups. When I read Eric's blog I immediately felt at home: the principles were the same even though some of the practices were different.

The foundation of TPS (Toyota Production System) is that people need to be (and feel) productive and society needs people to produce value. This value is evident in Lean Startups. We are all engaged in creating valuable (we hope) services for society in some form or other and simultaneously meeting our own need to feel significant and productive.

Another basic principle of TPS is respect for people. One form of respect is not wasting the time of people who are creating new products and services. Another form of respect is inviting customers to be part of the process of creating those products and services. At times on this lean startup mailing list I hear an undercurrent of "ha, ha, I got you to give me feedback on this fake landing page even though I gave you nothing in return," which is a violation of this principle of respect. Overall, though, Lean Startups seems far more respectful to me than the "build something big and shove it down customers throats" model I have participated in (and failed with) over and over.

TPS focuses on eliminating waste. Rather than try to create value faster (I'm thinking of Charlie Chaplin in "Modern Times" or Lucille Ball's Candy Factory scene), a lean organization creates more value by eliminating waste. This principle appears throughout Lean Startups, starting with the biggest waste of all in a startup--building something no one uses.

A TPS tactic that familiar in lean startups is reducing inventory. If you can split a feature in half and get good feedback about the first half, do it. The lack of inventory enables quick changes of direction, something seen in Lean Startups in the pivot and in TPS in the ability of a single production line to create multiple vehicles. I haven't seen an equivalent of the systematic elimination of work-in-process inventory in Lean Startups, however. (Those who are interested in work-in-process might want to take a look at Work in small batches and Continuous deployment - Eric)


Some specific TPS practices appear in Lean Startups. A/B testing is set-based design. 5 Whys is straight out of the Toyota playbook. Conversion optimization is a form of kaizen. Whether practices work directly is not as important as whether the principles are alive, though. I see the lean principles throughout Lean Startups.

Is the "lean" in Lean Startups an illegitimate attempt to steal some of the "lean foo"? I don't think so. It doesn't look precisely like manufacturing cars, but the principles are shared between the Lean Startups and a lean manufacturing system like TPS, and to some extent even the practices. I expect the cross-fertilization to continue, even as Lean Startups discover what is unique about the startup environment and what calls for a unique response.

Business ecology and the four customer currencies

Lately, I’ve been rethinking the concept of “business model” for startups, in favor of something I call “business ecology.” In an ecosystem, each participant acts according to its own imperatives, but these selfish actions have an aggregate effect. Some ecosystems are stable, others malign, and others grow and prosper. A successful startup strives for this latter case. I think this concept is necessary in order to answer the truly vexing startup questions, like: “Should startups charge customers money from day one?”

Let’s begin with the four customer currencies. I had a lot of use for this concept back when I worked on game design and virtual worlds. In order to maintain game play balance, game designers have to take into account the needs of customers who have an excess of four different assets: time, money, skill, and passion.

If players with more money than others can simply buy their way to the top of the heap, a multiplayer game fails – because this makes the game un-fun for other players. The same is true if kids who have an unlimited amount of time on their hands are guaranteed the top spot – this isn’t going to be very fun for the busy professionals who want to play only casually. Chess is only fun for those who have the requisite skill to play well – and even then, only if there are ranking systems to make sure that players of relatively equal skill play each other. If you could buy a higher chess ranking or, worse, simply grow it by logging more hours, that would ruin the system for everyone. And passionate players are often the backbone of game communities – especially online. They run the clubs, forums, groups and mailing lists that make the game more fun overall. If they are barred from participating (say, because they lack the skill needed to prevent advanced players from killing them all the time), the game is worse off.

Each of these four currencies represents a way for a customer to “pay” for services from a company. And this is true outside of games. Constructing a working business model is a form of ecosystem design. A great product enables customers, developers, partners, and even competitors to exchange their unique currencies in combinations that lead to financial success for the company that organizes them.

Here’s the ecosystem we built at IMVU, just to give one example. We cultivated a passionate community that nurtured a skilled set of developers. Those developers create an incredible variety of virtual goods: 3D models, textures, homepage stickers, music, and much more – more than three million in total last time I checked. This variety entices millions of end-users to invest their time and passion with IMVU, providing many incentives for a small fraction of those users to become paying customers. Those paying customers provide IMVU with sufficient profits to reinvest in the core experience for everyone. It’s a working, growing, ecosystem.

Having a balanced ecosystem is what game designers strive for. But startups strive for something else: growth. Thus, business ecology is concerned with both ecosystem design and finding a driver of growth for that ecosystem. In a previous post, I covered the three main drivers of growth: Paid, Sticky, and Viral. When a startup finds a working value-creating ecosystem that supports one of these drivers of growth, watch out. They’re off to cross the chasm.

And this is why questions like “Should a company charge money from day one?” are nonsensical. Some companies definitely should. Others definitely shouldn’t. In order to tell which is which, you have to understand the unique ecology of the business in question.  Let’s look at some examples:

• In a traditional business, customers pay money for a physical artifact (a product) or a service. Companies use that money to market the product or service to more customers. This is the simplest ecosystem and simplest driver of growth. A business that strives for something like this should absolutely be charging money from day one, in order to establish baselines for their two key metrics: CPA (the cost to acquire a new customer) and LTV (the lifetime value of each acquired customer). In other words, the minimum viable product is designed to answer the question: does the product generate enough demand and margin to support a growing ecosystem?  
• Now consider a traditional media business. By paying money to content creators (ie writers, producers, talent), the business uses builds up assets that are of interest to other consumers. Those other consumers pay for this content sometimes with money, but more often with their attention. This attention is valuable to yet another set of people: namely, the traditional businesses (see above) who are using marketing to grow, and are looking to advertise to new prospects. The value of the attention that the media company collects determines how profitable it is. In the old days, these media companies would then themselves plow this profit back into marketing and advertising, and grow. Today, many of these businesses are suffering because the ecosystem no longer balances thanks to the Internet. (Sorry about that.) If you’re starting a new media company, does it make sense to charge from day one? Probably not – you need to be finding an audience, making sure that audience will trade you their attention for your content, and – most importantly – establishing a baseline for how much that attention is worth to advertisers. A minimum viable product in this category must answer the question: does my media content or channel command the attention of a valuable audience?
  
  
• Let’s look at a viral growth company, like Facebook. They are a classic case of a company that doesn’t seem to care about charging customers money. Here’s Andrew Chen’s description:
  
  
  

  “it strikes me that consumer internet companies often don’t care much whether or not they have viable businesses in the short run. If you are building a large, viral, ad-support consumer internet property, you just want to go big! As soon as possible!”
  

  This is a common sentiment, but I don't agree. I think it uses the phrase “viable business” in too narrow a sense. When Facebook launched early-on at college campuses, it was immediately apparent that they had a viable business, even though they weren’t charging customers for anything. Why? Because they were collecting truly massive amounts of attention and they had an amazing driver of growth. Those two factors made it relatively easy for them to raise enough money to avoid having to build a profitable business in the short term. But that doesn’t mean they didn’t have a viable one. The ecosystem worked, and was growing. Figuring out how to turn that attention into cash seems to have been pretty obvious to Mark Zuckerberg. For a true viral ecosystem, the minimum viable product is designed to answer the question: can I unlock viral growth mechanics while still keeping my ecosystem alive? As many viral companies have found to their chagrin, quite a few viral products are fundamentally useless. Although they grow, they don’t actually collect enough of any customer currency to be viable.
  
  
  In fact, the viral metaphor is actually more apt than many people realize, once you look at it from an ecological perspective. Facebook is actually quite rare – many other viral products didn’t really build their own working ecology: they colonized someone else’s. That was true for Paypal cannibalizing eBay, YouTube and MySpace, and could still be true of Slide, Zynga, or RockYou – we’ll see.
  
  Now, Andrew’s excellent piece that I quoted from above correctly diagnoses two situations where consumer internet companies often get in trouble:
  
  
  
  1. They focus too much on short-term revenue, getting caught in a local maximum via constant optimization. They aren’t really engaged in customer development, they aren’t getting inside their customers’ heads, and they aren’t crafting a robust ecosystem. For a consumer internet company in particular, this is often due to a lack of design thinking.
     
     
  2. They get focused solely on growth. This isn’t helpful either, as countless companies have shown. If you haven’t figured out the ecosystem, growth is useless – whether it is a acquisition-only viral loop, like Tagged, or an advertising blitz like countless dot-bombs.
  
  
  

• Let’s consider one last example, a sticky-growth company like eBay or World of Warcraft. Here the goal is to create a product whose ecosystem makes it hard for customers to leave. eBay offers their customers an opportunity to monetize their skill and passion via online trading for hard currency. World of Warcraft offers beautifully balanced and addictive game play, for which customers trade all four currencies in bewildering combinations. Like eBay, these investments are best understood as trades between players, which is what makes multiplayer game design so much harder than its single-player counterpart. What these products all have in common is the question their minimum viable product is attempting to answer: does this product have high natural retention built-in?

Understanding the four customer currencies allows us to avoid these problems, and also unify a number of different concepts that have been floating around. Take the minimum viable product, for starters. How should the word viable be understood? Here’s the original definition I proposed for MVP:

“the minimum viable product is that version of a new product which allows a team to collect the maximum amount of validated learning about customers with the least effort.”

Of course, this begs the question: what are we trying to learn? Now I think I can answer this question with some certainty: we want to learn how to construct a business ecology harnessed to the right driver of growth. And how do we validate that learning? By creating a model of the ecosystem we want, and showing that actual customer behavior conforms to that model. And, of course, if customers don’t behave the way we expect, it’s time to pivot.

That necessarily means that different types of startups will be seeking to learn different things. Minimum viable product is a tactic for mitigating risk. It doesn’t say anything about which risks it should be used to mitigate. Startup founders need to use their own judgment to ask: which is the riskiest assumption underlying my business plan? In each of the ecosystem examples I gave above, the tactics of the minimum viable product are quite different. In some cases, Tim Ferriss-style landing page tests will suffice. Others require Steve Blank-style problem/solution presentations. And others require an early product prototype. The level of design required will vary. The level of engineering quality will vary. The amount of traditional business modeling will vary.

And now we can answer the biggest question of all: how do we know it’s time to scale? Or, to borrow Steve Blank’s formulation, how do we know it’s time to move from Customer Validation to Customer Creation? I think I have a solid answer here too: when we have enough data that shows our business ecology is value-creating and also ready to grow via a specific driver of growth.

Founders struggle with this question. Successful startups don’t. In almost every case I’m aware, the question never had to be asked. When an ecosystem is thriving and growing, it takes work just to keep up with its scaling needs. This was true at Facebook, eBay, and Google – and at countless other successful startups. Marc Andreessen has already coined a phrase for what it looks like: product/market fit. One clue that you don’t have product/market fit – you’re trying to evaluate your business to see if you have it. It’s probably time to pivot.

These concepts have important implications for any lean startup. My whole goal with the lean startup movement has been to learn how to tell the difference between value-creating activities and waste in startups – and then to start eliminating waste. In an entrepreneurial situation, this is hard, because artifacts that we are creating (products, code, marketing campaigns, even revenue) are of secondary importance. The real value we create is learning how to craft profitable ecosystems. Even then, it’s the learning that’s the real value. So, when evaluating any activity, ask: is this helping me learn more about my startup’s ecosystem? If not, eliminate it. If so, ask: how could I get even more learning while doing even less work?

Most of all, beware one-size-fits-all startup advice. In order to figure out what applies to your unique situation, focus on the principles. Who are the customers? What currencies do they have? And what problems do they need solved? Look for a balanced ecosystem and a driver of growth. And be sure to hold on to the reins once you find it.