Notes on ‘Lean Analytics: Use Data to Build a Better Startup Faster’ by Croll & Yoskovitz (2013)

Read in Jan 2017
Re-read in Dec 2022

Re-read after ~5 years.

Still one of the most practical and useful books I’ve ever read when it comes to tech and business. It’s hard to believe 10 years have passed since its original publication, given its content is still fresh.

As a subsequent thought, it’s frankly depressing to realize how little the digital economy has changed in these past 10 years from 2013 to 2023 (the authors mention Amazon, FB/IG, Quora, Twitter, Pinterest, Reddit, etc. – basically, most of the names in the book are still dominating their relative niches a decade later; also, the analyzed most popular business models for startups have remained exactly the same), compared to the massive changes that happened in the previous 10 years from 2003 to 2013.

The book’s content is organized in roughly six different macro-sections: Analytics strategy, Business models (a selected list of models getting analyzed), Customer development, Metrics, B2B startups, Intrapreneurship. The fist two are especially remarkable (5-star content).

***

A good metric changes the way you behave. This is by far the most important criterion for a metric: what will you do differently based on changes in the metric?

Drawing a line in the sand is a great way to enforce a disciplined approach. A good metric changes the way you behave precisely because it’s aligned to your goals of keeping users, encouraging word of mouth, acquiring customers efficiently, or generating revenue.
Unfortunately, that’s not always how it happens.

At one company, Alistair saw a sales executive tie quarterly compensation to the number of deals in the pipeline, rather than to the number of deals closed, or to margin on those sales. Salespeople are coin-operated, so they did what they always do: they followed the money. In this case, that meant a glut of junk leads that took two quarters to clean out of the pipeline—time that would have been far better spent closing qualified prospects.
Of course, customer satisfaction or pipeline flow is vital to a successful business. But if you want to change behavior, your metric must be tied to the behavioral change you want. If you measure something and it’s not attached to a goal, in turn changing your behavior, you’re wasting your time. Worse, you may be lying to yourself and fooling yourself into believing that everything is OK. That’s no way to succeed.

***

A good metric is a ratio or a rate. Accountants and financial analysts have several ratios they look at to understand, at a glance, the fundamental health of a company. You need some, too.
There are several reasons ratios tend to be the best metrics:
• Ratios are easier to act on. Think about driving a car. Distance traveled is informational. But speed—distance per hour—is something you can act on, because it tells you about your current state, and whether you need to go faster or slower to get to your destination on time.
• Ratios are inherently comparative. If you compare a daily metric to the same metric over a month, you’ll see whether you’re looking at a sudden spike or a long-term trend. In a car, speed is one metric, but speed right now over average speed this hour shows you a lot about whether you’re accelerating or slowing down.
• Ratios are also good for comparing factors that are somehow opposed, or for which there’s an inherent tension. In a car, this might be distance covered divided by traffic tickets. The faster you drive, the more distance you cover—but the more tickets you get. This ratio might suggest whether or not you should be breaking the speed limit.

***

Don’t just think “mobile first”. Think “search first”, and invest in instrumenting search metrics on your website and within your product to see what users are looking for and what they are not able to find.

***

Once, a leader convinced others to act in the absence of information. Today, there’s simply too much information available. We don’t need to guess—we need to know where to focus. We need a disciplined approach to growth that identifies, quantifies, and overcomes risk every step of the way. Today’s leader doesn’t have all the answers. Instead, today’s leader knows what questions to ask.
Go forth and ask good questions.

***

Zach Nies suggests going even further, segmenting customers into three groups. “‘A customers’ are your really big customers who negotiated a big discount and expect the world from you. ‘B customers’ are customers who are fairly low maintenance, didn’t get a big discount, see themselves as partners with you, and provide useful insights. ‘C customers’ cause trouble, are a pain to deal with, and demand things from you that you feel will damage your business,” he explains. “Don’t spend too much time on the A’s—they sound good but aren’t the best for your business. Bring as many Bs on as customers as possible. And try to get your ‘C customers’ to be customers of your competitors.

***

You need to know which aspects of your business are too risky and then work to improve the metric that represents that risk.

***

We sometimes remind early-stage founders that, in many ways, they aren’t building a product. They’re building a tool to learn what product to build.

***

Your job isn’t to build a product; it’s to de-risk a business model.