Building a great product is a matter of two questions: How should we measure progress? And what should we build next?
That first question is the most important because how you collect data and what you count as success influences everything else; if your organization is measuring shit then they’ll build nothing but shit. And most product orgs suck and churn out garbage projects because they waste so much time thinking in terms of junk data and half baked user inputs to inform their decisions.
(Show me what your org measures and I’ll show ya the crappy product that comes out the other side.)
The problem underlying all this is that when it comes to building a product, all data is garbage, a lie, or measuring the wrong thing. Folks will be obsessed with clicks and charts and NPS scores—the NFTs of product management—and in this sea of noise they believe they can see the product clearly. There are courses and books and talks all about measuring happiness and growth—surveys! surveys! surveys!—with everyone in the field believing that they’ve built a science when they’ve really built a cult.
(No great product has ever been made because of the answers collected in a dumb user survey.)
So how do you measure progress then?
I guess customers could tell us the answer, right? Well, no. Sure, you can talk to customers to see how they struggle but they cannot tell you why they struggle. They’ll have terrible ideas for improvements like “I really wish AI could show me all the relevant things on this page” or “I want more dashboards” where the answer is always much simpler than that. Customer feedback is a geiger counter: they can tell you about the problem coming your way but not how to prevent it.
(Customers, like data, will always mislead us on what to build next.)
See, I don’t think you can build a great product for customers. Yes, yes, yes; you can make billions of dollars building something for customers and go live on a beach in the south of France. But you’ll have built junk in the process; the product will suffer if you build it for customers. You’ll spend every waking moment trying to measure user happiness and score feelings in a spreadsheet and not improving the product.
In every product org it feels as if folks mistake qualitative data—stuff that can’t be measured like feelings—with quantitative data—stuff that can be measured like numbers or time or temperature. They’ll say “this user is 4.5% happy” and, okay, great. Now what? This numerical value sure is bullshit but it’s not even helpful bullshit because these numbers never explain why things suck.
(Just look at the product and it will tell you why it sucks.)
It comes down to this annoying, upsetting, stupid fact: the only way to build a great product is to use it every day, to stare at it, to hold it in your hands to feel its lumps. The data and customers will lie to you but the product never will. And most product orgs suck because they simply don’t use the products that they’re building; they ship incremental nothings without direction because they’re looking at spreadsheets all day long filled with junk data nothings.
See, I don’t know much about product stuff. I have no experience as a product manager, no experience running teams or building a company. Take everything I say here with an enormous silo of salt. But: I don’t care what the data shows me and I’m not sure I ever will. You can show me charts and spreadsheets all day long and I will not care. Tell me what your gut says instead after relentless experience of the product every day. This is the only way to see the world clearly.
Perhaps arbitrary, perhaps a bit naive, but the answer is vibe-driven development. If you have good experience of the product, your vibes will lead you down the right path of what to build next. I think this is why small orgs make better things faster than large orgs; they’re all about the vibes. Large orgs are bloated and frozen in place because they spend all their time talking about bullshit numbers instead of looking at the product. Whilst smaller orgs typically aren’t run by the numbers, they’re so focused on the product because they have to.
So when we use data to drive development it always leads us down the wrong path, it forces us to look in all the wrong places for the answers. Customers can tell you what sucks, sure. Dashboards and spreadsheets can show you attrition or whatever, yes. But these inputs can’t build a vision of your product for you. They’re mostly distractions and any moment with your eye off the product is a moment lost to making it better.
You can only build a great product if you care more for the vibes than for the data.