By Merlin Carter
Most of us know what an MVP is supposed to do — in theory. The idea of an MVP is to create a product that is simple, cheap, and quick to build, so you can test it with real users. If they like it, you can build a better version. If they don’t, you haven’t wasted too much time or money.
But if it’s so simple, why do some engineering teams get stuck iterating on MVP-quality code? Why is it so hard to throw the MVP away and start from scratch?
In this week’s discussion, Stephan and I try to clarify this conundrum with some concrete examples. As usual, you can read or listen to the discussion below:
The boundaries between an MVP and a beta version
Merlin: Hi, Stefan. I would like to return to a previous conversation that we had. It was about the difference between MVPs and prototypes. And I would like to come back to it because I think sometimes people still struggle with the boundaries and the definitions and when to switch.
So just to recap what we were talking about: We were basically trying to come up with a clear definition between prototype and MVP and beta version and what have you…
And so I think a good metaphor for these types of concepts is to look at hardware products that you might see on a platform like Kickstarter. Maybe you’re designing a new 3D printer:
- You do a prototype, which is just in your workshop, and it’s made out of spare parts, and it looks really messy. And that’s your prototype just to show people your idea
- And then, the MVP might be something like you did a limited manufacturing run. you’ve made it with proper machinery. It looks like a real product, but you’ve got a limited run for your Kickstarter donors to give it a test and try and give their feedback. So it’s basically just getting feedback from your user base.
- And then the beta version would be the first edition of the product where you actually do a larger manufacturing run and where you’ve taken the feedback into account, and you’ve improved it, and you’re ready to release larger batches of the 3D printer.
And so if we look at software, then I think it’s a very similar process. But it’s a lot easier to move or to stick in one phase. And the difference between going from prototype to MVP to beta is a lot less clear because it’s just all code, and you just keep adding to the code. So I guess the question I want to start with is, what are the signs of the boundaries between when you’ve stopped being an MVP and you’re ready to make a beta version. Like how do you know when to move from one to the other?
Stephan: I think it’s very good that we bring the topic up again. I think the main point is there will be no clear signal like “ oh, now you should do the beta version”. I would even doubt that a beta version nowadays in all of the cloud businesses is still something that people do, or that’s more like MVP and then continuous development. Maybe let’s have a quick look at what we expect from MVP? Or why are you actually doing an MVP?
The idea is to have a minimum viable product, something that is working already but that allows you still to quickly iterate and especially to learn fast. And I think this is the main point behind an MVP. So you’re not yet at a stage where say, oh, my product is exactly shaped in a way that the market accepted.
So you’re still in a phase of finding the product-market fit for a particular product. and I think the main question is, okay, when do you know when a product-market fit is there? and normally I would say you see it when you have suddenly increased user numbers, you get very positive feedback from your existing users, things go viral.
So you maybe don’t even need to do any advertising for that. Because people just say: “Hey, this tool is so cool. I can only recommend that you use it.” So this is typically some kind of indicator that shows, “oh… I found product-market fit”. And you could now say, “Okay, now we know what the market needs…let’s build a proper product out of that”.
The problem with that is… if you can do that (just throw everything away and start from scratch) …that would not necessarily fit your startup journey because if you found a product-market fit, the next thing that you want to do is to grow further.
And what will you do? You will develop your MVP further into a more sustainable, bigger, complex, and more value-providing product for the customer. So there is very often not the case of “hey, let’s throw it away”, but you just continuously improve and develop the product that you already have further.
Merlin: Yeah, that makes me think of the technical aspect of the boundaries between MVP and the first version. So let’s look at a basic made-up example…
I was reading about early MVPs for some famous startups, like, you know, Dropbox had a basic WordPress site, I think, and maybe had a demo video, but they didn’t have much infrastructure behind it at the beginning because they just wanted to see if people were interested in the service.
And I’m thinking of… maybe like an online shop where you have a very limited product catalog, and maybe you’re loading that from a CSV file. And the people who are introducing new products can add them to the CSV file. And then it’ll just keep loading the products and displaying them, and that’s fine.
And then you start to get more users… but no one is coming with a bell to the engineering team, ringing it and saying, “product-market fit has been achieved, officially end the MVP phase!”.
So from their perspective, it’s going to be more like, alright, this CSV is now 5,000 rows, or we’re getting write conflicts, or there’s going to be some kind of technical indicators you’re going to outgrow whatever temporary solution you used to find product-market fit. So can you think of more technical indicators that your MVP is getting too small?
Stephan: I think I have a couple of them. the first one is, how many incidents do we have, right. So are you regularly being pulled out of bed in the middle of the night because some kind of alarm started, and you need to go and fix it. That could be a very good indicator that you have a problem with the application. This can also happen with more mature products as well. But I think, especially if you’re in such an early phase, you will see, like, more incidents, more problems, the more users use your product. So that’s definitely one case.
And you will then figure out also bottlenecks, right? Because if you stay with the CSV example, that maybe works for the first 200 products, but as you said, like with 5,000 products, it just takes two or three or five seconds to read the whole CSV file to then show it to the customer, to the user. And I can promise you if such a situation happens, the product management, or like the CEO, even himself or herself, will just stand at your desk and say, “look Stephan, five seconds… that’s not acceptable… please improve that”.
And then you, as a software engineer, say, “oh, okay… let’s put it into a database”. It’s nothing new. You can just put it there and then use the database, which is… like, millions of times faster than just reading from a file, and this is how you actually learn. But I think the main point is you will learn by mistakes or that there is some kind of improvement needed… if there are any problems, I think this is the strongest indicator, the problems will grow exponentially the more users you have. And there will be times when you say, “look, we must throw everything away and just build it from scratch”.
That can happen as well. but I would always question whether this is the right direction.
A lot of teams have really confusing spaghetti code. They will excuse that and say, ‘it’s okay. It’s still an MVP…‘ how do you tell the difference?
Merlin: Yeah, that’s, it’s an interesting point… because I read somewhere that there was an opinion of, I think, a CTO, that a lot of teams have really confusing spaghetti code and maybe a lot of tech debt, and they will excuse that and say “well, it’s you know, it’s okay. It’s still an MVP so…” (giving the promise of future improvement) “oh, we’re going to fix it soon”.
But it never stays out of that situation because, you know, it gets more and more complicated, and no one wants to have to rearchitect it. So it sounds like you’re saying the transition from MVP to the first version could be: A) a gradual thing where you replace things bit by bit, or B) you reach a point where you just can’t look away anymore. And you have to throw it away… but how do you tell the difference then?
Stephan: Yeah, I think, maybe taking that quote from the CTO that you mentioned, I think the CTO, then, didn’t do a very good job in managing his technical product. Because if you end up in a situation where I say, look, we only have spaghetti code around, then you did something wrong earlier, right?
Because we are speaking about technical debt here. And I think the important point is… it’s completely okay to have technical debt. But that should be a conscious decision. Whether you take a shortcut here and there to just be faster, which is like the main thing behind an MVP, to find the product fit, you should definitely track that.
And you should be aware of, “Hey, we’re taking a shortcut here”, and everyone knows that if you take a loan from a bank, you have to pay interest. And it’s the same with technical software. So if you take a shortcut here and there, at one point, you will pay the interest, and the longer you wait, the higher the interest will be, maybe it comes with the cost of… “I have to throw everything away because it’s cheaper to rebuild the whole thing” compared to paying billions in interest.
What are the habits of a good CTO to keep an eye out for things sticking into MVP status for too long?
Merlin: Yeah, because at the beginning, I asked what are the consequences of confusing this distinction? And so the answer to that is, you have to pay your technical debt a lot later, and it’s going to be a lot more than what you anticipated.
And another thing. I was curious then, as you were saying, inexperienced CTOs will make this mistake, and they won’t keep an eye out.
Stephan: It can also happen that they make this mistake. But should be aware of that… Let’s phrase it that way.
Merlin: So what are the habits of a good CTO to keep an eye out for things sticking into MVP status for too long? Like how do you stop that from happening?
Stephan: Yeah, I mean, as I said. So normally, you see, you found product-market fit, and the MVP phase is over. If you see that users are starting to use your product and then you just go further in whatever direction they start to use it. I think this should definitely be the very first time when you see, okay, we are now developing that product further. And we are putting it on a more stable foundation. For example, where you’re introducing a database… That could be a very good indicator because what you see is that the current technology or approach that you have used so far doesn’t fit anymore… for the needs of the product.
And if you reach these points, that could be a very good indicator to overall rethink whether you’re still doing the right things… in how you develop the product further or whether there are already some shortcuts that should now be fixed… paid back, in terms of interest.
So maybe let’s take one very concrete example because I think it’s very abstract.
So we had an MVP discussion at one of our portfolio companies quite a bit ago. And there, the topic was around creating a marketplace where people can provide a specific standardized product. and then other people can actually say, oh, I’m interested in the product. I’m going to buy it. and the starting point was the question, okay, is there a need for that?
And, I know still that I had a long discussion with the product team there, and also the engineering team where we sat together and said, look, we can connect it now to the database where we have these products in it… we can actually create a fancy frontend and all of that stuff.
And we actually discussed it in a strong way, wouldn’t it also be enough to just create a static HTML table, which somebody, even like me, could create within an hour?
Make it a bit fancy and just update that once an hour and then just track how people would react to the offers that are on that marketplace.
Because this way, this is cheap because you can just put it together in one or two hours overall. and you can immediately test how users react to that. And this is actually an MVP.
And if you then see, oh, people like this product or whatever you built, then you can start to say, “oh, now let’s make it update in real-time”. Let them buy the product… because before it was only fake, but then you really start to develop the product further, and then you will also take this conscious decision, say, “Okay, how do we connect it now? How do we integrate it with the current system?”
But the main purpose was let’s make it easy, cheap, and simple to just see whether people want to have this kind of product.
What does the interaction look like between a CTO and a product manager?”
Merlin: Yeah. That’s it’s interesting. Cause I was about to ask you what the interaction looks like between a CTO and a product manager (or product owner) when trying to decide what constitutes an MVP. So it sounds like you had differing opinions. So what was the friction there? You were opting for something as simple as possible, technically simple. And the people you were talking to wanted to have more features in the very first MVP. And so there was a process of negotiation, right?
Stephan: Yeah. I think that the main problem, at least that I saw, was that people have not been willing enough to strip down things. So what they saw as this would be the minimum was far away from what could have been the real. Because if you think about creating something that is already functioning in a way of it’s integrated with the systems and so on if you come to that situation, you should at least challenge whether this is still an MVP over, that’s already something behind the MVP states, like more mature products or to say “I would say you should strip that down as much as possible with as less and minimal dependencies to the existing setup (if there is any already) to learn as fast as possible. This is like the main purpose of that. And, of course, you will make people happier if they can buy the product immediately or if they can sort the [data].
But at the very end, [you need] to figure out what, whether people are into that specific product, there is no need to have it because you can just provide them with that list in that specific case and say, “Have a look at that. Do you like that?”
Ask them and get some kind of feedback. And then, based on the feedback, develop that product or that approach further.
Do you remember when it stopped being an MVP… was that a clear-cut transition?
Merlin: That’s a useful example. And if we take it further, do you remember when it stopped being an MVP and became more of a first version. Was that a clear-cut transition, or was it a fade-in and fade-out?
Stephan: I would say, in that specific case, it was the point when we actually connected this marketplace to the existing system. At a point where people really have been able to buy these products in the marketplace. That was a point in time where I would say, “Okay, we’re leaving the MVP phase because we have proven that people are interested in a product overall”.
And now it’s the next step to improve the experience with that, right? So you put the foundation, or you integrate it. You can actually execute things automatically, and now the next step is then to prove the product… still based on feedback, analytics, and so on, but you have the first version that is really functional compared to a static list that is just pre-generated every hour, for example.
Did the company throw away everything?
Merlin: Yeah. And so when you connected it to the existing systems, did the company throw away everything they had done up to that point, or did they still retain some of it?
Stephan: No, if you take the very extreme example of having an HTML-created list, this was just thrown away. We reused some parts of the interface in terms of, okay, this is how it could look in the real product… but the rest was thrown away. And I think this was also not a problem because the total engineering investment was really small.
Also, there was no need to actually say, “oh, we already invested… like 500 hours into that… now we must continue to develop that further”. Because there was actually nothing…but a static HTML file in that specific case that showed a table of products.
The point is not to invest too much engineering resource before you even know if it’s even a viable idea
Merlin: Exactly. I think that’s a good example. You can really create a decent MVP with a very minimal technical investment. And you could even use No Code tools or what have you as another alternative. And the point is not to, not to invest too much engineering resources into it at the beginning before you even know if it’s even a viable idea.
Stephan: Yeah, this is exactly the point. So you try to figure out whether people are interested in a product and whether they would be interested in buying it. This is like the main purpose of an MVP. If you figure that out. out, then you’re on the way to building the real product, and the less time and money you need to put into finding this sweet spot for your customers, the better it is and the cheaper it is, and the faster you are. And I think that’s the main idea behind the MVP.
Merlin: Exactly — because I want it to make that point specifically. Because engineering, as we know, is especially expensive. And so if you can really reduce your own reliance on that while you’re trying to find product-market fit, then you don’t burn through your cash as quickly.
Merlin: Thank you. That was very insightful. I’m glad we went more into a detailed example, and I’m looking forward to talking to you next time.
Stephan ]: Likewise, thanks a lot. Talk to you soon.