If I were working at a start up right now, I would take a little time out and have my entire team watch this talk that Dan McKinley (@mcfunley) the Principal Engineer at Etsy gave about continuous experimentation. It reinforces a key lesson that is so simple, but so easily forgotten.
Here’s the video of his talk:
In the talk Dan outlines two projects, implementing infinite scroll and redesigning a drop down menu, and how two different approaches lead to very different results. The projects were both done by the same teams. But the difference was in how each project was managed. In the infinite scroll project, design, build, and measuring were done sequentially and each took up a considerable amount of time. However, because no measuring took place until the end of design and build, problems were not uncovered until many people had spent many hours on the project. Contrary, the drop down menu project was done in smaller, iterative sections, where slight changes were designed, built, and measured. If the results were good, another change was made. If the results were bad, they figured out how to improve or change it. Smart!
In the presentation, I love the part where Dan says, “we changed too many things at once”. I can see how this happens, I’ve been there, believe me!! There’s a huge temptation right now to implement a design pattern or feature, just because another company did. But you know what, unless you know the rationale behind why they did what they did, you might be setting yourself up for failure. For all you know, they could have implemented the change and be seeing horrible results. But you don’t know that, and you’ll probably end up with bad results too if you just copy them!
We must learn to justify every change we make and measure every change we make. That’s definitely not easy. And, it might require educating some people on your team about this. But in the end, it will be worth it.
And then, after you’re done watching that talk, I would then print out slide 72 of the presentation five times (image at top of this post) and post it all over your office so you and your team are always reminded about the value of working small so that you can see big changes.