My 24-hour startup challenge retrospective

On the November 17 I participated in the 24-hour startup challenge where I live-streamed the (attempted) development of a product. The day itself turned into a disappointment for me as I wasn’t able to achieve much. My Macbook was very slow, my connection was poor, I underestimated the work involved and how fast I would learn and I simply didn’t end up achieving much of anything.

I felt quite disappointed and demotivated, but I thought it would be instructive for me to at least do a proper retrospective on the experience.

What went well?

  • I wrote an article leading up to the challenge about the definition of startups. I believe it was well-received, it got some traction and I enjoyed writing it.
  • I was forced to get some practice doing self-promotion. This was a great benefit. I gained a handful of twitter followers and met other indie makers in the space.
  • I was forced to try to develop something and release it. In the end I failed to develop something meaningful, but this showed me that I was underestimating how difficult this would be for me and how much time it would take.

What did not go well?

  • In the end I wasn’t able to achieve much. I think it’s attributable to the following causes:
    • I underestimated how much time it would take to set up a development environment for the technology I intended to use. I never reached the point where I understood how to get debugging information on why my application wasn’t working.
    • I underestimated how much time it would take to get up to speed with a tech stack I had no experience with (node.js and AWS Lambda). I was much too overconfident and ended up spending hours going over tutorials.
    • Leading up to the challenge, I underestimated how long it would take to find and get in touch with a responsive market.
    • My internet connection was poor and unstable, which made everything about the challenge more difficult.
    • The Macbook I was using is a bit old and it could barely handle the live-streaming software which slowed down all other things I was doing.
    • The audio on my stream is not very good. I think the mic was picking up the Macbook taking off due to the extra load of the broadcasting software.
Me, struggling during the 24 hour startup challenge

What should I continue doing?

  • Continue to participate in events like this.
  • Continue to do retrospectives.
  • Continue to improve my development skills.

What should I stop doing?

  • Stop assuming you can reach a meaningful level of competence in a new language/skill in a very short amount of time. You can learn fast, but you’re underestimating how much there is to learn.

Insights & Provoking Questions

How might I leverage the honesty effect of streaming?

Streaming puts you in a situation where you assume you’re being watched, even with 0 real-time viewers. This changes your behavior and I suspect it keeps you a bit more honest than you otherwise would have been. I can’t prove this, but I felt like I needed to be careful with my words in constructing my arguments for why I was doing what I was doing. How might I leverage the honesty effect of streaming to become a more successful at efforts like this?

How might I methodically go through the product discovery process?

Below you’ll find three iterations I’ve done (after the fact) using the Lean Business Model Canvas. If you look through the iterations you’ll see that my idea has evolved somewhat. I’ve moved away from the thought of doing a productized service. I think a self-service app where you can enter your podcast feed and generate an Alexa skill perhaps makes the most sense. What would be the benefit over, say, a universal Alexa podcast player such as Anypod? Not much, provided the experience is good (and based on the reviews it seems to be okay). I think there’s one angle that’s worth exploring and that’s whether you can capture email addresses of listeners with Alexa. Many podcasts have CTAs with the goal of capturing email addresses. If Alexa can do that directly, that could be interesting. But here we enter the realm of solution-looking-for-a-problem which is always dangerous.

An overarching point needs to be made, though, which is that it’s difficult to come up with a way to methodically make your way through this process of product discovery. How do you surface and map out assumptions? Which do you test first? What areas can you ignore in the beginning? I’m not satisfied with the approaches I’ve seen so far online and I will keep my brain primed on finding a better solution.

How might I assess whether it’s worthwhile to continue with this idea?

The fact I was able to book 4 podcast hosts for interviews based on an email alone indicates to me that there’s potentially something there. I will conduct all the interviews and report back on my findings. My first interviewee told me that he’s come to expect SaaS apps to be almost free. He was interested in talking about an Alexa app, but he did not immediately see the benefit over the use of Anypod. He seemed most interested in getting discovered by more listeners, which would be a nudge towards going down the route of creating a universal player (aka an Anypod clone). I don’t see immediately see how I could create an unfair advantage over Anypod (although an in-app CTA which captures emails might be worth exploring). I don’t see a clear argument in favor of continuing with this idea at this point.

How might I maximize my rate of iteration?

My approach was to maximize my rate of iteration. I focused primarily on finding a market that would be highly responsive so I could iterate with them quickly. I assumed that building the application would not be the slowest link in the chain. In the end this is what bogged me down. (That’s not to say that the learning step would have been fast.)

Figure 1: The Build-Measure-Learn cycle with a conceptual shortcut going from Build to Learn.

My assumption was that the learning step would be the most time-intensive and thus the bottleneck to my rate of iteration. I did no anticipate that building would in fact be much of a hurdle. Seeing the other participants whipping out cool functionality within hours, I realize that I need to learn how to build apps fast in order to make sure this does not become the bottleneck to my rate of iteration. The two questions I’m asking myself now are:

  1. How might I rapidly build prototypal apps? (e.g. what tech stack is best suited for apps-as-prototypes?)
  2. How might I rapidly learn with a market for my prototype (e.g. what’s the best way to find a responsive, suitable market?)

I believe the first problem has been solved by any one of these amazing young developers that whip out fully functional apps within hours. So if I want to improve the Build component of my iteration strategy I need only learn what they’re using.

The second question, however, I haven’t seen approached by anyone. What I have seen is people iterating quickly within a market to which they belong, or a single market they’ve found themselves (perhaps by accident). I haven’t seen anyone iterate across markets quickly, or find markets based on their responsiveness. I’m also not entirely convinced of my approach here. Looking for responsive markets might not be the best way to maximize your rate of iteration. For instance, any market may be experienced as being responsive, provided you have enough warm contacts one phone call away. I need to reflect on this a bit more.

The rise of indie product development streaming

There was a lot of hype around the 24 hour startup challenge. A lot of people were streaming and I can only assume quite a few people were watching as well. While preparing my stream I slowly discovered the world of Twitch and all the gamers that have been streaming for years. They’ve really made inroads into this space and there are a lot of tools on the market and established best practices. I expect this to take off more. If your hardware can handle it, streaming an easy to step to make solo product development more exciting and more social. I think this will lead to more supporting software to be developed, like Twitch plugins specifically for developers. I also think that some indie makers are going to figure out how to make money while streaming like many gamers seem to have figured out now as well. Lastly, I don’t think the indie maker community has converged on solution for making watching streaming attractive. I don’t find myself compelled to watch anyone else stream and hour-long streams don’t seem very watchable to begin with. I expect some indie makers will start to figure this out as well, making their streams more fun and more consumable.

 

My 24-hour Startup Challenge

Introduction

On November 17 an online live-streaming event was organized where over 300 people participated by building a startup / product in 24 hours. I too participated in this challenge. In this post I’ve laid out my approach and included updates in reverse chronological order.

Although I was happy with my preparation leading up to the event, the day itself was a bit of a disaster for me. I had to deal with a terrible hotel internet connection, a Macbook that could only barely handle the live streaming software and a lot of difficulties (due to my own underestimation of the complexity involved) setting up a basic development environment. You can read my entire retrospective here.

Update: 24 hour startup: Challenge day

Summary

On the challenge day my plan was to create a basic Alexa skill out of a podcast I had selected. This should not have been very difficult to do, but I ran into problems with setting up my development environment and figuring out how to debug the Alexa skill. In the end I wasn’t able to even get the basic skill to work, so I just focused on creating a landing page.

Stream

The entire playlist of all streams can be found here: https://www.youtube.com/playlist?list=PL9EffBKUMi4qxK9vGSudatQo3ImoeGm9B

The streams for the challenge day itself are here (they are broken up due to the connection:

Assets

Google drive folder for challenge day

Key achievements

  • I didn’t achieve much at all during the challenge. Read my retrospective here.
  • Got a better understanding for how Alexa skill development works
  • Created a landing page for Voicast at voicast.io

Update: Friday (1 day before) – Reaching out to podcasts

Stream

Friday (1 day1 before event) Part 1/1

Assets

Google drive folder for 1-day-before: https://drive.google.com/open?id=1Jczx4be9_M_cnnTQU4ho9fH09p21F5SD

Key achievements

  • Reached out to 57 podcasts: 4 meetings booked
  • Had one interview with an “Aspiring Product Manager” user
  • 0 pre-sales

Update: Thursday (2 days before) – Reaching out to markets & testing assumptions

Stream

Assets

Google drive folder for 3-days-before: https://drive.google.com/open?id=1xVURxl-4h3F5uHro1Q8oWFY1kACg_3ES

Key achievements

  • Conducted interview with a social media marketing agency
  • Collected leads from 100 podcasts
  • Started the gather email addresses

Comments

None

Update: Wednesday (3 days before) – Brainstorming & Testing assumptions about the markets

Stream

Key achievements

  • Booked 1 interview (Social media marketing agency)
  • Joined many communities
    • Many not responsive
    • Many communities are active (e.g. FB ones), but don’t engage
  • Cold email still seems comparatively quite effective
  • Identified some potential areas for exploration

Comments

Some things that were a bit surprising to me:

  • Takes a long time to get accepted to certain communities
  • Lots of spam groups that are barely useful
  • It’s HARD to find an audience to interact with
  • Even if you find a community, telling them you’re trying to solve one of their problems doesn’t really elicit a response
  • Cold emailing & calling is probably better

Update: Tuesday (4 days before) – Testing assumptions about the markets

Stream

Assets

Google Drive Folder for 4-days-before: https://drive.google.com/open?id=1WVDm8bZS0X5rA6Wc53BMjft_tSGgUf83

Key achievements

  • Expanded market list to contain audiences that are active on slack/telegram
  • Applied to join 10+ slack groups
  • Reached out to 1 prospect in 1 market

Comments

I really need to pick up speed, because the challenge is coming up and I haven’t had any conversations yet with potential markets.

Update: Monday (5 days before) – Brainstorming responsive markets

Stream

Monday (5 days before event) Part 1/1

Highlights & Assets

Google Drive Folder for 5-days-before: https://drive.google.com/open?id=1WVDm8bZS0X5rA6Wc53BMjft_tSGgUf83

List of high-potential markets

  • Social media agency
  • High frequency trading company
  • Sales agency
  • Private equity firms
  • Public relations
  • Translation agency
  • Funds
  • Legal services
  • Funds
  • Defense industry
  • Think tanks
  • Broadcast media

Key achievements

  • Brainstormed a list of markets likely to be responsive
  • Clarified criteria for picking high-potential markets
  • Scored markets based on new criteria

Initial Post: Approach

I will be participating in the 24 hr startup challenge and I wanted to make explicit and public my approach.

When we make our commitments public we can leverage the sense of being held accountable by the people we share our journey with. We also become less biased when we are required to make our thoughts public and we know that they will be scrutinized.

I think the 24 hour startup challenge is a great idea as I’ve laid out in this article.

However I think it’s important to be clear on what your goal is for the challenge. My goal is the following:

Arrive at a validated idea for a startup. (Or, to get as far as possible in that direction.)

The 24 hour constraint, like many other types of constraints, promotes creativity and forces you to finish what you’re working on.

One way (and from what I’ve seen the most common approach) is to choose a problem that you personally have and to build a solution for that problem with 24 hours. That’s a fair approach.

However, by the end of the challenge if everything goes as planned, I’ll have one product that I can use to validate my startup idea. I think I can do better than that. 🙂

Former head of Y-Combinator, Sam Altman, recently tweeted this:

As a former product manager I think this is probably also true for products. Rate of iteration is key. Rate of iteration is key because every iteration you’re testing a hypothesis. Roughly speaking, the more hypotheses you’re able to test (provided they’re good hypotheses) the better shape you’ll be in.

When is a product idea validated?

A hypothesis can be validated or invalidated. Your product idea, before you talk to anyone, is just a hypothesis. It’s just a test. You should treat it as just a test.

The point of “getting out of the building” is to validate or invalidate hypotheses about your product, your market, your user, your business model etc.

Your product is validated when you’ve iterated on the hypotheses that define your idea and you’ve proven that they’re all true. You can do this before you build the product.

The difficulty is that not every experiment yield a binary validated/invalidated result. If people sign up for your email list, does that mean your product is validated? The answer is: a little bit. Here is an ugly drawing of how I think about this.

If your friends like your product idea — that doesn’t mean anything. If strangers give up their email address, that’s slightly better, but still not great.

What we’re trying to figure out, ultimately, is whether there’s a sustainable business model contained within this idea of yours. The holy grail of validation is of course: Getting people to pay. And you can get people to do this before you build.

That’s exactly what I’m going to try to do.

I’m leaning slightly towards products, as opposed to services, because I enjoy building things. So I will orient myself towards product ideas while keeping my eyes open for services and productized services.

So the goal is to arrive at a validated startup idea and I will measure this by the amount of pre-orders I generate.

To increase my odds of achieving the goal, the approach I want to leverage is to maximize the amount of iterations I can do.

Most participants I’ve seen are going to build a product during the 24 hr period and @patwalls has said that doing some research prior is okay. So I think it’s okay to do some validation prior to starting.

So instead of starting with one idea, I want to test out different ideas before I start the challenge. In order to maximize the amount of iterations I can do I want to:

Target markets where I can iterate quickly

What do I mean by that. I want to target people with whom I can easily jump on a call with or chat with in the build up to the 24 hr challenge and especially during the challenge itself.

Here’s an ugly drawing of how I’m thinking about this currently:

If I were to target a certain type of market with certain problem, but it would take me a week just to get them on the phone, I won’t be able to iterate quickly.

I’d rather select my target market beforehand based on which target market can I iterate quickly with. These need to be people that by virtue of what they do for a living are very responsive, often online etc. Perhaps salespeople? I’m not sure. That’s one of the first steps to figure out 🙂

My ideal scenario is that I start the 24 hour challenge with some pre-orders in my bank account and some of my pre-order clients a phone call away to iterate on the product with me.

There are some more criteria I have for my target market to increase my odds. For that I encourage you to tune in to my live stream later today where I’ll go over these criteria and brainstorm market ideas.

I will start a stream today some time after 1 PM UTC. Stay tuned!

https://www.twitch.tv/jessemsz