How to build a product with Reddit's Steve Huffman and Twitch's Emmett Shear

So, this is going to be the first of four lectures on how to build a great product. I apologize because none of the people here today can speak to that topic. My name is Michael, I'm the CEO of YC and help run the batch and program as most of you understand YC to be. This here is Steve, who's a co-founder and CEO of Reddit. And there's Emmett, who's a co-founder and CEO of Twitch.

Can you guys introduce yourselves in a couple seconds? You just did, but I can do it again. I'm Steve, yeah, I'm co-founder and CEO of Reddit. I studied computer science, I graduated from UVA in 2005. I was at Reddit for five years, I left for five years. I've been back for about a year and a half. My name's Emmett, I graduated from Yale Computer Science in 2005.

I started my first company right out of college. We made a bad copy of Google Calendar before Google Calendar existed. And- Google Calendar copied you. Yeah, that's right. And then we started Justin TV, which is the company that wound up turning into Twitch. And so I've been doing, basically continuously employed, working on that since 2006.

And then can you just, each of you guys say whatever the KPI you guys track for your business and approximately what it is, just to give folks a sense of why you guys might be experts at building great products. One of our more shareable ones is monthly active users, and we hit 300 million last month. Nice. That's a good number.

Our primary metric that we care about is a metric called minutes watched. It's the number of minutes of video that are consumed every month on Twitch. And we just crossed 30 billion minutes this year for the first time, which is very exciting. No, per month. So that's the main thing we look at, rather than uniques.

So what we're going to do for you guys today is I have about 12 questions I'm going to ask these guys. We're going to see how many of them we get through, and then we're going to leave the last ten minutes for questions about startup stuff, YC, or whatever else you'd like to talk about. My goal is to kind of focus this advice on early, early stage.

So these guys do a lot of stuff now that really is completely irrelevant to you. But hopefully they'll be talking to you about the things they did, bad or good, when they were starting. So I want to start with one of the major tenets of YC is talking to your users. And we repeat that a bunch of times, and then we don't really go into any detail.

So can you guys talk a little bit about how you talk to your users? Maybe the good and maybe some of the mistakes you've made as well. Sure, so most of the advice I give today will have two perspectives, one at Reddit and one at Hipmunk. because the companies were completely different and had different paths.

And I think kind of the ground rules for this discussion, the first piece of advice I would give is every company is really different and there are lots of different paths you can take. And what works in one place very likely won't work in another. So at Reddit, I mean, the product is talking to users, right? That was built in.

In the early days, we didn't have comments and we didn't have communities. So actually, we got a lot of emails for the first probably six months, was the way we communicated to our users. But when we added the comment feature, there was basically no option but to talk to our users. And the challenge then becomes, which users do you want to listen to? And what are they actually saying?

And what do they actually mean? Because they're often saying one thing and they mean something else. At Hipmunk, because we were selling plane tickets and hotel reservations, it was completely different. We didn't really have a good forum for connecting with our users. We used a product called Olark, which was basically this embedded JavaScript chat thing. And that was indispensable.

We still use it. It was users, if they got stuck, they could ask us for help. If they wanted to give us compliments or criticisms, they could do that. And we can often, both over email at Reddit and over Olark at Hipmunk, turn emotional users, angry users, often will become your most loyal users. If you can flip them around, there's this absolute value of emotion.

And so if you can find those angry users who are having a bad time and treat them well, they will likely turn into some of your most loyal supporters. So we found that whole cycle to be very valuable. The interesting thing for me between Twitch and Justin. tv, which are two very similar products, right? They're both live video products with chat. For Justin. tv, we were our own users.

We were building a product for us to build a TV show. And after that, we really sort of declined to actually speak to users in a pretty significant way.

And unsurprisingly, all of the good ideas we had were things we came up with when we were building it for ourselves, which sort of brings me to my first point of talking to users, is it's possible, especially very early, to build a product for yourself. And if you truly are building something that is for you to use, that can be super effective.

In fact, you don't necessarily need to talk to anyone else. Eventually, you'll need to because you'll want to learn about people other than yourself, but for the first three months, it's possible to build something that's just something that you want, but you have to actually really want it. You can't be lying to yourself about that. It can't be something that you think is cool.

It has to be something you, yourself, actually desperately want to use every day. And then we went through a long period in the desert where we were wondering about not talking to users, not really using our own product, and not building anything of value. And the thing we did with Twitch that I thought was really good for talking to users is we talked to them in person.

I'd done some amount of surveys and talking to users over email, and I found that getting people on, at least on Skype, or if not, literally in person, changed my depth of understanding of who they were. The other thing I would say about it is that we focused down really hard on which of our users we cared about.

And we decided it was the broadcasters, and we decided not only was it the broadcasters, it was the successful broadcasters. We didn't want to talk to people who had two viewers. We wanted to talk to people who had 200 viewers. And so we went and talked to a bunch of broadcasters on the platform, because we decided they were the important ones.

The other thing we did that was, I think, really valuable is, we decided it wasn't just people who are currently using our service, it was people who we wanted to use our service. And so we talked to a bunch of people who were streaming on other platforms, rather than just already on Twitch.

Because the people who have chosen to not use your product, especially if they've deliberately looked at your product, decided it sucked, and went somewhere else, are some of the best people to talk to. Because they know what's wrong with it. And so I actually think the most valuable things I learned, especially early on, were from users who had tried Justin.

TV, they'd tried Ustream, they'd tried Own3D, they'd tried YouTube, and they had a coherent opinion as to why ours was not the one they chose. And that was super valuable, because they really pointed at the details of where we were wrong. And often those details are smaller than you think. Memorably, one of the biggest issues was, we were not listed on the Team Liquid StarCraft fan website.

That was one of, we lost maybe 3,000 broadcasters over that. We had to email them and ask them to re-list us, because they just didn't have us there. Yeah, tiny, tiny issue, but really mattered to our users. It's easy to run your company for years, and then be like, I should have fixed this stupid one day fix, if you don't talk to your users.

So a lot of people talk about MVPs, minimum viable products. How many of you guys have heard that term, MVPs? So one of the challenges that I have is that when folks like you apply to YC and get into YC, even though you know the idea that you should be releasing something that's crappy, you don't do it.

And as a result, users slow down, and oftentimes that prevents you from getting into YC, or prevents you from building and talking to customers. And so I think that what's interesting is that Justin. TV and Reddit are actually pretty good examples of crappy MVPs. So can you guys describe how the products worked in the beginning, or maybe how they only kind of worked?

Yeah, we started, I think I wrote my first line of code on June 3rd or 4th, 2005 for Reddit, and we launched on June 22nd. I didn't even launch it. Paul Graham linked to us from his blog without telling me. And from then, it was on. And the good thing about that, for us in particular, is we didn't actually have a vision for Reddit, for better or for worse.

Which would affect us in a lot of different ways going forward. But as soon as we launched, that's when we started to find a path, and we just would kind of follow the users every day as what do we think is best for the users, what do we think is best for us, and just build towards that. And we built a lot of loyalty through those actions.

And that would not have happened if we were sitting in our apartment not Because we launched, and then for the next six months, we just added a ton of new features, only about probably 25% of which actually lasted more than a day or two, but there was stuff we thought was a great idea. We had these categories.

I remember in July at some point, I made Alexis, my co-founder, stay up all night one night categorizing every link that had ever been submitted to Reddit, and the next morning I was like, we launched, I was like, it's not working. Turned it off. But we could have probably, if we had lost, we could have had that feature until November, and then launched, and then wondered why it didn't work, so.

I think that Amazon actually has a really good saying about minimum viable products. They like to call them, what they're aiming for, a minimum remarkable product, which I think gets more at what you're trying to accomplish, which is a viable product brings into mind this whole ecosystem of support, like a viable organism.

That's got all the things it needs to live, and you don't actually want that. If you're launching something that's actually self-sufficient, that's viable in a real, this thing could go live on its own sense, you definitely waited too long. This thing is going to be on life support when you launch it, and you are the life support system.

What you actually want is you want something about the product when you launch it to be actually good. You want to get to the part where you've built something that maybe at least one person in the world finds remarkably useful or remarkably good. You don't know, really, if you've gotten there until you put it in front of someone, which is why you need to launch it so quickly.

But I think it's really important to remember Dropbox, for example, one of the big YC successes, did not launch an immediate product, because they were building a backup system. And backup systems aren't allowed to lose your data.

And so the last thing you want to do if you're building backup software is go rush something out that's going to lose your customer's data, because you will never get that trust back. And so, the minimum remarkable product for Dropbox was a video basically saying, look at this awesome thing we're going to build, and getting people to sign up for a newsletter.

And that worked because the visual idea of what Dropbox was going to do was super compelling. That would not have worked for Justin. TV. We really needed to get something out in front of customers immediately. Most importantly, because we really needed to produce this show. We had this idea, we're going to go build a show.

And we basically did the minimum possible work, maybe a little less than that, to get a show onto the Internet. And when we launched it, it could only have one channel. The way that broadcasting worked, the only way we could actually get it to work consistently is we had a Mac Mini running Parallels, which is a virtualization environment for Windows XP.

Where we had this piece of broadcasting software called. It got bought by Adobe later. OnFlix, OnLot, I can't remember. Anyway, that we were automating with Windows shell scripting to pick up the right, the camera input, and then auto reboot if it went down. Because it broke all the time, because it was a really buggy software. And that extremely jury-rigged thing worked.

It put a video stream on the Internet. And it was not a scale, we could not scale that up to thousands of streams. It was impossible. But we got something that at least was compelling to at least one person onto the Internet, that was compelling to us. And that was a critical step for us. Because before we did that, we basically weren't learning anything. At HitMonk, we again started in June, 2010.

And I had given my co-founder basically this ultimatum of what our MVP was. We have to have legit data, because we could scrape anybody. We needed legit data, real data. And when a customer buys a flight, we get revenue. And I was like, if we can do that in three months, we will continue with this business. If we can't launch that in three months, we're calling it.

And that for me, MVP, that was that minimum useful thing. If we had a customer who was giving us money, that meant we did something right. And unfortunately, we actually achieved that in three months, so I worked on it for five years. But I think it's good sometimes to set yourself deadlines and specific milestones to keep you focused and keep you pointed in the right direction.

So another thing that comes up a lot, and this is actually one of the things that hits when a company gets into YC, is we will often ask them, what are they doing for analytics? What are they tracking? And I think that this is another one of those MVP-like things. Everyone knows they should be doing it, and it's the first thing that gets cut off of any list.

So can you guys talk about how an early stage startup should think about measuring what analytics tools should they be using, and generally how you incorporate tracking into building product? All right, I have two very different answers, so reality is somewhere in between. At Reddit, we just didn't. We didn't know our traffic for years. We didn't measure anything.

We did product development by intuition for a very long time. Company ended up working out. I'm still not quite sure exactly how many users we have. And it's a huge, huge pain in the ass now. How many users do you have? When you get to 300 million, it's like, I, you know. No, in all seriousness, we have no idea how many unique people use that every month.

It's probably somewhere between 250 and 320, yeah. Anyway, at HitMonk, we were actually really good about it from day one, and we were really, because we didn't have that kind of instant feedback with our users, we actually got really disciplined about testing, and analytics, and measuring, and watching.

And that company lasted, honestly, quite a bit longer than I think it would have without doing that. And so, if I could do it again, there are a lot of corners you can cut when you're building your MVP. Like you can create all sorts of tech sins, and corporate sins, and I don't know, be completely dysfunctional, your users don't care.

But data debt can haunt you, so, because you can't make up missing data. And eventually, your intuition's going to fail you. So, I would think very carefully about what's the minimum viable data? You don't even have to look at it, just log it somewhere, so it's there when you need it. To reinforce that point, having historical baselines for important user actions, you will thank yourself later.

Every time I've ever been like, right, how many people are using that feature? And we don't know, and we can't find out. And now, even if we find out how much it is now, we don't know if that's up a lot or down a lot. We have no idea what the trends are. It's awful, and you can't make a decision for another three to six months as you build that baseline.

Actually, I got the best advice about this from Suhail, who runs Mixpanel, when he was convincing us to try Mixpanel for the first time. Which, by the way, didn't work, because Mixpanel at that time was only like six months old, and it broke when we tried to instrument everything. But we went back to- It's nice of me to use Mixpanel before you came to that conclusion, though. Yeah, you're welcome.

It works better now. It works a lot better now. We went back to it. We're still on Mixpanel today. We use a bunch of other stuff, too, because we're a little too big for just Mixpanel.

But Mixpanel, the advice Suhail gave me that was, I think, more important than the software or using Mixpanel in specific, even though I do think it's good software, was pick your top five to seven most important user actions, whatever those are, and just log those. You don't need to log everything. You really don't. Most of the things people can do with your product aren't important.

There's maybe five things people can do, like for Reddit, right? It might be vote on something, comment on something, submit a story, load a page, I don't know, some short list of things. And those are important. And you can kind of just ignore everything else. If they're not instrumented, who cares?

And so we instrumented, I watched a minute of video, I sent a chat message, I followed a stream, and I bought a subscription. And that turned out to just give, those four actions gave us a huge amount of insight into what was going on with our customers that we hadn't had before.

This is one of those areas where if I could go back and give myself just advice for like 30 minutes that would dramatically affect the trajectory of a company, it would be around this topic. And so, if I were in your shoes, I would just take half a day and just read best practices for collecting events and storing them.

But, and also, I know the other thing I'd recommend is use a third party service. Like use, collect logs, by all means, put them into your own system because I guarantee you, you'll want to do something the third party service doesn't allow at some point, and you'll be super annoyed if your data's only in the third party service, so collect logs or whatever.

But just in the beginning, you don't have time to mess around with building analytics tools. So just use Mixpanel or use Google Analytics or use whatever, anything. They're all better than what you need right now. Later on, you'll need something more complicated. You don't need it today.

So, one of the challenges that happens usually after a company, in the early stages, starts talking to their customers, is they realize maybe there's a big disconnect between what they thought their customers wanted and what they actually want. And at YC, this usually comes to us in office hours in terms of, we call the big redesign.

So, everything about our site's incorrect, we just basically need to rebuild it from scratch, or rebuild huge components of it from scratch, in order to start serving our customers. Needless to say, this is a very, how do I phrase this? Distracting and oftentimes not fruitful exercise.

But how do you see that type of challenge of, man, we got this feedback and we think we need to be going in a different direction now. But how do you deal with that kind of challenge? That's a big one. And I think you need to be clear whether you're talking about tech or the product. I'll repeat what I said before, is your users don't care about your tech.

So, if you can move forward, leave that for another day. When you're talking about redesigning your products, you're probably in trouble because whatever habits led you to build a product your customers don't like, if you haven't addressed those habits, you're probably going to build another product your customers don't like. So, I would make sure that, because I've been through lots of rewrites.

At Hipmunk, we rewrote our flight and hotels probably half a dozen times each. At Reddit, we rewrote the tech a bunch of times, and we're redoing the UI and UX now with better habits and better data. And so, you want to make sure that you know something new. But honestly, I would, if you're in that situation, I would also go through another mental exercise of, okay, what's the new MVP?

Right, it comes up in tech a lot, right, the second system syndrome. The same thing happens in the product, where you're like, all right, I've got this vision, I'm going to fix all of the products, all of the mistakes at once, and it's going to be perfect. That product never ships, it never ships. So, can you iterate your way there? Can you just start from scratch with an MVP and build it back up?

Those are much more likely to actually get you somewhere. But they're also harder to do and requires a little bit more discipline. So, I've also gone through a bunch of redesigns, big redesigns, where you have to rebuild the whole product. They've been mixed, actually. We've had some that were super successful. Twitch did a big redesign of our mobile app maybe three years ago.

From the first version that had basically been a cloned version of the Justin. TV app to a completely new version. It's basically the version you can use today. And that increased engagement by like 35, 40% per customer when we launched it, it's huge. Changes that make that big of an impact are actually really rare. Usually data errors. Yeah, yeah, yeah, they're usually data errors.

And in this case it wasn't. In this case it actually was a huge increase in usage. And so, absolutely, redesigns can make a big difference. But we weren't doing the big Twitch. We didn't do that redesign because we discovered, we couldn't move forward on existing products.

Because we got some insights as to what things should be presented at the top of the nav, and we redid the navigation to present those things at the top level instead of burying them three clicks deep, three touches deep, I should say.

Most of the time when I see people doing a big redesign, it's because they've fallen into this trap that engineers and product managers often fall into, which is, we have to fix these eight things. Let's just fix them all at once, it'll be easier. That's wrong. You should just fix them each one at a time individually. If you have eight things that are wrong with your product, that's awesome.

But if you have this huge create list of things, just fix them one at a time. Fix everything one at a time. Crank them out quickly, you'll get there way faster one change at a time than you will in a big rewrite. A big rewrite for a really early stage startup is almost equivalent to pivoting the company, and I just generally advise against it.

It's this weird sort of programmer product procrastination. If I don't know what to do, therefore I'm going to invent this big project to work on. We don't need to use that library, I'm going to build it myself. Very calm. Use the damn library. So I'd like to talk a little bit about decision making.

So maybe after you talk to your users, but before you decide what to build, usually there has to be some type of consensus building process, either functional or dysfunctional, between the co-founders. And what are some tips or tricks on how to come to a consensus? And then, maybe more importantly, how do you deal with the contrarian?

How do you deal with having to make a decision where you know there's going to be a disagreement? Well, I think the first thing you said is actually super important. It's something I want to reinforce. You talk to your users first, and then you have the ideas about the product. Almost everyone does it in the opposite order. It's called product validation.

If you find yourself thinking, I'm going to go talk to my users to validate my product idea, you've gone horribly off the rails. You do it in the other order. You don't talk to users to validate your product ideas. You talk to your users to have your product ideas.

And so, I just have a rule in general for people who want to have a voice in product design, is that they better talk to the users and look at the data too. If you haven't talked to users and you haven't looked at data, you don't get to have an opinion about the product. I'm sorry, the person who's actually done the work gets to have the opinion. You can have ideas, but they get to make the call.

But in the case you wind up where both people really have talked to the users, and both people really have looked at the data, and people still disagree as to what we should build. Which, by the way, is actually much more rare. Once both people have actually talked to users and actually looked at the data, it's usually relatively easy to get to consensus.

But it will come up where people still disagree. I have found it's better when you just have someone who's in charge. At the end of the day, you have someone who's the CEO, and everyone agrees this person's judgment will be trusted. And you let them make the call. Don't avoid conflict, talk about it, argue about it. But then, you have someone who just gets to make the call.

because anything else leads to just decision making that's far too slow for a startup. Yeah, maybe it's because I'm usually the one in charge. I just feel like I don't have a lot of disagreements. There are- You're the CEO. Yeah. Largely, I find the situation I'm in, I say one of two things a lot, which is I don't want to argue about it if we can just test it.

And I'm very willing to be surprised on this. You're probably right, but this isn't getting into my top three important things to worry about today. So, let's just see. It's easy to do that when you have more resources to let's just see.

But I think about Alexis and I over the years, and Adam and I over the years at HitMonk, I think as Emma pointed out, we very rarely had these philosophical differences over with the product. Sometimes it was timing, like should we build this thing now? Like what's the prioritization of these things?

But we always agreed on the general mechanics of what we were doing and why we were doing it, and so the strategy might change. But the actual, the details were very rarely not emotional. So if those are getting emotional, I think there's probably something else structurally wrong. You might be swimming upstream. That happens a lot.

The arguments I've gotten over the last year, like when we try to promote, we do cross promotion from mobile web to the app, it just doesn't feel great. It works, it works, and we like the numbers going up. But it's not really with the user's best intention in mind all the time. Sometimes it drifts into this, it's good for the company, but maybe not for the users.

And when you're in those situations, that's usually when the disagreements come out. And you're not really arguing about the future anymore. You're arguing about short term or long term, or best for the business, or best for the users. And then when you beat at it, and beat at it, and beat at it, eventually you come to a solution that actually feels right, right?

And when it meets your intuition, you don't feel like you're fighting the technology, you're not fighting the users anymore. Those are usually situations I look out for. So I want to get to you guys telling product story, but I want to cover one topic first, which is that let's take people past launch.

Where now you need to have some type of process around talking to users, looking at data, making a decision, building a product, releasing a product. Checking your success and repeating product development cycle. What is some advice you can give to early stage startups around how to set that up, what that cadence should be, maybe some pitfalls to avoid?

The first, just to take one little step back, is I see a lot of startups get too, they get too emotionally high when the numbers are going up. And they're not investing in this sort of process, because growth masks all problems. And then at some point that kind of stops, and you don't have any of these mannerisms.

I also see startups get a little too low when things are going right at first, and they kind of psych themselves out. So you want to live in that in between, right? If the data's really good, well, check the data, right? If the data's really bad, check the data.

Okay, so then in terms of the actual process, first of all, it's going to be different at every company, and it's going to change as you scale. I think it's just most important to have a way of working. To have, if you just want to start out really, really specific of, the way I like to think about it is, where do we want to be in a year? Okay, let's draw a straight line from that to here.

What do we need to be doing today to get there in a year? So that's our hypothesis. Do we need to validate anything about that? Probably a few things here or there. Do that. Did it check out? Yes, and then it's just straight line. I like to work on two-week cycles.

Check in every week on basically that kind of thing. Like, where do we want to be in a year? Are we still on track? Are we still doing the right thing? Does it still make sense? Is our assumption still holding to be true? If not, pop it up a stack. And then you give people whatever it is.

If you're an early startup, you give them 13 days to work. And if you're a more mature startup, you give them nine days to work. And then keep going after that. I found that that's worked really well for me. One thing I found actually is that your goal of talking to users shouldn't be course corrections on a week-to-week basis.

Like, I did a whole crap ton of talking to users at the very start of Twitch, trying to figure out what the users wanted, how to think about them. But then, actually, we could have gone, we didn't, because we had other reasons to go talk to users. But we could have gone six months without talking to a user. Because we actually, we knew what users wanted. They wanted to make money on the service.

Like, streamers wanted to be able to make money. They wanted to reach more viewers. And they wanted positive social feedback for what they were doing. And you kind of just check anything you were doing. Is this going to deliver this to our streamers? Like, yes or no? And if so, how much? Great, we don't have to talk to anybody.

You already know what they want. And you need to go back and revisit it and sort of talk to users periodically because you need to understand more of the nuances of, are there things about your service that are bugging them? Is someone else doing something really cool? But the goal of talking to users is not to get them to tell you what features to build, because users are really bad at that.

They actually have no idea what features to build. The goal of talking to users is to get to understand them really well. And users don't change that fast. So I actually, I personally prefer more in-depth time with my customers, talking to them for a while.

And then, depending on the size of the thing we need to do or trying to move, we might go just work and build and look at metrics for the next six months, and not actually talk to users that much for a while. Because if you know that what people want is to make more money, you can spend six months optimizing the sell-through rate on an object or on how much advertising you have.

And you just can have faith that it's going to be good, and you will be rewarded. I think the classic advice that's very difficult to follow is you need to have the courage of your convictions. So you decide what you're going to build. Might take six months to build it. And at Reddit, we just kind of do it in front of everybody and get flamed by our users the whole time.

Which is fine, like I'm used to it, but I have to kind of tell our product people, like just keep pushing, keep pushing, just chip a little bit every week. And then in six months, they'll get it. And at the same time, the completely contradictory advice is you have to know when you're wrong. You have to be able to identify when you're wrong.

You have to be able to say, all of you people are wrong and I'm right. And then at some point, you have to decide, wait, no, you're right. And if you can walk that line, you need to have this good combination of high ego and high humility and balance those. But I always try to just kind of run this process in my head. Wait, am I wrong? Nope, they're so wrong. Just kidding.

The making sure, as much as I said, don't talk to users. You don't have to talk to users every week. It's totally unnecessary. You do need to look at your numbers every week. You really need to know, and you don't need to look at them every day. It's actually pointless and almost just ego gratification to look at your numbers every day. But you should look at your numbers every single week.

And you should confront, are they up or down, and ask yourself why. And the holy grail of product design is for your numbers to move and for you to truly understand why they moved. We still struggle with that. There have been points where I felt like I understood it, and then it slips away again as the business gets more complicated.

But you're always trying to be able to explain fully down to atomic ground truth. We are up 20% this week because we got this many more new devices in, and our retention rate is this on those devices, and this is the conversion rates. And if you do the math, and you can explain it all the way down to this new feature moved this conversion rate on this page by this amount, and therefore we are up.

And if you can really do that, it's super powerful. It's also really hard. But you'll never get there if you don't build the intuition just looking at it every week. So in the last five minutes before we open for questions, can you guys walk through a feature that is visible on your sites now? And I'll let you choose. That's either bad or good. And talk about the process of. and how it came to be.

I have a lot. I'll talk about one of my favorite ones. This is from the early days of Reddit. So the early days of Reddit, it was just links. Everything was external. And eventually we added comments, probably about six months in. The first comment was, this is the end of Reddit.

The first of, that would be the first comment on every product we've ever released, and users started doing this curious thing where they would, so on any post, you could go, you could click on the link, or you could click on the comments page to go to the comments for that page. Our IDs for posts are just a counter that's in the URL.

So users would submit the URL that would represent the comment page for the URL they're submitting, right? So they would just increment, they would take the newest post and increment by one and submit that URL. So that you had a post that would link to its own comments page. And it was really cool, so we called those self posts.

And users were doing this, and I remember watching them doing it and thinking this was really cool. I would have never thought to do this. Sometimes they'd guess wrong, right? And so they'd submit a post to somebody else's comment page, which was funny.

And so because our user base at the time was fairly technical, the way we made this work is when you're submitting a link, you would just type in the URL area, self. And it would automatically do that little circular thing for you. And that was the beginning of self posts on Reddit. Self posts are now 60% of our content, I don't know, a few million a day.

And we would have never built that feature if we hadn't been watching our users. I don't think I, I mean, I'm explaining it now, but if I explain that feature without it existing, I don't know if it would make a ton of sense. And users, most of them didn't get it at first either. They were just like, comments are destroying the site, and there's not even a link here, it's just comments.

And it just kind of goes to show, I think, the best product ideas are the ones where your users are doing it anyway, and you just grease it a little bit. When you're not, that's what I talked about before, where you're either swimming upstream or everything just feels right. That was the classic example of it just felt right, and it completely changed the site.

I think my favorite story here is an advertising product, actually, which is funny. So all of our users told us really, really clearly, we want to show advertising and make money off of our streams. That was a clear desire, everybody wanted it.

And they also told us their least favorite thing about our competing competitors' products was that they would show these ads over the stream, and it would be disruptive to their user's experience, and they hated it. And it was very interesting getting these two pieces of feedback, because they both really wanted us to put ads on.

And sometimes different people, sometimes actually literally the same people, their biggest complaint about the opposing service was competitive service is annoying my viewers with all these ads, and this is a hard nut to crack because you kind of need to solve for both. You're not allowed to just give up as a product manager and pick one.

And I'm really proud of our solution, which was we gave them a button, we inverted it. And we gave the streamer the ability to hit a button to run ads on their own channel. And it's funny because normally you'd think of that as something that will, if you're uploading content to Facebook, you would never press a button on that to show more ads to your friends.

But because we were cutting them in on the revenue, they were actually heavily incentivized to hit this button. And we basically got this paid workforce of people placing advertising in the downtime of the stream.

And I'm really proud of that clever solution, and I think it points at something that I think is a really important principle of product design in general, which is that you want to just try inverting all of your assumptions. What we got stuck on that advertising problem for a long time, because we didn't want to make it disruptive to the customer experience.

We couldn't figure out how to automate putting it on the stream in a good way. And at the same time, we really wanted to show ads on the video stream. And we had this implicit assumption in our heads that we had to figure out where to put the ad. And once we gave that assumption up, it was much more clear what to do going forward.

And I think that's one of the most, I don't know, I was really proud of that product design because it was one of the things that made our broadcasters super happy, and you can't complain, you hit the button. How many times has that button clicked every day now? That's a really good question. I don't know the numbers offhand. A lot. It clicked very often.

People really like that button because it's a click the button, make $15 button. And like, you know. But they're also aware of the tradeoff, which is that their viewers don't like it when they hit the button too much. And so by putting that in their court, they get to make the optimal tradeoff for their stream in their situation. All right, those are my questions. I want to open it up to you guys.

I'm happy to talk about product, happy to talk about YC stuff, whatever you'd like to chat about. Go ahead. Talk about the process of discovering you've had product market fit, getting past that launch bow wave and discovering there's something sustainable behind this, like real core group of users that are really committed to this.

Just to repeat the question, how did you know when you had product market fit? So at Reddit, this is the nice scenario. We were growing, despite not knowing anything about our business, knowing little about our users, not really knowing how to run a business, not having a ton of product sense, being down a lot, we were growing. So that's the nice scenario. That's a good one.

At Hipmunk, our natural state was zero users. I'm not sure we ever got product market fit. The users who came to Hipmunk liked it, but we had to fight for every single one of them. It's possible to do that, but we were, again, I think that entire company, we were swimming upstream. And I remember the day we kind of realized that. We ran the survey of like, why do you use Hipmunk?

And we put a lot of effort into the product. We thought it was a really good product. Legitimately, you could find a flight a lot faster on us than our competitors. 35% of people said it's because of the logo. And 50% of people said because we had lower prices, which we didn't. So, I mean, it's like, we made it. The company ran for five years, it ended up selling, it's just fine.

But yeah, I think we were really fighting that one. I think this is a huge misconception, by the way. Most of the companies that apply to YC don't have product market fit. Most of the companies on demo day don't have product market fit. Most companies die before product market fit. Like, a lot of people talk about product market fit as just like this organic step in the process.

Oh, you raise Angel, and then you get an office, and you hire some people. And you have product market fit, and it's just like you would be surprised at how few companies. Like, product market fit is typically described as like, you are overwhelmed with demand for your product. It's really obvious when you've hit product market fit. So your product is bad.

You can point at seven things that are broken about it. And you have all these features you know you need to build. And weirdly, it's growing really fast anyways. Like, it's really obvious in your product market fit, your product is growing like crazy.

And you are spending, you don't get to fix any things that are broken or bad about your product, because you're spending all of your time fixing it as it scales. And you are, so the metaphor I use for startups is kind of Sisyphean. You're like pushing this boulder up a hill. And you're pushing the boulder, and you're pushing the boulder, and you're pushing the boulder.

Product market fit is when you crest the hill. And your job doesn't get easier, but it does change a lot, because now you're chasing the boulder as it tries to roll away from you down the hill on the other side. And now you're sprinting flat out trying to catch up, which is hard in its own way, but it feels totally different.

Because whereas before you have this like, god, we're like, every inch is like, as Steve was describing with Hipmunk, every inch is like you're pushing this boulder and you're fighting for every inch, now it's like, crap, if we don't keep up with this, it's going to run away from us. And that's a totally different feeling, and you'll know. And here's the problem.

Eventually, you'll catch the boulder on the way down, and then you're like, shit, I'm actually pushing it uphill again. Yep, absolutely, that's just what happens. And it's, I guess, it's like asking how do you know if you're in love? Trust me, you know.

It's funny, I think one of the major problems that YC companies have is how do they stay small and lean and don't have a lot of, don't spend a lot of money, don't have a lot of management overhead, don't have a lot of bills until they reach product market fit. One of the typical mistakes is confusing an angel round, or a couple million dollars in the bank for product market fit.

It's extremely common. All right, other questions? How do you recognize your first group of user? Like, yeah. How do you recognize your first groups of users? Reddit was easy. They were just like Alexis and I. Like we, Emmett was kind of talking about the intuition phase of building products.

We were in that for a long time. Like our users, they read the same Paul Graham blog that we did. They were programmers like I was. And so it was really, really easy. Hipmunk, we were actually completely wrong about who our users were. And I don't know if we ever quite solved this problem. They solved it through acquisition, which is we built the product for ourselves.

Like tech savvy people who travel a fair amount and buy their own travel. But in reality, our customer, the only way to make money in travel is to sell to business travelers, because they're the only one who travels multiple times a year. And business travelers don't get to use things like Hipmunk or Kayak, right? They use American Express and Concur.

So it took us actually a couple years to synthesize that, and we were just like, fuck. I think there's two kinds of startups. There are startups that are building things for themselves, and where your customer is easy to identify, it's you and people like you.

And there are startups building things for a third party group of users, where you should arrive at which customers are your customers through an analytic mode, right, through thinking about the problem. So Twitch was an analytic startup, because I was a huge fan of watching game streaming, but actually, I wasn't much of a streamer. I've never been much of a streamer.

And I basically dug in on Justin TV and realized that 80% of the minutes watched were coming from our top 200 streamers. And I was like, they're the most important people. We lose them, we lose everything. And if we can get 200 more of them, we'll be twice as big. And so, it was sort of identified analytically that that was the right group of users, and those are the right customers.

And I think that you should just know which one you are. And the most important thing there is that if it's not for you, and if you're not legitimately building something for you and for you to use, and you aren't a big group of people in this case, you should absolutely use the analytic mode. And I don't know, it's different for every single company. But think hard about it, I don't know.

Yeah, but if it's not for you, your intuition is lying to you every time. And you have to supplement it with data, or talking to your users, or whatever. So shockingly, we were 100% on time. If everyone could thank these guys for coming out today, appreciate it. Thank you. Thank you. Thank you.