with @annieduke, @pmarca, and @smc90
Every organization, whether small or big, early or late stage -- and every individual, whether for themselves or others -- makes countless decisions every day, under conditions of uncertainty. The question is, are we allowing that uncertainty to bubble to the surface, and if so, how much and when? Where does consensus, transparency, forecasting, backcasting, pre-mortems, and heck, even regret, usefully come in?Going beyond the typical discussion of focusing on process vs. outcomes and probabilistic thinking, this episode of the a16z Podcast features Thinking in Bets author Annie Duke -- one of the top poker players in the world (and World Series of Poker champ), former psychology PhD, and founder of national decision education movement How I Decide -- in conversation with Marc Andreessen and Sonal Chokshi. The episode covers everything from the role of narrative -- hagiography or takedown? -- to fighting (or embracing) evolution. How do we go from the bottom of the summit to the top of the summit to the entire landscape... and up, down, and opposite?The first step to understanding what really slows innovation down is understanding good decision-making -- because we have conflicting interests, and are sometimes even competing against future versions of ourselves (or of our organizations). And there's a set of possible futures that result from not making a decision as well. So why feel both pessimistic AND optimistic about all this??
United States


00:00:00everyone welcome today 6 and see if I catch X Donnell and today Mark and I are doing another one of our book other episodes were interviewing Kenny Duke professional poker player and World Series champ and is the author of thinking in bets which is just out in paperback today the subtitle of the book is making smarter decisions when you don't have all the facts which actually applies to start up sand companies of all sizes and ages quite frankly I mean basically any business or new product line operating under conditions of great uncertainty which I'd argue with my definition of a startup and Innovation so that will be the same for this episode and he's also working on her next book right now and found it how I decide. Org which brings together various stakeholders to create a National Education movement around decision education empowering students to also be better decision makers so anyway market and if you hear about all sorts of things in & Beyond her book going from investing to business to life but then he begins with a thought experiment even though neither of us really know that much about football so what I'd love to do is kind of throw it.
00:01:00that you guys so that we can have a discussion about that so I know you guys don't know a lot about football but it's pretty easy to feel this one should do this thought experiment Pete Carroll calls for Marshawn Lynch
00:01:14actually run the ball so we're betting on someone we know is really good well they're all really good but we're betting on the play that everybody's expecting this is the default so he has Russell Wilson hand it off to Marshawn Lynch Marshawn Lynch goes to Barrel through the line he fails now they call the timeout so now they stop the clock to get another play now and they hand the ball off to Marshawn Lynch what everybody expects Marshawn Lynch again attempts to get through that line and sheeps Tails end of game Patriots went my question to you is are the headlines the next day that worst call in Super Bowl history is Cris Collinsworth saying I can't believe the call I can't believe the call or is he saying something more like that's why the Patriots are so good they're lying is so great that's the Patriots line that we've come to see this whole season this will seal Belichick's place in history it would have all been about the Patriot
00:02:15so let's go to divide things into like we can either say the outcomes are due to skill or luck and luck in this particular case is going to be anything that has nothing to do with G Carroll and we can agree that the Patriots doesn't have anything to do with Pete Carroll Belichick doesn't have anything to do with Pete Carroll Tom Brady doesn't have anything to do with Peter as they're stealing their 5th Super Bowl victory so what we can see if there's two different routes to failure here one route to failure you get resulting and basically what was only thing is is that retrospectively once you have the outcome of a decision once there's a result it's really really hard to work backwards from that single out come to try to figure out what the decision quality is this is just very hard for us to do they say oh my gosh the outcome was so bad this this is clearly I'm going to put that right into the scale back at this is because of Pete Carroll's on doing but in the other case they're like oh you know there's uncertainty what could you do weird right so you can try to take that you can say
00:03:15aha now we can sort of understand some things like for example people have complained for a very long time that in the data felt they have been very very slow to adopt what the analytics say that you should be adopting right and even though now we've got some movement on my 4th Down calls and when it going for two-point conversions and things like that they're still nowhere close to where they're supposed to be don't make the play as corresponding to the statistical probability that if you're on your own one-yard line and it's Fourth Down you should go for it no matter what the reason for that is if you kick it you're only to build a chick to Midfield so the other team is basically almost guarantee three points anyway so you're supposed to just try to get the try to get the yards like what have you ever seen a team on their own watch are the forest sound like at let's go for it that does not happen Okay so we know that they've been like super slow to do with the analytics says is cracked and is that that thought
00:04:15really tells you why because we're all human beings we all understand that there are certain times when we don't allow uncertainty to Bubble Up to the surface as the explanation and there are certain times than we do and it seems to be that we do when we have this kind of consensus around the decision there's other ways we get there and so okay if I'm a human decision maker I'm going to choose a path where I don't get yelled at exactly so basically we can kind of walk back with his if we allow any uncertainty the bubble to the surface and this is going to be the first step to find my understanding what really slows Innovation down what really slows adoption of what we might know it's good decision making because we have conflicting interest right making the best decision for the long run or making the best decision to keep us out of a room where we're getting yelled at yelled at or possibly fired so can I propose the framework that I used to see if you agree with it so it'd be able to buy to make it to me to grid and it's consensus vs. not consensus and it's right versus wrong
00:05:15Seance the way we think about it at least our business is basically consensus right is fine consensus and a consensus right is fine fine because you just leave everything else is just as soon as I can videos made by consensus wrong is really bad bad baby this considered Innovation stuff that you'll be talking about there's a consequence of that there are only two scripts for talking about people are pretty and not even a consensus directions when script is there a genius because it went right in the other is there a complete moron cuz I went wrong is that does that mean exactly that's exactly right and I think that the problem here is that what is right and wrong mean in your to buy to wrong and right is really this is just a turn out while ago is where we really get into this problem because now what people are doing is they're trying to swat the outcomes away and they understand just as you said that on that
00:06:15the census wrong you will have like a cloak of invisibility over here like you don't have to deal with it right so let's think about the other things besides can sense it so consensus is one way to do that especially when you have like complicated cost-benefit analyses going into it I don't think that people when they're getting in a car are actually doing any kind of calculation about what the cost-benefit analysis isn't it their own productivity vs. the danger of something very bad happening to them like what is the society we someone's done this calculation with all kind of done this together and so therefore like getting in the car is totally fine I'm going to do that has anybody who dies in a car crash you know this is pretty transparent another way to get there is a status quo so like a good status quo example they like to give cuz I can understand it at that you have to get to a plane and you're with your significant
00:07:15are in the car and you go to college while wrote this is so so you go your usual route like you go that way literally this is the route that you've always gone and there's some sort of accident there's bad traffic you missed the plane and you're mostly probably comforting each other in the car it's like what could we do you know but then you get in the car and you announce to your significant other I got a great shortcut so let's take the shortcut to the airport and there's same accident whatever horrible traffic you missed the flight and that's like that status quo versus non status quo decision but you're going against was familiar and comfortable exactly if we go back to the car example when you look at what the reaction is to a pedestrian dying because of an autonomous vehicle vs because of a human were very very harsh with algorithm for example if you if you get in a car accident and you happen to hit a pedestrian I can say something like you know what Mark didn't intend to do that cuz I think that I understand your mind is not such a black box to me
00:08:15so I feel like I have some insight into what your decision might be in some more allowing some of the uncertainty to Bubble up there but if if his black box algorithm makes a decision now all of a sudden I'm like get these cars off the road never mind that the human mind as a black box itself I'll try right we have some sort of Illusion that I understand sort of what's going on in there just like I have an illusion that I understand what's going on in my own and you can actually see this in some of the language around crashes on Wall Street to when you have a crash that comes from human being selling people say things like the market went down today when its algorithm they say it's a flash crash so now they're sort of pointing out like this is clearly in the scale category 8C algorithm spot we should really have a discussion about algorithmic trading and whether they should be allowed when obviously the mechanism for the market going down at the same either way so now if we understand that so it's a clear Matrix now we can say will okay human beings understand what's going to get them in the room and pretty much anybody who's
00:09:15get out living and breathing in the top levels of business at this point is going to tell you that process process process I don't care about your outcomes process process process but the only time I ever have like an all-hands-on-deck meeting is when something goes wrong like what city you're in a real estate investing group and so you invest in a particular property based on your model and the appraisal comes in 10% lower than what you expected look at everybody's in a room right you're all having a discussion you're all examining the model you're trying to figure out but what happens when the appraisal comes in 10% higher than expected is everyone in the room going what happened to your reality was just like we don't compare process so there is a two things when is you have to make it very clear to the people who work for you that you understand the outcomes will come from good process that that's number one and then number to what you have to do is
00:10:15try to lie in the fact that is human beings we tend to be outcome-driven to what you want in terms of getting getting there an individual's risk to a line with the Enterprise risk because otherwise you're going to get to see why a behavior and the other thing is that we want understand if we have the right assessment of risk so one of the big problems was with the appraisal coming in 10% too high there could be that your model correct it could be that you could have just to tell result but it certainly is a trigger for you to go look and say with the risk in this decision that we didn't know was there and it's really important for deployed resources I have a question about translate did they not investing contacts so it's an indivisible of marks Matrix even if it's a non-consensus wrong you are staking money that you are responsible for most companies people do not have that kind of skin in the game right so how do you drive accountability in a process driven environment that the results of oxygen matter you want people to be account
00:11:15yet not overly focused on me out, can you calibrate that so let's think about how can we create balance across three dimension that makes it so that the outcome you care about is the quality of the forecast so first of all obviously his demands that you have people making forecast you have to stay in advance here's here's what I think this is my model of the world here where all the places are going to fall so this is what I think so now you stated that and it did the weather the outcome is quote-unquote good or bad is how close are you to whatever that forecast is so now it's not just like oh you want to order you lost to it it sways your forecast said so that's peace number one is make sure that you're trying to be as equal across quality as you can and focus more on forecast quality as opposed to like traditionally what we think of as outcome quality so now the second piece is directional
00:12:06when we have a bad outcome and everybody gets in the room when was the last time that someone suggested we really should have lost more here
00:12:17are you saying that but sometimes that's true sometimes if you examine it you'll find out that you didn't you didn't have a big enough position it turned out okay well maybe we should have actually lost more so you want to ask both up down and orthogonal so could we have lost less should we have lost more and then the question of should we have been in this position at all after a company Works in xssss sells for a lot of money you do off and say. I wish we had invested more money you never ever ever ever never heard anybody say otherwise we should invest more money to someone said that like wouldn't you love for someone to come up and say that to you that would make you feel better I don't really understand that it looks simple in a poker example so let's say that I get involved in a hand with you and I have some idea about how you play
00:13:16and I have decided that you were somebody that if I if I bet X you will continue to play with me let's say this is a spot where I know that I have the best hand but if I bet a nextplus see that you will fault if I go above acts that I'm not going to be able to keep you coming along with me but if I bet Acts or below that you will so I bet ask you call but you call really fast in a way that makes me realize how I could have actually been textPlus see you hit a very lucky card on the end and I happen to lose a pot I should have maximized at the point that I was a Master Mechanic independent of the winner of the loss so you need to be exploring those questions in a real honest way and sing like me know what the next part of mine is going to be and then not having had the information that would then drive a better decision-making process around 9 to learning loss it's happening because we're not exploring that in the negative direction is
00:14:16do the sun wins as well so if you do ever discuss it when you always think like how can I press how could I have one more how could I made this even better how could I do this again in the future should we have one left out by fluke we should have actually had less in it and sometimes not at all because sometimes are the reasons that we invested turned out to be orthogonal to the reasons that it actually ended up playing out in the way that it was and so had we had that information we actually went to bed on this at all these was completely orthogonal like we totally had this wrong it just turned out that we ended up winning and that that can happen obviously that happens all the time but what is that communicate to the people on your team good bad I don't care I care about our model I want to know that we're modeling the world well and that were thinking about how do I incorporate the things that we learned because we can generally think about Stephen and to have stuff we know and stuff we don't know there's stuff we don't know we know obviously so will we don't we we don't worry about that cuz we don't know we don't know it but then there's stuff we could know
00:15:16I said we can't now but things like the size of the universe are the thoughts of others will actually be we don't know that I have a question about this what is a time frame for that forecast to let do you have like a model of the world a model of a technology how it's going to job that's going to play out in some cases there are companies that can take like years to get traction you want to get your customers very early to figure that out right see if I get that data but how much time do you give like how you size that time frame for the forecast I caught that you're not constantly updating with every customer data point if they are also giving it enough time for your model your plan your forecast to play out you have to think about very clearly in advance what's my time Horizon how long do I need for this to play out but also not don't just do this for the big decision because there's things that you can forecast for tomorrow as well so that you end up bringing it into just the way that people say and then once you've decided okay this is the time Horizon of my forecast and you would want to be thinking about what are four
00:16:16how to make for a year or two years 5 years for the specific decision to play out and then just make sure that you talk in advance at what point you'll revisit the forecast weather so we want to think in advance what are the things that would have to be true for me to be willing to come in and actually revisit this forecast because otherwise you can start as you just said like you can turn into it right away because I know you're like one bad customer you suddenly like over rotate on that when in fact it could have been right in your forecast here are the circumstances under which we would come in and check on that you've already got that in advance so that that's actually creating constraints around the reactivity which is helpful to Christians are more people understand the logic of what you're describing is people getting exposed to these ideas and kind of expanding and importance and some more people to understand that stuff but there's two kind of General Saccone emotion-driven works or something the people just really have a hard time
00:17:16someone as you understand this could be true if investors CEO product line manager in a company you know kind of anybody in it one of these demands such as you can't get the night consensus right results unless you're willing to take the damage write the rest of the night consensus Ron results but people cannot cope with an a consensus right outcome they they just emotionally cannot handle it and it ain't even like to think that they can and it's such a traumatizing experience that it's the touching a hot stove they will do anything in the future to avoid that is that just maybe that's just simply flat-out human nature and so to some extent the intellectual understanding of your doesn't actually matter that much because there's no motion override it so that would be a pessimistic view on our ability as a species to learn these lessons or do you have to have a more optimistic view of that cuz I think that if you move this a little bit it's a huge difference to sort of have to tax that you want to take one is how much can you move the individual to sort of train this kind of thinking for them and that
00:18:16means naturally they're thinking in forecast a little bit more that when they do have those kinds of reactions which naturally everybody will they were right the ship more quickly so that they can learn the lessons more quickly I actually just had this happen I turned in a draft of my next book on the first part of my Nextbook to my editor and I just got the worst comments I never got back to 24 hours I was like you know what she's right now I still had a really bad 24 hours and I'm the like give me negative feedback like Queen because I'm a human being but I got to it fast like I sort of got through it pretty quickly after this I mean I've been on the phone with my agent saying I'm standing my ground this is ridiculous and then he got a text the next day be like no she's right and Rewritten and now I can get to a place of gratitude for having the negative feedback but but I still had the really bad day so it's okay and it's okay like we're all human like robots
00:19:16number one is like how much are are you getting the individuals to say OK I improve 2% that's so amazing for my decision-making in my learning going forward and then the second through line is what are you doing to not make it worse because obviously for a long time people like to talk about I'm results-oriented if it's with the worst sentence it could go outside. I have heard that a lot what's so bad about all you care about is why did you let her live that's fantastic asking people to do intellectual work results of the worst thing that you could say to somebody so I think that we need to take responsibility and the people in our orbit we can make sure at minimum that we are making it worse and I think that that's so that's a pessimistic and optimistic I don't think anyone's making a full reversal here
00:20:16is there sometimes known the journalist there are no stories in the end of stuff happening why am I not reading about any of it in these newspapers me the story of something is happening is not an interesting story he said they're only two stories is Shepherd's he said one is oh the glory of it and the other is over the shame of it through profiles of a co-founder you'll ever read it's just like what a Super Genius for doing something presumably not consistent right or what a moron like what a hopeless idiot for doing something that consensus and wrong and so and so since I've become more aware of this like it's actually gotten it's gotten very hard for me to actually read any of the coverage of the people I know because it's like the people who got a consensus right there being lavish with too much raise is wrong or being.
00:21:16for all kinds of reasons the traits are actually the same a lot of cases the so I guess as a consequence like if you if you read the coverage it really reinforces this bias of being results-oriented sounds like it's not our fault the people that want to read a story that says well I tried something and it didn't work this time right option Siri just quickly the pass preserve the option for to run place so if you want to get three tries to send instead of two for strictly for class management reasons you pass first ESPN SportsCenter that night it's a crotch Mystic or pessimistic that the narrative the public narrative on these on these topics will ever move on super super pessimistic on the societal level but I'm optimistic on on it for educating people better that we can equip them better for this so I'm really focused on how do we make sure that we're equipping people to be
00:22:16able to parse those narratives in a way that's more rational and particularly you know now there's so much information and it's all about the framing in the storytelling and it's particularly driven by what's the interaction of your own point of view would you think about it as part of some point of view for example versus the point of view of the communicator of the information and how is that interacting with each other you know in terms of like how critically are you viewing the information for example I think this is another really big piece of the pie and and someone actually related to the question about journalism which is that third dimension of the space we talked about two Dimension which is for the outcome quality and how how are you allowing that you're exploring both down side and upside outcomes in a way that's really looking at forecast how are you thinking directionally a neutral but then the other piece of the puzzle is how are you treating Omission vs. commission so one of the things that we know what this would this issue of resulting
00:23:16here's a really great way to make sure that nobody ever results on you don't do anything if I just don't ever make a decision I'm never going to be in that room with everybody yelling at me for the stupid decision I made because I had a bad outcome but we know that not making a decision is making a decision we just don't think about it that way and it doesn't have to just forget about investing their own personal decision so you know it's really interesting remember is giving somebody advice who is like thing is like 23 and you know so obviously you know your newly out of college had been in this position for a year and was really really unhappy in the position and he was asking me like I don't know what to do I don't know if I should change jobs and I said well you know so I did all the tricks you know time traveling and imagine it's a year from now do you think you could be having this job no okay well maybe you should go and choose this other maybe you should go and try to find another position and this is what he said to me in this I think shows you how much people don't
00:24:16lights at the thing you're already doing the status clothing choosing to stay in that really is a decision so he said to me but if I go and find another position and then I have to spend another year which I just spent trying to learn the ins-and-outs of the company and it turns out that I'm not happy there I'll have wasted my time and I said to him okay well let's think about this though the job you're in which is a choice to stay in you've now told me it's 100% in a year that you will be sad then if you go to the new job yes of course it's more volatile but at least you've opened your evil and that the range about comes up but he didn't want to do it because it doesn't feel like staying where she was didn't feel like somehow she was choosing it so that he felt like if it went to the other place Jagged up sad that somehow that would be that's his father in a bad decision for my case is spending a little too personal but in my case it was a decision that I didn't know I had made to not have kids and it's still an option but it's probably not going to happen and my therapist kind of told me that my not deciding
00:25:16what is a choice and I was like so blown away by that that it had been allowed me to then examine what was going on there in that framework and we have to not do that for other areas of my life where I might actually want something or maybe I don't but at least it's a choice that there's intentionality I appreciate you really want to thank you for that because I think that people first of all should be sharing this kind of stuff people feel like they can talk about these kind number to should I have kids or should I have kids or this the lawyer that for you right live or an the thing that I try to get a crosses you know we can talk about investing it like I'm putting money into some kind of financial instrument but we all have resources that were investing that's right I'm tired energy your heart can be whatever your friendships relationships are so you're just
00:26:16deploying resources in like for the time that you're talking about if you know if you choose to have children you're choosing to deploy certain Resources with some expected return at some of that good so bad and if you're choosing not to have children that's a different deployment of your resources toward the other things and you need to know that their limit everything is in a zero-sum game no but approaching the world and that fact that Evolution has approached the world it's a zero-sum game and our tool kit makes it a zero-sum game means that we need to have still view everything is a zero-sum game when it comes to those trade-offs and resources losing something every time even in a nonzero game I don't feel like the world is a zero-sum game in terms of like we can most of the until after eight or nine way we can both win too but it's a zero-sum game to go back to your therapist it's a zero-sum game between you and the other versions of yourself that you don't shoot Ashley or an organization and the other versions of self it doesn't choose exactly so there's a set of possible Futures that result from not making a decision
00:27:16swell so on an individual decision let's let's put things into three categories clear mrs. near misses and hits there's some that would just be a clear Menace throw them out and there are some that I'm going to sort of really agonize over and I'm going to beat out think about it I'm going to do a lot of analysis on it and so the ones but you would be coming yes go into the hit category and the other one is in your mess I came close what happens with those near misses as they just go away so what I realized is it On Any Given decision let's take an investment decision if I went to you or you came to me and said will tell me what's happening with the companies that you have under consideration on a single decision when I explained to you why I didn't invest in a company it's going to sound incredibly reasonable to you so you'll only be able to see in the aggregate if you look across many of those decision that I tend to be having this biased toward missing towards we're not going to do it so that I don't want to stick my neck out
00:28:16this for you is incredibly hard to spot because you do have to see it in the aggregate because I'm going to be able to tell you a very good story on any individual decision so the way to combat that and again get people to think about what we really care about yours forecast not really outcomes is actually to keep a shadow book the anti portfolio should contain basically all of your near misses but then you have to take a sample of the clear mrs. as well which is nobody ever looks at because I didn't the near-misses tend to be a little and your periphery if they happen to be big hit China portfolio and the way that we do it is we make the investment we take an equivalent we take the other equivalent of that vintage of that size that we almost had but didn't do they put that in the shadow portfolio and we're trying to do some apples to apples comparison Finance to return the shadow portfolio May well I'll perform the real portfolio and in finance firms that's because the shuttle portfolio maybe hire variance
00:29:16weather hitting are the ones that are less there less spiky there was foul there less risky when you decide not to invest in a company you actually model out why that is by the way it's often a single flaw that we've identified we would do it except for ex just make sure people like those ones that people are just reject out-of-hand the actual sample we were Jack's 99 for everyone we do yes literally to sample you just take a random sample of what is ask Aztec pushing against your model you're just sort of getting people to have the right kind of discussion so all of that communicates the people around you like I care about your model
00:30:1634 question on did you talk about instead of groups of groups of decisions I have to have this discussion with a friend of mine has a decision you think about Ambassador or not he said it's totally the wrong way to think about the sushi thing about this is is this one of the 20 Investments of this kind of his class size that you're going to put in your portfolio when you're when you're evaluating an opportunity you are kind of definition definition of talking about that opportunity but it's it's very hard to abstract that question from the broader concept of a portfolio or a basket I would suggest there is actually just doing some time traveling that has people are really down in the week let's imagine it's a year from now and what is the portfolio look like of these Investments of this kind so I'm a big proponent of time-travel I'm just making sure that you're always asking that question what does this look like in a year what does this look like in 5 years are we happy are we sad if we imagine that we have this what percentage of this do we think we'll have failed we understand that
00:31:16one of these individual ones could have failed so let's remember that and I think that that really allows you to sort of get out of what feels like the biggest decision on Earth because that's the decision you have to be making and be able to see it in the context of kind of all of what's going on send testic what the most powerful things my therapist gave me it was a simple contract it was sort of like doing certain things today is like stealing from My Future Self blew my mind so beautiful so beautiful and I had never thought of because we were on a Continuum by making discrete individuals like soda on the path on all today don't know this woman in the future I haven't met yet wow like the idea of stealing from her it was like I got a really I was Mother's Day
00:32:16give me tonight I totally hate this is actually prospective late and playing really good decision hygiene which involves a couple of things what is some of this good time traveling that we talked about where you're really imagining what is this going to look like in the future so that that's metabolized into the decision to is making sure that you have pushed back once there's consensus reached great now let's go disagree with each other than that the next thing is it in terms of the consensus problem is to make sure that you're listening as much input not in a room with other people so we had a when somebody has a deal they want to bring to everybody that that that goes to the people individually they have to sort of right there thoughts about an individually and then it comes into the room after suppose the car while I just sort of effects of consensus anyway so now this is how you can come up with basically what you're forecast is a futurist the den is absolutely memorialized because that memorializing of it access the prophylactic first of all gives you your forecasts which is what you're
00:33:16push against Danny while you're trying to change the attitude to be that the forecast is the outcome that we care about is a prophylactic for this emotional issues right which is now you it's like okay well we talked about this we had already have her here right we had a good week I really thought about why we questioned if somebody eat out let's see if someone has the outside view what would this really look like to them by eliciting that the information individually we were less likely to be in it inside you anyway we've done all of that good hygiene and then that acts as a way to to protect yourself against these kinds of issues in the first place again you're going to have a bad 24 hours but you can get out of it more quickly and get to a place where you can say Okay moving on to the next decision how do I how do I make better sweater lotion
00:34:16but we can move as individuals so one form of optimism would be more of us movies individuals the other for about two missing could be either we'll just always be room and he's probably sick domains for the rare individual who's actually able to think about this. Correctly like there will always be certain people who are like much better poker than everybody else there was simply can't or won't get there might be able to take the time and have the discipline and willpower to kind of get all the way there but most people can't walk that that's okay like I mean in evolutionary standpoint that kind of thinking was selected for for a reason write like a better for survival likely better for happiness cuz I have to figure out how often is that a lion that's going to come eat makes people shouldn't be playing in the World Series of Poker
00:35:16how much does the poker like you probably have a great time doing that and it brings you a tremendous amount of enjoyment and you don't have an interest in becoming a professional poker player just be proud of that I think that's amazing like I play tennis I'm not saying all but you know I'm just playing we could get all I'm just playing in like USTA like 3.5 like I'm really happy with my tennis I think it's great so I think we need to remember that like people have different things that they love and this kind of thinking is I think that I would love it if we could spread it more but of course they're going to be some people who are going to be up in this category more than other isn't that's okay but not everybody has to think like this I think it's all right so one of the things I get asked all the time is like what we can't really do this because people expect us to be confident in our choices yeah confused confidence and certainty so I can express a lot of uncertainty and still convey confidence ready I'm wearing these three options I've really done the analysis here's the analysis
00:36:16this is what I think I think the option is going to work out 60% of the time option b is going to work out 25% of the time and options he's going to work out 15% of the time so option A is the clear winner know I just Express so much uncertainty in that sense but also a lot of confidence. An awful lot of confidence in my analysis this is my forecast and all that I ever ask people to do when they do that is make sure that they ask a question before they Bank the decision which is there some piece of information that I could find out that would reverse my decision that would actually not that would make it go from 6257 I don't care modulating so much so that you can actually change point is that organizations can then bake that into their process that decision to let them the next time they get to it right or wrong they make a decision and if the answer is yes go find it or sometimes the answer is yes but the cost is too high could be time it could be exactly so then you just don't and and then you would say what
00:37:15Dan dewald recognized as a group we knew that if we found this out it would change our decision but we've agreed that it would the cost was too high and so we didn't so then if it revealed itself after which do not set well you've talked a lot about how people should use confidence intervals and communicate which I love cuz we're both neither finished so I love that idea one thing that I struggle with though is again in the organizational contacts like if you're trying to translate this to a big group of people not just one-on-one or small-group decisions how do you communicate a confidence interval and all the variables in it and inefficient kind of compressed way The Odyssey Part of communication in organizations is emails and quick decisions and yes you can have all the process behind the outcome but how do you then convey that even though the people were not part of that room of that discussion I think that there's a simpler way to express uncertainty which is using percentages and obviously sometimes you can only come up with the range but for example if I'm talking to
00:38:15my editor and this is very quick in an email I'll say you'll have to drop by Friday 83% of the time by Monday you'll have a 97% of the time those are inclusive fries another way without making it so wonky donkey so I'm just letting her know most of the time you're going to get it on Friday but I'm building like my kid get sicker I have trouble with a particular section of the draft or whatever it is and I set the expectations for that way and tastic I mean we've been trying to do forecasting even for like timelines for podcast editing and episodes and I feel frustrated because I buy have like a set of Frameworks like if there's accents if there's more than two voices if there's a complexing room tone like interaction feedback sound effect I know all the factors that can go into my model but I don't know how to put a confidence interval in R pipeline spreadsheet for you know all the content that's coming out in the range what's the earliest that I can get it and you put a percentage on that and then you think about the latest day they're going to get it
00:39:15I have no idea what's wonderful about that is that it's a few things one is I've set the expectations properly now so that I'm not getting yelled at on Friday like where the hell is that exactly and I think she's half the battle at what happens is that because we're short we think that we have to give a certain answer I end up boy who cried wolf right so that if I'm telling her I'm going to get it on Friday and you know it's like your own best judgment sometimes even 25% of the time I'm late she just starts to not put much stock in my and what I said anyway so that's number one number two is what happens is that you you really kind of insect other people with this in a good way where you get that it just moves them off of that black and white thinking so I'd love that one of the things that I love thinking about in this is a different from the deadline or kind of give me this range is that I think that we ask ourselves and my sure and other people are you sure way too often that it's a terrible
00:40:15listen to ask somebody cuz of the only answer is yes or no so what should we be at the house or are you sure are you I have a quick question earlier you mentioned uncertainty and you as an organization build that uncertainty in by default a little bit about time traveling in the usefulness of time traveling so one thing they're like to think about is not over value the decision that's right at half the things that are right sitting in front of us right so you can kind of think about it like how are you going to figure out the best path what is it that you know as you think about what your goals are and obviously the goal that you want to reach is going to sort of Define for you at what the best path is if you're standing at the bottom of a mountain that you want a summit let's call The Summit your goal all you can really see is the base of the mountain so as you're doing your plan and you're really worried about how do I get the next little bit right how do I start but if you're at the top of the mountain having attained your goal now you can look at the whole landscape you get this beautiful view of the whole landscape and now you can really see what the best path looks like
00:41:15we want to do this not just physically like standing up on a mountain but we want to figure out a cognitive way to get there and that's to do this really good time traveling and you do this through back passing and pre-mortem and now let's look backwards instead of forwards to try to figure out this is now the headline let me think about why that happened so you can think about this like it's a simple like weight loss goal I want to lose a certain amount of weight within the next six months at the end of the six months I've lost that weight what happened you know I went to the gym I avoided bread I didn't eat any sweets I made sure that you know whatever so you now have this list Dan in pairing with that you want to do a pre-mortem which is I didn't get to the top of the mountain I failed to lose the weight I failed to do whatever it is and then all the things you can do to counter burger exactly because that's going to reveal really different things to reveal some things that are just sort of lock right let me think can I do something to reduce the chant the influence of luck there then there's going to be some things
00:42:15how to do with your decisions like I went into the break room every day and there were Donuts there and that's why I couldn't resist them so now you can think about how do I counter that right how can I bring other people into the process and that kind of thing then there's stuff that's just you can figure out if it's out of your control is turned out of slow metabolism and now it happens is that you're just much less reactive and you're much more Nimble because you've gotten a whole view of the landscape and you got a view of the good part of Lansing and the bad part of landscape but I'm sure is he told you people are very loath to do these three more times because I think that the imagining a failure feel so much like failure that people that you should positive positive visualization in about an idea of options are the process you should be that it's a clear timing it there's wonderful research by Gabrielle attitude that I really recommend that people see that the reference is RR in my book
00:43:13in across domains what she's found is that when people do these the sort of positive fantasizing the chances that they actually complete the goal or just lower than it usually is to this negative fantasizing Islanders research that shows that when people do this time travel in this backwards thinking that increases identifying a reasons for success or failure by about 30% you're just more likely to see what's in your way so like for example she did it she did like one of the simple studies when she is she asked people who are in college you know who that you haven't talked to yet she had one group so you know it was all positive fantasy it so it's like I'm going to meet them and I'm going to ask him out on a date and it's going great in the room of Happily Ever After and whatever and then she had another group that engaged what if I asked Mountain Leasing know like they said no and I was really embarrassed and so on so forth and then she Revisited them like four months later to see which group would actually gone out on a date with the person that
00:44:13Russia Iran and the ones that did the negative fantasizing were much more likely to find out on a date and tastic so one of the things that I say is like look when we're in teams to your point we tend to view people is naysayers right but we don't want to think of them as downers so I suggest divided up into two processes have the group individually do a back half of the group individually write a narrative about a pre-mortem and what that does is when you're not doing a pre-mortem it changes the rules of the game we're being a good team player is now actually identifying the ways that you fail I love what you said two modes of the way I of getting into these two mindset right where you're not stopping people from feeling like they're a team player and I think that that's the issue as you said it's like don't sit there in like you know crap on my goal is it what what are they really say you're not being a team player so change the rules of the game you had this line in your book about how regret is an unproductive the issues that it comes after the fact that before so
00:45:13that I don't want people to do is think about how they feel right after the outcome because I think that then you're going to overweight regret so you want to think about regret before you make the decision you have to get it within the right time for what we want to do instead is right in the moment of the outcome when you're feeling really sad you can stop and say going to care about this in a year and about yourself as a happiness stock and so if we can sort of get that more ten thousand foot view on our own happiness and we think about ourselves as we're investing in our own stock our own happiness. We can get to that regret question a lot better you don't need to improve that much to get really big dividends you make thousands of decisions a day I fear you can get a little better at this stuff if you can just dobias a little bit think more probabilistically really sort of Wrap Your Arms Around uncertainty to free yourself up from sort of the emotional impact and outcome a little bit
00:46:13is it a such a huge effect on your future decision-making that's amazing any thank you so much for joining 6nz podcast

Transcribed by algorithms. Report Errata
Disclaimer: The podcast and artwork embedded on this page are from a16z, which is the property of its owner and not affiliated with or endorsed by Listen Notes, Inc.


Thank you for helping to keep the podcast database up to date.