Hi everybody and welcome to in good company. I'm Nicolola Tangan the CEO of the Norwegian S wealth fund and today I'm in particularly good company with Saul Pearlmter who I would argue easily is the cleverest person we ever had on the podcast because Soul won the Nobel Prize in physics for discovering that the universe expands at an increasingly rapid pace. Now you also written a book called Third Millennium Thinking which teaches us how to use scientific method in order to navigate this increasingly uncertain world. So big welcome to to this podcast.

Thank you. It's good to be here.

I thought we could start with the book and kind of the scientific thinking. So what is third millennium thinking? Well, it's a bit of a of a odd name because what we really want to capture is the direction in which we think the best of our scientific style of thinking has been helping our whole society uh be able to do better in working through problems together. And we want to try to capture what does that really look like so that people um can realize that there's so many elements of it they could all be using in their daily life and also they could be using when they're talking to other people and working out problems together. M

and uh in some sense I'd say that we've learned by now how to solve really dramatic problems and and difficult problems and interesting problems in the world. The one that I feel is the leftover problem that we can make a huge difference if we can solve is just how to talk to each other, how to work problems out together so that we can actually use all these other techniques that we've learned

because the first time I met when we when we spoke you said uh you know Nikolai we can now solve all the problems in the world uh you know climate how to feed people but we don't manage to because we don't talk to each other. I mean it's remarkable. I think we actually live at a incredible moment in in history and prehistory and uh and in fact you know maybe even cosmic history um where we are the first generations on this planet um who have the ability to solve planetary sized problems. I think you know the uh the idea that there could be a pandemic and we actually know what to do about a pandemic. Um, we have billions of people living on the on the planet, many more than when we were children. And we at the time when we were children, most of the world was going to bed hungry. Today, that's a very small percentage. And we've we now know how it's possible to feed a planet. Uh, we can even handle things like, you know, climate changes that have happened throughout history have wiped out civilizations at different points in different ways. Today, for the very first time ever, we know how you could stabilize a a climate. Um, and that we can actually manage that. We could even manage the uh the, you know, the thing that killed the dinosaurs, the the you know, possibility of a comet or asteroid hitting the earth. And and that one wiped out, you know, most of the families of species on the planet. That's something even that we have the possibility of of being able to solve that problem.

That's a pretty pretty pretty incredible starting point, right? So it seems like the one thing that we should all be enjoying today if we had had that a moment to just to breathe and ask each other what is the world that we want to live in this would be the moment where we could all be saying you know turn to each other and saying okay now finally we can build a planet that we just all be proud to live in and at that very moment we're having a hard time

interacting with each other and communicating well enough in a productive way so that we can do these work.

So what is a scientific method? It's a so when you say the scientific method I mean often many of us have been taught something called the scientific method in in in school which was just this hypothesis testing concept. Um but that's just one piece of this whole culture of a grabag of a whole bunch of different ideas that together make a an approach to think about the world. It includes things like thinking of the world probabilistically that um we don't tend to know we very rarely know anything absolutely true or absolutely false. Generally, we have a fairly strong sense that this is very likely to be true. Something else we would bet our lives on being true. Something else we're not sure is true. And that's actually a very useful differentiation. It's not uh black and white, yes or no. It's this one I'm going to bet 90% on, but this one I'm only going to give, you know, 70% uh bet on. And that makes us very powerful. It turns out we actually can do a lot more that way than than in just saying absolutely known or absolutely unknown. How do you think about your life in probabilistic terms?

Well, in our day-to-day lives, it comes up all the time. Uh we are having to make you know, well, some of them we just are we're very used to. We we make a bet on whether it's safe to cross the street and and you know, we are pretty good at that. Um some of them we occasionally are forced to think very hard about when we have to make a health care decision. So, you know, should you uh take this medicine? Should you uh you know, people have to decide whether to get a certain operation. um those you actually have to think through very carefully what the odds are and what the what the uh but and how much you should risk but those are things that we I think would so recognize that we do but it is hidden in many many of the other just day-to-day activities that we do and when we make decisions as a group we don't usually remember that and we don't remember that actually we should be using this as a way of not getting overly attached to any part of the of the argument but being willing to consider well I could be wrong I mean 75% 80% sure I'm right but there's 25% odds that I'm wrong in this one if in which case maybe your point might be the right point in this particular discussion and uh and it makes it a much more of a fluid discussion much more possible for groups to think through problems in a without being attached to the to their position

you talk about something called individual humility and collective arrogance what do you mean by that

so the these parts of the story where you uh need to be able to understand that most of what we did as scientists that was has been has made successful is to consider the possibility that we're making a mistake and that we're getting something wrong. And that is probably 95% of an experimental scientist's life is looking for where are the mistakes this time in in the experiment they're running in the uh in the um theory that they're working with. Um if they're day-to-day, you know, most measurements have some error to them. You have to figure out what the amount of error is that's that is permissible for the particular measurement you're making. Um but and that's part of the mistakes that are there in front of you. what you're really looking for is you're really hoping you find mistakes in the fundamental theories that you're working with. So if when you find out there's something wrong with our understanding of gravity, that was really exciting. Um and that was, you know, what what made Einstein uh you know, one of the things that made him famous. Um and and most scientists, they're constantly trying to figure out and push the edges of what is it that we are fairly sure about, but what things would we be would be amazing if it turned out that the world was a little bit different than we thought. And that is actually where our strength comes from, that ability to be constantly questioning. Um, and it sounds like a weakness to always be doubting, but it it turns out I think that's really where our superpower lies.

Now, that's easier if you work in a team, right? So, what's so special about teams?

There's a there's a couple aspects to this. One is that I think it's very easy, very hard to think outside of your own head when you're by yourself. And as soon as you start talking to other people, it opens up the range of possibilities. Even more difficult, but even more important is talking to people that you think that you disagree with. And so, science has built a whole tradition of taking your work and putting it in front of people who are going to give it a hard time and are going to show all the flaws in it. And that's actually one of the most successful ways to figure out where you may be going wrong. Our society as a larger society seems to have lost a lot of that skill um or or that memory of of why it is that you talk to people that you disagree with and where the usefulness of it comes in.

Why has society lost this? I think we happen to go through different waves um in in our um in our communication with each other and some of them tend to be um oh they tend to corner people into um groups where they are only talking to people who already agree with them and then they find that the other groups sound scary and the other groups that from the outside it's very easy um for for a group that was that's talking among themselves and you're talking with a different for the other group to look like they're just bad in some way, evil or or out to do something that you that you don't trust. But when people actually communicate with each other and think through problems together, almost invariably they start discovering that the actual people themselves that may be in these separate bubbles of communication turn out to really share almost all of the same big goals. And the differences are much more just the question of what do they think the answer is to some factual question u rather than really their priorities being completely different.

But if you work with um somebody whose job is to shoot down your arguments um do do is there less room for ego in science than in other parts of society.

Well I always describe the the aspirational aspects of science as opposed to the what actually happens day-to-day.

Dayto day people do things wrong dayto-day. they they don't follow the prescriptions of of you know I think what this best idea of what science and this third millennium thinking that I'm describing has offered us um as a whole though um that's the aspirational goal to be open to being to listening when somebody gives you criticism and it's very difficult to do not every scientist does it every time but in the big picture eventually people listen to the the referee's comments on papers they they listen to the uh the objections raised in a conference when they're giving a talk um And it ends up actually making a big difference. It means that, you know, they they may not want to hear it the first three or four times, but eventually they think, you know, I've got to answer that question. And that sharpens up their thinking.

So, it's kind of tied into the whole confident humility.

Absolutely. Concept.

Absolutely. And, you know, I don't think of the the the typical image of a scientist as being a humble uh uh you person. But

are you humble?

In this particular respect, I think um I aspire to being as humble as I can, right? I know that I've done things wrong and I've figured things I've I've missed made mistakes and I've caught them sometimes myself but other times I've only caught them because somebody else was there. Um and which parts of you are less humble?

So so I think the part that that's less humble is is actually sort of an interesting one. Um I there's another side of the culture of science which I think has to be um arrogant in a very interesting way which is that you I think science has benefited by having a culture that allows you to believe or maybe even fool yourself into thinking that you can solve difficult problems

long enough to solve them. And and the problem is that most of us I don't think are built to stick to a problem as long as it really takes. And so I think we tend to um think well I tried very hard on that problem. I spent at least a day on it. Maybe I spent a week on it. Um and then you give up. Um whereas the culture of science has I think led people to think we can solve that problem and if we didn't figure it out that week we'll keep working on it and we see some progress maybe by you know a month or so from now we'll try something different and then another month and eventually people figure out these these problems. And it takes a certain kind of um confidence, a certain kind of of and I you can you can call it arrogance. I think I think to to manage to stay with a problem long enough to to do these problems that are worth that are worth our solving.

Well, you have to kind of you have to be a bit of a diehard believer in your own brilliance, right?

Exactly. And and it's a very strange thing because you want what you really want is this funny balance between being very humble and very willing to be wrong

and yet very can do that we're going to be able to figure this thing out so that you if it doesn't work you think there's got to be a different way we can do this and that is I think one of the real secrets of this of how science has worked this this in some sense it's the the breaks um of the of of making mistakes that so many of the things that science involves is the skepticisms and the doubts that keep you from falling into certain kinds of mental traps. But you can't drive a car with just brakes. You you need, you know, you need an accelerator pedal. And I think that accelerator pedal is this can do sense that we can figure this out. You know, it's a pro. It's a hard problem, but once you identify the problem, we have a reasonable chance of of figuring out how to solve it.

What makes a good team? What I've seen that for me at least that that's been uh the thing that's made a huge difference is that combination of people who come in with a wide variety of skills um but who aren't so ego-driven that they can't uh just join a group together and bounce ideas with each other and listen to each other. And that combination of people who are very capable but are also able to to enjoy the the team nature and and and thinking together. That's the thing that I usually am looking for in a in in in a group along a little bit with that you know you always want the person to be very skilled and capable and but you also want them have a little that can do spirit that they think that it's worth staying in the room with the group of people and and worrying the problem for that extra time that you know most people wouldn't. Increasingly um Nobel prizes are worn by teams.

So the in general science has become less and less of a you know single person activity you know with the image of the lone scientist putting on their lab coat and going down to the lab and disappearing. Um that's never that's not been my experience at all. And uh and even rather small groups are still often groups um of of people doing projects. And a lot of science just requires enough different expertise and different parts and the scales have gotten big enough that they often are fairly big teams. They're you know the uh the projects I was doing were smallish in the sense of maybe 30 you know 30 people uh you know all working together on something. Um but the very next stage of those same projects in in more recent years are now hundreds of of people on those projects.

What are the challenges in terms of splitting the work and managing that whole process?

It's very tricky because you you um have all sorts of balancing acts to do. So there's the the fact that you want groups of people to share a lot of of their approaches and other resources. They might share their software uh with each other to work on it. But there's a danger um then that if they've shared too much, then you don't get the um the independent comparisons that that you can often find the errors with. So you you in some sense need to encourage um sort of splinter groups to be working on things while the whole group is is coming to a consensus at the same time. And that these balancing acts actually go throughout the whole third millennium thinking story that in that book uh is that most things you're doing you're having to get that right ch balance between several different things you're trying to achieve that aren't necessarily obvious that you would do them at the same time.

How does chamber music come into this? Because that's one of your teams. So I was I was thinking about uh this from the point of view of um you know who have been some of my more influential teachers in in my life and uh and you know some of them you know it's obvious uh a my research adviser was you know very influential as when I was going my PhD in physics and and others you know are in that category of you know teachers that in certain courses that you've taken that that I I still remember but perhaps one of the more uh um interesting ones is for those people who've taken studied a an instrument um in their lives um your your teacher of that instrument has stayed with you for usually had stayed with you for many many years um while you were growing up. And so in my case my violin teacher was a was a very influential person. Her name her her name was Francis Stuffy and she um was able to teach a certain degree of this this mix of extreme care and and and precision but with a spirit and a and a and a goal of making music not of uh of of just doing something right, you know. And

do you still play?

And I do still play. And the and the thing that I was always interested in when I was playing was not solo music. I was interested in playing string quartets and other kinds of chamber music. And I think the pleasures in that are are very much some of the same pleasures as in group uh efforts for like science solving a scientific problem that you have a group of people who are all contributing something but they have to listen to each other and and the then the degree of of paying attention back and forth I think is what makes that so enlivening and it's also I think one of the things that makes a group science project fun.

What is blind analysis? One of the classic problems that uh that we that we've become aware of um actually in in the larger world, we see it in this thing called confirmation bias. Um where you it's very easy when you're let's say trying to see what uh what some news story is telling you. Um to fall into the danger of only reading the news stories that already tell you something that you believe and not reading the ones that are giving you um information that would disagree with something you believe. And when you do happen to read an article that says something that's disagrees with something that you that you think is right, um you would you look for all the things all the mistakes it's making. Whereas when you see an article that's saying, "Oh, somebody did a study and it and it agrees with something you that you like." You don't look for its flaws in the same way.

And this happens actually not only in like politics and and and when you're reading the newspaper, but in a science project. Um so when you're measuring some you know some uh you know graph the points for a graph and it's going to agree with a theory or disagree with a theory and you're very excited to see what the results are. Um there's a real danger that and you and you see it has happened now in in history. You can go back and look through the the uh graphs of of the past and you see that there's a danger that people accept the graph when it's when they think it's showing you what they were sort of expecting to see either because it confirmed a theory or it disagreed with theory whichever it was that they were kind of expecting it to show and they're more likely to look for the errors in that in that graph um if it's showing something that they don't expect. So that means that there's a real bias towards having graphs come out in papers that are just what the scientist was expecting to to get because those are the ones that they don't go hunting for the errors in

the and in recent years that was that's become something where the physicists started to become aware that this was a trap that people were falling into and one of the whole ideas of science is to recognize these traps and then to figure out how to solve them. So the way the physics the physics community has started to solve this particular trap is they've started using this technique called blind analysis where you don't let your see let yourself see the results the the final graphs with the real uh you know the real uh values on the graph um until everybody has agreed that you've looked for all the errors in the measurement. And then when you've checked for all the errors, then you open the envelope and you see what the numbers correspond to and and you see whether the the answer is what you were expecting or what you wanted or if it goes against what you wanted or were expecting. And it's a it's a kind of dramatic moment. Um so uh you I've been in we've been in the group meetings where we were sitting there was the end of the day and one of our students had was about to unblind their data that they've been working over the last year and a half and if they came out a certain way they would have a wonderful PhD thesis. If it didn't, it would be a disappointing uh result. And uh and it was all going to depend on what happened when they opened this uh they opened this blind blinded result. And uh I remember one time particularly was like late at night. We had gone for a long long meeting and it was past dinner time and we were trying to decide okay are we going to open it now or should we wait in the morning and do it when we're fresh, you know, because if it comes out wrong it's going to be really devastating at the end of the day. Um uh maybe we should wait until we're you know sort of you know well rested etc. And we all looked at each and said each other and said, "Nah, let's look at it." And so we decided that was the time that we're going to unblind and we were going to have to accept whatever we got.

And was it good?

That time it was good.

Not every time, but that time it was it was

the whole concept of confirmation bias is interesting, right? I mean, you got one political view and then you only watch Fox News, you got another point of view, you only watch something else.

Exactly.

You know, this is I guess what creates more of the polarization in society.

No, it's it's a a real problem. And and also we tend to, you know, if you are a Fox News watcher watcher, when you see a New York Times article, you assume that it was you're very skeptical about it. And if you're a New York Times reader and you watch a Fox News story, you assume that there's something something got something wrong in it. And you tend to look for all the errors um in the other but not in your own uh your own reporters. M and confirmation bias is very very problematic also for investors because you made an investment and the only piece of news you you you read is the stuff that's confirming your already existing you know view.

I mean I think it's you know clearly it's one of these deep problems that for all of us for almost anything we're doing we can fall into that trap. I I been describing it also, you know, like when you're trying to decide, let me use the medical example again. Um, you're trying to decide, you know, whether or not you something is a good idea, bad idea for medical treatment. And there's a real temptation to um hunt on the websites until you find uh one that that um says what you want to hear.

And I think that, you know, the really what you need to do is you need to first uh without looking at what it says about your particular case, look for which websites do you trust more about everything else besides that one. So that you choose your website blinded without knowing what it would say for your particular case and then you read what it recommends.

So listen now I have bought a million dollars worth of Apple stock.

Yes.

Okay. And we have four people around in the room here. Uh what's the best way to analyze and question uh and research whether we should own Apple or not. So to begin with um the fact that you have four people in the room um could be a real asset, right? Because they they could um have different sources of information and be able to bring different

Well, I'm sorry. Should we have four people in the room? First of all, let's start from to let's start from total scratch.

Okay, so I I got a million million dollars of Apple stock. How should we decide whether that's a good investment? All right. So obviously um part of the story is what where is the information in in out in the world that would give you the most uh uh chance at doing good predictions in in this case and you know in this in this story of course you have to decide you know where do you expect the most information to lie. Is it going to be with the people who are doing the technology development? Is it going to be with the people who are doing the consumer uh research? Um will it be actual consumers themselves that are missed by the consumer research? And depending on what where you think the the you know the most information sits um you might want to bring different uh people to the table. Um my guess would be um that in many of these of these topics you you want a fairly broad range of sources of information some that you would not have even thought of and that actually you know requires reaching more people than you might typically uh do if you're trying to bring information in. You cannot have them all um get together and start uh walking around the room with the first person um saying uh all the best reasons for why it is that they would recommend the investment or not um or what form to do it in. Um and then have the next person talk and the next person talk because they will be so influenced by what the previous person says that there's a real danger that you just get heard thinking that people lock in on what sounded like a very good argument and they don't want to embarrass themselves by bringing in some other information that they have from a different source. nor do they want to sound like they're they're arguing. They don't want to sound like they're conflicting necessarily with the other uh people or the previous person, you know, in the room. So, it's much better to get all the information written down independently so people provide it separately and then aggregate it. Then people look at it together and try to figure out okay now how do we work with this all this information at the same time. Um so there are a number of tools that people have used for playing these games in different uh different kinds of organizational um activities that that get the best of that when they when when you can. Um I I particularly like um some decisions that need both some values in play as well as the factual issues. Um some of those um seem to be done very well with using random samples of the population which of course is a little bit different from uh from probably what you'll be doing in an investment case. Um but even there you never know. It could be that having a broader um random reach would reach the source of information that it never would have occurred to uh you know if you were just stepping back and and trying to choose your 10 people.

I think one of the one of the really uh bad examples of this bias is you know in the Norwegian Supreme Court uh the oldest and most experienced uh member of the court speaks first.

Ah yes

and you have the lowest supposedly disagreement of any court because of that.

Now it sounds like the exact uh opposite of this concept. Yeah. Because now you're

in Sweden just around the corner. Y

uh is the youngest person who speaks first. Is that right? Yeah. And you have much more disagreement.

That's fascinating.

Is that unbelievable? No.

That's really interesting. That's really very interesting.

But how can this last for years and years and years?

I mean you you you hear about all of these kinds of stories where you where you realize those biases people should be jumping on and asking how do how do we fix that? I mean the other one I was I remember reading about was I for where it was where you they compared the um I think the sentencing um after lunch and before lunch.

Yeah. Absolutely. Right. And it was

totally if you if you're hungry bang you are guilty in prison you go.

Yeah. Which is you know which is also crazy you know that that that would be allowed to go on for a long time. Yeah.

Incredible. Okay. So now we have we have this uh investment case potential. Uh we have people from all kind all over the place. They written down what they think. Okay. So nobody's biased by anybody. So what do you do then? How do you get some productivities agreement going here?

So I do think that um now this is the time where I think bringing people into a room and encouraging them to actually bounce ideas off each other and to be thinking very hard about um the possible ways in which the world could be different than they think. So here is a technique uh that we're was actually I think might have been developed actually originally for business purposes was this technique called scenario planning. Um this is something that uh uh that I've um been Peter Schwarz was somebody who I I came across who was teaching this uh and uh he's has has a book on this actually um but the their the goal was to try and identify driving forces that could change the future in in some topic area. Um I mean in this case you know I guess if we're talking about Apple stock right it's it's a presumably consumer uh computing you know um and then trying to uh just have a group of people that come from a wide a wide set of backgrounds just begin by just writing down all of the things that could drive the future um with respect to it. Is it you know going to be resource uh availability? Is it um is it going to be um economic uh growth? Um is it going to be economic disparity of you know different levels of income? Um will it be uh you know odd things like climate uh change? You know how could that possibly affect uh you know the consumer computing? Um will it be AI uh you know coming to fruition in some new way? Um and so they you write down a whole group of these and you choose a few of them even sometimes just two that look like they'd be um completely unrelated to each other. and and then consider the extremes of those two um in all four possibilities. So you know let's say um a stagnant uh economy where there is not much growth versus a very rapidly growing economy and then in the other direction maybe um you know you might have chosen you know climate uh uh strong you know large climate change and you know and and climate stasis you know where things stay roughly the same and use look at all four of those combinations and ask what would be our best in this case investment in each of those situations because it forces you to think through um the the logic of um of where where what things are robust and what things aren't robust, whether or not those particular futures are the right answer. It's it's a it's a very useful way of being able to consider a wider variety than you typically will do. I mean, if you're just stuck thinking about the future that everybody else is talking about that you all reading in the in the same newspapers,

how do you incentivize people to to disagree and to find mistakes and and flaws? Well, there's one of the reasons why I think um intentionally um allowing for different groups competing as opposed to trying to get everybody to be in one collaborative group um does make a difference because then you naturally get teams, you know, and the teams are naturally um they naturally have the incentive to look for the things that the others may have gotten wrong. And I think that's actually been one of the very interesting ways in which science has progressed that I think you've seen that people have some uh real pleasure in being able to find something that nobody else found and one of the way one ways that shows up is they they find the flaws in the earlier thinking. Um, and I think that if you can manage to do that in any organization where you you encourage there to be a bit of friendly competition going on, uh, where people are trying out ideas and that they're encouraged to be disagreeing with each other, that that's not seen as a as unfriendly thing to do, but it's just that's part of the the process. Um, I think it's a culture that really helps

in in investments. Uh, the best investments are typically uh the cases where you are right and nobody agrees with you, right? So really counterintuitive uh findings.

Um is that also the case in science?

Absolutely.

So what are some of the most counterintuitive

findings that you know of?

Well, I mean the uh well the only reason

but we're coming back to yours in astrophysics. Right. So

right the only reason I won a Nobel Prize was not because we had done what was really fun great experiment. It was because it came up with a real surprising result.

Yeah. And it is very surprising and you know that's that's like a cliffhanger because you have to you have to continue to listen to get to that one.

Yes. Yes.

Okay. We'll leave that. We'll leave that. uh in the meantime

but but I will say that you know in general um the you know the people who do well in the sciences um usually are doing well because they come up with something that nobody else has seen um and that that excites everybody that that you know they all say wow you know we didn't realize that the world was working that particular way um and that's really important to know and that's where the excitement is not somebody usually um it's not somebody who was just showing more of the same of what people already thought

why do we find uncertainty so hard.

I think that we have a a very lovehate relationship with uncertainty, right? Because uh in some sense it's what people love about so much so many games and so many uh things that they that they're going to do for fun. They they love that sense that it could go this way, it could go that way. Um nobody would would really want to, you know, play a sports where every single time you knew exactly how it was all going to come out. um that you en everybody enjoys that sense that something a little bit surprising could happen here and they look for that surprising thing to to occur. But we also are very easily scared. I mean it's very easy I think for us to feel like if we don't know that we're going to, you know, get to the next stage exactly the way everything is. Who knows? It could be bad. It could be that we wouldn't have enough to eat. Could be that we don't, you know, have a place to sleep. And and I think that that is probably an evolutionary uh you know deep down uh you know advantage um that if you if you are always being careful that you're not going to lose the fundamentals that that you need to live then probably you'll look out to you know what do your hunting and your farming early enough so you'll have food for the uh for the next season. Um, so I can imagine that that could have been a very useful cultural and and maybe even psychological thing to evolve,

but it's it's has to be balanced. You know, you you need uh to be u not driven by fear to the point that you don't step out of your cave and and out of your house and you don't try out ideas. Um because if you just just use the old ideas, the world changes. And the old ideas don't necessarily track um what we what what the world's doing. And they and they're not even the best ideas. Often they're they were good enough models of what was going on at the moment when somebody came up with them. But

there could be much better ways of understanding the world. And that is really what you want. You don't you don't want to miss those. And sometimes you need them

in in um I did a a degree in social psychology and I did my dissertation on on gut feel in investing. Well, I mean gut feel, nobody believes in gut field, but we call it pattern recognition. They always say they believe in it. Is there a room for pattern recognition in science?

Oh, or gut feel.

Oh, absolutely. Right. Because we we know that that our brains are doing a a very interesting combination of work when they're trying to understand what's going on out in the world. Um, some of it is the logical stuff that we're very conscious of that we're very aware that, you know, okay, I've just seen this and I've seen that. So I'm putting them together and I'm betting that probably that means that uh you know this aspect of of the world must be the case. But some of it you you're really puzzled. You don't see how is it possible that this thing and this thing turned into this thing and people work on it and work on it work on it and then some days they go to sleep and they wake up in the morning they think hey I think I know how that works. And we we think and this is some evidence that uh that the unconscious mind also has part the it gives us other ways of solving problems maybe a little bit more like the way uh these neural networks do in you where you don't you can't really track down exactly which thing was the source of the pattern recognition but in fact it does a very good job of figuring out patterns that we otherwise wouldn't catch. I I I often think that it's it's really good to try to play these two against each other. So um you you because some of the pattern recognition is wrong that we just we recognize things that we think we see a pattern and it's not really true. So we use our very thoughtful rational mind to go back and analyze and and use things like statistics to tell whether that pattern could have come appeared just from random noise um and that the the randomness looked to us like a pattern. Um so that's very um useful to apply your rational mind to your you know your logical rational mind to the pattern recognition side of the mind to to weed out the good from the bad parts of what you've come up with. But also the other way around, sometimes you need to feed the um the uh the pattern recognition mind by focusing uh the the rational mind for a long time on a problem and eventually that it seems to force the uh the pattern finder to go working while you're sleeping.

Yeah. Interesting. Uh we uh did a podcast with Magnus Carson, the chess player.

Yes. And so he uses his um kind of gut feel to choose for instance three different potential moves and then he then analyze them properly.

Ah that's that's very similar in that

absolutely absolutely

now he thinks that uh you know spending more than 10 minutes on analyzing is more than that doesn't really add a lot uh which is interesting. Now that's then that's probably

but then he's very quick,

right? I was gonna say it probably gives you a sense of exactly where his where his depth level is for his analysis that you know at that point he realizes he's no longer adding uh adding enough information. Yeah.

How do you teach this? How do you teach critical thinking?

We were not really sure how teachable it was when we began all this. So um there was a group of u well three faculty. I had a there was a social psychologist in the public policy school. That was Rob McCun and then there was a philosopher John Campbell and the three of us um started meeting um we we put a sign up saying uh are you uh are you embarrassed watching our society make decisions? Come help invent a course, come help save the world and about 30 students, graduate students, posttos started showing up um every week at the end of a Friday and we

So this was how you spent Friday nights?

It practically was it was for how long? It was Friday afternoons uh and and often went past dinner. Um and and this went on for like like nine months uh that we were meeting and what we were doing is we we were walking through what would be like a whole collection of ideas where if you had all those ideas clearly enough um it would help you think about problems in the world and then for each of them we started asking how could you teach that idea? Is there a way that you could get it across so that it wasn't just for one topic that you taught it, but anytime you read a newspaper article or anytime you walked down the street and you had to make a choice, um you would find yourself using the ideas and often that meant that we came up with like activities and games and and uh sometimes just good discussion questions and that mixture um is actually the way we end up teaching the course and I and I think it ended up uh it ended up being both fun but I think also a little bit effective. They copied that course in other universities as well. Right.

Exactly. So now it's starting to spread to other universities. We began it at Berkeley. It's now been taught uh at Harvard and University of Chicago and Irvine. I think now Colombia is picking it up.

True to be mandatory across the world.

I think that it's one of the things that people have it's one of the secret ingredients of what science really consists of that I think could help everybody. And so I

because because the world is more uncertain than it's ever been. Right. So you need more structure and vigor.

Exactly. And and I think that you know we've known for a long time that thinking tools are really important. That's I think one of the reasons we teach expository writing to students everywhere. Um they're not all planned to be writers, but I think the focus of how you think about a problem when you write is a very important thinking tool. But these are a whole set of other thinking tools that I that feel like they're very important in this technological scientific world. um and even a world where you just have to make probabilistic decisions with uh with other people and that so much of this has to do with how you can think together with people um in a productive way that you want that vocabulary of ideas to be shared by everybody. So I would love it to be as a standard basic course in every university.

Now um your mother was a professor in social work, your father was a professor in engineering and you are married to an anthropologist. So it doesn't get more cross domainish than that. Hey, what what did disc I mean what do discussions look like at your you know uh family dinners? Well, well, I remember uh at one point somebody was asking me, you know, how much influence was there of of having parents in these in these areas and I was thinking, well, my father was obviously a very big influence just because as a a scientist, uh, and you know, he would be doing calculations and I'd, you know, be seen the graphs and the and the calculator and I and just and slide rules back in those days. Um and and I think I was attracted to the fact that you can learn things about the world um that are very precise and and and organized and uh and I think I was enjoyed watching and and I was attracted to that. But at the same time I realized that I think probably just as influential uh was watching my mother who was a social professor. She studied public policy and public administration. Um and and then in her case, the work was often done in larger collaborations and just watching how she would work with a collab with collaborators. Um I think was was a whole different education and and it was a real pleasure to see groups of people who really enjoyed figuring out problems together and thinking together. And I think that was one of the things that was as important for me growing up as watching my my father as a scientist. M

how does AI inhibit critical thinking or does it? I think of it as a as a two two-edged. I mean that it can do both and I mean much like you know we I think we might have felt that way about calculators you know originally that we weren't sure should every every student be using calculators because shouldn't they know how to do multiplication and you know division and and and in fact we still teach how to do additions, subtraction, multiplication, division um first but then we we send them loose with the calculators. The tricky thing about AI is that it can give the impression that you've actually learned the basics before you really have and that there's a little danger I think that students may find themselves just relying on it a little bit too soon before they know how to do the the the work themselves, the intellectual work themselves. So I think that's the the danger. Now, the positive is that when you know all these different tools and and approaches to how to think about a problem, AI can often help you find the bit of information that you need um to use these techniques that we're teaching. Um and uh and so I think you know ideally what I'm what I've been asking in this this round as we're teaching the uh this set of we have like you know 24 concepts that we're trying to teach in this critical thinking course that for each one of them I'm asking the students to think very hard about how would you use AI to make it easier to actually operationalize this concept to really use it in your day-to-day life

but also how would you use this concept to tell whether or not AI was fooling you and whether the AI was was was putting sending you in the right direction or the wrong because many of them are just tools for thinking about where are we getting where are we getting fooled um and we can be fooling ourselves the AI could be fooling itself and and then could fool us and as we know at least the current generation of AI is very good at being overly confident about uh about what it's saying uh to telling you and then you believe oh well it's a it's a it's typed right there right in the screen it must be right um and yet you need to have that same sense of of gauge of of how much do you trust this result as you would when you're trying to figure out how much do I trust a statement I'm making or a statement that somebody else is making.

And I think that's the the game that you have to play um at least with the current version of AI. Um now of course AI will be changing and we'll have to keep we are constantly having to keep asking ourselves is it helping us or are we getting fooled more often? Are we letting ourselves get fooled?

So should we move out into space?

Glad to.

Is that where you enjoy living? So, I've I've been uh I've been really liking this the fact that um as a cosmologist, you get to you get to be in this really interesting place because you're you're using uh our understanding of the very smallest elementary particles and forces to understand the biggest largest structures um in the universe. And so you get to be sort of nestled right in the middle of scale while you look out at things that are tremendously larger than us and look down into into things that are tremendously smaller than us and and we're in that nice sort of middle place that we get we get to look at both.

Stupid question. What's the difference between a cosmo cosmopologist and an astrophysicist?

So the astrophysicists um are generally studying the physics and and or of of almost anything that you see in in the sky. Um and uh whereas the cosmologists are are specifically asking the question of um what was the evolutionary order? What how did things uh come about in this order from here to here to here that led to where we are today?

And you study both.

I do both.

When you look at when you think about um space, how do you visualize it? If you close your eyes, just what does it look like? I think that for me I've been very interested in this the fact that Einstein's theory of relativity allows for a for three-dimensional space to possibly have a weird curvature to it. And this isn't something that our brains, you know, are evolved to to picture at all. And so you constantly are having to s force yourself to imagine that there's what looks like infinite space. You're traveling as far as you want some direction, but it's possible that there's curvature in it. You'll find yourself back somewhere where you were before if you just keep going in that direction. And and and I think I always have loved um boggling my own brain. I like I like that feeling of not quite getting something but almost being able to picture it. And I think that for me that's a bit of a of a of a of a odd pleasure, you know. So when you picture it, is it like black with with you like the planets being, you know, lights and suns or just how does it what does it look like? I mean in in some sense the current uh universe to me seems like it's almost entirely empty and every now and then um in this huge empty uh void um you see a little dot and if you go close up to it you realize that that dot is actually a whole pin wheel of a galaxy um with a hundred billion little points of light of stars in them and then you go wishing by it and you then it disappears into the background and goes back to being this little dot and then eventually you see another dot showing up way out there in the future in in in the horizon and you and you get closer and closer and you start realizing oh it's another galaxy and and I think that's for my picture of the current uh universe. Now, if you go way way back in time, um the universe we think was um the space was sucked out between all these galaxies and they were closer and closer and closer until everything was on top of each other and then eventually all the material was on top of each other and eventually you get to the point where it was this

possibly infinite soup of of elementary particles that are hot and dense and uh and during that period um it's it's like a a very thick uh almost like a pea soup because it has little clumps where where the galaxies someday eventually form and slightly emptier spots but it's you know basically a non-stop plasma of these of these elementary particles. So those are those are the two different extremes of of time.

But of course a naive kind of question is something how can something which is already in infinite expand further. How you

I mean that's one of the the biggest standard mindbggling questions that um everybody uh comes to and and you know I certainly have come to it over and over again. Um when you hear the words the universe expands um that's the first thing that comes to mind. Wait the universe is everything. How could it expand? And the only answer that that seems to make any sense is you have to picture even an infinite universe um and you ask yourself okay um today as I said there's galaxy and then there's a lot of space and there's another galaxy and there's a lot of space in another galaxy and in an infinite universe um that's expanding all those distances just get a little bit f further apart. So it we're it's not expanding into anything else. It's that we're adding extra space between all points. So, it's almost like inflating it from the inside. Um, that we're we're just putting more and more space between any two between me and you and between this galaxy and the next galaxy and the and the further galaxies everywhere. We're just adding a little bit extra space and it's slightly bigger um because of that. And uh it's still infinite. Um it's just now there's more space between all the points.

If Ellen Musk asked you to go to Mars, would you go? I'm somebody who um loves the fact that there are people who would like to go to Mars, but I would never go and because I because there's so many things I enjoy doing that I'm I and I enjoy coming across and getting to explore with people that I would hate to give all that up just for this one thing uh you know the the one exploration of that of that of just going to Mars. Uh but especially because you don't get to know that you got to go back, you know, no,

right? I mean, if they promised,

you have to assume there was a drone ticket there,

right? Right. Right. If you promised that you could you that you could certainly get back and you' be and you'd be happy and healthy and you get to do all the other explorations, then absolutely I would go to Mars. You know,

given the big bang happened so long ago, why does it accelerate now?

So, the original picture um that we had started with when we were doing the measurements that that uh that we were talking about that led to our current uh understanding. Um the original picture had it that the universe begins um with a very rapid expansion that we can explain where we think it comes from but then has been slowing down ever since because gravity will attract everything else. And then if that were the case then the only thing that we really need to know about to to figure out the future is to measure how much it's slowing down because that tells us how dense the universe is and that's gravitationally slowing the expansion. So that's what we thought we were um setting out to measure back when we started that project. Oh, 25 no 30 some odd years ago. Um and then when we made when we actually finally got to the point that we could make the measurement um we found the surprise that actually the universe is not slowing down, it's speeding up. So what that led to is a expectation that there's some property um perhaps of empty space itself. But right at the moment, we're thinking uh it could be a new substance, a new field through that's spread throughout space that actually powers a a more rapid expansion and acceleration. And that's what we're calling dark energy.

And what is that?

And that's the mystery of the past 20 years. So we've seen uh that the this acceleration is happening. And so what we've been starting to do for the past 20 years is try to measure um properties of that acceleration to see if we can figure out what is what that dark energy could be. Um there there's well a huge number of theories. I think there was a I think it was estimating that in the last 20 years 25 years there's been a on average a new um theoretical paper about the dark energy written every 24 hours um published for you know um in the last 25 years. Um, and so there's way more theories than there is uh than there are constraints and measurements. And it's taken us 20 years or so of developing these projects to the point that we're just now um about to be entering the next 5 years or 10 years. Um we'll be making all the measurements that we were hoping to make for 20 years ago that we we're intending to start telling apart what could be the stark energy.

We talked earlier about the importance of believing in yourself. Now you spent three years without any breakthroughs uh researching uh these kind of things before you published in 98. How do you keep a team motivated to just kind of go on and on?

Yeah. Yeah. No, it was worse than that. In fact, um we started the project. So the n the 98 result started in 80 87 was when we first proposed the measurement and we thought it was going to be a hard project. We thought that was going to take three years because we were going to need 30 of these exploding star supernova to make the measurement with. At the end of three years, we had zero supernova, not 30. Um, and it was only after five years that we had um really de excellent first one that was well measured. And then but by then we'd learned how to make the um how to make batches of of these measurements. And so for the next three years, we collected a dozen or or more a year until we finally had the numbers that we needed to to to get the answer. And when you have one of these, how long time do you have to measure it and to

Oh, the well the these exploding stars, the supernova, they're they're amazing tools because you can see them across the universe and they and the the kind that we're using is all the same brightness and they make a great measuring tool, but they're a terrible thing to work with because um they don't let no warning about when they're going to explode in any galaxy uh around. They only explode every few hundred thou a few hundred years um in any given galaxy that you're that you're looking at. Um and uh and they rise in just a couple weeks and they fade away within, you know, a month or so. And so, and you have to catch them during that rise. So, you can measure them at their peak brightness. And uh and so they're the the most annoying research tool that you can imagine. And that's why it took so long for us to get to the point of knowing how to work with them as a as a very standard uh tool where we could turn out many of them all all the time and and study them. Because we talked about because we briefly talked about this when we when we discussed the importance of having a backup uh you know for these books and you said well you knew when you when you look at these you know these explosions you better have film in your camera right because

exactly and we used to do uh you know all sorts of things to make sure that we were that we would be that you know on those rare nights that we happen to be there at the telescope and there was a supernova that everything was going to work and we had teams of people flying out you know from one part of the world to another because that's where the telescopes were at that time that we had to fly to and we had teams of people back at the lab who were collecting the data from the parts different parts of the world and giving instructions for what they were seeing and what they recommended. Um so it was it was a bit of a show uh during during that period.

Talking about show I mean you and you raced against another team right?

That's right. So how did that uh impact the process?

It well at the beginning we we were it was funny because in the early days um we were trying to say this is a great project. everybody should be doing this. And then pretty soon another team was doing it. And that meant that now we were in a bit of a race because um there were only a few telescopes in the world that had the capabilities of doing what we wanted. And we were now both applying to use the same telescopes. In fact, we would sometimes pass each other in the airports going to these telescopes. Um and so it was a very toughly fought uh uh you know race.

Were you were you friendly with them? I mean did you say hi when you passed them at the airport?

Well well yes. Um but there there was you know some parts of it that were very conflictual and very you know people give each other a very hard time at the conferences. Um um but the for example the other leader of the other team and myself we would commiserate with each other about how difficult it was to keep our whole teams working uh effectively um you know independently and uh and we did at least on two occasions uh where we had we all depend on the weather when we're doing astronomy from the ground. And so we, you know, one one time they uh had terrible weather and that meant they were going to lose um everything they've been working for for that that particular sequence of searching and then following. And so we took some observations for them um so they could keep going. And then there's another time where we got bad weather and we traded times, we traded nights with uh with my uh counterpart, the leader of the other team. Um so they could help us be able to stay on our uh trajectory of of how many nights we had to keep observing.

And then you and then you share the prize

and then and then at the end the two teams came up with the same result um and announced it within weeks of each other. and and so in the end it was accepted as a a joint discovery and so uh the two the two teams shared the prize.

It's kind of beautiful. No,

I think I mean in some sense at the time, you know, everybody was saying to me, "Oh, you've got to you know, get you get yours out, you know, for first because otherwise you're going to have to share the prize and etc." And I and I was thinking, but in the end, you know, it was a relatively small community of people who were doing this kind of work and we were all dependent on each other in some sense for different things that we were contributing. And so it was the right it it was a very warm outcome in the end because it felt like it honored a big fraction of the community got honored by the by these two teams.

S is there an objective truth?

I think that the way science has been able to make all of its progress is by taking very seriously the idea that there is a world out there that is the thing that we're trying to figure out and that we don't get direct access to it. We get to see what we can see through our eyes, through our instruments, and through our tools and our measurements. And we build up our models of what that ex what that objective truth is of the world out there. Um, but it's going to do its thing whether we get it or not. And our models are going to be not quite right. They'll almost never capture everything. And for one thing, they've got to be simplifications just for our brains to understand some aspect of the world that's out there. We probably can't get it take it all in all at one time. So, we're always using little windows, little models of of what's going on out there.

But at the same time, um I think we make progress because we all believe that it's there no matter what we think. And that we can't just go to our separate corners of the room and say, "Okay, you believe what you want to believe and I'll believe what I want to believe." And you know, our science doesn't have to agree. Um in the science, there's a real need for us to agree because we have to um because that's where we figure out where we're making our mistakes. And it improves our picture of our model of what the reality is out there in the world.

And so the objective truth of the world is actually I think what provides the the link between our different uh our different projects that gives us the chance to figure out things that are more right and more likely to be wrong.

Is the world more uncertain than before?

Oh, it's I think it's always been uncertain. um you know in some sense I think that's the human condition that we that we grow up we don't know um you know what what's going to happen and we don't know what's going to happen to us we don't know what's going to happen you around us um and in some ways uh what we've been doing through civilization is making more and more little pieces of it manageable so that we can be pretty sure that those we can control um and but you know it it's always going to throw us uh curves that where we don't know um you know what's going to happen and our job of course is to be able to be nimble enough to manage what all what all the uncertainties are and not to have them throw us that not make them scare us but make us feel like that's where we thrive that's what we live on we live on playing managing the the the uncertainty

what's the benefit of being of youth in science

I think that you get so much advantage of having bringing in new generations year after year decade after decade um in into any field, any topic because um there's at least a half chance that they won't get stuck in the mistakes that the previous understanding made and they also are acting a little bit like when I said that it's very important to go find the people you disagree with and so they can give you a hard time. It's also true that there's a pleasure in usually in the young trying to show the the old uh the older that they that they made a mistake.

And do you never think hey who are you to teach me? I've got a Nobel Prize. I'm a superstar and you're just a youngster straight out of go, you know, university

all the time. Of course, what happens is that is that anytime somebody gives you a hard time, you you your first reaction is to say, "Oh, come on." You know, um first of all, you know, that that other team, they they don't know what they're talking about. They're giving me they're there's, you know, showing they're claiming I made a mistake. Um you know, I'm sure they made a mistake, you know, and then some of the time you you have to actually look at and go, you know, I think maybe they are right. Same thing with the, you know, fresh, you know, young Turk that comes into the thing and they they say, you know, okay, it's our turn. We're gonna we're gonna take over and do this thing and and you think, "Oh, come on. You you don't know how to do that." And of course, in the end, some of the time, they they're the right they're the right ingredient that you need at that moment.

And given that, what is uh what is your advice to young people? Oh, well I mean I my my sense is that if you can at all um put aside all of the scary aspects of the news that is often presented to you because the news is designed to be scary because I think that's how they make they get get you to watch. Um and if instead you can say we have always lived in a complicated um uncertain world and we are just getting better and better at learning how to ride on it and and work with it and we have this option opportunity and this option to work with other people to make a world that we that we want to live in. I think this is your moment. I mean, as as a a young person coming in this generation, you should not get turned off by all the doom and gloom that you might be hearing from the from the elders. I mean, you should be saying those are the challenges that we can deal with. We now know the how to do all sorts of things that are on the scale of these fears and these goals. Those are things we can we can manage. And so, you should jump jump in and join and try to bring the elders with you. I mean, try and get them to uh to be a little less worried but a little bit more constructively engaged, you know, in trying to think of how are we going to make the world the world we would like to live in.

Well, that's a perfect place to end. It's always been a privilege to be allowed to interview you. Big thanks.

Have a pleasure.