[Music]

Hi everybody. I'm Nicolola Tangan, the CEO of the Norwegian Sovereign Wealth Fund.

And today I'm in really good company with Sir David Spiegel.

Well, I would say he's the world best statistician and for sure the best communicator of risk that I've ever seen.

Written lots of fantastic books and is particularly known for making statistics, which is uh difficult uh really accessible for most people.

So big thanks for joining us. So David,

no great great pleasure to be here.

Having worked with risks, what what's the main thing that you have learned about human psychology?

So you know I am a statistician but I I have done my best to learn from psychologists that I have worked with and I suppose just from observing how people react to un risk and uncertainty.

I tend to think of the broader idea of uncertainty rather than just risk.

everything to do with not knowing about what might happen in the future or even not knowing what's going on at the moment or what's happened in the past.

Um, all these things we are uncertain about.

Some of them, you know, usually things do have an upside and a downside and so they could be considered risks depending on how they occur.

But I think, you know, what I've learned is that uh people, you know, have to live with uncertainty.

When you ask people, they say, "Oh, I don't like uncertainty."

And then when you ask,

but some but some people like it, right?

Well, some people like it.

Some people are a bit more bold than others.

But the point is that when you actually then go a bit further and say, well, do you want to know what you're going to get for Christmas?

Do you want to know how a match is going to end?

You know, if you've recorded it or something, do you want to know, you know, do you jump just jump to the end of your uh, you know, series on TV to see the last episode and see what happens?

Or also the thing I ask is, do you want to know when you're going to die if I could tell you?

And no, they all say no.

Or some would qu some would like to know when they're going to die.

Some are so in a way uncertainty averse uh that they really like to have everything planned.

But most people realize that you got to live with uncertainty and think of a life without uncertainty without some risk.

Think how awful that would be.

What are the people who hate risks the most?

Oh, there are some there are some people I think who are very cautious who uh would like to have everything planned out uh who want to feel that they can control all contingencies and have mapped out the possibilities and of course this is impossible.

Um and this is you know when I when I talk to audiences when I say the first thing is not not only that we have to face uncertainty but we have to face actually deeper uncertainty that we can't even list the possibilities of what might happen to us in the future.

we have to deal with that you know you know cloud of unknowing um and that's the part of human life and uh I'm interested in strategies that people use personally to to deal with that

how is it linked with the big five so my

impression is that introverts like risks less uh Americans like it more uh old people perhaps less than young people

just how do you how does that

yeah I haven't actually looked at that I

mean people will have looked at that because people have studied risk and proness from risk aversion.

What they found is that there's not a single characteristic.

I think this is why it's not part of the big five a single risk characteristic in people's personalities.

Some people I I've known people who are incredibly sort of what I would think consider reckless physically with what they did with their bodies.

They were extremely cautious with money.

Uh other people uh may be you know uh very bold um socially and take all sorts of uh risks uh in in changing jobs in changing friends going into new environments um

but again may be very cautious about their physical health and so I I I I there is not a single risk scale where you can put everybody on.

It's much more multi-dimensional than that.

It's much more multi-dimensional than that.

Why do people fear the unknown so much?

Oh, I mean I suppose if I was an evolutionary biologist I might say

I mean the we we fear the unknown risks more than

you know more more than the known right

which is why co covid was so scary because we hadn't seen it before.

Exactly.

Exactly.

And that's been known you know for ages that unknown unknown risk Ellburgg paradoxes that people invest in you know looking in the 1950s that if you couldn't actually say how big the risk was people were much more averse to being exposed to it and that's been known for ages a well known that idea of risk aversion to a well-known um to uncertainty about the risk has has been you know 70 years ago I think Daniel Ellburg did his original um you know study of that and um

and I think quite reasonably because um if you don't know what the possibilities are and roughly how likely they are then all sorts of other uh perhaps rather deeper attitudes to caution and precaution come in and people might start being really hedging themselves against and quite reasonably against major losses and we can see that going on you know in all all parts of our life.

and I think quite reasonably because um if you don't know what the possibilities are and roughly how likely they are then all sorts of other uh perhaps rather deeper attitudes to caution and precaution come in and people might start being really hedging themselves against and quite reasonably against major losses and we can see that going on you know in all all parts of our life.

Um what kind of habits can help us interpret risk in a more rational way?

Well, first of all, I don't like the word rational.

So, I I in my book, I hardly use the word rational at all because I think this um claim that's been, oh yeah, we can look at risks rationally and deconstruct them thing.

I actually in real life, I think that's pretty nonsensical because and I've taught this stuff for decades and I've taught decision theory.

I know how you should be doing it and I know you can't do it in practice because it for the for the theory to work, you have to be able to list all the possibilities.

You have to list all the options.

you have to look at the probabilities of the of the possible outcomes and their and their value to you and then according to you know economic principles you should maximize your expected return and things like that.

Well, it just doesn't work.

It be it fails at the first step that you can't even list everything that's going going to happen let alone except in really simple sit cir circumstances um put um numbers on everything.

So I I I I think r nobody can be rational.

It it just doesn't exist.

I think so I really don't like thinking in those and this is some objective.

No, I think one can try to be in a sense reasonable in a much broader sense of uh you know I suppose I how I would say is think is first of all you have to use as much imagination as you can.

the things might happen that you didn't think of, but really it it you're really um opening yourself up to problems if you haven't at least made a big effort to envisage the possible futures to consider quite extreme scenarios and that requires diversity of inputs.

I'm hopeless at this.

I have no imagination at all.

Absolutely disastrous.

So, but I know that if I were if I were, you know, God forbid, in an important, you know, making important decisions for society, I'd want advisers with a with a real range of different inputs.

The um the example I like is is Barack Obama when he was faced with the decision about whether to send in the seals when it was suspected that Osama bin Laden was in compound in Ababad and he had a a very diverse team of advisers who didn't speak to each other and some were you know they may have been set up as kind of red teams they they were really pessimistic 30 to 40% chance he was going to be there others were real gung-ho 80 to 90% chance he's there and uh he had to put all that information together but I think that's what uh someone who actually has to take the rap the real decision maker should be open to a diversity of opinion when there is no clear correct answer.

The um the example I like is is Barack Obama when he was faced with the decision about whether to send in the seals when it was suspected that Osama bin Laden was in compound in Ababad and he had a a very diverse team of advisers who didn't speak to each other and some were you know they may have been set up as kind of red teams they they were really pessimistic 30 to 40% chance he was going to be there others were real gung-ho 80 to 90% chance he's there and uh he had to put all that information together but I think that's what uh someone who actually has to take the rap the real decision maker should be open to a diversity of opinion when there is no clear correct answer.

The um the example I like is is Barack Obama when he was faced with the decision about whether to send in the seals when it was suspected that Osama bin Laden was in compound in Ababad and he had a a very diverse team of advisers who didn't speak to each other and some were you know they may have been set up as kind of red teams they they were really pessimistic 30 to 40% chance he was going to be be there others were real gung-ho 80 to 90% chance he's there and uh he had to put all that information together but I think that's what uh someone who actually has to take the rap the real decision maker should be open to a diversity of opinion when there is no clear correct answer.

The um the example I like is is Barack Obama when he was faced with the decision about whether to send in the seals when it was suspected that Osama bin Laden was in compound in Ababad and he had a a very diverse team of advisers who didn't speak to each other and some were you know they may have been set up as kind of red teams they they were really pessimistic 30 to 40% chance he was going to be there others were real gung-ho 80 to 90% chance he's there and uh he had to put all that information together but I think that's what uh someone who actually has to take the rap the real decision maker should be open to a diversity of opinion when there is no clear correct answer.

So I think the first thing is we have to acknowledge there's no correct way of doing this.

We have to have a diversity of opinion and we have to work in in a a combination of an analytic approach which I love.

I'm you know based in maths I I've done maths and stats.

I love de trying to deconstruct uncertainty, looking at the sources, analyzing data, building statistical models, and it's great, but it's never enough.

It never tells you what to do.

You also have to have judgment.

A bit of an increasingly numbers are emotional.

They're weaponized in debates and so on.

Just how do you counteract that?

Oh, I I think it's, you know, it's well known numbers, you know, it's a it's a complete myth that they're cold hard facts.

Even before they're weaponized, we know that someone has made a decision to collect that particular data.

There's lots of judgments has gone into every analysis, every sort of measurement.

They are there's always judgment behind every statistical analysis and it's good as that's made very explicit.

Um, and so other judgments can be added to it.

People may disagree about the fundamental tenants, the fundamental assumptions that are always underlying any statistical analysis.

So, it's a mixture.

It's not thinking fast and it's not thinking slow.

It's has to be a combination of the two.

Now, you started out in medical statistics.

just how did you how did that come about?

Oh.

Oh, very naturally.

It's just because there's a good job going and um and and cur curiously the job my first you know a real job after um after university I taught in America for a year and then came back and it was um it was 1978 and it was working on artificial intelligence in medicine 19 late 1970s.

It was a booming idea.

It was then largely called computer aided diagnosis and computer aided prognosis and it was building statistical models to enable um you know diagnosis but it included computer interviewing of people with stomach complaints and things like that.

It was really advanced and the tech was terrible but the ideas were absolutely modern and and you know the problems the issues about how to integrate this in with medical practice were in there.

So I was working on uncertainty in AI for much of the 1980s and we thought we'd solved it.

Ah, how wrong could we be?

But um because it's still a massive topic.

Obviously the machine learning techniques that people use now have developed, you know, beyond all imagination and they are really are incredible.

However, they are really struggling with uncertainty still.

However, they are really struggling with uncertainty still.

What were what was some of the strangest stats that you've seen in in the medical sector?

Oh, I don't know.

Some of the mind-boggling

the mindboggling ones.

Yeah.

I I suppose it's the the ones I've been involved in actually four major public inquiries into health scandals in the UK.

That's kind of where um I developed quite you know a slightly higher public profile I think.

So ones where you know over 30 babies died with heart surgery at a center more than you would expect to have died.

And then of course the second one was Harold Shipman, the mass murderer um who murdered, you know, at least 250 and possibly 400 of his patients over a 20-y year period.

And uh we were brought in, he had been caught by then, but we were brought in to say uh could he have been detected earlier or not?

could he?

Yes.

Yeah.

We can clear he could have been he could have been um detected after a few years if somebody had been looking at the data because he had so many excess deaths, but nobody was looking at the data.

So nobody could be in

are people now looking at data properly across across hospitals across

it's got it's got better but it's still it's still slow.

I'm in char I'm on a a group in the NHS that's only now building a a really rigorous statistical monitoring system for adverse events in maternity units and there's been endless maternity scandals in the UK and uh finally we've got a system based on it's essentially a statistical process control system but that we applied to shipment and then people took what we' done which was adapting industrial quality control to medical outcomes and then applied it to for example intensive children's intensive care in the UK has got a monitoring system based you know almost precisely on the work we did for shipment for early detection of problems

I'm in char I'm on a a group in the NHS that's only now building a a really rigorous statistical monitoring system for adverse events in maternity units and there's been endless maternity scandals in the UK and uh finally we've got a system based on it's essentially a statistical process control system but that we applied to shipment and then people took what we' done which was adapting industrial quality control to medical outcomes and then applied it to for example intensive children's intensive care in the UK has got a monitoring system based you know almost precisely on the work we did for shipment for early detection of problems

now you uh you are a leading kind of public communicator of of u statistics why is it important that people have a grasp of this field

why is it important that people have a grasp of this field

oh what are the big wrong decisions people are making

so we only have to look at some of the I

without mentioning any names uh well do a few Okay,

we only have to look at what's happening in America at the moment to see what happens when high level public discourse is not based on evidence.

Uh it's not based on numbers.

It's it's just based on on people saying what they feel like saying and regardless of of the evidence behind it.

Give some examples.

What are what are the most horrifying ones in your mind?

Oh, well, I I I I think the um you know what RFK is saying about vaccines at the moment, you know, because he's got a built-in bias.

I mean, vaccines are not perfect.

They're not perfectly safe and they're not perfectly effective.

So, I'd be the first one to say the term vaccines are safe and effective is is actually misleading.

However, they are of enormous value and um and he's got his particular I think biases there.

And uh and of course, I'm not going to talk about Trump and his way he was setting chars and things like that.

So, you know, it seemed to be, you know, how he originally did that seem to be based on what a you know, 20-year-old intern might do on a on a you know, on a in a spreadsheet.

And so, I It it just upsets me when I see um you know the enormous

Oh, apart from of course sacking the head of the Bureau of Labor Statistics when he doesn't like the numbers.

So all these things deeply upsetting to a nerdy statistician who I don't I I don't want to tell anyone what to do.

I don't want to tell anyone what the right policy is.

All I want to do is say please respect the evidence that we've got.

just respect it and it it doesn't tell you what to do or whatever, but just try to respect it.

And and it's not just of course politicians, it's social media, it's conspiracy theories everywhere.

The um lack of concern or the I think deliberate lack of understanding of what good evidence is and how a piece should be used is deeply upsetting to a to a nerd.

Why is it happening?

Oh god, this is beyond my look, I'm a statistician.

I'm not a great big sociologist.

Okay.

But give give us some more kind of examples of where you think the world is going totally bananas and and moving away from from facts.

Yeah.

Yeah.

I I I I think of obviously there's enormous blame on social media on the algorithms on the recommendation algorithms that that mean that something oh wow that's looks impressive and it's almost certainly wrong.

And so often they are based on numbers.

You know, people love numbers and they kind of think, as we said before, they kind of think they're cold, hard facts.

But no, and often the numbers are actually not completely wrong.

It's just that they're grossly misinterpreted and exaggerated and one-sided, cherrypicked.

And uh you know, a a wrongly ch a wrong number that's that's blown up and people making some bold claim uh is around the world, you know, is just triggered by the algorithms everywhere.

And it looks good.

It looks impressive.

and trying to backtrack on that is really really really is really difficult and that's why um one of the things you know I'm on the board for the UK statistics authority and one of the things I try to hammer all the time in the communication of official statistics boring old official statistics is um to try to preempt the misunderstandings that people will make you know because once everything's out there it's really difficult to counter the misinformation misclaims which might be made accidentally or deliberately Um, you can't stop every people saying everything.

You can't stop misclaims.

Um, but you can of if you can preempt them, if you can understand by knowing your audiences, by listening to people's concerns, even the people you don't like.

Um, to to know how what might be said, you can get in there and actually say this this data means, you know, I think we can interpret it to mean at least this, but it does not mean this.

And uh that I think is going to become is becoming a more common trend in the communication of official statistics to say what things don't mean.

And uh that I think is going to become is becoming a more common trend in the communication of official statistics to say what things don't mean.

What was the most important thing we learned from COVID?

Oh, the importance of data.

The importance of data in it.

Which data in particular?

Oh, everything.

Good.

We wrote a whole book, you know, with every chapter on a different data source.

There was so much.

which I mean I was working you know round the clock really analyzing data and communicating about it because I didn't have an official role which was great which meant I could get out there with the media and trying to explain things and I again I never said what should be done it was only trying to explain the numbers and you got had everything you had the vaccines you had the roll out you had um the testing you had of course the disease the infections

which I mean I was working you know round the clock really analyzing data and communicating about it because I didn't have an official role which was great which meant I could get out there with the media and trying to explain things and I again I never said what should be done it was only trying to explain the numbers and you got had everything you had the vaccines you had the roll out you had um the testing you had of course the disease the infections

but what a what a fantastic time for a statistician I mean it must be must be paradise for you

I mean it must be must be paradise for you

it was well I don't paradise isn't quite the right word but it was it was very exciting, very challenging and incredibly rewarding.

Um, and

important.

I mean, statisticians have hardly been that important before.

Exactly.

And I've had I keep on I now even now get people coming up and say, "Oh, thank you so much for the work you're doing during COVID."

And the media had to learn.

I you know that it wasn't just me or other statisticians who were out there talking about the numbers and yet we always at the beginning at least asked well you know who's to blame and what's going to happen or what should be done and we'd have to say I'm not going to say no no that's not my job you'll have to ask somebody else all we're doing is explaining the numbers and after a while the media learned that their a this is what the audience actually wanted an unbiased unaggendered discussion of the numbers and they loved it and so you know as I as I was really popular to be honest.

I you know that it wasn't just me or other statisticians who were out there talking about the numbers and yet we always at the beginning at least asked well you know who's to blame and what's going to happen or what should be done and we'd have to say I'm not going to say no no that's not my job you'll have to ask somebody else all we're doing is explaining the numbers and after a while the media learned that their a this is what the audience actually wanted an unbiased unaggendered discussion of the numbers and they loved it and so you know as I as I was really popular to be honest.

Well, you you in your book you talk about the five FCON rules.

You know, you tell people what you know, you tell people what you don't know.

Tell tell us about these five things.

Oh, yeah.

Well, this is from this is for communicating evidence in a crisis.

This is all derived from John Krebs when he was head of the food standards agency in the UK when he faced crisis after crisis.

He had foot and mouth, he had mad cow disease, he had everything one after the other, total disasters.

And he developed sort of playbook that he then afterwards he wrote and he said this is what I did when I was talking five points and I you got to have everyone should have these written tattooed on them I think first what you know so you know be really clear about what we know and then you say what you don't know you say where the areas of uncertainty are you admit them straight away second not first but what second and then you say um you say what we're doing about it you know we are learning more we're doing experiments we're finding out we're collecting data we are learning we are learning then you tell people what they can do in the meantime time that you may want to be cautious.

first what you know so you know be really clear about what we know and then you say what you don't know you say where the areas of uncertainty are you admit them straight away second not first but what second and then you say um you say what we're doing about it you know we are learning more we're doing experiments we're finding out we're collecting data we are learning we are learning then you tell people what they can do in the meantime time that you may want to be cautious.

then you tell people what they can do in the meantime time that you may want to be cautious.

You may not want to eat beef.

You may not want to do everything except you.

So you give people adi advice, self-efficacy in the meantime.

But the final one is the most important.

You say, "We will come back to you and our advice will change as we learn more."

So you emphasize the provisionality of what you're saying.

Now this is this is both deeply trustworthy because it's true.

Um, it's also, as far as I can see, absolutely impossible for politicians to do.

They just, it's just not in their vocabulary at all.

This idea of provisionality.

This idea of provisionality.

Why is it so difficult?

Oh, well, they think they have to be absolutely confident about it.

They say, "Oh, if we're not certain about everything, nobody will believe us.

They'll just listen to somebody else.

We have to be absolutely certain about everything."

And I think they believe it.

And our research with psychologists has and and not just our work but other research we've done randomized trials for different ways of messaging strongly suggests this is a complete myth that if you actually do you're in a position of authority you do actually admit some uncertainty that there are pros and cons etc etc um that you are trusted more and what's more important you're trusted more by the people who were initially skeptical

And what's more important you're trusted more by the people who were initially skeptical

absolutely

by the very people you're trying to reach trust you more because you're finally listening to their concerns.

You're finally acknowledging that um there are issues out there that perhaps vaccines aren't completely safe and effective.

So what that means is the common political way of communicating which is I think put into practice by communication departments in in in government which hammer through the message bam bam bam are actively decreasing trust in the group they're trying to reach those who are most skeptical people who believe them already there's no point.

So they're making it worse by their attitude.

And I I since we did these trials and actually saw data on thousands of people showing that trust was improved in the most skeptical way if you gave a balanced trustworthy message including uncertainty.

I it totally changed my mind.

It absolutely convinced me about this.

I also believe it's correct on an ethical point of view because it is correct but it's also purely from a practical point of view.

It should be more effective.

No, totally agree with you.

We we uh I've done a podcast with Rachel Botszman who is a specialist on trust and and indeed this is very very important but it's a bit um in the public sector generally they never apologize either right it's kind of tied into the same thing I think but

it's kind of tied into the same thing I think but

well what did we not learn from co

oh um

a lot of it was uh not being flexible enough um getting tied into you know we were told to you know wash our hands and wipe surfaces and things like that and within about a month we knew this was pretty useless scientifically and yet nobody ever said, "Oh, what you can this is actually this is not the point of point.

It's fresh air that's more important than ventilation."

Nobody said that for a year and and so I think what we didn't learn was that you need to be agile and flexible and take people with you by acknowledging the uncertainties and that you change course.

So the the guy we worked with who was uh you know most impressive I thought um was Jonathan Vanam the deputy chief medical officer and we worked together when the UK you know did admit that the Astroenica vaccine was causing these very nasty um well which was actually I think pretty well first detected in Norway um causing these nasty blood clots particularly in young people and uh we worked on the communication of that and where we showed that the the benefits of the vaccine went down massive when you got younger, but the risks went up.

So the the guy we worked with who was uh you know most impressive I thought um was Jonathan Vanam the deputy chief medical officer and we worked together when the UK you know did admit that the Astroenica vaccine was causing these very nasty um well which was actually I think pretty well first detected in Norway um causing these nasty blood clots particularly in young people and uh we worked on the communication of that and where we showed that the the benefits of the vaccine went down massive when you got younger, but the risks went up.

And so there comes a point you just shouldn't give the vaccine stratified by age.

And he then said to the public, he explained all this, used our graphics, went through the numbers, taking treating the audience with respect, admitting the evidence had changed, and then said, "We're changing policy."

And everyone said, "Well, fine."

And and he said, "Oh, we're adjusting our course.

It's not a U-turn.

It's adjusting our course."

And um and there was no push back from the media.

There's no accusations.

people really accepted it because he he actually showed the evidence to the public as he was explaining it to them and he could understand it.

Politician would have been hopeless at it because he wouldn't have understood what was going on.

He wouldn't be able to explain it.

And uh so that to me showed that um you know if you can get good scientists actually doing the communication and they're good and they're reliable and trustworthy this can have an enormous impact on the public and on um well public trust in authority I think

And uh so that to me showed that um you know if you can get good scientists actually doing the communication and they're good and they're reliable and trustworthy this can have an enormous impact on the public and on um well public trust in authority I think

uh the world is totally overflowing with um with data.

How do you um distinguish kind of the signal from the noise so to say?

Well, sorry.

That's my entire career.

You're asking me to explain what being a statistician means.

Um, that's a statistician's job, you know, trying to split the signal from the noise.

And of course, you can't ever do it.

And, you know, what's a signal, what's noise is never absolutely a black and white thing at all.

But um it's by trying to understand and this is a standard statistical you know thing that would have been said for the last century the sources of variation just like in you know pre-war you know the 19 all the statistics developed in um in Rothamstead breeding stations for uh for for plants and uh it was understanding the sources of variation what led to the variation between the crop yields what factors led to it and of course there is in a way unavoidable variation which tends be called noise or random error.

And then there's ins predictable variation which is due to factors that you might be able to control.

And that's what statistics has worked on for about the last hundred years.

And um and it's it's not been bad.

It's got some pretty good techniques for you know largely regression methods and so on.

Um when and so it's actually done quite well.

But you notice that that is different really from a a strict machine learning blackbox approach which just throws the data in and tries to extract a prediction you know a classification possibly with some uncertainty or a prediction of what what's going to happen where um there's no you know real understanding there of where it came from.

People again you can't people will have real difficulty for explaining why a piece of AI came up with its conclusion.

again people are really working on that now just like they are with uncertainty but um you know if you have rather slightly more basic statistical methods uh that you do I which I that's why I support them uh you can uh then generate a much clearer explanation of why you came to that conclusion and what are the important factors

again people are really working on that now just like they are with uncertainty but um you know if you have rather slightly more basic statistical methods uh that you do I which I that's why I support them uh you can uh then generate a much clearer explanation of why you came to that conclusion and what are the important factors

in which uh in which area are you the most impressed by the improved predictions?

Oh well I I I mean AI has is in terms of what you might call rather tightly controlled um areas um has done brilli brilliantly.

I mean the people at Google deep mind who started you know started on games like chess and go and things like that and then moved into um what obviously protein folding um and then and also sort of medical diagnostic from images in terms of breast cancer and eye problems.

they've worked with the Morfields Hospital is is tremendously impressive in that way in that they can take a really quite a big area but these are all tightly constrained problems.

Yeah.

you know there's a block of data you've got an image uh you've got uh you know a set of data and then you produce an outcome and it's just brilliant at that where I'm much more skeptical is about I think you know very uh unfounded claims that oh well we can just put your medical record into AI and it'll tell you x y and zed and so I'm much people make have made a jump from these kind of quite tightly constrained problems which are which are just brilliant into much more general problems.

And it's hardly surprising they do that when we look at large language models, how effective they are at coming back with a what is quite often a reasonable response to very generic issues because they've been able to mine, you know, vast amounts of stuff on the web.

Now, how good that would be about and and of course it can come up with medical list of medical diagnoses and things like that.

Yeah, it's just it's fine.

It can make suggestions and it's can be extremely effective on that.

But another if you take another I mean you've been around for a while I mean just uh during during that period the accuracy of weather forecast for instance is just mindbogglingly different right?

Oh, weather forecasts are fascinating what's going on because there's a real compet there's a real competition going on I think fairly friendly compet about the totally different philosophies for weather forecasting because traditionally they it's been kind of applied mathematicians and physicists you know building huge model weather models based on Navia Stokes equations and third order differential equations and they build massive models of the of the atmosphere and then make a prediction you know six 10 a week 10 days ahead um But they don't use any data apart from the initial conditions really and then they vary the initial conditions and that produces in an ensemble model and they produce a probability of what's going to happen.

the the alternative approach which people like Dean Mind have taken is to throw out all our knowledge of physics, the atmosphere, everything that's been learned in the last 300 years and just get and just throw it all out and take a the massive amount of data and to do a pattern recognition essentially and make a prediction which has no ability to explain why it's come up with conclusion particularly a pure black box and they're doing really well.

the the alternative approach which people like Dean Mind have taken is to throw out all our knowledge of physics, the atmosphere, everything that's been learned in the last 300 years and just get and just throw it all out and take a the massive amount of data and to do a pattern recognition essentially and make a prediction which has no ability to explain why it's come up with conclusion particularly a pure black box and they're doing really well.

Which one is going to win eventually?

Well, I I'm I'm really pleased that the UK Met Office has got both teams working and collaborating because it it's I think it's going to be complimentary in the end, but I'm kind of because my background in statistics rather than applied maths, I'm kind of secretly on the side of the blackbox machine learning people because I just love the thought of just putting the data in and out it comes with a with a prediction uh without any ability to say why.

It's saying it's just saying well in the past when this pattern was there this is what happened.

Well actually maybe that's as good as you can do.

So I I think um it's it's a fascinating competition going on at the moment which I'm watching with glee.

Would you have liked to be a weather forecaster?

Oh well I kind of I kind of think if I had to study something methology would have been I mean one but what I'm interested in is the I don't care about the meteorology.

That's why I quite like the databased approach.

What I'm interested in is the first main nature paper from deep mind on this didn't have uncertainties in it and I think it it's really important to have uncertainties when I um you know I all the time I look at probabilities of rain and things like that I you I use those uncertainties if I don't have them I I really um would feel a bit lost and so um what I'm

What I'm interested in is the first main nature paper from deep mind on this didn't have uncertainties in it and I think it it's really important to have uncertainties when I um you know I all the time I look at probabilities of rain and things like that I you I use those uncertainties if I don't have them I I really um would feel a bit lost and so um what I'm

So you so when you look at the forecast you don't look at is it going to rain or be sunn you you look at what's the probability of rain

okay

and what probability you need to bring your umbrella.

Well, exactly.

I don't know.

It depends how that's person very personal about how it depends what I want to do if I want to have a picnic or not.

So, uh I know I need the probabilities and so um this is uh you know this goes back to the 1950s with Glenn Brier developing a scoring rule but for probabilistic precipitation forecasts and uh it's it's terribly exciting I think and the the skill of these probabilistic forecasts is growing all the time and it will continue to grow.

Do you buy do you buy lottery tickets?

Do you buy do you buy lottery tickets?

Uh I bought one a few years ago when the the expected return was higher than the ticket price.

There had been so many rollovers.

It's still almost un completely impossible to win.

But I thought I've got to have a go at a gamble where the expected return is higher than the stake.

Where um in which cases do you look at probabilities where other people would look at yes or no?

Oh, um I got prostate cancer and so uh I I really interested in in forecasting effect of people with cancer and we've been involved in algorithms for um for breast cancer and prostate cancer building software to demonstrate those to people and they're all in terms of probabilities, you know, you know, roughish but not bad in terms of 10ear survival uh and how those will change depending on the different treatments you've got.

And I I think it's absolutely essential when somebody says, "Oh, my doctor told me I had 6 months to live."

I think what?

First, I never believe they said that anymore.

Maybe there's some doctors who'd be so stupid as to say that, but I can't believe that you have to show we don't know how long anyone's going to live.

We can put some broad bracket on it because we it's a survival curve.

We can put some broad bracket on it because we it's a survival curve.

First of all, I want to say I'm really sorry to hear that you've got canc but given that you are a mathematician, what are your stats?

Oh, mine.

Oh yeah.

Um is that's the problem.

I've got locally advanced prostate cancer.

So I I've got some minor metastasiz um olig metastatic it's called.

I got a few of them and um so I I and I but the drugs now the new drugs the new hormone drugs I'm on aberatone and um they just are so effective.

Uh now it's I've got my PSA is essentially non-measurable.

But the problem is we try to get the use the data from survival in clinical trials.

I can't that it's really depressing when I go back on the trials for apparatone.

They're terrible.

You get sort of immediate survival of 18 months or something.

I think what?

No, I'm going to live longer than that.

I'm sure because you know in the trials they were trying this on on very sick people. um and things have improved so much and the actually the um I'm I'm a the pe I'm at a earlier stage than the people who got the chance.

So there's no good data really on what are my survival prospects.

It's very difficult.

I I it's a shame and I wish there were um you know better you know databases.

I really wish I could just tap in and find out well out of a hundred people who are most similar to me what happened to them.

But for a start, we don't know.

We don't know.

It's only been given to people like me for a few years.

So, we got no long-term follow-up.

We don't know.

I mean, I my, you know, some people got it earlier.

My oncologist said, "Oh, I've had someone on this for 16 years."

And said, "So, that's the problem with something that's fairly rapidly changing.

When you're asking for a long-term prediction, you can't you can't say."

So, uh,

well, all I can say all I can say is, uh, fingers crossed.

And, uh,

yeah.

Yeah.

I mean it's it does seem a bit you know a bit pathetic for a statistician just to say well I you know hope I'm lucky but I hope I'm lucky.

Why are we so bad at predicting uh elections?

Oh elections.

Oh well for a start because when you go out and ask somebody the all the election things are just ask people what would you vote if you had to vote now?

I mean that's the question.

You're not even asking them what they're going to vote in the election.

You're asking them what what if you had to vote today what would you vote?

And um and that's a very biased measure of what you're trying to estimate of of you know what's that person will vote in 3 weeks time, two weeks time, one week or even you know 3 months time.

People change, people are not necessarily honest.

Um and uh so people may vote, they may not vote at all.

So I I think that the basic data source is always going to be biased.

Um, and so it's not like predicting weather.

You know, weather changes, but in a sense it's not changing because of what people feel.

Um, it's not like changing.

So I think it it they're never going to be great because they're trying to predict estimate something that you cannot predict.

You can't observe.

No.

You've been bringing um statistics into some new areas.

So for instance um anti-doping you know you mean the world anti-doping agency what kind of things were you doing there?

what kind of things were you doing there?

Oh then I I'm not sure what's happened then they had the idea of an athletics passport.

Um because they wanted to actually a passport which kept a complete record of their drug testing history.

Um and it was to try to allow to a certain extent for the fact that um you know there is individual variation between how people do respond um to drugs.

And so when you take a measurement from somebody um uh they were particularly interested in um people getting blood transfusions, you know, just before a um an athletic event.

And so you would be wanting to look at someone's red cell level or something like that.

Was it remarkably high?

Well, you only people vary anyway.

So to know whether it been pumped up high, you have to know something about their past history.

So essentially you have to have a model for um you know for someone's variability and their their natural baseline level for their hemoglobin level.

And so uh it was quite complex.

No, it was fascinating statistically and um and I I haven't I actually I would be interested to know whether it's what what the current situation is about that but it was in a sense trying to make the drug testing regime a bit fairer in order so it could adapt to the individual biology of the athletes

from uh drug testing to the financial industry.

Is there anything in the financial industry that puzzles you?

No, I've kept well away from that.

I mean I for a long time

why why why are you keeping well away from

I'm not interested for a long time of course because I was um I was funded uh from the charitable arm of Winon Capital Management hedge fund David Harding supported and he was great he gave us money and let us get on with what we wanted to and he was and I think I hope he felt it was a good investment but I I I I had nothing to do with the hedge fund um business and I've I've just never had any interest in money I hope it's just it just puts me off it's I just

I'm not interested for a long time of course because I was um I was funded uh from the charitable arm of Winon Capital Management hedge fund David Harding supported and he was great he gave us money and let us get on with what we wanted to and he was and I think I hope he felt it was a good investment but I I I I had nothing to do with the hedge fund um business and I've I've just never had any interest in money I hope it's just it just puts me off it's I just

why didn't why didn't you know why didn't uh Winton uh uh you know who's established a very uh successful firm why didn't he ask you hey David come and check out all things and make sure

no that was not part of the deal at all

no he had plenty of really good people

he's a mathematician right

yeah

yeah yeah

so he he had the whole place stuffed full of of of mathematicians really competent people much more competent mathematicians than me for a start and so um he for start I couldn't have contributed anyway and it would have it would have not it would have really inappropriate, I think, for for that to happen and and I just wasn't interested.

I couldn't care less.

Do you think people are worried enough about the really really bad outcomes?

So, for instance, um bioteterrorism, uh you know, the comeback of smallpox, that kind of thing,

nuclear war, you know, climate change, you say.

Um, well, nuclear war is relatively uh less dangerous than bioteterrorism.

Yeah.

Well, it could be on a larger scale, but yeah, it depends on the scale, but it's um nuclear war would not be great.

And so, um I I'm that's tricky, I think, because in the end, we're all going to die, you know, and you know, within a very finite finite period.

I from a personal point of view I would understand because I think I do that that people do not want to spend their time obsessing against about all the terrible things that could happen in the world.

It seems to me um that this is not a um not beneficial to your mental health shall we say to be really obsessed.

I I kind of hope there are people studying it, you know, more professionally and who are trying to counter it with appropriate regulation, appropriate policing and so on.

Um, but I personally do not want to spend my time waking up in the morning worried about smallox and bioteterrorism.

Um, if people are so in other words, I I think I can understand because I don't do it why this is not full of top of the agenda in people's concerns.

Catastrophic existential risks.

Um I for a start I I I but I'm hopelessly optimistic.

I think actually they tend to be overrated.

Um and so but that's maybe because I my particular personality is far too optimistic.

Was Norway lucky to find the oil.

Oh that's I don't know.

I think Norway I I don't know about how finding it but it was extremely sensible in how it dealt with it once it had found it compared with the UK.

um who so and uh you know which is why you why you've got your job at the moment to some extent and so um I think that Norway deal dealt with this in an absolutely brilliant way of seeing it as a as a a national resource to be in sense of you know um you know to be nurtured for the whole entire community rather than just in the UK to to sell off the rights uh in order to raise raise some money at the In shortterm view,

how do you look at climate risk?

It's difficult because I

so so you have people talking about it like u some places in America, you sit there in the middle of ash, rain, um forest fires and you're kind of in the middle of it.

You're sitting in the middle of it and you're saying CL, you know, there's no problem with the climate.

I know.

I know.

No, it's you know, it's just happening and that's it.

um you could just tell by the events which are just going to carry on.

There'll be you know ups and downs and but there are going to be more extreme extreme events.

I'm interested in uh in um attribute what what what it's brought to the four is attribution studies which is not so much about climate risk but about looking backwards and say to what extent was this caused by manmade climate change which is a really big growing area in research and it's going to be a big growing area uh financially when people start suing uh fossil fuel companies for um for events that happen and so the uh you know and but it all requires models.

You have to have a model of uh what we would, you know, how the climate has developed um with man-made uh you know, man-made interventions and how we think it would have developed had we not been throwing all this muck into the air um since the 1700s.

And uh and you see see how likely these events were under these two different scenarios.

And the relative risk can is the our meteorological office do have got an attribution center and they just they will give you a relative risk.

Technically that can be converted into a probability of causation to say the probability that this hot weather event that this tornado was caused by man-made climate change was x%.

And um officially uh once that gets above 50% by the balance of probabilities on a civil court case um you could say that man-made man-made climate change was responsible.

Now trying to then attribute it to particular companies I think is is rather more difficult.

But it it's brought from a technical point of view it's brought this fascinating idea of attribution.

And of course for that you have to have climate models and the climate models are used to make projections as to what's going to happen.

a lot of uncertainty and they the models take a lot of time in of dealing with that.

They also have what I think is really good independent teams coming up with different climate models which then they pull and then they make it even more uncertain.

They broaden the things out and um you know we're never going to know what's going to happen and uh one one should not state too confidently about what's going to happen but we know bad things are going to happen.

They broaden the things out and um you know we're never going to know what's going to happen and uh one one should not state too confidently about what's going to happen but we know bad things are going to happen.

Um, and what's to do about it?

Again, not my job.

Not my job, I'm afraid.

Talking about big things, uh, in, uh, 2021, you said that, uh, AI poses an extreme risk, but that it perhaps is overrated.

What do you think now?

Oh, I think it poses an extreme risk and I think it's probably overrated.

[Laughter]

What uh, could you explain?

Yeah, I mean it's amazing how how strong it is and it turns what you mean about extreme risk.

It's certainly going to take some jobs and certainly we're all going to have to adapt our jobs, our work to it.

It's also incredibly useful and valuable.

I use of course use it every day and um that I but in terms of sort of risks well people when they talk about this they're talking about sort of existential risks about you know um the the say self-aware AI that's going to start uh having essentially a will of its own and deciding that the these people get right get in the way of what it wants to optimize and um I I think of course that is a possible scenario

And and I think it's quite reasonable then that people respond by wanting additional overview of and guard rails on AI.

So it's like a lot of these things people say it's a bit like COVID before um you know co the modelers in the UK said oh there could be half a million deaths and that still gets quoted oh they said there were going to be half a million deaths.

No they said there would be half a million deaths if nobody did anything about it.

If we all just sat there and let it wash over us, there would be half a million deaths.

And there would have been.

But they're now being accused of saying, "Oh, they said there would be half a million deaths."

And there weren't.

So the point is that one could talk about there being risk.

But I think it's limited the value of that if we're all going to assume we're just going to sit here and allow ourselves to be taken over by killer robots.

So I think that what it does do is of course call for far greater scrutiny of um what's being done um in the o in openness about the guardrails put in um and so on.

I mean

I mean

but are you seeing but are you seeing the necessary guard rails and so on being put in place?

I I don't know enough about it.

Again, it's not really my area.

But when you look at AI, what are the type of things you look at in order to to gauge the risk?

Oh, again I don't know enough about I don't know enough about these sort of existential risks and how those could occur, the super intelligence and things like that.

I really don't know enough about it and I wouldn't want to claim to.

I really don't know enough about it and I wouldn't want to claim to.

And how do you use You said you use it all the time.

You said you use it all the time.

What kind of What kind of things do you use it for?

I use it in writing my book.

I use it for researching and I use it for coding and I use it for personal things like trying to work out where I'm going to go on holiday.

So, so I'll I'll use it all for all sorts of stuff.

Where does it where does it tell you to go on holiday then?

Oh well.

I I I I don't just say where should I go on holiday.

It's not quite that broad, but I no I use it all the time and it's and of course we we're almost forced to use it now because it comes up number one thing when we do a Google search.

So um I I I I think it's incredibly valuable.

It's unbelievable what it can do.

But I the guardrails of course are already in there in many ways in terms of you know violent speech and racism and all sorts of stuff that it can't do that are built in.

Um, and I want those the crucial thing I feel is that these should of course should be open and public and uh and they should be there should be a regulator to make sure they're being adhered to.

So, but lots of people are saying this.

I mean, there's a massive AI safety is such a massive because all the tech people are going on about it as well.

So, so I I'll leave them to it and hope that there's some some decent people involved in it.

But it is a crucial area, not my job.

Uh now if you do look at your job and and kind of your legacy, what what is the one kind of idea or principle that you hope you will be remembered for?

Oh, I don't know.

I've moved around rather a lot.

Um yeah.

Oh, one thing.

Oh, I think I I think it's the stuff I've been doing later.

I think it's on trustworthy communication of evidence.

Um in a way, it's not my I've done quite a lot of technical stuff.

That's where I get all my citations from, etc. and I get loads of those which is lovely.

But in the end, uh what I'm just obsessed by is the need for trustworthy communication about evidence.

Uh as we said, not that it tells you what to do, but unless I mean, we're s sort of doomed if we're if we don't use evidence in in an appropriate way and don't communicate it properly, we are just left up to thinking fast.

In Danny Carnean's turn, we're just left up to gut reactions and emotional feelings in order to for everything.

And I think, wow, that is just disastrous.

And where is the next step of this trustworthy communication?

Where is it?

Where is that kind of part of the science going?

Oh, uh, yeah.

Well, people are concerned about it.

They're obviously concerned about, you know, the quality of what's published in the scientific literature, which is a massive problem because there's so much junk out there, um, un from paper mills and so on.

Um I think that's well well I can talk about in the UK with actually this is now you've got a very high level there's a new there's a code of practice of statistics in the UK which is a pretty dull document but it's incredibly important there's a new version coming out which is really putting down you know the way in which all official statistics and even non-official statistics should be communicated to the public and based on these ideas of preempting misunderstandings of not being misleading of being open about limitations and so on and that's all down there and people have to adhere government departments have to adhere to it they are actually bound by it and so it's I think quite setting quite a good because in the UK we've got an office for statistics regulation which I think is might be fairly unique around the world an actual body that is there as a as an inspector as a regulator for statistics and uh I I'm a you know huge believer in that obviously And I would love to see that model developed elsewhere that you have got a body who can really tell people off when statistics are being misused.

My god, they'd have their work cut out in the US at the moment, wouldn't they?

My god, they'd have their work cut out in the US at the moment, wouldn't they?

Oh, that's for sure.

What's the big what's the big unanswered question that you still want to tackle?

Oh,

well, I'd like quite like to know understand consciousness and whether there is such a thing as free will, but that's again going somewhat outside my um in my my professional expertise, shall we say.

Um but it is in my book.

I do discuss it because I think it comes quite important when you start talking about whether to what extent is the world genuinely stochastic and random or whether in to what extent is it actually deterministic but staggeringly complex in a way that renders it unpredictable.

Um I think is an interesting issue.

Um, oh, oh, interesting question.

I suppose, you know, broadly it comes down a bit to what we're discussing before about whether I co was actually quite encouraging because in a crisis situation, um, good communication did rise to the surface.

There were obviously people on each side arguing.

I was right in the middle getting attacked by everybody.

So, I thought, yep, doing the right thing.

And it wasn't just me, but there was an enhanced respect for the mainstream media and largely people um you know were interested and uh you know there was a a whole body everybody was discussing the data.

It was very active community and I think it was it was as you as we discussed a very exciting positive time.

Now once a crisis has gone everyone goes back to their normal normal stuff and loses interest in all of this and um I suppose my qu my question I'm interested is whether that kind of interest and attention and wanting trustworthy information can be retained in societies that are becoming increasingly popular.

you know, follow populist politicians who object to authority, who are distrustful of what they call elites and uh experts and so on.

And as that happens, well, first of all, I suppose the big thing is can that be countered by having trustworthy experts, you know, people who do know something out there?

Yeah, I think it's so interesting.

According to Bill Gates, it would take roughly a billion dollars a year to make sure the world is really ready for the next pandemic.

And we the world is not spending that money.

Yeah.

And I the next pandemic will be different.

Um I and and it's people certainly wouldn't respond.

I don't think in the next pandemic people would accept lockdowns.

I just don't think they'll be politically acceptable.

So I think uh

so we all so so we'll all be Swedes in the next pandemic.

I think well the as Swedes as they as they said the guy the minister said well we practice social distancing anyway in Sweden.

So I thought that's great.

So um yeah I think we would be I think we would be more Swedes in the future.

Um and so but the crucial thing about that is that what it says is that you know you said you can spend a billion dollars but you can't change how you do what are you going to do about people.

I mean how people react is absolutely crucial and you don't necessarily change the way they react by just spending money on on on things.

So I I think that the the particularly in pandemics the the role of human reaction to the situation is which is the most important and the least predictable aspect um and uh and actually very quite difficult to research very difficult I think to learn from the past pandemic about exactly what worked and what didn't because things were so different across um every every bit of society with within and between societies.

So, um I'm not quite convinced that just throwing money at something is going to necessarily uh you know protect us against it.

What do you read outside statistics?

Oh, I read um some crime novels, but I'm interested in I really like history and biography um especially military history.

I'm obsessed with the Second World War.

So, I I spend my time visiting um you know, war sites around Europe and uh and and further and and in India.

So that's actually what I'm interested in.

Why are you why are you so interested in that?

I've been asking myself that why am I so interested?

I think it's growing up in this generation and you know born in 1953 in the UK the war was around us all the time in the films and the culture and the experience of of the adults around us and things like that.

We grew up with it as small kids absolutely obsessed with it and um and I never really have quite lost my my interest.

I I think partly because every time I read anything about it, I just sort of thank uh heavens for my well what's known as constitutional luck.

The fact that I was born when I was born into a society I was born into which was staggering constitutive luck.

Now, if you were to apply your if you were to put on your professor in statistics hat and give advice to young people using some numerical math or whatever, what what what is your advice to young people?

How should they think about their life in statistical terms?

Yeah.

Well, I do I mean some of them even read my books, which is I don't I had a 17-year-old come I nearly burst into tears because he came up and said, "Oh, I really like your book."

And I said, "Well, it's not actually aimed at teaching you.

I really I was so moved that he liked it because I actually think there is some stuff in there about um you know fa we have to face up to uncertainty.

We we don't know what's going to happen.

I didn't know when I was 18.

So I had no idea what was going to happen in my life.

I was as I said I had enormous constitutive luck where I was at that time.

Lots of opportunities you very secure situation everything paid healthcare university everything all paid for.

Um I was in a very really privileged situation.

Um and so but there's still of course massive unpredictability but I didn't mind.

I took I felt that I'd got a good upbringing which gave me resilience.

So the crucial thing and of course this is as relevant to corporations as it is to human beings I think is resilience.

It just it's the number one priority because that's how we deal with the deep uncertainty in all situations.

the fact that we don't even know we can't even list what might happen particularly some way in the future.

So all what we have to do is cultivate resilience which is an ability to deal with if not and learn from

benefit from anything that can happen whether it's good or bad and um for some reason I think I developed quite a lot of it.

I'm not quite sure maybe it's a nice secure um upbringing I had um and the the good fortune I've had but for some reason I think I got it.

I don't know and I think know again I would say this for young people that you have to go out and take risks.

Don't be reckless.

I say this to I do I I say I give talks in schools and I say you got to take risks now.

Don't be reckless.

You know look cover yourself from the damn major downsides.

You know just be careful but take risks.

So go out you know camping on your own in the middle of a moore but don't be stupid.

let people know where you're going and make sure you got the proper kit in Norway because everyone knows about that stuff, but not in the UK necessarily.

So, go and take go and have those adventures, but don't be stupid.

And so, and it's through having, you know, those sorts of adventures and taking some risks and being in situations where you're not quite sure what's going on that you develop resilience.

So, I just say to young people, go out there and have adventures, but don't be stupid.

David, I had a 90% probability on this uh podcast being very good, but it's even better than I expected.

I just absolutely love talking to you.

So, big thanks for everything you do to, you know, society and increasing the knowledge of stats and just for being such a wonderful communicator.

Big thank you.

Well, thanks so much.

This was somewhat outside my comfort zone, a lot of this, but so anyway, so I suppose I had to take the risks of of being there.

The biggest risk I've done for quite a long time is being on this podcast, I tell you that.

Great.

Big thanks.

Thank you.

Bye-bye.