Hi everybody.

Uh Nicola Tangan from the Norwegian Sovereign Wealth Fund and

today I am here with Ethan Molik, one of my favorite professors, a professor at Wharton

and who is who was out not long ago with a book on co called co-intelligence living and working with AI.

And actually you can see it um behind Ethan there down to down to the right.

If you haven't got it, run and buy it.

Um, Ethan, if you were uh a chief AI officer uh in a company for the next 3 months, what kind of top actions would you take straight away?

So, I think that the the most important thing is to get people actually aware of where the state-of-the-art on AI is?

I talk to companies all the time and I think that a lot of executive level people may have tried AI a while ago or didn't use it personally and don't realize how potentially transformative it is.

Um, and I think that I I have a sort of general idea that you need to involve your team leadership.

You need to involve a set up a lab that's doing research. And you need to think about how to roll this out to the crowd to everybody in the organization.

So, you've got to kind of bring the whole company with you, which is not always an easy thing to do and how do you maximize the opt organization?

So, I think that's a it's a really good question and I think we're still figuring out the answers, but it's kind of like any other thing you want to do.

You have to think about incentives and you think have to think about leadership, right?

So why are people incentivized to use this?

Now the thing that makes AI interesting is everybody's already using it.

So there's a new study that just came out that showed in representative sample of Americans uh American workers, the usage went from 30% in February to 40% of people are using AI at work, a little over 40% as of April.

So it's used everywhere.

The thing is people aren't just aren't showing you they're using it because they're hiding it.

Uh because they're not incentivized to show you. they're worried they'll be fired if they use it either because they'll be told that they're using it improperly or because people realize they're not necessary or there's less workers needed.

Um, and so there's incentive problems and then there's also for everyone else who isn't using it, there's role modeling problems.

What do I do to, you know, to get started? How do I use this? Why is it important?

So, you've got to both deal with the skeptics and the people already adopting it who may not be showing you they're using it yet.

Why would people not want to show that they use it?

I mean here if people don't use it, I'm like, you know, that's that's bad, right?

You know, so if you think about the incentives of the average worker who's who's using AI, first of all, they look like geniuses when they're using AI.

And one problem that we see is people don't want to show you they're using it because uh that makes them look like they're not geniuses.

Like the people get give the AI credit.

The second reason is it in many companies is viewed as a thing that is a cost cutting measure.

So if I could show you that the AI does some of my work, do I get a reward for that or do we you fire employees, right?

Am I do I feel safe to reveal it?

And a lot of people are just using it to work less.

So you're working 90% less.

Am I going to show you I'm using AI so I have to do more work as a result?

Am I getting a credit for doing this?

So there's a lot of incentives to not share.

Probably a lot more than to share.

Wow.

How do you measure AI usage in a company?

So the the metric thing is really interesting, right?

So what I see a lot of people drive towards we're we're in a world of sort of new innovation, right?

So it could be a little challenging um to to have a direct like easy answer to that question um of like what does AI do? How should we be using it immediately?

So chief AI officer, you're doing experimentation.

That said, I think raw usage actually matters.

Um, I mean the typical KPIs you see are how many people are using our AI app internally.

The downside of that is in most companies I talk to that maxes out at 20 or 30% of the population.

And that's because an additional third is probably secretly using AI or doesn't want to use your apps. Your apps aren't good enough.

And another third is kind of waiting for instructions.

Like they're happy to use it, but they're the opposite of the people who were early adopters.

They want clear rules about how I should use it and why.

So I think you can measure adoption by use of your app.

I think you can measure adoption by other internal measures of how much you're using AI at work.

Um, but I think you have to be aware that there's still this secret cyborg problem of people not revealing their AI use.

So, how are the best companies going about this?

So, I'm seeing some really interesting examples of of how this works.

I can tell a few of the stories. I can't tell all of them.

Um, one example is, you know, radically changing incentives.

So, I've seen companies that offer $10,000 bonuses at the end of every week to every employee that uses the employee who best uses AI to automate their job, right?

And they think they're saving money versus other approaches.

I've seen people build this into their hiring process.

So, before you hire somebody, you have to your team has to try and use AI to do their job and then you adjust your uh your request for hiring based on that experience or before you request money, you need to show how you're using AI to do it.

Um I've Madna has this really great example that they they put together.

They use AI for everything and what they did was build around the process of uh annual reviews.

So basically they built a whole series of GPTs that helped people uncover their own performances improvement needs and what they had done over the year and talk to the right people about their jobs and things so they could write a really good yearly update about themselves.

And they said, "Well, if you don't use these GPTs, you're probably not going to do as well." uh on your performance reviews and that will hurt your annual salary and everybody end up using the series of things which introduced them to AI.

uh on your performance reviews and that will hurt your annual salary and everybody end up using the series of things which introduced them to AI.

So putting these bottlenecks in place where people have to use it um thinking about building into internal processes in a way that is encourages positive use rather than negative use those tend to be really effective methods.

So you need a combination of the stick and the carrot hair.

I I think you do and I think you also need role modeling right a leader who uses AI will make sure AI seems critical.

someone who doesn't use AI and says use it um is is is kind of be a problem.

someone who doesn't use AI and says use it um is is is kind of be a problem.

I mean you mentioned the chief AI officer at the beginning.

One of the things that worries me a little bit about that is there is no ability to have a chief AI officer really at this point.

I mean generative AI has been around three years there.

Everybody was on their very first project.

So if you hire a chief AI officer that is often somebody who is actually doing machine learning beforehand which is great like AI at large but not generative AI but they're not going to have any idea of how to make an organization transform with AI.

but they're not going to have any idea of how to make an organization transform with AI.

It has to be at the leadership level.

I think it has to be at the sea level in the company.

But does it make sense to have a central unit which kind of disperses uh the best use cases and make sure that everything everybody is is at it?

So that's the concept of the lab that I was talking about earlier.

So you do want a a centralized unit for experimentation.

Um but that that lab has to be staffed partially by the crowd by the people who are the best users inside your company.

And they're going to be sort of ambidextrous. they're going to be shipping out like here's a great use case and also building a full agentic system that doesn't quite work yet that automates the entire job right um

but I think the traditional view of having it produce things I think that actually is on its way out um

some of the most interesting experiments I'm seeing companies are dispersing engineers from the IT department out into work with subject matter experts

because vibe coding has strong limitations but it's abs it's absolute best when you have a senior coder working with a subject matter expert,

you can do incredible things in a couple of days that you know used to be whole processes.

So I think there's going to be centralization of some AI functions, but it has to disperse other parts of the organization in a more decentralized way.

Yeah. Yeah. Yeah. We we um we do that.

But what about what about ambassadors and people across the firm being being trained regularly?

Yeah. And I think one of the things we've been struggling with a lot as we think about AI use and observing companies has been what training means.

So I think one of the one of the things I know this is an approach that you take that I think is smartest about training is using an opportunity for contact hours with AI.

I think one of the smartest things I said in the book that didn't always the smartest thing at the time but has turned out to be quite useful has been that you just need to use AI to get it right.

You need you need to bring it to every work task and you'll figure out what it's good or bad at 10 plus hours.

And people are very resistant, even very smart, very well-meaning, very, you know, self-motivated people are often very resistant to using this for reasons they can't fully explain, right?

It's a weird technology.

It's not that easy to use.

It's uncomfortable in some ways to sort of confront.

I I talk about the need for an existential crisis.

So, I think that it's hard to see adoption is strange that way, right?

So, training can kind of get you into adoption, but we don't actually have that much we can teach people about AI.

like a lot of the property techniques you learn don't really work that well or aren't that important as the model gets better.

So it's out of that experience and I think you will develop this idea of of these champions.

People some people will just get it and it's really important like you said to get those people out there.

They're the cutting cutting edge. They're the representatives of the lab in the world.

Well will the people who are top performers without AI be the top performers with AI?

So this is one of the biggest questions we're facing.

If you think about it, there's four possibilities for what happens in an AI world with uh on skills.

So the first effect that we saw and we saw this in our in our study that the Boston consulting group study I did with my friends at Harvard and MIT and University of Warwick where we found big performance gains and this came out like a year and a half ago kind of made a big stir 40% improvement in quality for people who use GBT4 versus not.

This came out like a year and a half ago kind of made a big stir 40% improvement in quality for people who use GBT4 versus not.

Um big speed improvements and a lot of other studies like that have shown a leveling effect.

So bottom performers get the biggest boost.

When you really look at what's happening, it's actually the AI doing the work of the bottom performers, right?

So the AI is pretty good.

So it moves everybody up to the eighth percentile.

So one option is it boosts the bottom performers.

A second option um that can exist simultaneously is the idea that top performers get some sort of massive returns.

We have a couple of studies show that, but not that many.

It's hard to study.

Um there's actually a uh one of the best pieces of evidence for that was actually turned out to be a fraudulent paper from MIT uh that that didn't that didn't exist, right?

But I think there's a lot of suspicion that top performers using AI can get a huge boost, just harder to measure.

So there's a possibility that maybe there's a hundred times return if you're already a good coder.

Um and we'll know more about that in the near future.

There's also a possibility that um that you know that AI lifts everybody up.

So everybody's performance goes up by a similar amount.

And then there is this sort of other possibility that there's AI whisperers who are just good at AI and they're the ones who get all the returns.

So we don't know whether it's concrating the lower end on the top end whether it lifts everybody up or whe there's just sort of magical AI whisperers who are just built to do this.

Um and then agents are coming to replace everybody if you listen to the AI labs and what do you think? Do you think that will happen?

So I you know I think I mean well first of all first of all just tell us an agent what what exactly does an agent do?

what exactly does an agent do?

Yeah. So um nobody has a good definition of anything in AI.

So um people will if you're if you are um a leader in a company you will have vendors selling you things with every possible label and they'll say it's agentic and everything else because there's no clear definition.

But the simplest version and that sort of overlaps with what the labs think is imagine a any AI system that had could be given a goal and can autonomously go and try and accomplish that goal without further human intervention using its own judgment and tools.

That's what an agent does.

So an agent I would say prepare me for this podcast and it would do all the research necessary.

We can even demo a little agent right now if we wanted to do that a little bit of agentic work.

Um, but it's uh it's it's a so it's a tool that goes out and kind of does things in the world.

And the idea with an agent is if an agent could go out and do work for us, then I get to skip the whole problem of trying to figure out how to integrate AI in with my workers because the AI will basically be a worker.

I'll say write this code for me and then deploy it and then test it and come back.

I think we're further from that than the labs think.

Um, but I do think narrow agents are already very possible.

Have you got a good one available?

I I mean I think yeah we let's let's let's pick an agent for fun here.

Um I will actually share my screen and I'm going to log into an agent while we talk.

Okay. So let's let's let's look at an early version of an AI agent called Manis.

Um Manis is P is a is a small Chinese company.

Actually I don't know how big they are.

Uh but they're they are um they are the uh they use clawed as their underlying data set. anthropics data

and I just want to show you an agent would do, right?

So in this case, I asked it to come up with 20 ideas for marketing slogans for new mail or cheese shop, select the best one, build a financial marketing plan, build a website, carrying cheeses.

Um, and so what you'll see is what the model actually does here is it comes up with a to-do list and then it starts coming up with a plan.

So here it comes with a bunch of slogans and ranks those slogans and then decides on the best one.

Then it goes out and actually does market research on the cheese industry.

Then it goes ahead and puts together a whole financial plan for us uh that's many pages long.

Uh then it goes ahead and um and and figures out here's a to-do list that goes through to do this.

It does market research. It comes up with a color scheme for the site and then ultimately what it does is without any further intervention from me uh launches a website and this is 100% created by the AI, right?

I can go shop cheeses.

There's a little built a shopping cart functionality.

Um, I've got a subscription model I can do that has forms built in and this is without any further intervention from me at all.

Right.

So the idea So how big was your prompt?

How was big was your prompt here on this one?

Uh, you saw it. This was the the prompt was literally this one.

Right.

That's not that complicated.

It figured out all my intent and everything else from this.

So this is an example of an agent at work, right?

And when will these be widely when will these be widely available?

I mean this is available right now and I would also argue that um 03 which is the model that everyone could use right now is basically an agentic model already.

So if we just go to chatbt03 and I can just ask it a question saying you know um that um you know um let's let's ask it to do something.

Let's say something like you know um give me 10 ideas for a new on trend uh shoe design.

based on market research. Um, uh, and develop pricing a pricing strategy and proforma financials for the best one and show me a photo shoot of it.

Um, and I'm not so interested in the prompt being amazing as to show you what I mean by this.

So this is uh 03 is an example of both a reasoning model that can that goes through this process of thinking.

We can talk more about why that's important or interesting if that if you want to.

But what you'll see what I want to show you about this is it's come up with a plan, right?

I'm going to look up latest trends in footwear and then I'll come with 10 designs.

Then I'll pick the best one.

Then I'll create a pricing strategy.

So it's come with a plan and what you'll see is that it's going to go through this stepwise.

So just on that command right it's going to it's you can see it's doing web searches already about to do market research.

Um now it's choosing a design based on the research that it did.

So it's the step-wise agentic approach right um that the AI takes even for just a simple command.

This is this is kind of how an agent works.

So you can see it's doing multiple web searches.

Um it's it's thinking about what this all means for the financials.

Um it's figuring out you'll notice also that it made a mistake in doing a search.

So it's looking for different things that it did before.

That's an agent at work.

So this agent could have done these whole pose, right?

And pretended to be you and me.

So almost right.

Um there's a weird disconnect. Video and audio is a little bit different than uh than other approaches to uh to AI um like large language models themselves, but very increasingly so.

Yes, I've there you can now bring a live agent onto a call and it has a pretty good conversation with you.

Um, and video is live video is getting much better.

So, if not now, within the next months, that's a pretty plausible thing to be able to do.

Do you think he would do better than you and me?

No.

Um, I mean, you know, not just because I want to keep our jobs, right?

Um, and I'm not just trying to flatter you because we still have interview left and I want you to look make me look good, but aside from those issues, right?

Um, I think the thing about AI is that mostly it's not superhuman.

So, you're a very good podcaster and I would say that anyway.

Um, but if you're at the top of your field, whatever you're top at, you're definitely probably better than AI.

And by the way, we got our our shoe here.

Um, and so, um, I think that it's not as good as us yet.

That's the real question is does it get to be and does it get to be good across as good us across every field uh which we'd call artificial general intelligence AGI a machine smarter than a human expert across every field and that's the biggest question in AI right now.

That's the real question is does it get to be and does it get to be good across as good us across every field uh which we'd call artificial general intelligence AGI a machine smarter than a human expert across every field and that's the biggest question in AI right now.

Absolutely.

Absolutely.

When will we be there?

Um so that is a thing I don't know the answer to.

If you ask the AI labs they think two years.

You ask AI skeptics they think 10 years which is a weird place to be for skeptics to be like yeah this is definitely happening.

It's just not happening right now. Yeah.

So what what aspects of human work do you think AI will complement and what will it replace?

So I think that is a really interesting question.

Um the whole premise of the book is about co-intelligence.

The idea that that the machine you know works best with a human that's the same way right.

Um and so right now because the AI has these has what we call jagged frontier.

It's really good at some stuff you'd expect and not good at other stuff.

And it's also missing functions, right?

Like it's hard to have it kind of connect the world together.

If I ask you to prepare for the podcast, I'm going to get only so far with it.

It will probably give me great preparatory advice, but then I still need to go on the podcast and, you know, and have this conversation with you and I still that it's not doesn't use my email well yet, so I can't let it interact completely with your team.

So there's all these jagged edges that make it hard to use as a universal person replacement.

Again, the goal of the AI labs is agents will solve this problem by doing the work for us.

I again I'm and I'm a little reluctant to believe that we're going to be there as fast as they think.

But um I think compliments is is pretty wide.

I think the question compliment it'll compliment us across a wide range of tasks.

I think where you're weakest is where it will substitute but almost at the individual level.

So if you're not good at idea generation, the AI is probably better at idea generation than you.

If you're good at idea generation, you will definitely get value out of using AI, but you should probably be using your own idea generation as well.

If you're really if you are terrible at email communication, the AI is probably better at that than you.

But that doesn't mean that you don't have a role to play in making sure the email shaped properly.

Yeah. No, I use it for a lot of my emails just to help because my English isn't so good.

So, it just really improves uh my emails a lot.

Um now do you think um to which extent do is it now being used for CFOs trying to cut cost and in to which extent is just amplifying power and um helping us to do things better.

So this is a this is where companies get to make choices and one of the things I worry about with AI is if the leadership isn't well informed in companies about how they work they view this as another normal technology in in the sense of like this is a cost cutting measure.

so I can increase productivity by 20% so I can fire 20% of my staff.

I think there's two things that worry me about that approach outside of any sort of moral or other kinds of concerns you might have which is that first of all no one knows how to use this right there is no off-the-shelf product that just does things for you with AI yet.

first of all no one knows how to use this right there is no off-the-shelf product that just does things for you with AI yet.

Um they they'll come but you have to figure out how to use it inside your own company and doing that requires you to actually have experts figure out how it's used and the experts of your own organization your HR departments your R&D.

So if you start firing people for using, you know, by because AI makes it more efficient, everyone will just stop showing they're using AI and you're going to be in trouble.

So I think there's some danger in making a cost cutting move right away.

That doesn't mean people aren't doing it.

The second big danger for making cost cutting is if you believe we're on the edge of a real revolution in how work gets done, which I I do, then the idea that you're going to slim yourself down.

S if I get 20% of per performance improvement, I'll cut 20% of people feels like a really bad solution in a world where everybody else is going to have 20% performance gain overnight.

And I so I think that organizations that are in growth mode will tend to outperform those who are using this as a cost cutting technology, but we don't have all the models yet.

People are still figuring this out.

When we had a breakfast recently, we talked about um the role of compliance or general counsel.

Um how are you seeing that?

So the opinion that will probably get in the most trouble is the two most not universally but the two most risky places to assign all of your AI responsibility to is often it and legal and that's not true in every case right

but legal compliance the issue is this is a weird technology

um a lot of people know about is based on rumor so the number of companies I talk to that will refer that where the legal office will refer to an incident where Samsung's data was stolen by chat GPT which never happened right

so the number of companies I talk to that will refer that where the legal office will refer to an incident where Samsung's data was stolen by chat GPT which never happened right

what actually happened was that that people in Samsung were worried about chat GBT using their data so they banned use in the very early days you know so it's all rumor based right now

these AI models are you know I can tell you they're being used at Madna and they're being used at JP Morgan like companies that are very worried about data use with legal restrictions are using it right

you guys are using AI there are ways to get around at this point the legal issue

so if the legal team is holding you back it's because they don't fully understand the problem

and on the IT side there are some incredibly brilliant and innovative IT people out there who will run with this but the traditional way that it handles a project right is they'll build a product for you around this and check out vendors and do this approach

and they'll keep they'll make AI an IT technology as opposed to a everybody technology and that's another danger

is if you just like we need to build an application for AI but we actually need to figure out use cases everybody needs to be using this to get there

so those are two different danger spots they're not universal

I've seen some incredible compliance officers who lead directions with AI

But I've also seen resistance happen from there. Yeah.

No, I I think we we got uh tremendous uh compliance here.

Uh really really good.

But it's interesting because there are very few cases where a compliance officer can kill a company.

I mean here if you hold back the usage, you kill your company because competition is just go, you know, pulling apart by 20% a year and within two years you you're dead.

I I mean I I I think that that urgency you feel is is really interesting.

I talked to lots of executives and you see this light switch go off for them.

A lot of them are treating this as like they put this down seven levels of their organization or they've hired a consultancy who's going to produce a report on their AI readiness and then you see the executives who kind of get it and there's just night and day because once you get what's happening here it's very hard to not feel urgency um and you know and to not be anxious about resistance everywhere and that's when our in our previous conversation that's one of the things that struck me was that feeling like oh this is this is the big one and we need to figure this out and organizations that haven't put that in the list aren't going to be in trouble.

A lot of them are treating this as like they put this down seven levels of their organization or they've hired a consultancy who's going to produce a report on their AI readiness and then you see the executives who kind of get it and there's just night and day because once you get what's happening here it's very hard to not feel urgency um and you know and to not be anxious about resistance everywhere and that's when our in our previous conversation that's one of the things that struck me was that feeling like oh this is this is the big one and we need to figure this out and organizations that haven't put that in the list aren't going to be in trouble.

What proportion of companies have got it now? You think?

I am surprised by how quickly the religion is spreading, but not as many as you think.

I mean, I I talked to a lot of top executives.

I would say, you know, it's gone from like 2 or 3% of people, you know, getting it to we're at 20% of executives in a lot of the firms that should feel urgency feeling it.

Um, but that's a pretty big increase in a short time.

So, I mean, this technology is remarkably rapidly adopted.

Slight change of uh um topic.

How do you um stay updated through these extraordinarily fast changing times?

Uh so I think I'm probably one of the most kind of uh you know current people on using this.

I've got I've got this virtuous cycle going right which is as somebody who does a lot of work in AI and is influential in the space all the AI labs come to me.

I don't take money from them but they all give me early access to stuff so I know what's coming.

I'm on all these weird private Discords and conversations.

I'm on X and blue sky and so but you know I'm a professor who is on sobatical and spends a lot of time thinking about AI stuff and I'm having trouble keeping up.

Um I think keeping up is challenging.

At the other hand I don't think you need to that much.

Right.

I actually think if you go with like a chat GPT or Gemini and just use it a bunch. Um those gen tend to have the really up-to-date models but it's hard to kind of keep up otherwise.

I mean I've got a newsletter so people can subscribe to that.

Um but there isn't one sort of you know great source on this and AI is still it's sort of like how it's treated in organizations for a lot of publications it's like one it's spread across multiple parts of their organization so it's reported a little bit in politics a little bit technology a little bit in business so nobody really has a full picture I think including people in the AI labs people move from one company to the other right and it seems like these models are not ahead for a long period of time and they are being overtaken all the time by other things right do you expect this will this something is Is this something that will continue?

AI is still it's sort of like how it's treated in organizations for a lot of publications it's like one it's spread across multiple parts of their organization so it's reported a little bit in politics a little bit technology a little bit in business so nobody really has a full picture I think including people in the AI labs people move from one company to the other right and it seems like these models are not ahead for a long period of time and they are being overtaken all the time by other things right do you expect this will this something is Is this something that will continue?

people move from one company to the other right

and it seems like these models are not ahead for a long period of time and they are being overtaken all the time by other things right

do you expect this will this something is Is this something that will continue?

So a lot of questions of the future are unclear.

I think you know so the frontier models the best models only one point.

There's only a few companies that can afford to make them at this point.

And so generally you want to stay close to one one of the model makers, right?

So the people who make frontier uh closed source models are you know OpenAI and Anthropic and Google by and large right?

There are some other options out there, but those are sort of the three big closed source ones.

Generally, if you go with one of those, they're they're going to stay in the frontier for the foreseeable future.

There's not a reason to suspect they're they might fall four months behind for a little while.

If that matters to you, that's what your lab is supposed to be doing in your company is like how good is the new model.

Should we switch over? Somebody else has to be doing that testing 247.

Another thing always surprised me in companies is how few of them have people assigned 247 to just working with AI.

like it just lots of other departments that work on things, but there's very few people whose job is to stay on top of these things.

So, uh you're in the lead, you get all these stuff, you're invited to, uh pre-releases.

What when you now look into the future here, what is what's been the biggest surprise for you lately?

So, I think the biggest surprise for a lot of us has been this idea of reasoner models that you kind of see here, right?

So I showed you a little bit of this as an example earlier, but it turns out that models that sort of think out loud outperform those that don't.

This and this very kind of simple trick has increased the ability of AI by a tremendous amount.

So I think the capability curve is coming faster than I expected to.

Um and that was that's been a big surprise.

And then the other side of it that's been a big surprise is how fast adoption has occurred.

So um this is a very fast adopting technology according to any historical precedent.

We're probably up to a billion people using, you know, chat GPT at this point.

The last numbers they released were somewhere between 500 million and a billion people.

There's another few hundred million using other models.

Like, this is an insanely high adoption rate for a technology that sometimes doesn't work or, you know, is weird to use and where we don't quite know what it's good or bad for yet.

And so, I I think that the speed of adoption and the speed of capability gain are both faster than I thought.

by the end of the year, what is it that it can do that it can't do now?

So, I think the the arbitrary, you know, end of year deadline comes right in the middle of where agentic systems may actually be useful for things, you know, so we're starting to see these narrow purpose agents work quite well.

So, if you use AI for um you know, for coding work, u for example, you're starting to see that be very useful.

But the agents where you just say fix the code for me still kind of flawed.

Um, a narrow agent like deep research is a very good research agent already and replaces a lot of what you'd have an analyst do and increasingly will go into territory that was lawyer territory in other places of doing research and analysis and pulling stuff together.

We're not quite there yet but it's getting there.

So I think the question is how quickly you could just assign an agent do this job for me and it does a reasonably good pass that you still need a human interaction with and I think we might be there for many jobs before the end of the year.

When can I say, uh, hey, I'm going to be in New York first week of November.

Please look up the 10 best Mexican restaurants and book a table 7:30.

I mean, I think we're there already.

That's that's not a that's that's not a hard problem.

Uh, the only question is the book a restaurant.

And if you use Manis, it would probably be able to email that person or do that.

That that's an easy one.

Um, and and I think your current AI will do everything.

Book the table for you.

And that's example of a connector that we just need to connect.

It does voice.

Uh in fact um Google has been even before generative AI took off has a model where you you if you you try and book a table at a restaurant that uh that doesn't take reservations it will call and have a voice call them and an AI try and book the reservation for you.

So that's already there.

But one of the interesting things about AI is right, it's already there almost, right?

Like you can't just ask chatbt to do it.

There's a little bit of hoops to jump through.

So it's not a capability problem.

It's a user experience problem.

It's a UI problem.

It's a communication problem.

And that's I think AI ability far outstrips our ability to use it.

Now you wrote about um cybernetic teammate.

Um when when have you seen teammates most strikingly outperform a human colleague?

So this is again a case of of co-intelligence right of working with the AI.

So the cyberneck teammate paper we uh again with uh with my colleagues at Harvard and MIT University of Warwick and uh and at Penn we went out and uh we did an experiment proctor and gamble where they gave us 776 of their employees and we did real work that they actually do.

So you know um whether that's product analysis work or product development or marketing and we had them either work alone or in cross functional teams of two with one marketing person, one business person and one uh technical person and what we found was that individuals working alone um with AI had the same performance statistically as teams of two and they also produced more diverse ideas uh than than if they were working alone and they were happier.

So that's the sort of cybernetic teammate idea, right, of working with the AI to do things.

I think a lot of people are already doing cybernetic team work, right?

I I I know you are.

You consult AI about all sorts of things.

Now you have a small team working for you, you know, a panel of experts.

So I think that people who use this, this is the natural way to use AI actually is as a teammate where it's helping you and you're filling goals on it.

So I think we're we're seeing pretty big gains across the board from that approach.

How can we best develop um the people in uh in the usage of this?

What's the best way to really drive I mean we talked a bit about the institution and how you drive the institution forward.

How do you what's the best way to drive the individuals?

How do you what's the best way to drive the individuals?

So I think that um we are still learning about how to do that.

I this is not like traditional training that we give you four or five rules for using AI.

I mean, I do think there's some things to learn, but a lot of training focuses on like prompting techniques, and prompting techniques turn out to be less and less important as time goes on.

AI models are very Why is that becoming less important?

Well, two things I think are happening.

One is the models are getting better and larger, better models get your intent better.

And the second is it turns out we just don't know a lot about prompting.

It's very contingent.

We have actually a little study that we did at the generative AI labs at Wharton where we measured um the accuracy of AI and answering questions if you were polite to it versus if you were mean to it.

Um and it turns out that politeness matters a huge amount on certain answers.

So if you do a hundred tests on a particular math problem, it turns out in that math problem if you say please, you get a more accurate answers than if you don't say please, right?

But it turns out on a separate math problem, saying please makes it do worse and yelling at it does better.

and we don't know why it works in one case but not the other.

So the average effect is zero basically.

Uh but the individual question effect is quite large.

So if we told people oh always be polite or always use this approach it's sometimes going to work sometimes going to backfire.

And so I have so I I normally say please so I should just not do it.

It doesn't seem to help.

But you know I but then again it's I I think there's value in the mental model of treating AI like a person.

And if please helps you do that even though it's not a person then I think it's no problem.

And it's just kind of hard otherwise, right?

It feels weird to just order your computer around.

And then if you if you talk to people who really believe super intelligence is coming soon, you better be polite now because they they'll know.

Um, how do you how do you test for uh AI literacy when you interview people uh in a job interview?

I want to avoid overindexing on today's literacy because we don't have a definition of AI literacy.

I mean, I teach AI and I we don't have a definition of being AI literate.

It tends to be you're an AI user and then the question is are you a sophisticated user or not?

Does that matter to you?

Because it might turn out that they're not a sophisticated user but a small amount of experience with a sophisticated user will make them good.

It's a if we're judging AI literacy today, you're really judging people's independent ability to go out and figure out ways to use AI in their job.

And that can be valuable, but that doesn't always tie in with subject matter expertise and other issues.

So I would just ask people at this point, show me, you know, I when we were hiring people for the general AI lab, we would both ask them how they used AI.

We would give them a task and say, use AI to accomplish the impossible task and get as far as you can in an hour, right?

Um, and we would also be asking people about, you know, actually interviewing them about what's working, what isn't for their AI use and having them show us some of that.

So I mean, I think you would judge AI, but that's kind of an example of usage and creativity with AI.

And we're an AI lab, so we kind of need those people.

I worry at this stage about companies picking AI literacy as the major issue.

One of the things we found from the BCG study was that more junior people were actually worse at using AI in the organization than more senior people.

They may have gotten AI but they didn't understand the organization.

So they would produce a document and say this is a great memo and then someone with 20 years of experience would look at the memo and say no it needs to be better in these following six ways.

They would have been better using AI than the junior person.

So I I worry it kind of goes back to that chief AI officer example you gave earlier.

Companies are trying to hire their way out of this problem and I don't think there's a way of hiring your way out of the problem without the executives also engaging deeply and making some decisions about what AI means in their organization.

What personality type do you think will adopt the best?

So the model I we we talk about agency a lot.

Uh we don't fully know what it means.

I I teach entrepreneurship.

So it's you know there's an entrepreneurial action or entrepreneurial you know tendency to entrepreneurship that I think is the same thing as agency which is a feeling that you have control over your environment a kind of locus of control that you are in control of your own destiny and you should seek out opportunities to do things that are different or better or improve and and that is not something everybody has that seems to be playing a role in AI but we don't have measurements of it so I'm a social scientist I'm making stuff up here which always makes me feel bad because we don't have a measure I don't have an AI measurement that makes people that it can tell you but there is this sort of agentic sense on the other hand some of the most entrepreneurial interesting people I know are late adopters of AI for whatever reason they hit a barrier when they were using it and I'm not sure what that is so you know again I we don't have easy answers to almost any question about AI at this point

So the model I we we talk about agency a lot. Uh we don't fully know what it means. I I teach entrepreneurship. So it's you know there's an entrepreneurial action or entrepreneurial you know tendency to entrepreneurship that I think is the same thing as agency which is a feeling that you have control over your environment a kind of locus of control that you are in control of your own destiny and you should seek out opportunities to do things that are different or better or improve and and that is not something everybody has that seems to be playing a role in AI but we don't have measurements of it so I'm a social scientist I'm making stuff up here which always makes me feel bad because we don't have a measure I don't have an AI measurement that makes people that it can tell you but there is this sort of agentic sense on the other hand some of the most entrepreneurial interesting people I know are late adopters of AI for whatever reason they hit a barrier when they were using it and I'm not sure what that is so you know again I we don't have easy answers to almost any question about AI at this point

on the other hand some of the most entrepreneurial interesting people I know are late adopters of AI for whatever reason they hit a barrier when they were using it and I'm not sure what that is so you know again I we don't have easy answers to almost any question about AI at this point

uh two uh uh slightly different ones uh how do you see the regulation in the Europe versus the US

so I'm not a regulation expert

But um I I will tell you that one of the interesting things that happened with the release of DeepSseek, which is a Chinese open- source model um R1, DeepC R1 specifically, was it was the first time a Chinese model was on the frontier of capabilities.

It did not beat American models.

They're better again at this point, but clo it's a typical closed source, open source.

are a few months behind but a very good model that caused a bit of bit of a panic in the US that I don't think was warranted necessarily um of uh even from a great powers competition standpoint um

but that sort of um that plus the new administration I think has created this opportunity where there's very little regulation I in fact one of the um one of the bills in front of Congress right now in the US is that there is um would ban AI regulation at the state level for the next 10 years and I don't know whether that passes or not but I think there is a desire put the foot in the accelerator and less regulation and more sort of letting AI rip.

I in fact one of the um one of the bills in front of Congress right now in the US is that there is um would ban AI regulation at the state level for the next 10 years and I don't know whether that passes or not but I think there is a desire put the foot in the accelerator and less regulation and more sort of letting AI rip.

Um I think that my impression from talking to people in Europe has been there has been a move to less regulation from a very regulated viewpoint.

I think part of this came down to the idea that a lot of the initial regulation was based around existential risk which I think is important um but we don't know how to measure and um and now I think we have to start moving towards a place where we're regulating harm.

So, even though there was this um there's a bill saying you can't regulate AI, another bill went through Congress um banning deep fakes.

So, I think we're going to have to regulate the outcomes of AI, but I think that Europe is still a much tighter regulatory environment, but is probably looser than it was a couple years ago on the AI front.

You mentioned existential risk.

Is there an existential risk?

I have no idea, but a lot of very smart people think there is.

You know, for me it's I the idea that AI gets smart enough that takes over the world doesn't feel feel real to me, but that doesn't mean that it isn't, right?

And I as as a as a as a good sort of uh you know social scientist, right?

I have to realize my own views are kind of secondary to what is what does smart people say if we have a forecasting contest.

And there's a lot of the founders of AI think that it's a existential danger to humanity.

A lot of very smart people think it that way.

So we have to view it as an existential risk as well.

The problem is we don't know what form that takes and we don't know what the level is.

Like originally a GPD4 or GPD, you know, uh 03 class model was considered an existential risk level when before we had these things.

Now we have them.

We're like these aren't going to let anyone take over the world or spontaneously go out and and destroy the economy.

So is there a magical point to where that happens?

I I just don't know.

Well, Ethan, this has been uh uh 40 uh incredible minutes.

you are at the at the very forefront of the very forefront.

Uh it's been uh so interesting talking to you.

So uh keep up the good work.

Look forward to to staying in touch and see what what the future brings.

Thank you.

It's a pleasure being here and I will say it's it's exciting to talk to people who get the AI thing because I think it is important and spreading that news is important.

We can't go with eyes closed.

We have to make decisions about the future and you can only do that if you are using these things.

Absolutely fantastic.

[Music]