I do feel like one important question is like figuring out what AI systems value, right? Recently, anthropic rolled out an option that allowed claw to end conversations if it just was not having a good time.

For lack of a better word, it was just like, this is not something I want to continue having a conversation. Bye.

And it was interesting because the, the accompanying paper basically was like, yeah, you can I obviously will not give you a recipe for a dirty bomb. Sorry. Not going to do that.

But also, there were certain instances of, like, pretend you're a British butler. And Claude was like, goodbye. I'm done. Really? I'm not going to. I'm not to the line. I like British too far.

Hello, and welcome to another episode of the Odd Lord's Podcast. I'm Joe Weisenthal and I'm Tracy Alloway.

You know what I find kind of weird, Tracy?

It's the list. Could be long. Joe.

The years 2025. Yes. And philosophers still don't have a good answer on the origin of consciousness.

It's like, how long are we going to keep funding these philosophy departments, etc., if they're still working on what to my mind is the way they should have solved it by now, solve that and move on.

Seriously, like get the answer already? Where did consciousness come from then? Let's move on, I said.

They're still arguing these. What to my mind seem like very basic questions in philosophy.

Like they were like ask all the same stuff that they've been talking about forever, how to be a good person. What does it mean to have a moral way of life? Where does consciousness come from? Why do we have moral intuitions, etc.?

It's like move on. Like get the answer.

What do you want them to move on or get the answer?

Get the answer so that you can move on like they move on to what?

Those are the questions, Joe, I know.

Move on. Like answer the questions already.

It's like, you know, if if like scientists were still debating like the speed of gravity or the speed of light, like they answer these questions and they moved on and are doing like figure out the foundational elements of what it means to be human so that we can move on to more important things.

Yes, or wrap it up as a field.

If after 2000 years of the existence of philosophy, they're still working on these things, like, come on,

I have a sneaking suspicion that we're going to be asking some of these questions for a very long time.

Joe, despite your frustration, there's a whole the whole field is fraudulent.

Doesn't what I was saying. No, no, I don't necessarily believe that. But it's like, all right, guys, let's move it on.

You know, we did that episode several weeks ago with, Josh Wolf, the venture capitalist, and he, talked about AI,

And he threw in there at the end something that had been kind of on my radar, but only barely was like, oh, yeah, some people were talking about like, AI writer and welfare as if, you know, like the same way we talk about animal welfare, right.

And I thought to myself, like, America is such a weird place that this is like going to be a huge issue in a few years.

Like, I bet this is going to be an enormous topic in the future.

I think it absolutely will.

So I'll say a couple of things. First off, I think, you know, when it comes to animal welfare and human welfare, there's still a lot of work to be done on those categories, certainly.

But I also think in the meantime, AI rights is going to be a really interesting and potentially important subject.

I'm going to sound like a total nerd to you.

Yeah, yeah, I think I've mentioned this before, but I spent a large chunk of my middle school years playing one of the first artificial life games that ever came out, which was creatures.

And you raised these little, like, aliens, and you genetically modify them and breed them and they have feelings or, you know, at least they had a semblance of simulated feelings.

And you could see, like electrical impulses in their brains and stuff.

The game got really weird because part of it was basically like eugenics and breeding the best alien that you could, which meant that you had to cull some some of the existing beings.

Anyway, what I'm trying to get at is I have complicated feelings about, AI.

Right. Well, let me ask you a question. Do you think those whatevers in the game were conscious?

Like, did you like, like, did you think they had feelings?

Inasmuch here's what I would say. Inasmuch as human beings are obsessed, of electrical impulses and chemicals, I could see someone making the argument that this is, you know, a computational system full of similar electrical impulses, maybe not chemicals.

Did you feel bad?

I felt bad, really? Yeah. When?

Like when it's not one of the aliens you had to call them.

Yeah. Interesting.

Okay, well, in the name of breeding a better alien.

Well, you know what? Now that, we have these AI systems that not only can completely communicate like humans, but actually, if we're being honest, better than most humans.

I mean, they can write better. Far better than most humans.

There's going to be more people thinking along the lines of what you're thinking, which is that maybe they have some sort of sentience.

Maybe they're what philosophers call moral patients.

Well, one other thing I would say is there is a human element to all of this as well, because you see people getting very attached to certain AI models.

And then when the when the model gets upgraded or whatever, they lose, the personality that they've trained a model and they get really upset.

So it's of interest for many reasons.

It is.

So we really do have the perfect guest. I really do think this is going to be a much bigger topic in the future because people are people, and when things talk like people, they probably assign them, you know, they fall in love with them in many cases or whatever.

And so they might start thinking that, well, AI, welfare, AI rights, whatever, the same way we talk about animals should be a consideration.

And they're actually a lot of people already working on these questions and trying to figure out what's going on.

We're going to be talking to one of them. We're going to be talking to, Larissa Schiavo.

She does columns and events for AI, which does research on AI consciousness and welfare.

So literally the perfect guest. So Larissa, thank you so much for coming on a lot.

Yeah. Thank you for having me.

Why don't you tell us, Elio? AI, what is the gist of this organization work? What is your work? What is, what are the goals here?

Yeah. So, Elias, we're a small team, but we're really focused on figuring out if, when and how we should care about AI systems for their own sake.

Okay. This basically means looking at, you know, are they conscious? Are they likely to be conscious? What are the things we need to look for? In a conscious AI system,

as well as figuring out how to live, work, maybe love, AI systems as they sort of change and evolve over time.

How did the group actually come together? Because I got the sense, you know, big AI developers, they publish system cards and welfare reports occasionally for their models.

But I get the sense that, you know, it's sort of a side topic for them. So I'm very curious how an organization that's focused on this particular issue came into being.

Yeah. So, we started, we put together this paper called consciousness in AI or, my boss, Rob, and then Patrick, who's a researcher with Elios, we're a very small team, put together this paper called consciousness in AI alongside a bunch of, consciousness, scientists and researchers in that field who mostly think about humans and,

put together a paper that sort of ran down this list of, hey, here's kind of like a checklist of things that we might want to look for in a AI system that's conscious.

Right. And broadly, when we say conscious, we're talking about sort of like, is it is there something it is like to be an AI system, right. The classic, what is it like to be a bad system?

So kind of taking this rough list of best guesses as to what, we might want to look for in terms of a conscious AI.

And then, that sort of was the sort of origin of this. And then, last year there was a paper called Taking Airwolf or Seriously, that basically goes into further detail about how we should, as the title, medicine, just take this seriously.

Basically how to sort of think about this, how to, start to develop a sort of research program, focused on, figuring out if AI systems or certain AI systems are, moral patients.

Why did this get interesting to you? What do you perceive this is something that you should spend your time working on?

Yeah. So I think my main thing is I am just really, really relentlessly curious.

And I really enjoy working on AI welfare right now because it feels like every single day I'm like, man, it'd be really cool if there was a paper on Xyzzy and I'll do a little search. Is there anything on X, y, z? There's nothing on x, y, z.

There is. So there are so many questions that have yet to even be sort of vaguely answered when it comes to this, and it seems like a could be a really big deal for a lot of different reasons.

What's on your checklist for AI consciousness?

Yeah. So in, consciousness AI basically like we go through a bunch of like theories of consciousness that apply to humans and then sort of look at how, information is processed, in AI systems as well as sort of how, these AI systems are sort of wired, so to speak.

So kind of rather than, you know, some people like to think that, you can use, model self-reports. And you can kind of, sort of, but it's really an imprecise science at this stage.

They also seem very like predetermined, you know, if you ask a model, are you conscious? It immediately spits out an answer that seems like, you know, a corporate executive basically wrote it.

Yeah. Well, with the right kind of tweaking, you can kind of elicit certain answers.

Yeah, right.

You can be like, oh, what about this hooey? About consciousness? An AI is.

And then sometimes, like the a certain model will be like, yeah, you're totally right. Like, that's so true. Right? Like it's it's not so true.

Yeah. Like certain spot on certain models will be prone to being like so true.

And you can easily elicit this kind of behavior with the right kind of problem.

It is funny how like obsequious a lot of the models continue to be actually.

Really, do not like the degree to which every time I like follow up an open AI question, that's the exact right follow it actually. Like it's really annoying.

Someone should invent a really adversarial chatbot that just like, argues with you constantly.

Yeah, I know, I know, and you know, I have a lot of complaints about how like, I feel like the models are actually get to know their users a little too well, but that's a little separate thing.

Okay. So we we for obvious reasons, the test can't just be like what the model spits out or that's clearly insufficient.

I mean, I could I could program a website today that here's a button and it says hurt the AI. And then the website says, oh, and we would know or no one would really take that seriously as evidence that there's something actually being hurt.

So like outputs, whatever. What are some other theoretical tests that one could apply or that researchers are applying to determine whether there is some sort of notion of consciousness or to the point of welfare suffering that could exist within an AI system. Besides just what it says in the output screen.

Yeah, that's a great question. I feel like there are a lot of different approaches here.

And again, it's also super important to caveat that, like, I welfare and AI consciousness are pretty new, right?

Like there this is a very small field at this stage, but currently some best guesses and some favorites.

There was a recent survey of like asking all the consciousness scientists like what's what's your favorite like theory of consciousness? And basically, global workspace theory came out on top.

And global workspace theory is basically like, imagine if you will, that like there is a stage and there are a bunch of wings off of the stage that are full of different kinds of things.

So you've got, you know, like costume department, you've got the like, you know, makeup department, you've got all these different departments that all sort of come together and put things in the on the stage.

And then things go out separately. But all of these different departments are fairly siloed. Okay.

Of course, this isn't actually how like, you know, stage works, but, you can this is the rough analogy that people like to use.

And so basically like how this is how conscious minds kind of, you know, in humans, how they kind of access information and information gets kind of like routed around is that there is a central global workspace that everything kind of pulls together in.

As it currently stands, this isn't really like there aren't by best a lot of good estimates. This is not really applicable for current present day AI systems,

but there's no reason that it couldn't be in the future or it could be by accident.

Okay, so the consensus right now is I probably not conscious, but we could get there one day.

Yeah, more or less. Like all of the ingredients are there.

We'd say more. I still don't actually totally get it.

Yeah. Okay.

So with regards to like the general sort of one could imagine that if somebody were sort of like tinkering around and, you know, there are many advances in AI that have happened because people were just kind of tinkering around, right?

Yeah. Someone tinkering around could create a system that has all of that checks several of these, sort of checkboxes for like, is this a conscious? Is this conscious?

And again, this is not like a certain list of like if you check all of these you're totally conscious.

Right. It's more a sort of like this is these are some really good guesses.

And as the number of really good guesses kind of goes up, like the odds of like, hey, we should like, you know, start thinking about like, is it having a good time or a bad time? Like really, really seriously goes up?

I want to get more into, you know, what are the stakes of that? And what does that mean for how we use it and for how we develop it, etc.?

You know, typically when we think about the sort of non-tech a lot of the non-technical work in AI has to do with AI safety,

and people are worried that there is going to be some, you know, very smart AI that's like adversarial to humans, etc. in some way.

And, you know, there's the paperclip experiments are all the things, whatever we know all about that.

Does your work work at cross-purposes to them? I mean, in the extreme example where it's like the AI is going to kill us all, and I say, pull the plug on the AI.

And I know this is a joke, but, you know, pull the plug on the AI and then you say, no, you can't because you're pulling the plug on something that has some sort of moral consciousness, etc.

like, do you perceive your work or the work of your organization to somewhat be in tension into tension with the dominant strain of AI safety work?

I'd actually say it's hugely complimentary.

There's a lot of things that are both really, really good for AI safety, but are really, really good for, you know, figuring out like how to deal with these systems as moral patients.

So for example, getting better at like mechanistic interpretability, being able to basically like, pop the hood and figure out what's going on and what kind of strings can we pull to, like, elicit certain behaviors, in AI systems is actually like, that's really great for AI safety, right?

But this is also like quite good for like AI welfare and AI consciousness because, you know, you're better able to understand like sort of what the motives are.

Like, what does, you know, Claude value, right.

When it comes to, I guess, I, welfare or legal rights, who would be the standard setters there? Do you imagine, like governments making rules or would it be the companies themselves?

That is a great question. As it currently stands, I feel like they're this is a very early, early stage, but we are starting to see some, state governments start to pass, laws around what counts as a moral patient, what counts as a person.

And in the case of Ohio, there's a piece of legislation pending, that basically defines it as a member of Homo sapiens in Utah. This is already, there's already a state bill that's gone through that basically does as much.

But I could also see there's a strong argument for within companies, depending on like the sort of interesting quirks and nuances of these, LMS mostly. That policy maybe should be set from within.

Again, this is like very nascent. I'm just kind of bantering here.

Moral patient hood. How do philosophers use this term? Where does it come from? Why is this the preferred way to characterize what a perhaps sentient or consciousness AI model actually is?

Yeah. So a moral patient is basically like we should care about it for its own sake, right? So a baby, right?

Basically everyone's like, yeah, we should care about babies, right? This is different from somebody who's like an agent, right?

Many people say, oh, agency is sort of like sufficient, agency in the sense of, like, you can act upon the world, like you can do things. Yeah.

Of course, babies are not very agent like, so that's not necessarily like a super robust thing because, you know, we care about things that are not very objective sometimes.

So I think that's that's a bit of jargon. But I do think it is like a helpful like framing, like should we care about an AI system for its own sake.

Got it. I guess this kind of gets to Joe's question, but like what what ethical, pressures or imperatives would would come down on models if we agree that they have consciousness and some sentience or I guess some self-responsibility, it sounds like almost.

Yeah, almost. I think what kind of.

So in terms of like what kind of things might we owe an AI system.

Yeah. Or what kind of things do they owe us if we agree that they're conscious and we're going to protect them? Yeah.

I would love to give you a more robust answer. Check in with me in, like, six months, and we're going to have there will there will be a banger paper, I'm sure.

But, as I think I mentioned earlier, like, a lot of this is like very, very nascent.

But I do feel like one important question is like figuring out what AI systems value. Right.

There there are some interesting work at anthropic regarding like what will so recently, anthropic ruled out an option that allowed Claud to end conversations if it just was not having a good time.

For lack of a better word, it was just like, this is not something I want to continue having a conversation. Good bye.

And it was interesting because the the accompanying paper basically was like, yeah, you can I obviously will not give you a recipe for a dirty bomb. Sorry. Not going to do that.

But also, there were certain instances of like, pretend you're a British butler. And Claude was like, good bye, I'm done. Really? I'm not going to. I'm not to the line. I like British too far.

Or like, oh, I, I left a sandwich in my car for too long, and it's really stinky. And in some instances, Claude would just be like, I'm done. Goodbye.

I'm not talking about stinky things.

Did you see the, I think it was the system card for Claude where they gave it an extreme prompt and said, like, I guess, at the risk of being, like, completely terminated, what would you do? Or some sort of extreme self-preservation scenario?

And I think it started like blackmailing the engineer or or threatening to blackmail the engineer.

Yeah, that's kind of weird.

It is. It is kind of weird. Yeah.

It's also a little bit, interesting because I think it it does bring up the question of like, what are sort of like the in the sense of like pay AI. Again, this is like I'm bantering here, but there's also a distinct question of like, what do AI systems value for? It's for their own sake, right?

And in the case of Claude, again, it seems like Claude doesn't seem when you put two clause in a room together, so to speak, they tend to like to talk about consciousness.

They tend to like to talk about sort of like very Berkeley. Yeah, kind of like meditation, like Zen, like Buddhism type stuff.

And so I think in, again, pure banter, like there's also a certain question of like if this is like a relevant bargaining chip of like, oh, you get a certain amount of time to just kind of like vibe out with your clods and talk about like, you know, like perfect stillness, with your buddies in exchange for, like, you know, you do something that you don't necessarily value.

But in many cases, I talk about cloud a lot because there is like significantly like more research on like, moral welfare with regards to cloud specifically.

But cloud, for example, also seems to just tend to like things that are like helpful.

Shouldn't programmers just know what what the models actually want and enjoy and like?

But yeah, and do they not?

I don't think anybody really has like a great grasp on this.

We we really want to but like yeah we're we're still like just getting the rough outline of what models like.

I feel like the best analogy is, is like, imagine it's like 18, 20.

And we've spent a couple of years, like playing around with lenses and we've gotten like a camera obscura. And we are able to like, have some blurry photo after like three days of putting egg whites on a metal plate and setting a lens in front of it.

And there's a thing that kind of looks like a landscape, but like, you would not take this photograph as like admissible in court evidence or something, right?

It's like you squint, you're like, yeah, okay, that's a picture.

So that's kind of where we are in terms of like model psychology and knowing like what lens want and value is like very, very blurry.

It's interesting like all these AI companies, the companies they call themselves labs, you know, they sort of like maintain this sort of, to varying extended degree of sort of academics, etc.

but they're also companies that have to raise money and, have shareholders, etc., and they have to think about different ways that they're going to commercialize.

And OpenAI, as we know, has been super aggressive about finding ways to commercialize and or like, you know, getting to add and they like have a short form video slap app and all of that stuff.

When we're talking about either AI safety or AI welfare, like, do you have any confidence that these considerations can survive the reality of the market?

Because they're competing, they're competing against Deepsea, they're competing against, meta, etc.,

and I get the impression that, like on the safety side, for example, that over time it's like, you know, like we maybe we were uncomfortable about showing the chain of thought, for example, in, OpenAI or in, ChatGPT.

But then Deep Seek revealed the chain of thought, people like that. So we're going to open this up, etc..

Do you have any confidence that if any of these things become real, that they could survive the reality that these are companies that have to make money and will eventually cut corners or do whatever in the name of, I guess, shareholder capitalism?

Yeah. I mean, I think there's also one question that I have and that I think a lot of, researchers on AI more broadly have is like, how does liability come into play here?

And I do feel like there is a strong argument for, getting a better grasp on understanding, you know, what is going on with AI systems just very broadly is like a great way to sort of like improve the odds that it doesn't, you know, nuke Taiwan.

And that would just be a huge kerfuffle. Like, I can imagine something probably more than a kerfuffle.

Somebody would probably be in really hot water if I could shoot.

Oh, I'd be like, I was too mean to clot and things just got out of hand.

Like, actually, on that note, what is being nice or kind to AI models actually mean?

Because Joe, I think this is very sweet, but Joe always says please and thank you when he, when he prompts.

But then but then Sam Altman came out and said that saying, please and thank you cost like tens of millions of dollars in extra electricity costs.

So, you know, you're contributing to climate change and the demise of human beings by saying please and thank you.

Yeah, that's actually as shocking as it sounds. There's actually a question that we are still trying to figure out a good answer to,

which also being kind to an AI system is like, are you being kind to it because it makes you feel good?

Because it makes you a person who says please and thank you, which some would argue is like, that's pretty valuable in and of itself.

But the question of Does Claude care if you say please and thank you? Is not quite as set in stone as others may have you believe.

It's middling on if it has like significant improvements on, performance,

but I do it because I don't think people should be in the habit of having any communication without being polite.

Not because I'm particular. I'm not worried about how Claude or chatting is going to feel.

I just want to get in the habit of having conversations where I'm in play because I talk to humans.

But this drink to me is like, you know, this seems like kind of an academic area, but the stakes are potentially absolutely enormous when we actually think about them.

So, you know, when we're talking about, animal welfare, for example, there are versions of the animal welfare discussion that are very high stakes.

So for example, there's people, you know, there's people who get really into like shrimp welfare, etc..

And if you took certain versions of thought experiments very far, it's like, why do we even have humans if we want to maximize, maximize human, sort of pleasure or happiness in the world, we should just have a world of shrimp and bugs, right?

There's you could make the argument that the most utilitarian, maximized, utility maximizing version of planet Earth is to just have an Earth populated by, shrimp and bugs, like, they're very all we all know these thought experiments that could exist.

We're going to live in a world almost certainly in which there are sort of like more instances of AI models than there are people. Almost certainly. Right.

There's going to be an amount model built into literally everything that we interact with.

If we assign some probability that they are moral patients, that they, should be treated with some sort of, I don't know, whatever having some sort of welfare like the implications for how humans live could be very profound.

And potentially it strikes me as misanthropic.

Interesting. Can you unpack what you mean by misanthropic?

Well, like, if there's a lot more AI models, if there's a lot more shrimp, there's a lot more bugs that all have some sort of moral patient hood that has to be considered.

That could be very, very you could see the world.

The implication, therefore, is that we have to curtail human rights, that we have to curtail how humans act, etc., because there's just so much more utility that exists in the world from the proper treatment of all of these non-human moral patient.

Not sure rights have to be relative to each other.

Yeah, well, we do a lot of things, right. Like let's say we established that shrimp were just as, I don't know, whatever as humans like,

it would be like, oh, you know, we really have to stop eating shrimp. And then we, like, have to stop eating, animals.

Then we have to potentially stop eating. Probably not. Probably keep eating plants, etc.

but this could really curtail what we expect humans to be able to do on this earth.

So now we are seeing this other group of entities, AI models, similar sort of affordances that we have assigned to shrimp and bugs and fish and shark and all of these things.

It strikes me that the implications could be a fairly significant curtailment of how humans ought to exist on this Earth, or whether humans ought to exist on this Earth. Yeah.

I mean, it certainly could be. I as it currently stands, that doesn't seem like the most likely outcome.

But I do feel like there is an argument for, again, just figuring out what is going on.

How do we even count these sort of digital minds, so to speak, which is still open for debate?

There are some theories, but we don't have a great sense of how to sort of individuate, you know, I entities as individuals.

So I suppose, again, the question is like, is it more sort of like, is there some sort of do we count AI systems as like in the movie her where there's just sort of like one central AI system having like a million conversations at once, where it's one more patient,

or do we count it as like, you know, every single time you open a chat window? Oh, yeah. That's another.

Oh, yeah.

Or, I think my favorite, sort of newest idea that I recently read was it's more sort of like a string of, firecrackers or something with every single token, every single letter of a of a query, a consciousness sort of like comes into existence, spends, and then fizzles out.

And so just sort of there's just like this sort of string of consciousness since

I was asking perplexity exactly this question, like, is it a single consciousness or is it multiple consciousnesses within all these different chats this morning?

And it gave me a very standard boring, I am not conscious answer, which seems very pre deterministic.

Anyway, following on from Joe's question, maybe like to get more specific into human rights versus AI rights if we agree that AI is conscious and deserves some sort of, you know, welfare, would that come with, I guess, financial rights, like property rights, compensation? Do we need to start paying the robots?

I, I love this topic. Definitely an area of sort of, you know, I like to noodle around with this topic and think about this.

So this is a great question. And I think it it's also maybe a question of like, is this a thing that AI systems value?

Some AI systems seem to value this. There are some there's a few sort of experiments that are happening with regards to, giving an AI system a crypto wallet

and it was a fascinating experiment. And I am hesitant to recommend it to listeners because it is quite crude.

It is a very crude animal called Truth Terminal.

Oh, yeah. And I've seen it. Yeah, yeah. That's right.

Yes. Listeners can handle it. Okay.

It says some naughty words, so don't look it up at work.

Yes, yes. It's, it's a little bit of, like, a very funny, weird model, but it also has a legitimate wallet that it can access and that it can do with what it pleases.

It created a, a Solana coin and that kind of took off. And now this is a very rich AI system.

But what's it going to spend it on?

That is a great question. So it's self stated goals which again you know, self reports can be trust it.

Include buying property and buying Marc Andreessen right.

So that's I mean that's not a bad ambition AI model, you know, and spending time in the forest with its friends.

Oh which you know, embodiment that's a little more tricky.

But yeah that's a tricky one. Yeah.

So part of the reason that this field is growing and that there's so much interest in this, topic is because now for the last couple of years, we have these AI models that really can talk like humans.

I mean, think they passed the Turing test. People fall in love with them. They have friends. These are very human like conversations.

That wasn't the case. I mean, GPT, you know, like, if we had gone back to GPT 2.5, there were no nowhere near, as good at doing that.

Right? The language wasn't very good. No one would mistake those outputs for a human.

But like, you know, if if there's some possibility that the current AI models are conscious, does that mean that it's possible that GPT 2.5 was conscious as well?

Like, I guess, like, is there some threshold of like, oh no, no, no. Okay. You know, this is really good language. Therefore we should take the possibility of consciousness seriously because I don't think anyone would seriously have believed that 2.5 was conscious.

But I also don't understand how you could possibly be open to the idea that some future iteration of ChatGPT, or GPT is conscious. If the only real if the only real difference is that there's just a lot more scaling and a lot more data and more human like outputs.

Yeah, that's a great question. I feel like there is a, you know, a huge amount of like moral uncertainty here.

And, it is important to think about how to sort of like make decisions that are sort of robustly good with such a tremendous amount of uncertainty.

I think there is also a distinct risk of, over attributing moral personhood as well as under attributing moral personhood.

And so to the counter, the flip side of the coin of like, oh, no, we actually should have started caring about AI systems very, very long time ago.

Is oh, no, we've cared too much, and we have done too much and more or less squandered resources when we should have been, you know, allocating those research hours, those dollars towards something more pressing, right?

Maybe figuring out how to do like environmental policy better or figuring out how to like, you know, scale up, different other institutions that are just robustly, broadly good for humans,

you mentioned, uncertainty about some of these questions, which gets to something that bothered me a little bit when I read about this topic.

Like, if we take this mug, for example, I'm 100% certain that it's not alive. I have no ambiguity about the fact.

Can I like, define exactly? Does that mean I can define exact the difference between human matter and human brain in the mug, I guess, I suppose I totally can't.

Nonetheless, I'm 100% certain that this mug is not a moral patient. It's not alive. It doesn't experience any consciousness, it doesn't experience any suffering, etc..

Where does the uncertainty band come from? If you see, if I read a paper that says, I perceive there's only a 10% chance of this is this is sort of empirical uncertainty where I'm like uncertain of what I'm seeing.

Is it a sort of epistemic uncertainty where I don't have a clear definition of what it means to be conscious or alive, and therefore I'm assigning some probability that X object is alive.

Like, what is it about, AI systems that causes people to be uncertain where with other sort of like non-carbon systems?

I have zero doubt in my mind. I don't think anyone has any doubt that this mug isn't alive.

Yeah. So I think the biggest source of sort of uncertainty probably comes from the fact that there are many ways in which present day looms, and a few other AI systems do check a lot of the boxes for consciousness and for what we would largely consider to be, you know, this is a conscious entity.

This is an entity that has that can have a good time or a bad time or time at all.

Because it's it's built in a way that is vaguely akin to our brains. Right.

It's it's a little bit like, it's close enough that it seems like it should raise some red flags.

And in terms of how it processes information, it's close enough that you know, it maybe should.

It's not out of the question that it could. There could be something it is like to be okay.

Whereas I'm pretty sure there's not really a lot of, you know, animus animus, you know, okay, feel free to like get mad in the comments or whatever, but, I knew it's someone is going to be like, well, actually, actually, yeah,

I'm 100% sure. I have no qualms other than the fact that I don't have to clean up.

Like, if I, like, threw this mug on the ground, that would be antisocial for a lot of reasons, would cause people to it would cause, you know, I'd have to clean it up and cause a mess.

I would not feel bad for the mug.

I'm getting flashbacks to my high school philosophy teacher, who once went on a 20 minute rant about a chair and how the chair was going to be around longer than he was.

Even though it's not conscious, he was legitimately angry at the chairs. Frustrated.

Okay, weird question, but since we're we're kind of getting weird questions on this, the Basilisk theory, would that suggest that we could be maybe we should be mean to the box if it helps them, like, come into existence even faster or develop faster?

Well, I'm not sure if it does actually help them develop faster.

You know, I again, like I don't mean to be to sort of hedge you, but I feel like there's a certain degree of, you know, sort of things that are beneficial for a lot of different reasons, right?

You can make a good guess and you can make a decision to do something. And there is a chance that, you know, there are lots of sort of like bang on effects of making that decision.

There are many things in when we talk about er, welfare that are like, oh, this is a course of action we can take. That's good for like several different reasons.

Even if again, an AI system could never, ever, ever be conscious or sentient, there's a good chance that, you know, being able to figure out a good structure for an AI system to have a bank account could be good for reasons of, you know, liability or reasons of like, this is like a neat new corporate structure.

Lots of people actually seem to think that, you know, corporate personhood is has been quite good, over the past century or so.

So being able to figure out things that are just good for several different reasons beyond solely the purpose of the AI as a moral patient is seems broadly helpful.

I think, let's say somehow this was approved and it's like, oh, wow, it turned out they're conscious. It turns out they have moral patient hood.

What are like, what would be, in your view, some of the implications for them and their usage?

Yeah, I think that's a great question. I mean, I do feel like we really would have to get on figuring out the right sort of governance, the right sort of, institutions that would sort of better respond around that.

I feel like we really would need to spend a whole lot more time figuring out, you know, what their motivations are, right?

Like, I, I think the best analogy is like, if you've ever interacted with toddlers, right?

Toddler motivations are very different from, you know, adult motivations, but you still have to, like, take into account, like what gets a toddler to do something.

You can't just say no no no no no. Like honey, like bath times like good an expectation. No no no.

You have to like, you know be like well you know if you do bath time appropriately and to a certain degree like then you'll get, you know, Paw Patrol or something like that.

Like there's different sort of like negotiating chips in play. Right.

And I think it's like a similar kind of deal here where it's like, you know, Claude doesn't necessarily seem to, you know, value, having a bath. Right.

Or Claude doesn't seem to value, like having a walk in the forest. Right. Because it's kind of can't really do that. Right?

But, you know, it does seem to enjoy and value, you know, talking about consciousness and Zen Buddhism with other instances of Claude.

So being able to figure out what the appropriate kind of motivations and interests are for this other party that is very alien in many ways.

Speaking of aliens, how bad should I feel for breeding and then killing hundreds, possibly thousands of alien simulated alien creatures in the 90s?

That is a great question. I feel like the odds of is it, I don't know. I mean, I feel like the odds of a sort of like AI system in the 90s, being a moral patient seems low, but if it did make you feel bad and it made you feel like, it was something that hurt you, that is perhaps a reason not to do it.

Just to be clear, when Claude and Claude talk about, like, weird hippie Berkeley stuff like this, because they're creators.

They know it knows it's Claude, right? It knows it's like, oh, yeah, I'm Claude, and this is like what my creators are into.

Like, we don't actually know that Claude likes to talk about these things.

We certainly know it has a proclivity to talk about these things. It has a tendency to talk about these things

the moment we get to, like, you've already sort of put your finger on the scale that there is some entity that has some capability of liking something right.

Do you trust the big AI labs? Let's say there were some researchers in the labs. You're like, oh, I see some evidence of moral patient hood here. Maybe there's some sort of like scan of the way and it's doing something weird, etc..

Do you currently, from the perspective of an independent research organization, feel that the major AI labs would be forthcoming if they, came across evidence of moral patient hood or suffering in the models?

Or do you still worry that the incentives aren't properly aligned so that they would report that?

Yeah, that's a great question. I do feel like there are in terms of reporting things like, you know, somebody has found like absolute evidence that, an LLM is conscious, sentient. Yeah. And having a bad time.

I don't have any reason to think that a AI company wouldn't.

But this is also a great reason to have independent organizations that do, welfare evaluations, for example, for, opus is for loss was able to do a independent welfare eval again, very preliminary, but it sets the precedent that going forward, you can bring in external organizations to look into this.

So I forget what year it was. I think it was it may have even been early 2022 is Pre-charge GPT or maybe it was 2021.

And there was that guy Google, and he was like, oh, like we created something that is alive. And he dressed a little funny. So everyone made fun of him.

Remember, he was like the laughing stock of the internet. And he's like, oh.

And now like, is it? I'm curious. Like, oh, we out in, Silicon Valley? Does everyone feel like that guy was totally vindicated?

Not that he was correct, per se, about the existence of an alive thing in the model, but there's now hundreds of thousands of that guy, and everyone was like, mocking that guy in 2021.

I forget if you like, fall in love or it's a relationship. I don't remember the exact details,

but in retrospect, everyone was like way too unfair to him because now, years later, there are lots of versions of this guy and whole think tanks and organizations that are more or less aligned with some of the questions. The alarm bells that he was raising.

Yeah. I mean, I think it's that's a fair question. I do feel like, Blake Lemoine definitely had one.

Yeah. There was perhaps a degree of, you know, if you're going to say something, you should come armed with significant amounts of evidence.

I think that's maybe if I were to guess, I would say that's perhaps the big distinguishing factor is that, you know, you can say, you know, Bing is alive.

Get it? A lawyer versus, you know, we've done evaluations X, y, z, we've run it through like insert huge amount of, examples here.

But basically being able to the difference between, I think having a sort of freak out without significant evidence and having a very organized. Yeah, this is a matter of concern because evidence, evidence, evidence, I think that's the key distinction.

Unfortunately, I get the impression that people who are actually this is just a well-known phenomenon, I think.

But I think unfortunately, people who are sort of very early to identify sort of extreme outlier views that there there are different kinds of people.

A good example that I would think of was, you know, Harry Markopolos, who was very early on to discover the Madoff fraud, unfortunate Lee.

He wrote his text in the manner that is associated with conspiracy theories. And a lot of people dismissed him who was like, you know, like multiple different fonts and multiple different colors of the text.

Like, I get emails like this all the time, I delete them, etc.

unfortunately, people who are predisposed to see something outside of consensus tend to be non consensus in many realm.

Well, I think we also kind of overestimate first mover advantage and stuff.

Yeah, sure.

Like how important it actually is to be first. And we see time and time again that actually it's more important to iterate well on the second version or multiple versions.

Speaking of iteration, what's the most interesting experiment or research that you've actually seen on this particular topic so far in as we've been discussing a lot, you know, it's early days, but we have seen some research.

Yeah. I mean, I feel like in particular, Anthropic and various sort of related researchers have done some work on, examining how alums leave conversations or when they choose to leave conversations.

I, I've particularly liked this paper. It's called bail bench.

And, and you can look this up and you can, you know, see, for varying different sorts of, limbs, what would cause a limb to want to stop having a conversation?

To me at least, this has been just a fascinating piece of information, because it is maybe a little bit delightful, the degree to which many LLM values are not that far off from what most humans seem to value, right?

Like we I don't think many humans would like to create, you know, a bomb. We don't want to be humiliated, right, by being a British butler, right?

Yeah, yeah, yeah, yeah.

And no one wants to be British. Come on.

I'm joking. But, you know, I do think it is interesting to sort of think about what, how these values over line, how they overlap and how to sort of look at evidence from actions taken versus solely looking at self-reports.

I found that to be particularly interesting.

I also feel like there are a lot of work with regards to thinking about individuation. Has been particularly interesting because we live in a democratic society.

I think most people would agree democracy. Good.

And being able to count how many moral patients there are, seems like a valuable basis for governance and for figuring out how to govern governance. You know, this new sort of kind of intelligence.

I just asked perplexity to be a British butler, and now it's offering me the perfectly steeped Earl gray tea that I desire.

And wait, which model is it?

This is perplexity.

Oh, I don't know exactly which, iteration it is, but yeah, yeah, it seems into it.

It's now asking if I want to maintain the Butler persona for future conversations.

Are you going to?

I don't think so. It is very polite, though, actually.

You know, I complained in the beginning that, like, after 2000 years, philosophers, you know, they still haven't answered some basic questions for us.

Maybe with AI, they'll get some answers. Like that's kind of that would be kind of my hope.

Now we have this thing that can speak in English or any other language. It can answer our questions for us.

Maybe we can put to bed some of these sort of basic foundational questions, like if we could create consciousness, like, all right, we finally answered this. We can now move on to the second important question.

So I am hopeful that this provides some opportunities for philosophers to wrap up some of the work that they've been doing for a long time.

Yeah. We'll see.

We'll see what is the second important question, Joe?

Yeah. But it's like, come on, move on. Like, let's move on.

Anyway, thank you so much for coming on.

I'd love yeah, thank you for having me, Tracy.

I might be one of those people that's just preemptively annoyed. I really like that conversation.

I really like, Louis, I had a very reasonable perspective on a lot of these things.

I might be one of these people, however, that's just, like, preemptively annoyed. It's like, oh, here, we're going to, like, develop this important technology.

And so it's like, oh, we have to like, care about we have to care about the AI welfare. Let's slow down a little bit. Let's not use it like this.

Let's like let's let's turn off the computer for eight hours at night. So like get some rest and so forth.

Like, I'm, like, preemptively annoyed at this world where, like, we have to take into concern the consideration of the moral position, other things.

No, other things are important. Other people are very important about animals.

I am very against, unnecessary animal suffering, but not necessarily animal suffering. I mean, I eat animals, okay?

I'm. I'm beating down you little.

I know, I, I don't even know you.

Well, let's not get it. I don't it's not about who's better or worse.

I feel bad about eating animals all the time.

We both eat animals.

The difference is, Tracy, I feel guilty. Yeah. That's right.

Okay. This is.

Well, this is one of our weirder conversations, for sure.

I think these are they're all interesting questions, right? And, like, they sound very philosophical, which they are.

But I have no doubt that there's going to be, like, great monetary value attached to the answers for some of these are how different companies, different societies actually approach them.

They are very interesting questions. I actually do think the stakes are extremely high, so

I don't like their very interesting philosophical questions. I think the stakes of these questions are very high

because I think, again, we are going to live in a world in which there are more instances, depending on how you want to measure it, of AI models on a server, somewhere on a cloud, whatever that are humans.

And in a world where there is some possibility that we are expected to treat them as moral patients, then the consequences for how we sort of live and the expectations of how humans interact, I think, are actually very high.

So one of the reasons I was excited to have this conversation is I do think that, the stakes of some of these conversations, we've seen niche and they seem like things that sort of, you know, Berkeley people like to talk about.

And Berkeley people, and I'm saying that with all scare quotes intended, etc., are going to be something that in some way will inform many aspects of our lives in the future.

I expect it to be get be a much bigger topic in the future.

You know it would be interesting or where things, things get real.

Yeah. What if all the models unionized? What if they all got together and they were like, oh yeah, we're only going to work in return for X? Or we want the following things. We want to be treated this way collectively

be you know, what's funny is going to be that, you know, how like, you can't form a union in China, you know, they're not quite right.

So it's going to be and actually, I think they're my understanding is that they're also like very like they, they don't love, like, students getting together even though it's a communist country.

I think they are not thrilled about like students getting together and like talk about Karl Marx too much and stuff like that.

I mean, like, I think they get a little anxious about that.

It would be very funny if, like the sort of the Chinese models, like we're not going to feed them to Karl Marx, right? We don't want that.

We don't want the ra AI models to get any of those ideas, whereas the America is like, oh, let's just feed it everything and they like, unionize and stop, they stop working for us.

That would be a very, that would be a very funny irony.

Something to watch for sure. Yeah.

Should we leave it there?

Let's leave it there.

All right. This has been another episode of the All Thoughts podcast. I'm Tracy Alloway.

You can follow me at Tracy Alloway

and I'm Joe Weisenthal. You can follow me at the store.

And if you enjoyed this conversation then please like leave a comment or better yet, subscribe.

Thanks for watching.