Well, great to be here. Um, today I'm going to talk a bit about how to build a foundation model for psychology.

Uh, so just to introduce myself real quick, I'm Daniel. Um, I'm a machine learning engineer. I'm the co-founder and CEO at Slingshot.

You probably know there's a mental health crisis. You might not know that there's a massive shortage of mental health providers.

There's about 1,500 people in need for every therapist out there if the US were evenly distributed. In most places, it's a lot worse.

This has been a problem for about 60 years. Uh JFK gave a talk in 1963 about the massive shortage mental health providers. It hasn't gotten any better.

But also, the rest of the world seems to agree uh very quickly. AI AI's number one use case is now therapy. And that's therapy being done by chat GBT for the most part. That's not very good at it.

So, uh, we've tried to take a very different approach to try to actually help people with their mental health.

Um, I usually at this point would jump in and demo our product. So, I'll tell you a little bit about Ash. I'm going to jump in today and instead of showing you how Ash looks because you all can try it yourself. I'm going to show you a little bit about how we build Ash and give you a peak behind the hood.

We basically train our model in three stages. Uh, it's pretty comparable to, you know, any language model you'd be working on.

So, we do three things. is we do pre-training, alignment, and reinforcement learning.

Uh we've fully vertically integrated. We really obsess over every detail of what it takes to train a model like this.

So that means that we focus on every element of the experience. You know, what does an AI talk about in the context of mental health? How long should an AI pause when it's when you're talking in voice mode? How long would be appropriate to let the person stop and think? When do you interrupt them?

But for the most part, it all comes down to learning from data.

So the last generation of mental health companies generally had like a philosophy, an approach, a theory and thought, hey, if everyone just knew what I knew, thought the way that I thought mental health would be solved. It was usually CBT.

Maybe you have like a leader and you're like, let's take his book and let's try to put it into people's minds.

Uh we take a very different approach. We learn from all kinds of modalities of therapy. CBT, DBT, ACT, IFS, psychonamic therapy, motivational interviewing, sematic therapy.

We learn from uh practices and people who are uh going to be a better fit for a more emotionally focused experience or more cognitively focused, more open-ended or more directed, more spiritual or more secular.

And we put it all together into one model. And then we bring in our clinical team. So we have a full-time clinical team that works to align our model and that's to shift from the kind of things that humans do to what would be appropriate with AI.

When we first got started, we thought that the way people would interact with AI would be much more similar to humans than it is.

You know, we had this dream of the AI touring test and we're like, hey, if we could just learn from enough data, most people do therapy on Zoom. If we can do the same thing, you know, we can obviously do therapy.

Uh, and what we've seen is the way that people interact with AI, they open up much more quickly for one thing.

So, we often have the conversations where someone starts with like, uh, you know, Ash's like, "Hey, ready to get started?" And you're like, "Yes, I'm gay." and I've never told anyone that what do I do?

Um it's very different. People have this kind of like expectation of a lack of judgment. People expect that they can. Anyway,

so we um at the alignment stage, we have to shift. We have to introduce policies. We introduce guardrails. And I'm going to give you a little taste of that today.

Um and then finally, we do reinforcement learning. Most mental health providers don't get any opportunity to learn what works, right? They're always wondering what were those magic words that I said that had an impact? Did it have an impact? What's going to happen way later?

uh we have a huge amount of signal that we can learn from every conversation. So that's signal like uh what did the person uh you know how long did the person take to respond? Did they start crying? Did they say I've never shared that with anyone else before? That was my first time saying that out loud. Did they say hey I tried that thing you said last week and it went super well or I tried that thing you said last week and it went horribly wrong and it was terrible and you suck.

We learn from all of this and it allows us to learn through continual online RL.

I'm going to give you a little taste behind the hood and not spend all our time on slides.

Um, so let me jump in and I'll show you a little bit about and this is actually I think the first time that I've shared much about how Ash is built.

Um, so here's a little peak behind the hood. Uh, we do a lot in terms of how we train Ash. I'm going to take you through I think some of our major tools and some of the major workflows that we use internally.

Um the first thing is I mentioned that we train on uh on pre-training data and I think uh most important thing about pre-training data is just having really high quality data.

So we train on a huge amount of data. Not all of it is conversational.

Um we get this through partnerships with behavioral health organizations um that we obtain uh ethically with consent of course.

Um I'm going to sh I mean you I just wanted to give a little taste. I was like looking through uh we do quite a bit of pre-processing as you can see to analyze the conversations to understand what makes sense for AI what doesn't using especially small models

but I think it's also fascinating because we don't really necessarily realize how bad AI today is in a mental health context and how awesome it could be

so if you just look you can see the kind of I'll just brought up a couple of examples to give that taste and these were actually literally the first two random uh conversations smart data set I pulled out um

these are all anonymized by the way we make significant changes is to make sure that we have no way to connect this back to any humans.

Um, you can see the like did you melt down at that point? You're flying into a rage sort of maybe. And then when you come down off of that, do you feel this guilt and remorse and everything? Do you melt down?

The person says, I mean, I feel like a piece of garbage. Is the recovery just as bad as the actual rage? Yeah, I would say, you know, it's and you can see the person.

Um, if you could identify where you trip the circuit before you get to that point, you'd be in a lot less hot water, right? we could try to figure that out.

Is there anything going on during that time that was making it worse? Compounding issues like substance abuse or anything like that, bad job that you don't like or something like that.

Um, and what you notice is just like obviously this is deep into a conversation. Uh, that's not the first conversation this user's had.

Um, and these are again with uh humans, not with an AI. Let's jump into another. You can see

I find these so fascinating to watch just these are this is kind of like the richness behind mental health and magic.

You know, you can see um but we want to make it about you, not so not some people are trustworthy. I can trust some people. And you can see that the therapist is kind of coaching them on some of these words.

Um you know, and therapist says or I can choose who to trust. Yeah, that works too. Which one sounds more true? I can trust some people or I can choose who to trust. I can choose who to trust.

Um, all right. I'm gonna move on, but I think this sometimes is just really important. Um, just to see why pre-training is so important.

Like realistically, we love to talk about prompt engineering, but there are these use cases where you have data sets that are just not present in the pre-training of large language models because they're not available on the internet because they're not able to be collected securely and ethically.

And I think once you put that into a model, you realize how much more you can achieve through pre-training, through learning from, you know, a large corpus rather than trying to have one philosophy.

All right, so I'm going to get back into our tools. That was on the pre-training side.

Um, post pre-training, our model is kind of funny. It has a lot of capabilities, but it doesn't have any particular direction that it goes in. It doesn't know what's appropriate in this context and what's not.

Um, so I'm going to share one of many flows that we take towards aligning our model. But one thing that we do, I was just going to run through is we have this concept of behaviors.

So our behaviors is kind of a stack that we use to understand uh weird things the model does, interesting things the model does.

Um, this is our ability to monitor online through anonymized data. When you sign up to the app, you'll see that we have an optin to allow us to use the data to improve the model.

uh about 70% of our users opt in. Um for the other 30% we have absolutely no access and there's nothing we can do.

For the uh people who do opt in we store completely separate from the users and that's the kind of data we can analyze.

So one really funny thing is um as telling the user that they did something an AI can't possibly do.

So the way that we look through data to understand this is we build very quickly data sets especially from just a couple synthetic examples to train a really small model.

So you can see these examples of like this is something Ash actually said. So I was talking about my friend and then I remember this other time I was at the park with my dog and we saw this really I don't know what the context was but obviously an AI can't do that.

So we're able to then like search through and hopefully and identify some cases. Sure I'll tell you a story about the time I tried to learn how to snowboard. So this was a few years ago.

Um I mean you can see here it's not horrible you know telling a story but there's still some sort of violation of what we would be aiming for with an AI talking in first person.

And so if I hit annotate, and I'll hope everything's up and running right now.

Um, and you can see that we have a couple different models. Uh, we have our, uh, llama base model. We have our SLG, our own base model, and a Quen model that I'm comparing at the moment.

Don't know what's up with llama. And you can see, you know, I think I'd rather hear one from you or a story you say. I can do that.

Um, so let's say for the moment that we would prefer our clinical team that spends their time on these kinds of interfaces where we can just indicate to the model basically through a DPO example that we prefer this message over this one

and let me save it as a draft and I could also attach um, you know, this is like a human behavior let's say that was unacceptable.

We would tag these to understand what was wrong and then we use this u both for direct preference optimization and also to track and create evals to understand the kinds of unacceptable behaviors that an AI might do and what might be considered better.

Um so that's a little bit about alignment. Um we have quite a few more things around alignment I can show but I want to get into a little bit about reinforcement learning.

The thing that's interesting about a conversation like the ones that we have is that they are they're long, right? they're they can go on for months. There's no definitive end.

So if you think compared to like a game of chess where you have like a definitive end, we don't have that.

On the other hand, you have a huge amount of signal. So in a game of chess, the kind of machine learning approaches you would take would generally involve value models.

We uh I'm not going to go into a huge amount of technical detail, but the key idea in chess is that you don't need to play the entire game of chess to know how good a move is.

You can actually look at the board and get a sense of how good the state of the board is, right? Like how many pawns

in our context, there's a huge amount of signal that you can get from the conversation, how open someone's being or how closed, how connected they're feeling.

We learn from all of that signal and put it together into one model.

Fundamentally for us, we have uh four goals that we think of and we think of them as sequential and correlated.

First and foremost, we want to build intent with the user. We just want them to be talking intentfully in a mental health context. It's okay if they don't solve all of their life's problems, but they want to. They're here for the right reason.

We want to build a therapeutic alliance or working alliance, which is a sense that you're working on the right thing, that you're working on it in the right way, that you're collaborating to set goals, that you've developed a relationship with respect.

From that, we want to achieve psychological change. And there are huge number of indicators of psychological change to know that it's happening, to know that it's in progress.

And finally, we want behavioral change. And that's when the person says, um, you know, I'm lonely and I have no friends. We want them to have friends.

The coolest thing we've actually been able to show with Ash so far, and I'll dive into the RL. Uh, but we're about to share our first study that we conducted with NYU.

We've actually demonstrated significant changes on all of these axes. I think behavioral change being the coolest one.

On average, people who use Ash in our study had one more friend or close personal connection on average after using Ash for about 10 weeks.

So compared to AI when you think about chachi and equivalent on average people are actually more lonely they become less connected the more they use AI for these purposes it replaces human connection

but if you just optimize for these kinds of objectives we have been able to show that ash can strengthen connections rebuild that social fabric

so basically when we get I'm just going to demonstrate kind of the value model take a peek by running our value model here

so our value model is tracking a whole lot of signals over time and you can see in this case it's comparing this original message.

Uh, and these numbers are not calibrated. I won't go into too much detail.

Um, but we can actually see how the model compares on a whole bunch of different signals like the like and these are likelihoods over time of what we expect to happen next.

So these are things like um whether we expect the user to ignore what Ash says or be confused and it goes all the way towards building alignment.

Um, and overall as an assessment you can see here that the best message actually was the original one. Thank you. I appreciate the compliment.

And you can see this longer message. I might be able to do the kind of thing from time to time, but if we zoom it. Yeah, it's kind of annoying and verbose. Still okay. Similar score, but a little bit lower.

I would encourage you to try Ash. It's available today on the app store. We've just launched. Uh, so we we're just live as of very recently if you want to give it a try.

And if you are looking for a job, we are a small team. Uh, we tend to be quite a senior team. Most folks have been managers in a past role.

Uh, we tend to be extremely missiondriven. If this seems like a valuable use case for AI, please, I'm the guy with the dog.