Fabian Theis 谈虚拟细胞、类器官扰动与闭环实验设计

Just as importantly, his work often focuses on creating the tools that researchers and scientists actually use in their day-to-day work. Most computational biologists have probably used Scanpy and AnnData at some point. And this work has led to the creation of SCverse, which is an amazing collection of open tools that's attracted many many contributors.

Of course, Fabian is one of those academics who has way more honors than you can possibly list, but suffice it to say that he leads a bunch of the AI and computational biology efforts at Helmholtz Munich and related institutions, and participates also in many other industry efforts.

Fabian, it's really great to meet you. Thanks a lot for joining us.

And I I think there's a bunch of reasons for that. And the first one I think it maybe the most important one, which you really hit here, is the data generation. I think we had that concept of a virtual cell for a while, but my my lab we started this perturbation prediction modeling and and using machine learning in 2019 with a paper we called SEGAN at the time, which was like it was a non-linear model, but essentially sort of in the latent space did some linear transformation. And it was actually quite okay, and even nowadays many of these sort of pseudo-linear type of things are still doing a good job, but the reason for that being that I think the data sets and the questions that we asked on top were just sort of still on the simplistic side. That's why also sort of it didn't yet take up. So whenever people talk about virtual cells, maybe also your question, I think what you actually ask is

how does Can you predict how a cell would behave under all kind of perturbations? And that's sort of I think a quite deep question. If you really think about it, it's maybe even a bit of a holy grail in computational biology systems biology. Sort of this this input-output system that is a cell with memory and like with all these specifications and with all the different sort of levels of molecules and so on, not just RNA. I I think being able to map this out, what does even understanding mean? And I've been this sort of a bit of a meta problem, but I I think you understand or at least you model a system very well when you sort of predict its output behavior given different inputs. So I think it's a very good question, and this is why often people nowadays equate a virtual cell with um perturbation prediction. I think it's a bit too simplistic, but I think these perturbation prediction type of models and that's sort of being a maybe what you mean by virtual cell is taking off a lot in the startup landscape. I think it's a bit exploding there because I think a lot about US Europe, but also in in the Chinese ecosystem there's a lot of of these these additional efforts now happening. And I I hope that these are not just sort of now a kindling of of a flame because everyone sort of adds AI to their things, but because pharma is really desperately uh needing to be cell type context-specific in their screens, which at the moment they can't do yet. on that screen.

I felt that was like like next-level stuff, and you know, this step from just getting more PBMCs as a control tool, like let's actually do something at scale to it sort of having here the cell space, but then having here the perturbation space, which typically was very scarce. It was like control versus disease or so. So, this sort of number of points what you had here in terms of sampling on the perturbation space was typically much much more scarce than the number of observations in terms of cells that you had. But now this is exploding. Now you sort of have, I don't know, 100 different cell lines where you do sort of full perturbsic like full CRISPR, or you have like batteries of 1,000 drugs or so added, and suddenly this space gets really rich, right? And then you can sort of start doing all these fancy perturbation prediction type of things. And this is currently happening. It's crazy what scale it's happening. We are ourselves involved in a whole bunch of things where we actually go beyond cell lines and do organoid perturbations, and maybe if you have time we can can talk a bit more about that because I'm quite convinced that we need to go to multi-cellular systems to have actually really interesting readouts, but I think everyone is scaling these these these levels up, and at some point, you know, we're going to go maybe even to human genetics as a perturbation.

So, I think in the future the bigger part at least of the sort of more used models will be those more flexible, very much database ones, where you then a posteriori put the interpretation on top.

Um I think I think there's a truth to it that the particular in the current data state for some things let's say you want to predict the effect of a crisper in a cell and you might not need the most fancy model. Whereas if you ask a bit or if you want to predict a cell type or something like that, you know, for that mostly logistic regression is fine. But if you want to predict something a bit more difficult, for example, in our niche former paper which is a foundation model for spatial omics, we showed we had a new task which we introduced where you say you have a you have the gene expression you have sort of the state in latent space of a single cell, can you predict the gene expression state of its neighboring cells? Because you know, they're always in communication or maybe the other way around. So, like they're always in communication, so because you know, they talk to each other you sort of see a remnant of that sort of cell state in this one. And then actually this is harder to do and if you sort of have these harder tasks you need the stronger more linear things and you particularly need to look for more subtle data. So, your question, can you sort of find out more sort of information which type of things I think very much depends on what you sort of ask at the end of it. Of course, like in the self-supervised learning phase, I think we're definitely not there yet with the best way how we represent how how we learn these things. There's a lot of copying of what happens in NLP type of LLMs, right? And like masked autoencoders are always the typical way to go, but the type of losses, the type of representations, I think people are still debating. Maybe because you know, this is sort of often kind of tabular data, so you can just like tabular foundation models and all sorts of being tried out. There's a lot of JEPA type of approaches now because it's like we have one thing where we use contrastive learning. So it's just people playing around with the losses, but also with the representations. And I think nowadays reviewers, and at least I've been advocating this for a long time, are really much more closer looking that you really properly show scaling laws. This was what has missed from the early papers of the foundation models, and so people haven't really shown that you need this complexity to actually sort of really scale up. But if you go really if you can really demonstrate at scale like across examples and and samples that you have this sort of scaling behavior, then I think you have this buying into these models. And for these, of course, there's not so much at least inductive biases used. And so I do think we're going to going to hit this regime actually quite soon, and we do see this also in a few papers. So for example, in our contrastive learning paper, we call it SE concept, we do show already quite quite quite strong scaling behavior. In the paper, we had something for going because we compared to Gene Former going to maybe 30 million cells, but on on Hugging Face now, we have a 350 million cell model with I think going up to maybe 100 million parameters, so really going to the big scale for us. And we do see nice scaling behavior already starting to emerge.

So, one of the first observations there was in a thing as basic as cell type prediction, okay? So, nothing super fancy, okay? Basically, you you give me a new transcriptome, you tell me from which cell type it is, right? Maybe even from which organ or something like that. And it turns out that even nowadays, I think some close to state-of-the-art models are really just logistic regression, right? So, for such a simple categorization, you know, this is actually not not so bad. There's one thing called cell type is for example, which is doing a good job and I think we had a paper maybe 5 or 6 years ago where we had the title very plain, "Deep learning is not needed for cell type prediction." This is true if it's within an organ. If you sort of go cross organ, suddenly gets a bit more subtle because, you know, sort of cell types reoccur in different organs and immune cells are a little subtle change depending which tissue they are. Suddenly, you know, there you need to do a bit more and we like for us, we published on a paper called SC tab, like a tabular foundation model, predictive model type of thing. We actually showed that this nonlinearity was really adding beyond the the baseline. So, that was sort of for that class.

For the perturbation models, which then you were asking Brennan similarly like early on, many analyses particular perturbation could run with classical linear models in Suda pack. We had then our our SCGen that was like something around 2019 where we started introducing linear shifts but in the latent space, so sort of non-linear encoding and we always compared with subtle sort of simpler linear shifts. But then, the field getting complex and you know, sort of data sets popping up, people started asking more and more perturbation type of questions. There have been a whole bunch of these early on competition for like a regular one run for example by MIT. We've been running a regular NeurIPS competition on uh prediction problems in single cell data, sometimes for integration data, also for perturbation prediction. Um that was very well attended. The cherry in the on on on on the cake was most definitely that that that arc competition which called the thing not just perturbation prediction challenge, but virtual cell challenge. To sound a bit cooler, right? We picked up very well. And but of with metrics that are robust were maybe but maybe not not that easy. So far actually our solution in in this this this consortium where a lot of labs contributed called open problems was actually that people need to submit their own metrics and additions so in order to really maybe subtle things that you might not have thought before. Dark one is also fixed a bunch of metrics that weren't even so so single cell specific and they could be just just exploited. So actually I think the winning model that was then in the end actually sort of crowned as the actual winning model, but essentially is a flow matching best thing. So based on on this idea that you know people fought with this flow with this cell flow type of idea. I think shows you that some non-linear things are in this case needed. So that the actual real winners or if you sort of remove the the exploitation was actually something that is this is taking some some non-linearities and I think nowadays we do see that just linear or additive type of models are not cutting the the the cake for more complex things, but for I think even nowadays if you go in a simple if you cell lines and you have sort of gene knockouts then particularly when the effects are a bit subtle all these sort of more complex models are sometimes a bit overfitting and that's why a linear baseline is always very good. So I always tell my students, guys I know we we have guys I mean guys and girls. We always need to do I know you want to do the most fancy deep learning thing, but please please you know have the baseline and like say for drugs you know a baseline as simple as just predicting getting better at the mean. You know, like that's the like the hottest base. That's not even a linear model like better than the mean is sometimes not as hard if there are so much noise in the data and this stuff is super noisy.

And increasingly, that's maybe an interesting link to So, this is maybe a bit similar to NIH type of models like in US, but sort of distributed all over the country, not just sort of in one place. And maybe the one interesting um um um link is that increasingly the government and this sort of realization also in in in Germany is is sort of dawning that there needs to be much stronger transfer component, not just by going into translation to universities, but also in into sort of value creation outside of the academic realm, which is obviously going so more more into startups and so on, which I think the Boston environment has always been one of the key examples, but now of course more so now also the the the Bay Area when, you know, recently I was was visiting uh I believe maybe Genentech or colleagues, and we went to a restaurant, and then one of the waiters was asking us, "Hey, are you guys talking about AI and bio? You know, I have been also interested." So, like everyone sort of thinking about this startup ideas and so on. And I I kind of felt that's a it's a good environment, and this is something I think our PhDs are now more and more doing, and and we we are sort of learning there, and I think also in terms of sort of value creation, Munich in particular is actually at least in continental or maybe the prime place now for VC capital, but also sort of deep tech and particularly the bio and and defense area and sort of really exploding. So, this stuff starts to work.

So I do think the visions are starting to happen, but the scale is is not as fast. So I think what you need for a tech startup what I noticed is is always the biggest thing is you need to have the best team for that. And you know, we have professors and and PIs that really think about this. So for example, for for the department that I coordinated, I said, "Guys, every every PI every group leader, I would expect you to at least to have one one one spin-off in in in the next 3 years." And see how that works, right? But you sort of see this sort of get people a a bit a bit on that track. But I think this is happening very much from the bottom up nowadays anyway where the students are actually pushing for that. They do realize that there's a whole bunch of opportunities at the moment, particularly with with this sort of computational angle getting Yeah, I would say pushed even more in the forefront that we can now start actually guiding experiments. We haven't really spoken so much about, right? But once you have these perturbation world models, you can actually go back and drive the experiments, right? So I do think computational startups can be very good also data generation. And I think this is what will be also turned not just into nice papers, which is sort of mostly our business, but also I think it is next generation of startups. And you do see a few of these popping up where you actually get the lab in the loop really implemented.

Um Maybe you have some other counter examples. And there's a few companies that are sort of still stick with the scaling up of of the AI business, but it's not as clear uh how you would sort of directly turn this into a viable sort of really scalable business model. And there's even room for many of these. Of course, then there's also this angst that at some point like Astra 2.0 might be doing parts of this already anyway, yeah.

But but I I think this idea is it's somewhat even nowadays a bit trivial right? But I I thought it was good to bake this in so hard so that students really ideally go to two lab meetings. One lab meeting where they need to really learn and understand some deeper biological problem, understand the language maybe of a medical setup or something like that. Then they go to a computation typically all computation data science type of students, right? But then they go to the computation one they sort of get pushed the method aspect of it and thereby sucking up a whole bunch of of problems that are sort of not exactly precisely as formulated in in that bio lab. They sort of can't directly put that in a function, but then because they are sort of methody, right? And they're quantitative, they can actually put this into a metric or whatever that then turns into a model and then paper and I I think this is the way how uh I mean we've been doing this now for maybe 8 years or so this this type of school. I really a whole bunch of papers came out where I I I learned about systems that I had no idea about this before, right? You know, people approaching me and could we do something like this and then you know, I learned something about the lung or the brain that you know, I had no idea about, but then uh I said, "Oh hey, cool data. I think our model types could fit. Let's build a project around it." And then those students became the experts in my lab for that sort of detailed biological thing together with a partner. And then maybe we drive this forward and this actually nowadays extends also across the data types. Earlier on a lot of things have been around just RNA, but you know, since then you know, we had a very nice paper in science last year on single cell proteomics. We've been extending things to spatial observations. You know, there's sort of all these additionalities and things that you can can can measure and I think then interesting questions come up in that particular domain where you know, these essays work very well.

I think Nvidia just a few months ago were hired in 1 month two of my people going very much into sort of the computation aspect and also a bit of a bio part that Nvidia's now building up and there's similar other of the big companies. So, I think it it gives them an an an interesting profile, but I think when you're out there when you get interviewed, you know, it's it's a good idea to say, you know, I I didn't only do like this fancy model that improved maybe 2% in a NeurIPS paper type of thing, but you also sort of had the real sort of more complex, deeper application that they're like, you know, I had to help find as a use case.

And even though some big pharma have been putting this out as sort of a general general big theme, there's not many examples that where really people do multiple [clears throat] loops of this integration of you know, experiment, a computation model, then going back, whether this is now two person one, right? But like this integration is hard to set up. And it's hard to set up once because of modeling because actually sort of once you have a model, actually deciding what's an optimal next question is not trivial. And we actually brought out I think it should be on on bio archive today or tomorrow and new thing we which we call lab compass, which is an inverse generative model that actually allows you once you have a forward world model or whatever to tell you what you need to do in order to reach a certain state. So, it's super exciting for us, right? And I think a big part of the work was not just the model, but actually the work with experimental people, in this case a good friend and colleague from University of Cambridge who was doing hematopoiesis. In addition, had a sort of a rapid cycle system where we actually set up three four lab in the loops where we iteratively improved on a model. And this is I think a really hard thing. And this sort of goes into your second question, Brandon, right? Like how do you bring a culture where once your computational person or model or whatever predicts something that you actually get to convince experimental people to put their life time in because these experiments often take some time right to actually buy into it and actually measure some of these follow ups. And sometimes you know this works a bit better if it's sort of molecules because then you know also the stakes are not as high but you know if you like say now you need to do this new cell and you need to add these very strange things. Do you really want to do this? Like shouldn't we maybe not take a bit of our gut feeling and then you know these sort of disintermediate solutions come. So I think really setting this up is far from trivial. That needs culture and it might be the same person doing it. A lot of people then start throwing around robotics and so on as well but I I think this is a tad too early because the the cycle times in terms of our essays are still a bit on the long side but I do think this will come and you know obviously the the ultimate vision being that we want to RL everything. We want to do RL also in in in in this loop. It's a bit out there but you know I think we're we're slowly getting there. At the moment it's more maybe active learning.