Matt Tur: Hi, I'm Matt Tur from Firstark. Welcome to the Matt podcast. Today my guest is Sebastian Raschka, an AI researcher and one of the best educators in the field. Well known for his in-depth technical blog posts and his book entitled Build a Large Language Model from Scratch. In this episode, we go deep on the state of LLMs in 2026 architectures, post-training, scaling, benchmarks, tool use, and what it all means for the next wave of AI. Please enjoy this in-depth but very approachable conversation with Sebastian.

Matt Tur: Hey Sebastian, welcome.

Sebastian Raschka: Thanks for inviting me on your podcast today. I'm excited to talk about anything AI.

Matt Tur: Wonderful. So we are going to go in the state of LLM in 2026 in depth including very much post training and reinforcement learning but I wanted to start the conversation with the Transformer architecture itself obviously the backbone of the entire generative AI revolution but also over 8 years old at this point and for all the tremendous progress in LLM based systems over the last year it also seems that there have been some interesting developments in terms of alternative architectures. So has anything caught your eye and do you think that's a world where the days of the Transformer architecture could finally be numbered?

Sebastian Raschka: That is actually a very interesting question to start with. I mean it's starting at the very beginning with the Transformer architecture. You said 8 years. I think it's almost 8 9 years because 2026 it came out in 2017. Quite a long time. And I think the question you raised is if it's like the final architecture. I think people probably ask that every year. Is this the thing we should be betting on going forward? Let's say in 2026. I would say right now yes, because it's still the state-of-the-art. So there is nothing really better in terms of state-of-the-art performance getting better quality results.

Sebastian Raschka: What we have seen so far though is alternatives that make it cheaper. So there are tricks to make the architecture cheaper itself like linear attention variants that are like a building block in the Transformer architecture. A big one was a mixture of experts which is essentially making the model bigger without necessarily making it more expensive to use and inference like keeping like that reasonable while expanding the size. you see all kinds of you I would say like levers tips and tricks hacks around that architecture but it's still kind of like the same architecture in the core and you can actually in fact take a GPT-1 or GPT-2 model and with a few I mean few lines of code almost you can transform it into the latest DeepSeek version 3.2 to architecture. It's not like a big leap. It's still the same scaffold.

Sebastian Raschka: At the same time, you have other alternatives popping up like diffusion models, text diffusion particular or Mamba models, state space models and so forth. They all try to address a problem that the Transformer has namely that it is expensive and big and expensive to run and train. But then of course there's no free lunch. These have other trade-offs. They are like cheaper to run in certain instances. If you take a look at diffusion models or text diffusion models, but then you don't get the same let's say quality out of it and if you want to get the same quality out of it in in this particular case, you have to crank up the den noising steps and then you end up with something very expensive. So right now I think we at that point where there is no free lunch we are still you know like we are trying to figure out what is the best next architecture right now. There's nothing on the horizon that would replace that. So my short answer is I would say right now if I were to build a state-of-the-art model that would be still a Transformer-based model.

Matt Tur: Great. What do you make of world models?

Sebastian Raschka: World models are also an interesting hot topic. So there is the whole world model aspect for more like images and physics and that stuff. So world models are basically models that have like an internal model of the world. So they kind of simulate something internally what you have externally like for example if you have like a chess playing model it has like an internal chess simulator built inside so it can kind of make better predictions or predict the next states I think that's particularly interesting for robotics.

Sebastian Raschka: But coming also back to LLMs there was also a paper by Meta it looked very promising to me as refinement or a next step for codebased LLMs. LLM for coding are still next token predictors. But in addition to that what they did is they also tried to predict the internal states of the variables like that was like a during training an objective to if you have a Python code to say okay this at this iteration when I if someone would step through the code this variable would have that and that value and so this is in a sense giving the model more context more information about the training data and it forces the model also to kind of like in quotation marks understand training data better so it's like instead of just you know brute force just what is the most likely next token, it has kind of like an understanding of what it is right now. I think that's also how humans work. When I for example as a human read through code, I'm also trying to visualize or verbalize or write down what are the states of the variables in my for loop. For example, at this first iteration, second iteration back then when I learned coding, I actually had like a paper notebook and was like writing down things with a pencil basically like these types of iteration. It's kind of like also this approach but for LLMs essentially and I do think that is something that is maybe more expensive to do but it is also something that might push the state-of-the-art a little bit.

Matt Tur: And what about small recursive models? What does recursive mean in this context?

Sebastian Raschka: So that was also a big topic in 2025. There was the hierarchical reasoning model and from that we had also another paper tiny reasoning models. And so they are interesting because they were getting very good performance for their very small size on the ARC benchmark. So ARC is like a benchmark almost like an IQ test like a logic puzzle where there are like different symbols and you have to you see a let's say an array of different symbols and you have to predict like let's say what's the missing thing here in the bottom corner and it's kind of like going a bit beyond text and beyond things usually on on the internet.

Sebastian Raschka: So in that sense I think the motivation between behind this ARC benchmark was to have something that really tests the capabilities of that model on something new that hasn't been shown during training like a new task and how well the model can take some examples from that benchmark and generalize to new tricky problems. Also different iterations of this ARC benchmark to make it harder and harder and harder. hierarchical reasoning model it became like popular because it performed relatively well on that benchmark compared to very expensive models like Gemini Chubbyd and so forth and it is a Transformer architecture and then there's the tiny reasoning model that is I think even simpler than the hierarchical reasoning model.

Sebastian Raschka: And so the idea is that you recurse so you have like a latent let's say storage vector or something like that where you refine the answer over multiple iterations instead of just doing a oneshot you let's say write an intermediate answer and the model looks at that looks is this correct or not and takes it another round and another round and and refineses that answer. It is not cheap either but the model itself is much cheaper. it made a lot of waves also because like oh we don't need these big Chubbyd Gemini type of models to solve complex problems and I think this is to some extent true but I think also that kind of underestimates the appeal of Chubbyd Gemini Claude and so forth and the appeal there is it's like one model that can do it all it's very general purpose you don't have to even really teach people that much how to use that model I can ask anyone like hey you know here's is Judgly be the interface and people who have never used it before they will be able to figure it out just typing some prompts and I can drag and drop an image there I can ask it a code problem you know it's doing a lot of things very well at the same time.

Sebastian Raschka: It's also a downside because it's this gigantic model which is very expensive so if you have a simple task that is very expensive to run such a big model and at such a big scale and so there is then this appeal to develop these special purpose models so tiny reasoning models hierarchical reasoning models they are very specific to a particular task task. For example, in the paper they had path finding like finding the path through a maze or something like that like a toy problem and then the ARC benchmark but each one was a different model. It was not like one model that could do all three things. In that sense, it is I think really hard to compare to something like Gemini or JJP.

Sebastian Raschka: At the same time, I do think this is a very interesting and promising direction because even though JPT can do everything, it's not always the cheapest thing. If you have a business problem, you are maybe manufacturing something, maybe you can start with a generalist model, but then once you know exactly what the task is and you want to hone in on it, maybe it makes sense to replace that expensive thing by something like that that is cheaper, you know, like a module that you can plug in and you can even have an LLM like CHPD or Gemini use those as tools. And so I think that's it's a great development, but I don't see it quite fair comparison to, you know, state-of-the-art LLMs. Basically,

Matt Tur: I want to come back to something you mentioned a few minutes ago, diffusion models, especially for text. Last year, I believe Google DeepMind announced one called Gemini Diffusion. So, what are those and how different are they from Transformers?

Sebastian Raschka: There's of course the big field of diffusion models coming from image models like not too long ago maybe two three years ago there was like the big hype around Stable Diffusion which was based on a research paper where they had like a model that replaced going back generative adversarial networks which were an idea for generating images.

Sebastian Raschka: And so the diffusion models were essentially instead of having generator and discriminator setup like two networks competing against each other, it was like a pipeline that was denoising starting with random noise denoising an image and coming up basically with so with realistic looking images and you could also have a text prompt and basically guide in terms of it's not like a random image you can basically guide what you want to generate. This is basically the modern generative AI image AI that we see out there. People were wondering okay can we do the same thing for text. So can we use this pipeline like this the denoising pipeline to generate text instead of using Transformers?

Sebastian Raschka: Or I mean I'm saying instead of Transformers diffusion models can be Transformers are often also Transformers because Transformers is the architecture. So LLM nowadays it's specifically auto regressive Transformers which means these are LLMs that are generating one token at a time. So like where the next token always depends on the previous tokens. And so with diffusion models you don't have that you have you generate everything at once in parallel but it might be very messy and then you have multiple iterations. You take that whole thing and denoise it basically refine it. What's nice about it is well it is fast because it's like a one iteration generates something and then you have a few steps that refine that which might be cheaper than using an LLM to generate a long response because then you have a lot of sequential steps and so let's say 16 denoising steps is fewer steps than having I don't know 2,000 tokens 2,000 steps that you generate something with.

Sebastian Raschka: The downside is well you have everything in parallel and there are nowadays a lot of tasks that require sequential processing. For example, if you think about reasoning models or tool use when you have a reasoning model and you ask or in general you ask a model to answer a question and the model maybe does a web search as part of its answer and so you have to kind of interrupt the generation. I think the diffusion models they have these downsides but like what you mentioned is Gemini I remember seeing the Gemini Diffusion website where they are saying something like coming soon and they compared their diffusion model to their latest I think they call it Flash the cheapest model and so as an alternative to the Flash model being even like I would say faster at the same performance level but they're not putting it out there as their state-of-the-art model. It's more like a cheaper model maybe for everyday use maybe for like the free tier or something like that. So it is a interesting direction to go into these diffusion models as alternative to the auto regressive Transformers but it is not I would say the replacement at the state-of-the-art.

Sebastian Raschka: I think one company will launch a big diffusion model this year. So there are diffusion models out there that you can use already but I I haven't seen anything at you know like Gemini Chip Anthropic Claude scale I I think but this year maybe we will see something like that.

Matt Tur: Great. Super interesting. In the the world of LLMs, you mentioned and that that triggers a question which is what I think a lot of people are wondering which is like are we seeing real architecture breakthroughs within the LLM world or are we effectively at this point polishing what we already have within the LLM world? What are you seeing that's moving the needle in terms of architecture improvement or optimization?

Sebastian Raschka: Improvement is not so much coming from the architecture anymore. It is basically the post-training. But like coming back to the architecture, I think it's still an interesting question because there are so many different architectures and almost no one uses the same one. Like they're all very similar but they are not identical. I think a lot of it is coincidental where there are some tweaks and if you look at the laws that in some cases on some training data in some training pipelines maybe like moving the normalization the RMS norm before after makes a small difference. I mean there are theoretical justifications for but also for example OMO 3 which is very transparent they moved the RMS norm placement.

Sebastian Raschka: So then Gemini had a post and prenor they had both on both ends and so there is some justification where okay ablation studies show this stabilizes the training but well assuming a stable training it's not going to I think make your model magically perform better I mean this is just like people tune their cars a little bit by you know putting in different air filters and something like that. So I think it's on that level where you can make small tweaks but it's not really changing the engine itself.

Sebastian Raschka: The one thing though what we've seen is a lot of large architectures now using that I think that's a new 2025 thing. Of course, is not invented in 2025. That was I think going even back to the Google Pathways paper in I don't know 2022 three something like that. And then Mixrol had a big I think it was 2024. Then I think it was pretty quiet. I mean around there was only I think the Chad GBD model which was rumored and to be ane. But now this year really almost everyone has an OE out there like every every open developer.

Sebastian Raschka: I would say Deepseek kind of like restarted that trend in 2024 in December with Deepseek version 3. They had an even model before but I think this is like the one that everyone looked at because that made such a big splash that people like oh what they are doing is maybe sufficient it's the right thing let's not you know try something crazy let's like iterate on that so there were a lot of companies adopting straight up the DeepSec architecture so there was like I think Kimmy had DeepSe architecture scaled it up I think to from 670 billion to 1 trillion parameters and then even the European Mistral AI company used the DeepSeek version 3 architecture for their new Mistral 3 model. It is something that is working well.

Sebastian Raschka: But then DeepSc itself they iterated on that too. So they have DeepSe version 3.2 where they changed attention mechanism. They had a multihat latent attention which is already a nice tweak to they added sparse attention where sparse attention again it's not new but they had their own flavor of it to make it cheaper. The idea I think is to get better modeling performance through the training pipeline while tweaking the architecture so that benefits can be of course main like absorbed by the architecture but then also at the same time to bring down the cost of running the architecture because we've seen with GPD-4.5 which was also just 2025 GPD-4.5 was rumored to be a bigger model a bigger version of GPD4 but it was not very popular because it was too big too expensive and so they kind of abandoned ed and went a different direction with GPD5.

Sebastian Raschka: And so I do think well I wouldn't expect bigger architectures. I would expect more efficient architectures tweaks getting the same modeling performance for less compute because then you can have more tokens for the same cost and the tokens they give you better performance like inference scaling and so forth.

Matt Tur: But you see room for progress there like you're not in the pre-training is dead camp.

Sebastian Raschka: I would say pre-training is not dead but pre-training is boring. So, it's not where the lowhanging fruit is anymore. I think the low hanging fruit used to be in pre-training, but now you need really good pre-training still, but it is, I think, harder. I mean, I wouldn't say harder, but you can get better bang for the buck elsewhere almost. I would say pre-training, I don't think it's dead. It's just not, let's say, the most popular thing to spend money on right now. I think it would make more sense to put a lot of that budget into post-training right now.

Matt Tur: Okay, let's go into post-training. So in your blog post, you mentioned that 2025 was the year of RLVR and GRPO. So you had like a nice timeline where you say 2022 was RHF which gave us JGPT plus PO. 23 was Laura SFT. The 2024 is a year of mid-training and 2025 the year of RLVR and GRPO. So would love it if you could walk us through those techniques. So fair to say so both of those belong to the world of post-training. Let's pick RLVR and let's start with the definition. What what does RLVR mean versus regular RL?

Sebastian Raschka: I would say RLHF is the biggest leap in RLM we have seen in a long time because that was taking GPT from GPT to ChatGPT. you know like the RLHF that reinforcement learning with human feedback and in that sense it's almost like LRVR which is reinforcement learning with verifiable rewards took that other leap basically from just simple chat model to a reasoning model both RLHF and RLVR have the RL in it so both are based on reinforcement learning but I mean this reinforcement learning is a bit different from the reinforcement learning that plays go it's almost like is a special thing in simpler thing in in the context of LLMs.

Sebastian Raschka: But the idea is that instead of doing next token prediction, just predicting what's the next token, it's more like looking at the full answer and then based on that answer, you give a reward like in RHF, it's you have multiple answers and then you say which do you prefer or in the case of LR, you look at the full answer and then let's say it's a math problem, you say the math problem is correct, the final answer of the math problem is correct or incorrect. That's like the main difference between next token prediction and pre-training. And then the RL here.

Sebastian Raschka: So RLVR was kind of like popularized by DeepSync R1 which was based on DeepSync version 3 and that came out R1 came out January 2025 and with that they also introduced the GRPO algorithm you mentioned but it well they go well together because they make it more efficient the whole thing but it doesn't have to be. So you could technically do LRVR with a PO algorithm that was back used back in RHF. Now why I think it's such a powerful combination is well it just makes things more efficient with RHF you had to have people ranking answers. So because the the goal was essentially to train a model that prefers one style over the other. So, for example, for safety, like reducing swear words. If there are two answers, use the one with fewer swear words or if you have an explanation, maybe use the explanation that is simpler to read and these types of things. But you always have to have someone who compares these answers and says, "Okay, this answer is better than the other answer."

Sebastian Raschka: What you do then though is during the RLHF, you train a reward model, another LLM that provides this information for you. So, at that point, so you can replace humans looking at these answers. So you have this other model that does it automatically as part of your loop. It's more expensive. Now you have two models essentially. And then there's also a value model. So the value model is internally kind of like a reward model, but it gets also updated to make some predictions as part of the reinforcement learning signal. And so you have basically three models in memory. And if you have CHPT style training like large models like or even like DeepSe three or 600 billion parameters, you have three times 600 billion parameters. and you have to keep them all in memory. It's very expensive.

Sebastian Raschka: And so in LRVR with GRPO, you replace two of these models. So you have three models for RHF with PO, you replace that reward model by verifiable rewards. So instead of having someone say, oh, I prefer this this answer over the other or using an LM or that, you have now tasks that can be automatically verified. So for example, math. You can have a math parser. It could be like something like Wolfram Alpha. You have the correct solution and you have the LLM solution and you just parse out that part that you can compare algorithmically and then based on the correctness you can give a reward for the reinforcement learning. So you eliminate already one big LLM that you have to have to train and have to have in the loop and the other one you also eliminate. So there's the value model that assigns a value to each of the responses during the training and in gpo. So that's the gRPO part. You just compare them relative to each other. That's where the R and GRPO stand comes from like the group relative policy optimization.

Sebastian Raschka: And so yeah and and this makes it much more feasible to train it. It's just cheaper. And they show that it is actually really powerful. So you can take a base model even skipping supervised finetuning and RLHF and just do this LRVR and you get a really good reasoning model out of it. the Deepseek R1 model, you can still do supervised fining in RLHF and it's recommended to do it. The reasoning behavior comes from that RLVR.

Sebastian Raschka: There are of course papers showing that the base model already has reasoning capabilities and I think this is actually partly true, but it is hard to say for sure because yeah, you don't really know what's in the pre-training data anymore. So there's also a lot of reasoning data. So reasoning data is essentially just data which has this chain of thought format which means that the model writes intermediate steps like it explains its own answer. A lot of the pre-training data has already the style of data in it and then it's hard to say does the reasoning behavior come from the pre-training corpus or is it from the RLBR.

Sebastian Raschka: And in my experience I think a little bit of both. So for example I took the Qwen 3 model as part of my book the Reasoning from Scratchbook and I trained it just for 50 steps with RLVR and it goes from 15% so 5% accuracy on Math 500 to 50% on MA Math 500. So it it takes us three times bold leap in terms of accuracy by only doing 50 reinforcement learning steps. And I think it's not really learning that much in these 50 steps in terms of how to do math better. I mean yeah it does but it's not learning new knowledge about math. The knowledge is already there in the pre-training and this just unlocks it. It's just like a step that maybe shows the model how to use its own knowledge basically. So so that's how I think about it.

Matt Tur: Fascinating. Just to unpack some of this, you can understand the reasoning steps that led to the explanation. You mentioned in some of your writing the label process reward models, PRMS, and the fact that this is not successful yet. Can you unpack that that part?

Sebastian Raschka: So there's an outcome reward and a process reward. And the outcome reward is mainly like is the final answer correct or not? But then there's the whole explanation of the reason model whether it leads to the correct answer. And so there's also research like hey why would why should we throw out everything the model generates and only look at the final answer. Can we get something useful out of this intermediate explanation?

Sebastian Raschka: And the intermediate explanation is useful for several reasons. I mean one is it has been shown like that this helps the model to generate the correct answer whether it the explanation is correct or not but is a different aspect but just the fact that it generates these intermediate steps is correlated with a more accurate answer then the hypothesis is if we can improve that explanation maybe it gives even a better answer like even if maybe it even drives the accuracy higher if you want to learn something it's not enough to just see the final answer you want to the steps that lead to the right final answer. Process reward models, they are also focused on training the model to reward the models based on that explanation.

Sebastian Raschka: And so my statement that it is not so let's say promising or useful was mainly based on the R1 paper where they had a final paragraph at the bottom. I mean this is already a year old but they had a paragraph at the bottom that that I think the headline was something unsuccessful attempts. They tried it and they found it wasn't worthwhile because of reward hacking. the model was because usually you need another model to grade the responses and they can be susceptible to reward hacking and it's hard to train that model and it's not reliable and then that whole thing it's not really worth it according to their experiments.

Sebastian Raschka: There are a lot of people who try to make it work and I think it is promising and we will see it working at some point I think. So it's just like right now it's still tricky to make it work but I I'm I'm quite sure we'll see it as part of the standard repertoire at some point and there was for example end of the year last year was the Deepseek Math version 2 paper where they had actually a nice study they had something like that where they had like an a second model that was checking the answers and explanations of that first model and they it's almost like a turtles all the way down they had yet another model so they had three models they had one model generating the answer, one model to like grade the answer and intermediate steps and they had one model to grade the greater basically to say oh is the grader actually doing a good job. So there were like three models in a row. It sounds a bit excessive but based on the performance of that whole setup it performed really well.

Sebastian Raschka: So they were cranking up also the self-refinement steps and iterations and they got gold level performance on some of the math benchmarks. One could say okay maybe well cheating the data was public or whatever. I don't know that's a different question but the fact that this performed better than the model without it tells me whether let's say it's really gold level performance is a different question but it is doing better than just the plain model. So it is actually adding usefulness to the whole process and I think we will see more of it. it's just expensive cuz now you have to have more models, more training, more stuff. But that's what I meant also earlier with that's where you make the bigger gains rather than scaling the model size. I think that's one of those things where you will see more progress coming from.

Matt Tur: And speaking of math, I think that's one of the key questions going forward for RL whether you can expand this beyond math and coding to other domains. What's your take on that?

Sebastian Raschka: I think what's so attractive about LRVR is that you don't have to have humans checking the solutions. you have a verifier that deterministically checks for math is the answer correct like giving two fractions are the fractions the same you know that what two numbers with decimal points are they the same if I round them up and so it's like very easy to check programmatically algorithmically.

Sebastian Raschka: And the same for code so you have code and you like code problems and you can so in that case they you can compile the code if it compiles or you have unit tests it it checks it works it's very nice to check it's no there's no subjective aspect it's very objective If you can say okay it compiles it doesn't compile it's very clearcut the question you had is does that what happens now in general to other fields is it like specific to math or code and I think we will see also expansions of that to other fields I I'm personally not an expert in other fields so I don't know what that would look like for medicine.

Sebastian Raschka: I mean I have like a computational biology background I know a little bit about the drug development pipeline and so forth but it is I think it's not quite as clear of what the reward looks like. But you can also be more creative. So you don't it doesn't have to be strictly verifiable through an algorithm. It can be verifiable maybe through an LLM, you know, it could be something like that. So for example, I can see I don't know for research maybe training a model to give correct citations and you can maybe check the citations. you could have another model that goes to the URL and say, "Oh, this is in indeed the correct paper giving the correct title of the paper or something like that. It's a correct URL and stuff like that." So, I think there are lots of these things where we can expand RLVR to and train on those things. So, I think we will see a lot of that.

Matt Tur: The thought that crosses my mind as you describe this is that I've heard people say that RL is very difficult to scale is very finicky and hearing what you describe about different techniques put together. I'm starting to get a sense for for why is that why it's complicated? Is that because it's a basically a bunch of different things and models talking to one another that it is hard to scale or is there another reason?

Sebastian Raschka: Well, I just implemented before we recording this GRPO LVR from scratch in a Jupyter Notebook. I wrote up a chapter 39 pages. So it is not super complicated. I would say it's like you can fit it into Jupyter Notebook it works and it trains fine. What I'm trying to say is if you can figure out pre-training the scale at pre-training, you can figure out this because if you also look at the numbers of how much it cost just in GPU hours, Deepseek version 3, they had like a $5 million price tag on that given the I think $2 per GPU they assumed. Whether that's a correct assumption or not is a different question. But if you compare it relative to the cost of R1, I think R1 was about $300,000 when they trained it had they had a number in the nature version of the paper. So it's basically more than 10 times cheaper than pre-training.

Sebastian Raschka: So and it's the same infrastructure where you have to make sure I think the complexity comes from making sure that GPUs don't crash. If they crash that you can resume and so forth and also during pre-training you might have bad losses where you want to you know re-roll the old checkpoint and the same things apply. you have multiple models but yeah you are right there is a bit more I would say well trial and error in in RLVR where just due to the nature of the updates and so forth you can't I mean that's what I observed when I was training my models you often get I mean not that often but every so and so many hundreds or thousands of steps the model gets bad you know like so the model works totally fine and you train longer and long and suddenly the model is really bad and so you just go back to the previous checkpoint point but it's not new in terms of a new thing that's happening all the time in pre-training as well.

Sebastian Raschka: But I think vanilla grpo the original algorithm it is pretty flaky like where it is you have to babysit it over the course of the year many people had these tips and tricks where some people were saying remove the KL divergence term like if you just drop it for math it performs better remove the standard deviation the normalization term or if all the rewards look the same you can skip them to make it faster you know like there's like there's a lot of tips and tricks like these tricks of the trade that make it more stable and I think like if you apply all of them together it is actually a pretty stable okayish algorithm.

Sebastian Raschka: Just like last week Nvidia also had a paper on GDPO I think GDPO so they were focused on also algorithmic improvements with respect to multiple rewards so if you have more than one reward could be something as simple as you have the accuracy reward but you usually have also a format reward because you want the model to put in the final answer in you don't have to but you can put that into these think tags so there's like a token a think token it's more honestly for stylistic purposes I think the advantage is some people develop models that are hybrids which are capable of a normal mode and a thinking a reasoning mode so thinking stands for reasoning the appeal here is you don't always want to use reasoning modes because it's expensive it uses a lot of tokens and sometimes you have a simple answer and you don't want to spend 2,000 tokens on the simple answer.

Sebastian Raschka: And so you can with these think tokens steer it a bit. For example, in Qwen 3 you can add empty think tokens. So you have a opening token and a closing token. If you add that the think whatever in between is empty and then the model will not generate any reasoning chain of thought. And long story short so during training you can teach the model to adhere to these different formats. So then you suddenly have a second reward. So one reward is correctness. Is the answer correct? The second reward is does the model output something that fits my formatting here? And then you have two rewards and how you combine them. Usually originally you just add them up together, but then there were some yeah downsides in the GRPO in stability. And so GDBO had some algorithmic improvements to yeah improve the stability. And so there are lots of these little tricks over the year to make RVR more stable. But it is a new a newer paradigm. So it it just takes a few iterations to find the canonical one.

Sebastian Raschka: It's similar to you know optimizers with Adam. So Adam is I mean right now there's AdamW there was SGD and all the other RMSProp and how they were called and they kind of of all converge to LMW by adding more and more tricks and I think that's the same right now with LVR with GRPO where adding more tricks to get to something that kind of like is pretty stable across a lot of different scenarios.

Matt Tur: It's fascinating to hear you talk about tips and tricks and and different techniques that triggers the thought that you had a like a nice way of putting it in your blog post and taking a a step back for for a second from the weeds. But you talked about a a meta lesson for all the things in 2025 and where progress actually comes from. Do you want to get into that? I think that'd be interesting.

Sebastian Raschka: And so meta lesson would be essentially here that well the whole I think we are talking right now for half an hour about different things. So I think the theme would be well there's no one thing that fixes it all. It's a lot of little tricks and tips tips and tricks all over the place and if you add them up that will give you the progress but I yeah I think there's no magic lever no magic I guess bullet that gives you everything. It is kind of of tweaking things here and there and making things more robust. I think the tweak was the Transformer architecture back then and now it's essentially let's you know make it even better I guess refining it and a little bit of post-training here a little bit maybe improving the quality and pre-training maybe some architecture tweaks algorithmic tweaks it's all a little bit of everything basically.

Sebastian Raschka: Yeah it's also that I would say you don't have to know all these things because like in in practice it's like a big big team at a big company and everyone has a specialty like everyone is either like on post-training team or the pre-training team. It's not that one person has to know everything and all the tricks because that would be really impossible. And so I think it's also just due to the nature of work because it's so much work. It's a lot of work to train these big models that you kind of like separate these roles and then everyone is working on can work on everything at the same time which is also nice and then you bring back together all these improvements into the model.

Matt Tur: Yeah. any confident in the industry's ability to keep coming up with tricks and tips going forward?

Sebastian Raschka: That is a good question. I mean, if I look at DeepSeek for example, because I mean, I'm I'm always picking here Deepseek in this podcast because I think they have a really nice trajectory of models. I wish I could also talk more about Gemini and Chad Chubidi, but they don't really release the yeah like techniques, so hard to talk about it. So picking on Deepseek here I mean if you look at version three and then R1 and then they had version 3.2 model with the sparse attention mechanism and then also this Math version 2 with a self-refinement and everything. So they do have right now still a track record of improving things and they are rumored to release a new model in February the Deepseek version 4 but I think all so far I think we are still on that trajectory where we haven't run out of ideas.

Sebastian Raschka: So I think the only things we're running out of is really benchmarks. So the improvement on benchmarks it's kind of like harder to measure and I think maybe well maybe it's not the oneshot problem anymore where it's not really answering knowledge question. It's not really solving math problems in in one iteration of the benchmark. It is maybe more like the agentic cycle like where you have like a more like a objective that is not let's say answer the question but more like design something blah blah blah and then it goes off and how long it can or how long it needs or how long it can run until the problem is solved and I think it's maybe more towards that how we measure progress rather than whether we get 90 or 95% or 97% on a benchmark.

Matt Tur: Yeah. you had this nice expression benchmaxing in your in some of your post. What do you mean by that?

Sebastian Raschka: So benchmarks maxing is so I'm often often reading things on X because it's like where a lot of the AI community is. I think benchmarking is one of the ones that came up in 2025 like new generation term and so loosely how that what it means is essentially that well it's almost like exploiting the benchmarks like people train models do well on the benchmarks but it doesn't really translate to real world performance like a popular example was the Llama 4 model I mean based on rumors I heard they had a separate model just for the benchmarks the leaderboards but let's say even that aside if someone let's say trains a model on leaderboard board performance, it doesn't mean the model necessarily performs better in real life because leaderboards are susceptible also to style.

Sebastian Raschka: And so with leaderboards, the tricky part is because humans compare which model they prefer. And if I have, let's say, a very complicated math problem, I ask an LLM and let's say I don't even know the answer and then I well, it should help me with my tax report or something like that. And there's one LM that gives me a really nice explanation. Maybe the result is wrong, but the expression is really nice, easy to follow. I probably like that one. And so I would probably give it a thumbs up because, oh, it's understandable. It's reasonable because I don't know what's correct because I'm not a text expert. And so, um, and I think that's one problem with leaderboards. It rewards the style more than the correctness because there is no correctness check. It's like, yeah, you as a expert, you have to know whether it's correct or not.

Sebastian Raschka: And so also LLM developers when you're training the LLM it kind of gets biased to follow a certain style and the style of people who use those leaderboards and then in that sense you end up with models that have let's say have been benchmarks they have been getting really good benchmark scores but they might not do better than previous models and then it's kind of like yeah a tricky thing you know it's like how hard to measure progress this way.

Matt Tur: And you think people do that just out of largely economic incentives like the companies need to raise more money and people need to have successful careers and therefore they want to look good. Is that the is that the driver?

Sebastian Raschka: I mean I don't want to accuse anybody. I don't know for sure. I I I mean I only know what is known on the internet. I read on Reddit a few times that Llama 4 was a separate model. So there might have been I know some company leader decisions that have led to that. I I honestly don't know. And maybe incentives getting good headlines and that stuff, but well, I think the open weight community is a pretty smart community. So, it's like I think it's not worth worth risking something like that. And I think most people don't risk it. It's just implicit. It just happens. It's like if you iterate too many times, it's a it's a classic deep learning problem or machine learning problem.

Sebastian Raschka: But the nice the beautiful thing here was actually it's not a big concern because it happened to all the models. So all of the models performed like five 10% worse on this new data and it was pretty consistent. So if you were to rank those models, the ranking would still be the same. So in LLM terms, let's say CHPT and Gemini, let's say they cheat on the benchmarks and the models are 10% worse. But if the ranking is still the same, let's say Gemini is still better than GPT, then it it's not a problem if both of them do that. So and I think we have right now that in LLMs where I wouldn't say they are cheating they are just using the data a lot and from using the data a lot well the data leaks in a sense so you kind of of like you're biased in a sense but then if they're all biased then it's again fine because the ranking is still the same.

Sebastian Raschka: But I think yeah the problem still remains the benchmarks are saturated and it's hard to demonstrate or detect or have any type of notion of progress. It's really right now I honestly personally I stopped looking at the benchmark numbers. I just use the model and see for a few days and I see if it's better or not. Like I can't say okay this is better by so and so many%. It's more like oh I use it and it I feel like it's doing a better job. I can't even put it into words. And I think that's the challenge we have right now. How do you put that into words to communicate the progress? And I think that will be in the upcoming years the more difficult problem to solve. How to actually evaluate what you're using? I mean the the the power of LMS is that they are so free form but that's also the downside for evaluations because evaluations if you want to be numeric and precise well free form is not so easy to deal with basically.

Matt Tur: Super interesting. So going back to tips and tricks to make sure that we cover the state of LLMs in I guess early 2026. So we talked about post-training. How much in the recent progress in the last year or so do you think comes from non-archchitectural and non-postraining stuff and I'm thinking in particular inference scaling and then tool use?

Sebastian Raschka: I do think a lot. I yeah I mentioned previously post-training but honestly I think one of the biggest drivers this year has been also the inference scaling so inference scaling essentially means you don't change the weights of the model you just expend more compute during using the model during when the consumer or the person the user uses the model.

Sebastian Raschka: And a beautiful example or chart was back in October 2024 when OpenAI came out. They had like this chart where they had two subgraphs. One was for scaling the training and one was for scaling the inference and you could see for both both were going up with a similar like increase. And so you can basically invest either more money during training which is a onetime cost and then you have with a fixed size you never have to pay money again later on but then that breaks a bit in a sense because reasoning models they generate more tokens so they are also expensive to use during inference. So inference scaling includes that it includes like models that generate more tokens because if you generate twice as many tokens it's twice as expensive because now you have twice as many steps. That is one form of inference scanning but it can lead to more accurate answers.

Sebastian Raschka: The other one is parallel sampling. You just have the mo you ask the model multiple times more like a majority vote. And most people if you see the benchmarks they do that. It's called best at something like that like best of five or something or best they say I think an at sign five or something running it five times and then selecting the answer. So you can do majority vote but it's five times more expensive now because you have to have to run the model five times. There are also methods where people have a judge model that judges these results. Like if you can't do majority voting, you can have a score and then score the highest answer. It's a bit brittle because well that model can also make mistakes.

Sebastian Raschka: But there are all these types of tricks or self-refinement where you have multiple iterations. Self-refinement is also like basically like this iterations where you you have one LLM write the answer and then you say okay take a look at this answer and then it oh I made a mistake here and it self-refineses. I mean reasoning models do that internally as a chain of thought also sometimes but you can also have an explicit version of that or another really cool paper that came out in January was RLMs and so what they do is they take that prompt so instead of processing it all at once they chunk it up into several smaller prompts or the LM decides it learns how to or sees how it should chunk it up in code and then runs a prompt on each of those again. So basically making one prompt into smaller prompts and then having multiple requests.

Sebastian Raschka: And this I would say is also a form of inference scaling because I well it depends but I do think it's it can be more expensive because now you have more more LLM calls and each one if you want to go deep you can end up spending more tokens not more tokens in one request but in the sum of all the requests but there are all these things I think that underappreciated in a sense because they are not I mean in this case it was a popular paper but often inference scaling is talked about that much. But I think it is a big driver of making LLMs perform well. And I think why I think that is if you use DeepSseek locally or use the platform, let's say you use a local LM and use CH GPT, I think CHB of course is a really good model, but I do think the leading open weight models are not that far behind, but if you use them locally, they don't feel as good.

Sebastian Raschka: And I I think that's because JBD has a really good interface. Like the the platform they have, it's not just running the LM, it's maybe cleaning up your prompt. Again, this is like I would say a hypothesis or like I'm I'm guessing here, but like instead of having the LLM learn, of course, it can it does learn how to deal with missspelled words, but you can also just clean that up the input in certain cases where that might improve the accuracy. And I think all these little engineering tricks, not just inference scaling, but just cleaning up the prompt, how to manage the context, the history and everything, I think that all contributes to a lot of progress that is felt by the user.

Sebastian Raschka: Yeah. And then another example would be tool calling too. I think that was also a big one in in 2025. I don't remember when Jet GPD introduced tool calling, but might might have been early 2025 or 2024. Um but GPTOSS so they had that open source model in let's say summer 2025 and GPTOSS has tool calling support. So tool calling means that the LLM can call a web search or it can call code interpreters and so forth and and that is very very powerful because I think this is one of the ways you can mitigate not totally mitigate but let's say reduce hallucinations because then the LM suddenly doesn't have to remember everything anymore. You can outsource a lot of things that are hard to tools.

Sebastian Raschka: So like we humans do right we humans we use calculators we use the web search we don't try to memorize everything and so I think in that sense that is really a big unlock the only problem is well you have to trust the LLM to run on your computer so which is why right now I think it's more like confined to these proprietary proprietary LMS like Gemini and Chetchd because well it's not your computer it runs on if it goes somewhere executes some code and messes it up well not your problem but I think we will see more of that in the upcoming years when the open-source tooling kind of like I would say gets more robust and people have more trust in running that on on their own computer maybe in a Docker container still but yeah something like that.

Sebastian Raschka: I mean right now a lot of people already run code agents on their computer they're usually kind of of more restricted but I think people are more and more trusting these to do things they wouldn't have trusted them to do like a year ago and give them permissions but as LMS get better people develop more trust or I mean run it in a secure virtual environment and I think that will be a a lot of also leaps we will see even though the LLM doesn't get bigger or anything like that and oh you can actually go to the GPOSS release block and they did have benchmarks to show how the performance on the benchmarks is with the same model with tool called enabled and disabled and you can actually see there is I mean it's not like two times the the performance it's maybe 1.2 two times the performance. But you can see there's definitely a jump in in capabilities just by allowing the model to use tools basically.

Matt Tur: And that's part of where you see the world go, right? This combination of like open source model and private data. I think you call that the edge in your in your blog post.

Sebastian Raschka: What distinguishes different LM right now? They're all kind of similarly good. I would say like open weights. There are a lot of LM that are similarly good. I mean, personally, I don't use all of them all the time, but so I usually use one LLM at a time, but like if you use or compare Chachi, Gemini, Claude, Grock, I think they're all pretty much on the same level like and and I think that's because they're trying to do everything like they they're generalist models for a general person to do a lot of things. I mean, cloud is a bit more specialized to code now, but the other ones they are more like general models and I I wouldn't say one is significantly better than the other. they have like small I mean differences and so forth.

Sebastian Raschka: But so if you want to really distinguish them and make them better in in certain industries I do think yeah the private data is what helps like all the treasure troves of data that a finance company has over the years over 100 year or 50 years collected or medical data like medical records from patients I think JDP had like a contract now to process them to make it secure and private but I honestly think these companies they don't want to just give away that data At first they can't or they I mean it makes also sense you just really as a patient or customer I would feel really bad if someone gives my health data to some other company that I didn't agree with or agree with with this sharing but then also the companies they don't want to just give away all that data because once they do well then maybe they become then really kind of obsolete in that sense where it's all all their treasure is basically all their what makes them different from other companies is now taken basically.

Sebastian Raschka: And so over the last month I mean people reached out to me also I know for a fact that big companies are training LLMs inhouse really like big companies who have the financial means to train ChatGPT like model are hiring people who train LLM and I think that is also what we will see that instead of going to this big LLM provider and giving them the data people will try to make their own LLMs for their own company and private data.

Matt Tur: It's fascinating, right? So, sort of back to the future because initially people thought that they were going to train the models and then they kind of like gave up. But what you're saying is that you're seeing people going back maybe with a better state of open-source LLMs that they can build on as a building block. That's what you're saying, right?

Sebastian Raschka: Yes. And no, I think you're right. I mean, open weight and open source models are very or were very popular like a few years ago and are still very popular and I love working with them but I think well there's still a gap between a ChatGPT model and an open-source open weight model maybe now with Deepseek version 3 not so much but that's almost like a different community like the tinker community like me like small system well Deepseek version 3 would be way too expensive to run for me every day I would have to spend thousands of dollars on just hosting costs every every day every week and so I toy around with smaller special purpose models.

Sebastian Raschka: But what I meant is so first yeah the open source community in that sense will have maybe a comeback at these companies but I even mean a step further that they actually develop models from scratch like really big models and what's different from I would say the regular open source here is that it is really large scale it's like it's like a Chache scale data center large LM. It's not something you run on your computer basically. It's really like a big data center style LM. So I know that there are I mean I can't say any names but I know people are interested in that like they are exploring that whether it will work out or not I don't know but I think I mean right now if you are in college you are learning about LLMs that's the thing that's the big thing. So you start with open source, you start training small LLMs and you you work your way up.

Sebastian Raschka: Or and then at some point well you probably want to get hired either by Gemini Chat Gribbd or so forth and do the big model development there but not everyone I mean you can't have 100,000 people doing that. So there will be people distributed across different companies who will do something similar.

Sebastian Raschka: And also on the other hand people who are I think at OpenAI and Gemini at some point I mean big finance companies have a lot of deep pockets they will make it attractive to do something similar at their company too. So I think we will see right now it's very concentrated at these companies but we will see I think the knowledge being a bit more spread out where other companies will develop models too. we will probably not hear about it you know it's like on in the news or anything maybe the news of course but not there won't be papers there won't be big announcement because it won't be this general audience model so it's more something they will do internally right.

Matt Tur: Yeah especially if you're a big hedge fund or defense company like I assume that's the kind of companies we're talking about yeah those are have a very secretive DNA okay fascinating fascinating.

Matt Tur: What else do you see happening this coming year I was interested to to read your thoughts on continual learning which was sort of the talk of the town nurips but you view viewed it or you view it as a 2027 thing not necessarily a 2026 thing.

Sebastian Raschka: Yeah the continual learning is an interesting one. I think it was discussed there very heavily but also in general if you went to social media AI related topics continual learning was there all the time everywhere and to be honest with you I don't know why exactly it was such a hot topic in 25 because I don't think there was a big breakthrough I mean maybe it's the hope for breakthrough or the like hey nothing has changed so maybe we have to focus more on that to force some change but yeah so continue learning I think is an interesting topic because it is it sounds attractive if you have an LLM that self-improves or like an agent that does something fails and learns. I don't think anything like that is feasible this year.

Sebastian Raschka: I mean so right now I mean well you could technically do continue learning if you wanted to with the data you have like when you think even of RLVR um so you could technically it's just updating the model but the problem is still the catastrophic forgetting or also you don't want to train the model on garbage data. So people kind of do I think I mean I would call it continual learning but in a more controlled setting where well instead of just updating the model letting the model update itself they collect failure cases and data and then construct the data set and then do it in a more controlled manner but you can see based on the model releases it happens more frequently than it used to because I mean back then it was GPD-1 GPD2 GPD 3 and now it's GPD 4 4.1 4 I think two or five 51 52 and all the models they are iterating now or even the same with Deepseek so it's like this same architecture you iterate multiple time but still in a more controlled way.

Sebastian Raschka: And I think it makes sense because it's such a expensive thing to do you can't I don't even know how you would do continue learning if you have this model it is hosted in a data center you can't just update it it's a very expensive model you can't just update it on good luck you know so it's like a risky thing to do and you can't do it even as a single person you have to be really careful monitoring a lot of things. I don't know how that would work because right now we are still in this era where everyone uses the same model. People don't have custom models. When I go to Chetchup, I have the same model as you do. And yeah, the prompt is a bit different like the memory and everything but it's all in the prompt. It's the same model weights.

Sebastian Raschka: And as long as we have that I don't think yeah we will see anything like continual learning. I mean there are companies I guess like I mean Tinker API it is something where it is democratizing a bit like the training where through an API you can now train your model more cheaply or instead of having the hardware it's on their data centers people have their own copy but you know it's I think very very far from continual learning it's just making available what other companies have in terms of training on a large cloud instance without you having set it But I don't see anything 2026 that really makes continued learning like the big breakthrough in efficiency or I don't know. And so 27 is even ambitious. So I don't know. So maybe we'll see something there 2027 given that it is such a big topic an important topic. There's a lot of smart people thinking about this and working on. So, we'll probably see something or at least I don't know some ideas that are fresh or things that prototypes that work or interesting. But well, really hard to say anything concrete without having, you know, seen anything. So, it's a prediction that I just put out there. Maybe we'll see more continual learning stuff in 2027.

Matt Tur: But, well, with a grain of salt.

Sebastian Raschka: Yeah. Maybe becomes a self-fulfilling prophecy, right? And if you have enough smart people that decide it's the thing, then maybe it does does happen.

Matt Tur: All right. So maybe to close the conversation, let's talk about your work and how you do your work. So you you published a book this past year in 2025 on how to build LLM from scratch. I believe that you are writing the SQL currently how to build reasoning models from scratch. is at the is that the the title and you produce an incredible amount of of of work. So people should find you on your website on your Substack letter. How do you absorb all of this knowledge? To which extent are LLMs part of your workflow? Just curious how you work these days?

Sebastian Raschka: Good question. So I must say like well I don't have like a magic well approach or anything. I think I I think the thing I have maybe is I get very excited about things and then when I'm excited about something it goes very easy and very fast. I don't know it's like well if you notice maybe I write only about certain topics I don't cover image models at the moment for example because I am just very excited about LLMs and then I don't know it's just I can't help it. I get very excited, read all the things about it, write about it and you know that that's that's mainly I almost go by intuition basically what I.

Sebastian Raschka: And I'm kind of lucky in that sense that like with my blog what I find interesting other people also right now find interesting. So I think there's like a lucky coincidence that I honestly write only about things I find interesting. So I'm not kind of trying to force myself oh I have to cover XYZ because well it's something that should be covered. It's more like oh how does this let's say recursive language model work? Let's just read the paper and then I write about it you know like more like um yeah getting excited about things.

Sebastian Raschka: And yeah the book writing is well it's also a bit different because my blog is more research paper focused where I put all the like when I get excited about something I read about it and put it in there. For the book I'm similarly excited but that's more like a coding book where it's like the fundamentals. It's like because I think that's like to be honest the best way to understand something is to see to see it actually working. It's not any like handwavy figures. I mean there are a lot of figures like right now for chapter six just I finished it the other day 21 figures they take the most work. Maybe one day LMS can help me with that but figures help to explain the code and everything but code basically doesn't lie if it it either works or it doesn't work you know.

Sebastian Raschka: And I think that's a very useful way to learn also and it's just alo a lot of fun for me. It's like when I write code and I see it working it's very satisfying and you have something that actually works. So I should say I'm not building in that book any production level systems. It's called build an LLM from scratch or build I mean large language model from scratch but it's not like an LLM that you would use in production. It is it well if you make a few tweaks you could use it in production but the goal is code readability and teaching LLM basically to because I think to see actually what how do I format my training data? How does it get processed? What is the loss function? What gets updated? I think this explains so much more than if I say, oh, it does next token prediction and then hand wavy hand wavy here and there. You can actually literally see how it does that and what feeds in.

Sebastian Raschka: And the same with LVR. I we had like a hopefully not too bad explanation in this podcast at the beginning of a gpo. But if you actually see the diagram, I have the numbers in there, but the numbers could be wrong though if you have a figure and you draw arrows and everything. But then if you implement that in code and you get exactly the same results and the model trains and you you get 50% accuracy, it's like a nice thing where you can oh it's actually working. It's not just made up numbers. It's actually it actually works.

Sebastian Raschka: And also that's how I learn about LLM architecture. So I have this blog the big LLM architecture comparison with now I think 13,000 words because I keep extending it. I read a paper, I look at the architecture of an LLM. Do I really understand it? So I draw the architecture but then do I really understand it and often unless it's like a one trillion parameter model I often code the model and so I have still my GPT2 architecture and they are relatively similar and so if there's a new architecture I take the most similar one and make a few changes to it but then the beautiful thing here is so someone already implemented that in hugging phase the transformers architecture so I have a reference model I can run I run my model and I can see do I get the exact same logits the same numbers if I have the same prompt And so with that, you can actually selfch checkck yourself. Did I implement everything correctly? Are the results correct? And I think that that's just a lot of fun. It's a lot of work, but it's a lot of fun and it doesn't lie. It gives you the the correct answer.

Sebastian Raschka: Perfect. On some prompts, someone else extended that and I found a bug and now I have a better understanding how they implemented the yarn scaling. That is something you would never understand by just reading the paper. You have to really, I don't know, look at the code and to play around with that. And so yeah, so that's that's basically how I try to work. I try to combine you know like reading and coding and LLM I also use but I try to use it um so I I would say for blog writing on bookw writing not so much because honestly I for fun I tried it out it's just it generates okay text but it's I don't know it does not um I can ask it to generate text like me but it's almost like then I don't like it I end up editing it and then it's almost faster for me to just write it out the way I want it.

Matt Tur: And you had interesting thoughts on LLM burnout, how using LLM tends to deplete energy.

Sebastian Raschka: I would say first also the thing is it's not super satisfying if you just ask the LLM to do it. It's you know like cheating at homework. I honestly I understand there are jobs and people where it just matters how much you get done and how fast you get something done and then it it makes sense to use an LM to do the job for you. But I think that different types of people who enjoy different things. For example, I enjoy doing the work more than managing. When I was a professor, I well, I did research, but I had to also, you know, supervise other students. And I liked working with other students, but I noticed I actually like doing the research myself more than telling other people how to do research or like managing.

Sebastian Raschka: And so I think if you use only LLMs to just generate everything and I wouldn't say useless but I would feel maybe empty like okay you you use that pride I think the pride oh I did something that worked and it's cool and you're proud of this and so what I try to do is generally when I use LLM I try to make my work better not to like not necessarily to make more or make it faster I mean to some extent I do but then I try to like how can I make what I do kind of better. So what I use LM for is more like hey I use actually the GPD5 Pro when I written an article and put it in there. Hey, can you find any mistakes or typos? Often I have mis numbered mislabeled figures like oh like figure 11 12 15 16 and things like that I can check myself but it's just faster for an NLM to find all these things like how to make things better or are there any things that are unclear?

Sebastian Raschka: I mean I'm not a native speaker so sometimes I have problems with a sentence that I'm tired. just can't get it right and then it suggests me, oh yeah, that that is maybe not a bad way to say it. And I I would then take that sentence for example, like things like that. Um where yeah, I'm trying to make let's say the work better without fully replacing myself. I mean, maybe that's like shortsighted because LM will eventually be able to do everything, but I kind of like enjoy the work too much to just give everything away if that makes sense. So, and I have the luxury that this is still works for me. So I know there are some businesses where yeah it is really important to execute faster and you know so yeah.

Matt Tur: Thank you very much for your work very popular for the quality of your writing all your tweets you have like a big X following and your name keeps coming back in conversations about where people go to to learn. Thank you for doing this part. Really appreciate your time. That was super fascinating and very educational, very insightful. So really appreciate it. Thank you so much, Sebastian.

Sebastian Raschka: Thank you for inviting me. I had a lot of fun. I think it was maybe one and a half hours just talking about LLMs and AI. I mean, this is um well, that's the dream, right? Thanks for having me. Thank you.

Matt Tur: Hi, it's Matt Turk again. Thanks for listening to this episode of the MAD podcast. If you enjoyed it, we'd be very grateful if you would consider subscribing if you haven't already or leaving a positive review or comment on whichever platform you're watching this or listening to this episode from. This really helps us build a podcast and get great guests. Thanks and see you at the next episode.