Today we're going to be training an LLM from scratch. So, no pre-trained weights, no nothing that you can just grab online from like a transformers library. We're going to work purely on torch and some like very basic libraries. This will be like a good indication of how like actual like research engineers like in big labs like design their models.

As I mentioned like this is generally the first thing you think about when you create a new any new transformer model is what tokenizer you're using. I come from the voice world. So this is where that's the one of the most important things. LLMs don't see text they they work with embeddings or like vectors. What we're going to be using here is a character level tokenization just because it has the lowest number of possible tokens. In our case it's going to be only 65 embeddings essentially because it's 65 different characters that will appear in our training data.

The concept of biogram is very is a very important concept when when you're training transformers because you want your model to see as many possible biograms as possible. In our case, 4,000 biograms is like very doable. If we did try to train using a full tokenizer, this will never converge. The problem with character level tokenizers is that they don't really scale very well because the models need to understand correlation between different tokens. But that's our trade-off that we're willing to take because of course we're running a small model.

Transformers are using like these four different building blocks. One is multi head self attention. Attention is what the difference between that makes the transformers different than other neural networks is that they can actually attend to previous tokens and understand the relationship between tokens. Next one is the MLP or like the feed forward network which essentially takes this different relationships between those tokens and it combines them together to to be able to generate the logits.

Then you have the residual connections which are basically there for the model to not have to reinvent itself after every layer. Residual basically means that every layer that passes through, it just takes the previous input and adds a small difference. This way the model doesn't make a huge change on the inputs themselves and the model can be more stable during training. And lastly, the layer normalization is allowing you to scale down those activations in ways that allows those activation to not be exploding into very big values. So if one layer like multiplies your activation by 10x, the layer norm is going to push it back back to like normal values.

The objective of this training is going to be to create a Shakespeareian like LLM. As we mentioned before these models like are learning as next prediction. The way this cross entropy works is that you take your sequence and you just offset it by one as what the model needs to learn.

The way the models work is that you first start generally with as high of a learning rate as as you can afford without making the model become unstable. Usually you have this concept of a warm-up where you start with a very small learning rate. Then you start reducing the learning rate. That's what we call weight decay. You want the learning rate to be less when the model is like close to being perfect.

TQDM helps with like tracking like the losses. Another important part is evaluations to make sure that your model actually works well. It's very easy for models to like overfit especially like this small models. That's why we have this val loss which is a part of the data the model has never seen. When the loss starts going below 1.0 for this specific data set, that's where we're going to start seeing overfitting.

Greedy decoding doesn't work very well for LM. You would want to use temperature. Generally like a 0.7 temperature is like the best middle ground to make inference work well. And then you have top k sampling which prevents the model from predicting very unlikely tokens even though the temperature might get unlucky.

Question: Are reasoning models quite a bit different in training? Answer: The building blocks are very similar. This is very data driven. So you need very high quality data. The complication of reasoning models is finding this good chain of thought data. Reasoning is essentially just adding to the context of the model this logic that then the model can when it generates the response it can go back and attend to those reasoning tokens and get a better response out.

Question: How different is this versus like doing this with audio? Answer: It's surprisingly very similar. These vectors they don't have to correspond to specific token you can take these vectors from other places too. Instead of having a tokenizer for like video, what they do is they have another transformer that they call a video encoder. It will take those frames and then put them through this encoder transformer. You take those vectors out of this encoder and you're going to input them in the embedding layer of the transformer model. The model just cares about these embeddings. It don't care if it's text or if it's audio or if it's video.