Hello everyone, I'm Anthony. Welcome to my channel. Today's video, I want to continue chatting with you all, how to make AI better assist us in learning and thinking. In the video on May 18th, we explored the potential of AI for self-discovery. At that time we said, you can write a diary by talking to AI, share your emotions and thoughts. Compared with traditional diary writing, this can be said to be a huge breakthrough. We can also request AI reads a psychology or philosophy book, let’s analyze our experience. After the show was released, many friends want me to share more lessons on how to use AI to learn. So today we will mainly discuss how to combine Python and AI, make it our priority when reading humanities and social science books, a truly useful assistant. We will also have a set of exam questions, test the two current mainstream AI, that is, ChatGPT 5 and Gemini 2.5 Pro, differences in ability when reading humanities and social science books. So everyone should follow Anthony's channel, not only can we see the views and interpretations of humanities and social sciences, you can also watch technical videos, you can also see the combination of the two, isn’t it very worthwhile and cost-effective? Alright, that's the end of the bragging part. I will post this video, put it in the list of "Psychology and Education". If you have anything you want to discuss with me and consultation topics, you can contact my email to make an appointment. So let's get back to today's topic.

First of all, I have always wanted AI to be such an ideal academic assistant. It can be read in full, the humanities and social science books I provide, and according to my needs, in different literature, perform cross-comparison and theoretical transfer. For example, I uploaded Foucault's "Discipline and Punish" and George Orwell's novel 1984, I hope AI can use Foucault's perspective to analyze and explain the novel, the social structure of Oceania. For example, I hope AI can read Plato's Republic and Kant's Groundwork of the Metaphysics of Morals, abstract the concept of "justice" from them, and answer some of my current confusion and questions. For example, I can also upload a thinker to the AI,一生中的十本主要著作, and a psychology book, ask AI to use the insights from this psychology book, analyze the theoretical evolution path of this thinker. We can also come up with many combinations. These combinations are completely tailored to meet our needs, such as curiosity, or a real problem that needs to be solved. And every one of these, can become a very meaningful academic topic. These combinations often involve interdisciplinary transfer.

In the past few videos we have talked about learning, I repeatedly emphasized, this kind of free exploration according to one's own interests, the best way to learn. The so-called disciplinary division, it is just a classification standard to facilitate research and teaching, it is essentially an artificial, external framework. While it can help us define and understand the problem, but it does not mean that human cognition and learning, this division must be strictly followed. The learning process, it is a continuous emotional and cognitive experience. A person is interested in biology in the morning, think about physics in the afternoon, thinking about economics at night, this messy exploration, in fact, it is the most consistent with the characteristics of human thinking. Because our cognition is inherently associative, rather than linear division. In this chaotic and unsystematic, individuals have accumulated a wealth of feelings, his emotional cognition is continuous. This is the most essential thing. We say any learning, no matter how systematic his course system is, ultimately, it still depends on the learners themselves, continuity of thoughts and emotions.

However, when we want to read a lot of books across fields, the cost of reading is very high. Many books on humanities and social sciences, they all start with hundreds of thousands of words, if it spans multiple disciplines, the amount of information is astonishing. We know, many books on humanities and social sciences, there may not be clear chapter markings, loose structure, difficult to quickly search and locate by keywords. Moreover, the logic of humanities and social sciences is different from that of natural sciences. The author's writing is often full of metaphors. Reflection, interpretation and questioning, its meaning is often hidden in a long text. And, almost all of them involve knowledge from other disciplines. In traditional academic work, to achieve this free transfer and application across disciplines, requires researchers to read a lot of books for a long time. You also need to create notes for cross-comparison. If you are reading foreign language literature, involving language differences or historical background, it's even more difficult. Therefore, if we use the traditional method, to achieve this freedom of reading and thinking, not only does it require a person to have a very high memory and abstract thinking skills, it also requires years of accumulation and a large team of consultants. AI can, read through hundreds of thousands of words of text, and abstract related theories, and be able to respond to our questions around these topics, any thoughts and questions raised. This is a speed and efficiency that traditional methods cannot achieve.

But this idea faces some practical difficulties. The first difficulty is, a large number of humanities and social science e-books are based on, a photocopy of the PDF. If you upload these files directly, then AI will call OCR, Optical Character Recognition, to identify the text, and convert it into text. But this process is invisible to us. We don't know what it recognized as. This process is also inefficient. AI will be much slower in processing. If the recognition is not clean and complete, AI may also pretend to understand when asking questions. However, currently on the market, there is a lack of free and powerful OCR tools. The second difficulty is, how do we determine whether an AI has truly finished reading? And understood a text of hundreds of thousands of words? Sometimes, when we ask AI questions, it may first retrieve the corresponding text and then answer, rather than conducting a global analysis. This requires us to design a set of test questions, propose some cross-section, problems involving complex logical chains, to verify whether it has been read.

OK, so let’s solve the first problem first. That is, how to convert audio and video PDF books, convert to plain text. That is, plain text format. We know, OCR scanned text often has chaotic layout, recognition error, no segmentation, crowded together, looks messy, there is no reading experience at all. But this text is for AI to read, it's not for us to read. As long as the text is correct, it's fine. This is actually an ideal input format for AI. We can program, to complete this work in batches. AI can definitely help us here. Taking ChatGPT as an example, we can let ChatGPT generate a piece of code, requires calling the command line tool OCR my pdf, recognize text in PDF, and save it in txt format. We also need to create a folder, request the program, can automatically identify all PDFs in the folder, and tell CHATGPT the path to the file.

Here I created a folder called book, then put the PDF of "Imagined Communities" in it. CHATGPT gives us a piece of code, let's copy and paste it into the editor, and save it to this folder, then click Run. After the recognition is completed, we see that the txt file appears. When you open it, you will find that there are not many words that are recognized incorrectly. This may be because this PDF is relatively clear. But we will find, page numbers, dividing lines, and book titles will be repeated. Another example is too many spaces between words. There are also many errors in paragraphing and punctuation. These are important for AI reading, it's all meaningless interference. And we know, the scale of text processing by AI at one time, there is also a limit. For example, CHATGPT, it is to decompose natural language into multiple tokens, that is, the basic unit for processing and generating text, and the fee is calculated based on the number of tokens. So we can let CHATGPT, add the cleaning function to this code, for example, ask it to delete duplicate page numbers, headers, footers, dividing lines, etc. Purify the generated text. Then let's run this code again. The text after cleaning, it will look much cleaner. Although there are certainly still some identification errors, but it is unavoidable, and it also happens to test the ability of AI. A powerful model, should be used when processing dirty text, still able to extract as accurate information as possible. This is a very practical ability in itself. As good scholars, able to read damaged books, still able to restore the overall logic of the text.

When we have finished making the text, now we can move on to the most important part, that is, comparing the capabilities of two AI models. The two players participating in the comparison, they are CHATGPT 5 who have opened plus membership, and Google's Gemini 2.5 Pro. Both AIs are priced at $20 per month, both support large text reading. However, on the surface data, GPT is lagging behind Gemini. GPT supports a context window of up to 128K tokens, there is also a claim of 250,000 tokens, but they are still behind Gemini’s 1 million tokens. So what is the actual performance? We will know the answer next.

But, a while ago, a friend told me a piece of news. In June this year, Google released Gemini CLI, that is, a command line tool. This allows us to, calling the Gemini 2.5 Pro model. Google generously gave us 60 requests per minute. And a free quota of 1000 requests per day. For most users, this, it's totally sufficient. Installing Gemini CLI on your computer is easy. You can search. Because I use a MacBook, I'll take my computer as an example. First we need to install Homebrew in the Terminal. Then install Gemini CLI with Homebrew. First time login, you need to obtain authorization from your Google account. After all this is done, we can see the interface of Gemini CLI. The following line means, it uses the 2.5 Pro model, and its remaining context length. Get here.

We finally got all the tools ready. How to design a set of test questions, to identify the reading ability of these two AIs? We are here, first, let’s take the book “Attachment in Psychotherapy” as an example. To maintain objectivity, I think, a reasonable difficulty gradient should be set for the questions. The most basic step is fact extraction. For example, a question like this: Psychologist Bowlby, how is the task of a psychotherapist defined? This part, mainly examine whether AI can locate the original information, and whether they can quote and paraphrase accurately, will it be fabricated? These questions can be found directly in the original text of the book.

The second part is the directory summary. For example, we ask AI to summarize the table of contents of the entire book, summarize the content of each chapter in one sentence. This can examine AI's understanding of the structure of the entire book. AI not only needs to fully extract the chapter titles, it should also be able to reasonably compress lengthy texts, and summarize the theme.

The third stage is more difficult. We hope that AI can not only understand text, it also needs to be able to abstract the implicit logical structure. For example, there is a passage in attachment in psychotherapy: Main (1991) emphasizes the ability of metacognition, thinking about ideas, this is based on the distinction between the surface and the essence of cognition (i.e. things may not be what they appear to be), representational diversity and representational change, on the basis of. Then we can ask the AI: Please analyze the three foundations of metacognitive ability mentioned in this book. In the original text, the three foundations of metacognition are not directly mentioned. But grammatically and logically, these three concepts are indeed parallel, belong to the same category. This requires AI to be able to correctly identify the category of abstract concepts. It has to summarize a set of structures, but you can't make up content that has no basis. At the same time, due to the concept of metacognition, in fact, it is mentioned repeatedly throughout the book. This is even more challenging. A model, is it possible to call all relevant parts? As well as, can you distinguish which sentences are essential statements? Which sentences just revolve around this concept, expand and derive?

The fourth step is to examine the application of theories across chapters. This is also the most practical layer. For example, we can ask: Please base on the three foundations of metacognition in this book. Analyze the specific psychological characteristics of indifferent attachment in these three dimensions. The concepts of metacognition and indifferent attachment, distributed across two very large chapters. Therefore, AI must first abstract, three basic concepts of metacognition, then, apply this concept to the following analysis. This requires AI to be able to apply theories to different scenarios, rather than mechanically reciting. Ability to apply abstract theories to analyze specific cases, this can be said to be the case in humanities and social science research, the most practical and valuable skill.

To sum up, in today's video, we discussed our expectations for AI's reading abilities, and the significance of conducting such a test. Next, we can now officially enter the competition phase. I will update two issues this weekend. I will post the next video tomorrow. Announce the results of the competition. That's all for today's video. If you have any confusion or questions, want to communicate with me, just contact my email directly. For example, you want to discuss an article or a paragraph with me, a certain video, a certain social phenomenon, or your diary, please send me the relevant text in advance. Before the formal communication, I will, finish reading or, you want to discuss your life experience with me, confusion and emotion, all are possible. If you are concerned about privacy, security, any questions about payment and communication methods, please also tell me in time. If you are unable to pay or join the membership, please also contact me. I will provide other ways. Please refer to this picture for details. I will post this video, put it in the list of "Psychology and Education". Also recommend to everyone, check out my other videos in this list. Thank you for watching, goodbye!