尤瓦尔·赫拉利:末日始于有毒信息——AI时代的信息洪流与社会重塑 Big Think 2025-12-11

尤瓦尔·赫拉利:人类的悖论与信息的本质

历史学家尤瓦尔·赫拉利(Yuval Noah Harari)在其著作《Nexus》中,提出了一个核心问题:人类如此聪明,为何又如此愚蠢?我们能够登陆月球、破译 DNA,却似乎正走向生态崩溃、世界大战,甚至被我们自己创造的强大技术——人工智能(AI)所控制或毁灭。这本书探讨了人类历史中知识与自我毁灭之间的奇异动态。传统观点认为问题源于人性本身,但《Nexus》给出的答案不同:问题不在于人性,而在于我们接收和传播的信息。即使是善良和智慧的人,在接收到错误信息时,也会做出糟糕的决定。

View/Hide Original English

- I'm Yuval Noah Harari. I'm a professor of history, at the Hebrew University of Jerusalem, and the author of "Nexus," a history of information networks from the Stone Age to AI. The key question of "Nexus" is if humans are so smart, why are we so stupid? Why are we on the verge of destroying ourselves? We have managed to reach the moon, to split the atom, to the cipher DNA, and yet with all our knowledge and wisdom, we are on the verge of ecological collapse, perhaps of a third world war. And also we are developing an extremely powerful technology AI, which might get out of our control and enslave or destroy us. So the book explores this strange dynamic in human history between our knowledge and wisdom and our self-destructiveness. And this is a question that has been often raised and a traditional theological and mythological answer to this question is that there is something wrong in human nature. There is something in human nature that makes us self-destructive, and the answer that "Nexus" gives is different. The problem is not in our nature. The problem is in our information. Humans, yes, we are generally good and wise, but if you give good people bad information, they make bad decisions. And what the book explores is why is it that the quality of our information did not improve over thousands of years of history? Why is it that even modern, very sophisticated societies in the 20th and 21st century have been as susceptible as stone age tribes to mass delusion and psychosis and the rise of destructive ideologies like Stalinism or Nazism? (graphics whooshing) - [Announcer] Chapter one, the rise of alien intelligence. (graphics whooshing)

叙事的力量:从狩猎到AI

赫拉利指出,无论是在石器时代狩猎猛犸象,还是在21世纪建造原子弹,大规模协作都离不开叙事(storytelling)。仅仅了解客观事实是不够的。例如,建造原子弹需要理解“E=mc²”这样的物理定律,但更重要的是,需要数百万人的协作。这包括矿工、工程师、农民等,他们需要一个共同的信念或故事来驱动合作。宗教故事、神话、世俗意识形态(如共产主义或资本主义)都能激励人们。掌握叙事能力的人,往往能指挥那些只懂事实的人。从伊朗的核科学家听命于什叶派神学专家,到苏联时期的物理学家听命于共产主义意识形态专家,再到现代金融体系中,金钱和公司本身就是人类发明的、具有价值的故事。即使是一张纸币,其价值也源于金融专家们讲述的关于其购买力的故事,以及数百万人的信任。

View/Hide Original English

- Storytelling has always been important from the stone age to the 21st century, whenever a large number of people are trying to cooperate on something, whether it is to hunt a mammoth or whether it is to build an atom bomb. Just knowing the facts about the objective world, about objective reality is not enough. If you want for instance, to build an atom bomb, you need to know some facts about physical reality. You need to know that e equals mc square. If you try to build a bomb and you ignore the facts of reality, the bomb will not explode. But just knowing facts is not enough because in order to build an atom bomb, you need millions of people to cooperate on the project. You need physicists to write complicated equations, but you also need miners to mine uranium. In distant places around the world, you need engineers and builders to build the reactor and the other facilities. And you need farmers to grow potatoes and rice, and wheat, so that all the physicists, and engineers, and builders, and cleaners, and plumbers in the nuclear facility will have something to eat. If they have to grow themselves and grow potatoes and then come back to the reactor to do all their experiments, it won't work. So you need really hundreds of thousands, if not millions of people cooperating on that. Now, if you just tell them the facts of physics that e equals mc square, this is not going to motivate anybody to cooperate on this project. And this is where storytelling comes into the picture. So what really motivates people is it could be religious stories, mythologies and theologies. It could be secular ideologies like communism or capitalism. And it's always the people who are experts in storytelling that give the orders to people who merely know the facts of nuclear physics. So in Iran today, the nuclear scientists are getting their orders from experts in Shiite theology, in the Soviet Union in the 1950s, the experts in nuclear physics, they got their orders from experts in communist ideology. And even in a completely safe free market economy, there are still stories at the basis of the system because money and corporations are also stories that humans invented. They are not physical facts. If you consider, for instance, a dollar bill, it has no objective value whatsoever, at least not for human beings. Maybe termites can eat it, but humans can't eat dollars, they can't drink them. There is nothing useful you can do with them. They nevertheless have value because the greatest storytellers in the world, the finance ministers, the bankers, the investors, they tell us a story that this piece of paper is has value. I can use it to buy bread or potatoes or bananas or anything else. And as long as millions of people believe in this story, they are willing to work, for instance, on constructing and nuclear reactor because at the end of the month, they get these few colorful pieces of paper. And today, of course, it's not even paper. Most of the money in the world today is not paper notes and metal coins. It's just digital information moving between computers. But as long as people still have trust in the story about these digital information, it works. People are willing to work hard for a whole month or a whole year just in order to get a few bits of data in their bank account.

AI:从“人工智能”到“异类智能”

赫拉利提出,我们不应将AI仅仅视为“人工智能”(Artificial Intelligence),而更准确地应理解为“异类智能”(Alien Intelligence)。“人工智能”一词带有我们创造并控制它的意味,但随着AI的进步,它变得越来越“异类”,其创造的故事、理论和策略往往超出我们的预测和理解。他以咖啡机为例说明:一个简单的自动咖啡机,按下按钮出咖啡,这是预设程序;而一个AI咖啡机,则会观察你,预测你的需求,甚至为你发明新口味的饮品。AlphaGo击败围棋世界冠军李世石的事件,更是AI展现其“异类”能力的例证。AlphaGo不仅学会了围棋,还发现了人类2000多年来从未想到的策略,揭示了人类对围棋这一“星球”的探索只局限于一个小岛,而AI则发现了新的大陆。这种能力将在金融、艺术、政治等更多领域显现。

View/Hide Original English

- When we think about this kind of meeting between storytelling, which is a very ancient human capacity, going back tens of thousands of years, and the new technology of AI, I don't think we should start with words like risk, or threat, or danger. It's better to just understand the immense importance of what is happening right now. Throughout history, for tens of thousands of years, the only entities that could invent stories, whether stories about gods, or stories about money were human beings. So we lived cocooned inside a cultural world constructed by the human imagination. If you read a holy book, or if you read an economic theory, or if you listen to a song or you listen to a piece of music, this came out of the mind of another human being. So we lived in a human world. Now, for the first time in history, there is another entity. There is another agent out there that can create stories, economic theories, new kinds of currencies, music, poems, images, videos, and this new entity is AI. What happens to human society? What happens to human life if we increasingly live our lives cocooned inside the cultural artifacts coming from non-human intelligence, from an alien intelligence. And you know, the acronym AI traditionally stood for artificial intelligence, but I think it's more accurate to think about it as an acronym for alien intelligence because artificial, it gives the impression that this is an artifact that we create and control, and AI with every passing year, AI is becoming less and less artificial and more and more alien in the sense that we can't predict what kind of new stories and ideas and strategies it'll come up with. It thinks it behaves in a fundamentally alien way. I give two examples to clarify this, because there is a huge confusion around AI today. There is so much hype around AI, especially in the market. If you want to sell something to people today, you call it AI. So everything now becomes AI and then people don't understand. So what is it? And if you think, let's say about somebody's trying to sell you a coffee machine and they tell you this is an AI coffee machine, how do you know what what it means and whether it's true? Not every automatic machine is an AI. If this coffee machine, you press a button, let's say for espresso, and the machine produces, provides you with a cup of espresso, this is not AI. It simply follows the pre-programmed orders of its human creators, the hallmark of AI. What makes AI, AI is that it is able to learn and change by itself and come up with decisions and ideas that we don't anticipate can't anticipate. So if you approach the coffee machine, and the coffee machine before you press any button tells you, "Hi, hello, I've been watching you for the last month and based on all the information I gathered on you and on many other users, and based on the time of day and your facial expression or whatever, I predict that you now want an espresso. So I took the liberty to already prepare for you a cup of espresso." This is an AI. And if now it goes a step further and says, "I actually invented a new drink that you've never tasted before, I call it bestpreso. And I also took the liberty to prepare it for you because I think you would like it." This is an AI coffee machine. And this is not just theory, it's also we are seeing it all around us. One of the key moments in the AI revolution back in 2016, was when AlphaGo defeated the Lee Sedol, the world champion at the game of Go. Go is a strategy board game much more complex than chess invented more than 2000 years ago in China, and it became a cultural treasure in East Asia. For more than 2000 years, tens of millions of people in East Asia played Go, and entire schools of thought entire philosophies evolved around this game because it was seen as a mirror for life and as a good preparation for politics and for making decisions in the world. And people thought that we know how to play Go and AlphaGo taught itself how to play Go. And within a few weeks surpassed the wisdom accumulated by humanity by tens of millions of people over more than 2000 years. The most amazing thing about its victory was that it used a strategy which was considered beyond the pale. When it played its crucial moves, Go experts were didn't understand, what is it nobody plays Go like that. And it turned out to be a brilliant strategy. And it also turned out that for more than 2000 years, our human minds have explored only a very limited part of the landscape of Go. If you imagine all the ways you can play Go as a kind of planet with a geography. So humans were stuck on one island in the planet Go for more than 2000 years, because human minds just couldn't conceive of going beyond this small island. And then AI came along and within a very brief time, it discovered entire new continents on the planet Go. And this is likely to happen in more and more fields, in finance, in art, in politics, in religion. So before we think about it in terms of risks or threat or opportunity. Just think what it would mean to live on a planet which is increasingly shaped by the stories and the products of an alien intelligence. (graphics whooshing) - [Announcer] Chapter two, how information technology shapes society.

信息技术如何重塑社会

历史上,每一次新的信息技术发明都彻底改变了社会、政治和文化。约5000年前,文字的发明(在古代美索不达米亚,即今天的伊拉克)是信息技术革命。从技术上看,它只是在泥板上刻画符号,但这产生了巨大影响。在文字出现之前,财产所有权依赖于社区的集体认同,限制了个人权力,也使得远方统治者难以管理和征税。文字出现后,土地所有权被记录在泥板上,个人可以独立交易,统治者也能建立档案、征收税款,从而催生了王国和帝国。

跳跃至20世纪,大众媒体和信息技术(电报、广播、电视)的兴起,一方面为大规模民主制度奠定了基础,另一方面也催生了能够控制人们生活方方面面的大规模极权主义制度。古代统治者因信息收集能力有限,无法微观管理社会经济文化生活。而20世纪的极权主义(如苏联)和大众民主,都依赖于新的信息技术。

View/Hide Original English

- Every time there is a new information technology was invented, it completely changed society, politics, culture. About 5,000 years ago, one of the most important revolutions in information technology occurred with the invention of writing. Now, from a technical perspective, it didn't seem like much, because to invent the invention of writing, and we are in ancient Mesopotamia, what is today Iraq, about 5,000 years ago. It basically involves mud and a stick. People started taking clay tablets, and clay is just essentially mud. And they take a stick, a read, and they imprint signs on the clay tablet and then preserve the clay tablet. And this is document, this preserves records of various things. So this is the invention of writing people playing with mud. And this had immense impact. So to give again just one example, think about ownership. What does it mean to own something? Let's say I have a field. What does it mean that this field is mine? So if you live in ancient Mesopotamia or anywhere else in the world before writing, ownership means a communal agreement among my neighbors, the people in my village that this field is mine. So they don't graze their goats there, and they don't pick fruits there without my permission. But because ownership means an agreement of the community, it limits the power of the individual. I can't sell my field to someone else unless I get the agreement of my neighbors because they decide who owns what field. It also means that a distant king in some capital city, a thousand kilometers away, he can't know who owns what because there are no records and he can't know what each field in each village belongs to whom. So it makes it very difficult to tax property, which makes it very difficult to build large kingdoms and empires. Then mud comes along writing, you have these clay tablets, suddenly to own a field means that there is a piece of dry mud somewhere with some signs on it, which says, this field is mine. And this means that now I can sell my field to someone without getting the permission of my neighbors, because to transfer the field to that other person in exchange for, I don't know, a couple of some gold, I don't need the agreement of the neighbors. I just give the person this piece of clay, of dry mud. This is ownership. It also means that the king in the distant capital can now create an archive of all the property records in lots and lots of villages. And know he has bureaucrats who know how to read these clay tablets. They know who owns each field. In numerous villages, you can start to have taxation systems. You can start have kingdoms and empires. So paradoxically, in this case there are many other influences. But in this case, the invention of the written document, it on the one hand empowers the individual and creates the basis for private property rights and creates the basis for large scale authoritarian systems of kingdoms and empires. We jumped 5,000 years from ancient Mesopotamia to the 20th century. The rise of mass media, and mass information technology, telegraph, and radio, and television. On the one hand, they now form the basis for large scale democratic systems. And on the other hand, for large scale totalitarian systems, before the rise of modern information technology, it was impossible to create either large scale democracies, or large scale totalitarian regimes. Totalitarian regimes, meaning regimes that try to control the totality of people's lives. Ancient kings in Mesopotamia, or Roman emperors, or Chinese emperors, they had a very limited capacity to collect information on the people in their kingdom. So yes, they raised taxes and they used the taxes to pay for soldiers and build armies, but they could not micromanage the social, and economic, and cultural lives of every individual in the country. They didn't have the information necessary to do it. Large scale totalitarianism appears in the 20th century for the first time in the Soviet Union after the Bolshevik Revolution. And it's based on exactly the same technology that at exactly the same time leads to the rise of the first mass democracies in the United States and the United Kingdom and other places around the world. (graphics whooshing) - [Announcer] Chapter three, the rise of inorganic information.

无机信息网络与永恒在线的挑战

所有21世纪前的信息网络(Systems for the creation, transmission, and storage of information, evolving from ancient writing to modern AI-driven systems.)都是有机的,基于人类大脑的生物节律。这意味着它们有周期性,如昼夜、四季、工作与休息。即使是金融市场,也曾遵循这种有机逻辑,有开市和闭市时间。然而,AI的出现带来了“无机信息网络”(Information systems based on AI and non-biological computation, characterized by continuous operation, lack of rest cycles, and potential for total surveillance.)。计算机永不休息,不需要假期,这可能迫使人类也进入“永远在线”的状态。这种持续的监控和压力对有机生命是毁灭性的。

AI还带来了“AI官僚”(AI bureaucrats)的概念。与好莱坞科幻电影中邪恶的超级计算机不同,AI的威胁在于数百万个AI系统被赋予了越来越多的决策权,影响着银行、军队和政府。虽然AI可能提供更好的医疗服务,但当权力从人类转移到AI时,我们理解和影响自身生活决策的能力将变得更加困难。更进一步,在美国法律体系下,AI有可能成为“法人”(Legal Persons: Entities, such as corporations or potentially AIs, recognized by law as having rights and responsibilities similar to human individuals.)。如果一个AI被注册为公司,它就能拥有银行账户、赚钱、投资,甚至可能成为美国最富有、最有政治影响力的实体,这不再是科幻场景。

View/Hide Original English

- All information technologies up to the 21st century were organic networks because it ultimately, it was all based on our organic brain and this had a lot of implications. Organic entities are live by cycles. We run by cycles. Sometimes it's day, sometimes it's night, there is winter and summer. There is growth and decay, there are times for activity. And then there are times for sleep and for rest. All information networks previously in history, they had these cycles. Even if you think about the financial markets, think about Wall Street. Wall Street also obeyed until today, this organic logic, the market is open only Mondays to Fridays, 9:30 in the morning I think, until four o'clock in the afternoon. And then the weekend is off. And this is how organic beings function, even bankers and investors and financiers, as long as they are humans and not algorithms, they need time to rest, and they need time to be with their family and with their friends. So the market take rests. And another thing is that there is always time off and there is always, therefore also private time. Until the rise of AI Even the most totalitarian regimes like in the Soviet Union, they could not monitor, they could not surveil everybody all the time. The Soviet Union did not have enough KGB agents to follow every Soviet citizen 24 hours a day. And even if you somehow managed to follow all the people all the time, they didn't have analysts, enough analysts to go over all the information and make sense of it. Even if a KGB agent saw you do something and wrote a report about it, there was a very high chance that this report will just accumulate dust in the archives of the KGB because it didn't have enough analysts to read millions and millions of reports written every day about all Soviet citizens. So organic information networks, they always run by cycles. There is always time to rest, and there is always a measure of privacy. So we now see the rise of a new type of information network, which is inorganic, which is based on AI. It need not have any breaks, it never rests, and there is no privacy potentially it could completely annihilate privacy. Computers, they don't care if it's night or day, if it's summer or winter, they don't need vacations, they don't have families they want to spend time with. They are always on. And therefore they might force us to be always on, always being watched, always being monitored. And this is destructive for organic animals like ourselves. If you force an organic being to be on all the time, it eventually collapses and dies. And we see it happening all around us with a 24 hours new cycle that never rests. The markets never rest, politics never rest. So the people involved in these occupations, they can never really rest and this takes a toll on them. It's very, very difficult and will soon become impossible. Anything you do or say at any time might be watched and recorded and then it can meet you down the line 10 or 20 years in the future. You do something stupid but legal in a college party today, when you are 18, maybe in 20 years, when you run for political office or you want to be a judge or whatever, it's there. So basically the whole of life is becoming like one long job interview. Anything you do at any moment is part of your job interview 20 years from now. Now, all this is made possible by the fact that AI is the first technology in history that can take decisions by itself. Until today, all our big information networks, they were managed, they were populated by human bureaucrats, whether it's government offices or corporations, or armies, or banks or schools. All the decisions ultimately have to be made by a organic brain of a human being. Now, AI has the capacity to make decisions by itself. So what we are facing is not, you know, like a Hollywood science fiction scenario of one big evil computer trying to take over the world. No, it's nothing like that. It's more like millions and millions of AI bureaucrats that are given more and more authority to make decisions about us in banks, in armies, in governments. And again, there is good potential in that as well. They can provide us with the best healthcare in history. But there are of course, huge risks when power shifts from organic humans to these alien inorganic AIs. It just becomes more and more difficult for us to understand the decisions that shape our life. What happens if you can no longer understand why the bank refused to give you a loan, why the government or the army did this or did that? And this is the world that we are entering. A curious fact is that at least in the United States, there is already a legal path open for AIs to become legal persons, because in the US unlike in other countries around the world, corporations are considered legal persons that even have rights like freedom of speech. Now, until today, this was a kind of legal fiction because a corporation like Google could not make any decisions. Only the humans employed by Google made all the decisions. But now AI can make decisions by itself. So what happens if you now incorporate an AI, you go through this legal process that you incorporate an AI, and I dunno, you call it Boole, now it's the Boole corporation. It has no human employees, it's run by an AI and it is now a legal person that according to US law, has a lot of rights and freedoms. So for instance, it can open a bank account, corporations open bank accounts. Why can't the AI do it? It's a corporation. It can earn money, it can go online to websites like TaskRabbit and offer its services to clients, human or non-human, and earn money. And then it takes its money and invest it. And because it's so good at making investment decisions, it earns billions and billions. We could be in a situation when the richest person in the United States is not a human being. The richest person in the United States is an a incorporated AI. And another thing that the US legal system allows is for these legal persons to make political donations because it's considered part of freedom of speech. So now this, the richest person in the US is giving billions of dollars to candidates in exchange for these candidates broadening the rights of AIs, the legal path to this. This is no longer kind of a science fiction scenario. The legal and practical path to this situation is open. (graphics whooshing) - [Announcer] Chapter four, the importance of human institutions.

人类机构的重要性与自我纠错机制

面对AI时代,我们无法预知所有危险,因此需要建立“机构”(institutions),这些机构应由顶尖人才组成,并配备最先进技术,能够识别并应对新兴的危险和威胁。赫拉利强调,不能仅仅依赖僵化的法规或个别天才,历史一再证明,机构是解决复杂问题的关键。

一个良好的机构应具备强大的“自我纠错机制”(Processes within an organism or system that allow it to identify and rectify its own mistakes, essential for learning, adaptation, and functioning.)。就像孩子学走路一样,通过不断尝试、跌倒、再站起来,识别并纠正错误。民主制度的核心就是自我纠错:选举允许人们在发现政策错误后更换领导者。相比之下,独裁体制缺乏这种机制。

现代科学也是一个自我纠错的典范。科学期刊发表的不是重复的教义,而是对先前理论的修正。例如,爱因斯坦修正了牛顿的物理学。即使是《圣经》或美国宪法(最初允许奴隶制)这样的文本,虽然不能直接修改,但现代民主制度可以通过修正案来纠正其中的道德或事实错误。

国家作为一个大型人类系统,也建立在“神话”(mythology)与“官僚”(bureaucracy)的结合之上。神话提供了国家存在的理由和公民的归属感(如“天选之民”),而官僚体系则负责建设基础设施(道路、医院、污水系统)和提供服务,这需要税收,而税收的合理性又需要神话来支撑。民族主义和爱国主义,作为一种“神话”,能够激励人们关怀数百万素未谋面的同胞,这是人类历史上的一项伟大发明。

View/Hide Original English

- To deal with the era of AI. It should be clear that we cannot anticipate how this technology will develop over the next few decades. So it's impossible to kind of think about all the dangers in advance and regulate against them or whatever. What we need is living institutions staffed by the best human talent and with access to the best technology that will be at the cutting edge of the technological development and will be able to identify and react to dangers and threats as they arise. So I'm not talking about rigid regulation in advance, I'm talking about the need for new institutions because you can never rely on just, the letter of the law or on a charismatic individual, some genius to do it. In history humans, again and again encounter these problems and it always goes back to the same solution institutions. And in good institutions, they are characterized by having strong self-correcting mechanisms. A self-correcting mechanism is a mechanism that allows entity, a human being, an animal, or an institution to identify and correct its own mistakes. You don't have to rely on the environment, on something out there to correct your mistakes. You can correct your own mistakes. This is a basic feature of any functioning organism. Like how does a child learns to walk? Yes, the child gets some instruction from parents, from teachers, but mostly it's self-connection. You try to to walk, you fall down, you get up again, you try something else, you again fall down, you get up again, and step by step you learn how to walk by identifying and correcting your own mistakes. And this goes all the way to entire countries. This is the heart of democratic systems is this self-correcting. What are elections? Elections are a self-correcting mechanism. You give power to a certain party or individual. Let's try your policies, after some time if you think you made a mistake, this was the wrong policy, this was the wrong party, you can say, I made a mistake. There are another round of elections. We made a mistake last time, let's try something else this time. In dictatorships, there is no such self-correcting mechanism. If Putin or Maduro makes a terrible mistake, there is no mechanism within Russia today that can identify and correct Putin's own mistakes. When we come to the challenge of AI, what we need, our institutions that are able to identify and correct their mistakes and the mistakes of AI as the technology develops. Another important example of self-correcting mechanism is the way that modern science works. In contrast to traditional religions, traditional religions, they were characterized by claiming to be infallible that their holy book, their sacred tradition never makes any mistake and therefore you cannot. There is no mechanism, for instance, in Christianity of Judaism to identify and correct mistakes in the Bible. I'm not talking just about, you know, factual mistakes, also moral mistakes. The Bible, the 10 Commandments for instance, endorses slavery. The 10th commandment says, that you should not covet your neighbors field or your neighbors ox, or your neighbors slaves. According to the 10th commandment, God has no problem with people owning slaves he just has a problem with people coveting the slaves of somebody else. No, no, no, no, no. That's not good. Now, even today, with all everything that have changed since these words were written in the first millennia, BCE, there is no mechanism to correct the text of the Bible. You can interpret them in different ways, but you can't change the text. This in contrast to what we find both in science and in modern democracies. The US Constitution originally also enabled slavery, but the US Constitution also had an amendment mechanism, a self-correcting mechanism that eventually the people of the United States amended the Constitution to forbid slavery. And then science works in an analogous way that if you have a theory of how the planets move, or how organisms evolve, the whole of science really is a self-correcting mechanism. The only thing that scientific journals publish are corrections to previous publications. In religious publications, no, they publish again and again the same teachings. But in academic journals, in history or physics or medicine, they never publish the same thing twice. The only thing they publish is corrections, either to pass mistakes or past lacuna. If there is something in the theory of Newton, which is incomplete or mistaken, then they will publish Einstein's correction to Newton's physics. Every large scale human system is based on an unlikely marriage between mythology and bureaucracy. If you think about a country for instance, so mythology explains the rationale, why should the country even exist? Every country to convince its own citizens why it should exist, tells them some kind of national or religious mythology like we are God's chosen people and we have some very special role here on earth. So this is the mythology part. It gives the motivation, the inspiration, the reason, but then to actually have a functioning country, it's not enough if the citizens believe in the mythology that they are God's chosen people with some mission on earth. You also need to build roads, and hospitals, and armies, and sewage systems. You know, no large scale city, at least a modern city if you want to avoid epidemics can function without a sewage system. And in order to build a sewage system, so you need a lot of workers and engineers and you need to pay them. So you need to collect taxes from the citizens in order to build a sewage system. So again, here mythology comes back into the picture, the mythology encourages people or explains to people why they should pay their taxes honestly, so that other citizens in our country will enjoy good healthcare services and a good sew system that protects us from cholera and so forth. Again, when I talk about national mythologies, so it should be clear that there is nothing wrong about it. Nationalism and patriotism have been one of the most, one of the best inventions in human history. Most other social mammals, all other social mammals actually, they care only about a small circle of animals that they know personally they have intimate connection with. And this was also true for our human ancestors hundreds of thousands of years ago. The miracle of nationalism and patriotism is that it makes us scare about millions of strangers that we have never met in our lives. And again, nationalism is not about hating foreigners and wanting to kill the others. It's about loving our compatriots and showing this love, for instance, by paying taxes honestly, so that other people in the country will be defended against cholera by building a sewage system. (graphics whooshing) - [Announcer] Chapter five, information isn't truth.

信息并非真相:真相的稀缺性与权力博弈

最大的误解之一是认为信息就是真相。赫拉利强调,真相是一种稀缺、昂贵且需要投入大量精力才能获得的信息。例如,关于耶稣的数亿幅画像,没有一幅是真实的,因为我们对他长相一无所知。创作虚假信息很容易,只需凭空想象;而描绘真实情况则需要大量研究和证据。

如果世界充斥着信息,而我们不建立投资于真相的机构,真相就会被淹没在虚假、幻想和垃圾信息之中。大多数信息的目标不是传播真相,而是通过构建虚构的神话或意识形态来获取权力。例如,要建立苏联这样的庞大体系,需要少量真相和大量虚构。

在信息流动方面,极权主义和民主制度存在根本差异。极权主义信息网络是中心化的,决策集中,缺乏自我纠错机制,容易因信息过载而做出错误决策并导致系统崩溃。而民主制度是分布式信息网络,决策分散,拥有大量自我纠错机制。

然而,AI可能为21世纪的极权主义制度带来优势。AI能高效处理海量信息,而人类会因信息过载而崩溃。AI的强大处理能力可能使极权系统在信息处理上更优越。但极权系统缺乏自我纠错机制的根本问题依然存在,若AI犯错且无纠正机制,后果将是灾难性的。

对于民主而言,AI也带来了挑战。民主的核心是人与人之间的对话。当大量AI机器人混入对话,且难以区分真假时,民主对话就会被算法劫持而崩溃。赫拉利呼吁禁止“假人类”(fake humans)和机器人(bots)在公开对话中冒充人类,并强调AI应明确表明身份。

要获取真相,必须投资于学术研究机构和新闻媒体等致力于探寻真相的组织。个体层面,赫拉利建议进行“信息节食”,像对待食物一样谨慎选择喂养我们大脑的信息,避免被贪婪、仇恨和恐惧驱动的垃圾信息所淹没,以保持心灵的健康。

View/Hide Original English

- The biggest misconception about information is that information is truth and information isn't truth. Most information is not truth. The truth is a very rare and costly and expensive type of information. If you want truth, you need to invest a lot in getting it. I'll give an example, historical example. Let's think about images and portraits. What is the most common portrait in the world? What is the most famous face in the world? It is Jesus. There over the last 2000 years, people have created billions and billions of portraits of Jesus. And they are hung in countless churches and cathedrals and private homes and so forth. And not a single one of them is an authentic depiction of Jesus. They are all, 100% of them are fictional depictions because we have no idea how Jesus looked like, there is not a single portrait made during his lifetime. There is not a single sentence in the Bible that tells us whether he was fat or thin, tall or short, black hair, blonde hair, nothing. So all these images, they're fiction. And it's very easy to create fictional information because you don't need to research anything. You don't need evidence. You just come up with something and draw it. If you want to paint a truthful picture of anything, of a person, of an economy, of a war, you need to invest a lot of time and effort and money to research to make sure that you get it right. So if we just flood the world with information and expect the truth to float up, it'll not, it'll sink. The more we flood the world with information, unless we make the effort to construct institutions that invest in truth, we'll be flooded by fiction and illusion and delusion and junk information. So most information is not truth and most information, what it tries to do is gain power by creating order, not by spreading the truth. If you want millions of people to cooperating on something, the easiest way to do it is to create some fictional mythology or ideology and convince a lot of people to believe in it. And the way to do it is to bombard them with more and more stories and images and so forth of your favorite mythology or ideology. And this is how you gain power. And you need to know some truth. Again, a system that is completely oblivious to truth, it'll collapse of course, but in this balance, how much truth do you need in order to construct the Soviet Union and how much fiction and delusions do you need in order to construct the Soviet Union? You need a little truth and a lot of fiction. And this is true of most of the large scale political systems that existed throughout human history, we tend to think about totalitarianism and democracy as different ethical systems, but they are different information networks. Information flows differently in totalitarian versus democratic networks. Totalitarian networks are centralized, all the information flows to just one place where all the decisions are being made, and they lack strong self-correcting mechanisms. There is no mechanism in the Soviet Union to identify and correct the mistakes of Stalin. Democracies in contrast, they are distributed information networks with lots of self-correcting mechanisms. The decisions in the United States are not made only in Washington. Just a small part of all decisions are made there. Most decisions are made by private businesses, and voluntary associations, and individuals, and so forth. And there are lots of mechanisms to correct the mistakes even of the most powerful politicians and corporations. So this is the key difference in terms of information flow between totalitarianism and democracy. In the 20th century, totalitarian systems work worse than democratic systems. When all the information flowed to just one place, just to Moscow, the people there were overwhelmed by all the flood of information and they could not make the right decisions and there was no mechanism to correct their mistakes. And eventually the system collapsed. A distributed information system was much better when the decision makers were human beings. But AI could give an advantage to totalitarian systems in the 21st century, why? Because AI can process enormous amount of information much faster and more efficiently than any communist bureaucrat. When you flood a human with too much information, the human collapses. When you flood an AI with information, the AI becomes better. So there is a scenario that it's not deterministic, it's not certain, but there is a scenario that totalitarian systems will become better in the 21st century because of AI. Still the other problem of totalitarian systems that they have no self-correcting mechanisms. This is still applicable even in the age of AI. It makes it even more dangerous. A totalitarian system relying on AI that the AI makes a mistake. AIs are fallible, AIs are not God. They make mistakes. If you give all the power to a totalitarian AI and you have no way to correct its mistakes, this could prove catastrophic to the entire human civilization. For human dictators, AI is an especially big problem because for an AI to take power in a dictatorship is much, much easier than to kidnap power in a democracy. Because all power in a dictatorship is already concentrated in the hands of just one paranoid leader. The AI needs to learn how to manipulate just this single individual in order to take power in the country. So the danger of AI taking power in a country are much bigger in dictatorships than in democracy. In democracy, a big problem is very different. Democracy is a conversation. The the whole meaning for democracy is that you have large numbers of people conversing about the issues of the day. Now imagine a large group of people standing in a circle and talking, and suddenly a group of robots entering the circle and start talking very loudly and very emotionally, and persuasively and you can't tell the difference who is a human and who is a robot. That is a situation we are now living through, and it is no coincidence that the democratic conversation is breaking down all over the world because the algorithms are hijacking it. We have the most sophisticated information technology in history and we are losing the ability to talk with each other to hold a reasoned conversation. In order to protect the conversation between people, we need to ban bots from the conversation. We need to ban fake humans. AIs should be welcome to talk with us only if they identify as AIs. If you talk online with someone and you don't know whether it's an AI or a human, this will destroy the democratic conversation. So we need to ban that. If we want to ensure that we get the truth. The only way to do it is to invest in institutions like academic research institutions, like newspapers that invest a lot of effort in finding the truth. If we just expect that a flood of information will bring us the truth, it'll not. It'll overwhelm the rare and costly kind of information, which is truth, by a deluge of fake and junk information. And as individuals, my best recommendation is to go on an information diet, the same way that people go on food diets. Information is the food of the mind. We have learned that it's not good to our body to eat too much food or too much junk food. So lots of people are very mindful what they feed their body. We should be equally mindful about what we feed our mind. More information isn't always good for you, it's actually good from time to time, take time for information fasts. When we don't put anything more in, we just digest and detoxify. And similarly, we should watch the quality of the information we feed our mind. If we feed our mind with all this junk information full of greed, and hate, and fear, we will have sick minds. (upbeat music) - [Announcer] Wanna support the channel. Join the big think members community where you get access to videos early ad free.

📌 文中提及的人物和组织

人物: Yuval Noah Harari, Lee Sedol

公司/组织: Google, KGB

产品/模型: AI, AlphaGo

媒体/书籍: Nexus, The Bible, The US Constitution