原生架构：重塑音频感知维度
在当前的人工智能技术前沿，Google DeepMind 正在推动音频处理范式的根本性转变。传统的系统往往依赖复杂的级联管道来处理声音，而新一代的架构选择将音频理解能力（Audio Understanding: 允许模型直接处理并解析原始声音信号特征的底层架构）深度烘焙进基础大模型之中。我们最新发布的端侧模型 Gemma 4 已经原生支持了这一能力，允许开发者在边缘设备上实现高效的声音解析。与此同时，在生成式多媒体领域，Video 3.1 Light 拓宽了视频生成的边界，而核心焦点则落在 Gemini 3.1 Flash Life 身上。这是一个真正的全双工（Full-duplex）、端到端（Sound-to-sound）的实时多模态对话模型，它摒弃了过往由于文本中转而流失的声学细节，能够精准捕捉对话中的情绪、语速以及微妙的上下文变化。
这种底层架构的跃升，直接带来了业务层面的降本增效与能力释放。通过 Google AI Studio 构建的 Echo Script 演示完美证实了这一点：开发者只需向 Gemini 3 Flash 预览版 发送单次结构化 API 请求，就能完成以往需要多套系统协同的任务。系统不仅能提取精准的时间戳、通过声纹识别出特定的发言人，更能无缝处理多人叠音的复杂场景。在多语言处理上，当测试者用德语、法语、日语或普通话发言时，系统不仅提供了完美的英文翻译，还精准剥离出了隐藏在语音中的情绪特征。例如，系统成功将一段内容极度欢快的德语识别为“高兴”，而不是被特定语言的刻板音调误导为“愤怒”。
Original English Source
All right. What's new in AI audio? ... I joined the team in November, literally the day before Gemini 3 was released... most recently on the open model side, we released Gemma 4... there's audio understanding in the Gemma 4 models and you can do that on device on kind of edge devices as well... most recently with video 3.1 light on the gen media model size and then on the audio models, we recently launched Gemini 3.1 flash life, which is our kind of full duplex, you know, sound to sound real-time conversational model. Also multimodal so you can ingest real-time text, voice, vision... Gemini 3 is incredibly good at understanding audio. And you know, that's not just transcribing it but really you know, understanding kind of all the nuances that are in there... our goal is to build models that deeply comprehend, richly transcribe and robustly reason through audio. Seamlessly handling you know, a large mix of different languages, dialects, accents, and modalities... transcribing even people that are talking over each other... Echo Script is kind of Gemini 3 flash preview to sort of analyze, you know, audio recordings and extract sort of information out of it. Uh it is built with Google AI Studio... we can extract a lot of information out of the audio within one single, you know, request to the model... we're extracting timestamps, uh we're labeling the speaker, kind of identifying the speaker, we're identifying the language, and sort of the emotion of it... happy, you know, to introduce myself as Grace. Um now you can see this was in German. Um normally it would classify my German as angry. Uh but, uh here, you know, I I guess I'm very happy...导演指令：降维构建语音生成
在建立起强大的音频解析基础后，我们将这种深度感知能力反向应用到了语音生成领域，彻底颠覆了传统的实现路径。传统的文本转语音系统（TTS: 依赖预先录制并分类的大型音频数据库生成声音的技术）通常需要用户在海量的声音库中按照性别、语言和口音进行繁杂的搜索与过滤。而 Gemini 采用了一种极为创新的策略：系统中仅预置了约 30 种基础声音，但赋予了开发者通过系统提示词（System Prompt: 引导和约束 AI 模型输出特性的底层控制指令）来直接“导演”这些声音的能力。
具体而言，开发者只需输入类似电影剧本的导演笔记（Director's Note），描述出特定的声学情境和角色情绪。例如，在演示中我们设定了“在克莱尔郡海岸拥挤的酒吧里，用浓重的爱尔兰口音”发音，系统立刻将标准的美式基础声音转化为了极具沉浸感的爱尔兰腔调；在另一个案例中，通过输入新加坡小贩中心的背景设定，系统完美还原了带有浓烈地方特色的“Singlish”口音，流畅地推销着海南鸡饭。这种架构让模型直接在潜空间中理解声场和口音的本质，从而用极低的存储与训练成本，实现了无限延伸的个性化声音生成。
Original English Source
And so, with speech generation it's a bit different. You know, if you've used kind of other TTS providers before, you probably have a huge library of, you know, different voices that you sort of, you know, you filter by gender, by by, you know, accent, by languages, what have you. But so, in in in Gemini, you have, you know, just I think it's like 30-ish sort of base voices. And then what you do is you you kind of direct that voice to act in a certain way. And again, because we have that audio understanding, we can we can basically modify the voice to, you know, act in a certain way, to, you know, use a certain accent. And so, we can go from kind of a small set of base voices to a very specific, you know, kind of voice that we're looking for... we're building sort of the the audio profile, the scene, we're setting the scene, we're instructing sort of this director's note... deliver the lines with a strong authentic uh, Irish accent... Ah, you wouldn't believe the size of the thing until you saw it with your own two eyes... give it kind of a Singaporean sort of C. Wah, you must try this chicken rice lah. The chili is damn sure confirm plus chop you will love it. Faster queue before the uncle close shop, okay? ... The model really understands what, you know, these these different scenarios sound like and then can modify the speech generation to to be like that.零延迟交互：智能的端到端闭环
在前两项技术的底层逻辑支撑下，我们最终迎来了完全原生的多模态交互平台。借助 Gemini 3.1 Flash Life，开发者现在可以通过网页套接字（Web Socket: 提供全双工、低延迟双向数据交换的网络通信协议）以最高每秒一帧的速率，向模型实时输入文本流、音频流乃至连续的屏幕视频帧。这里最核心的技术突破在于，逻辑推理机制（Reasoning: 模型进行逻辑分析与问题求解的过程）已经被完全融合进音频模型架构的深处，它彻底抛弃了必须先将语音转录为文本才能调用大语言模型思考的过时级联机制。这意味着模型能以类似人类神经反射的超低延迟，直接给出带有自然语音节奏的情感音频响应。
在这个高速多模态基础设施之上，各类创意工具的集成也变得前所未有的顺畅。我们新推出的 Lyra 3 音乐生成系统被直接引入了实时链路中，该系统包含专供生成 30 秒片段的 Lyra 3 Clip 和能生成带主唱歌词全长曲目的 Lyra 3 Pro。为了验证系统潜能，我们构建了 Life Jukebox 演示应用：在这个场景下，Gemini 作为一个具备听觉和视觉感知能力的 AI 电台 DJ，能够与用户实时自然对话。当用户提出“创作一首关于英国创业圈的带有狂热能量的德国电子 Schlager 音乐”的复杂需求时，AI 不仅能顺畅接茬，还自动调用了后端的 Lyra 3 工具生成了真正的定制化音乐作品。这标志着人工智能从被动的文本处理者，正式跃迁为具备深度跨模态解析与实时创造力的智能实体。
Original English Source
And then, you know, finally sort of the the native audio sound to sound multimodal real time. So, we just launched a couple weeks ago Gemini 3.1 Flashlight. So, it is a speech to speech is kind of real-time multimodal model. You can ingest text, audio, video in real time through a web socket connection and then you get real-time audio response back... the thinking and the reasoning and the intelligence is baked directly into the model. So, that's, you know, different from a cascading pipeline where you would actually go through text to then go through an LLM to get the intelligence... You could also ingest your screen, so you're basically just ingesting video frames in addition to the audio at a maximum frame rate of one frame per second... Uh we recently released Lyra 3. So, yeah, it's it's a music generation model. Um but so, it it now actually can generate music with lyrics. There's two separate models. There's a Lyra 3 clip which is a 30-second kind of jingle generation model and then a Lyra 3 Pro is the full-length uh song generation model... I've kind of built this application um called Life Jukebox... we actually give the the real-time Gemini Life model uh a tool to then generate a song using Lyra... we get maybe a German techno Schlager about the UK startup scene? ... Manic German techno Schlager with a British startup twist. Cooking up a proper banger for you.📌 文中提及的人物和组织
公司/组织: Google DeepMind
产品/模型: Gemini 3.1, Gemma 4, Gemini 3 Flash, Lyra 3, Google AI Studio