World Labs 收购机器人初创公司 Scenix 与推进空间智能

我们正在构建 AI 的下一个前沿，也就是我们所说的空间智能。

英文原文

We

are

building

the

next

frontier

of

AI

which

is

what

we

call

spatial

intelligence.

作为 Scenix，我们正在开发一种我们称之为“real-to-sim-to-real”的管线。我们可以通过在数字世界中以可扩展的方式生成数据，来替代我们在真实环境中需要的所有数据、所有评估。

英文原文

>> As

Cynics,

we

are

developing

what

we

call

a

real

to

see

to

real

pipeline.

We

can

replace

all

the

data,

all

the

evaluation

we

need

in

the

real

environment

by

using

the

data

that

can

generate

at

a

scalable

way

in

our

digital

world.

想想人类的智能。我们脑海中会进行大量的模拟。为什么？模拟扮演着一个非常重要的角色，而真实世界的数据却无法扮演，那就是反事实推理。

英文原文

>> Think

about

human

intelligence.

We

do

a

lot

of

simulation

in

our

head.

Why?

There's

a

very

important

role

simulation

plays

that

real

world

data

doesn't

play

which

is

counterfactual.

Reason

我们正在构建的是一个一致的世界，在空间上、在时间上、在不同视角上以及在不同交互类型上都保持一致。我的北极星指标就是我希望机器人能够运转工作。

英文原文

>> what

we

are

building

is

a

consistent

world

consistent

both

over

space

over

time

over

different

viewpoints

and

over

different

type

of

interactions.

My

north

star

is

I

want

the

robot

to

work.

我们生活的世界可以是多元宇宙，我们创造技术来让人们、构建者、开发者能够在不同的空间内行动。你认为我们能否制造出具备像人类那样本领功耗效率的机器人吗？我们离这还有多远？是大概5年，还是遥遥无期？

英文原文

>> The

world

we

live

in

can

be

multiverse

that

we

create

technology

to

allow

people

builders

developers

to

act

within

different

spaces.

Do

you

believe

we'll

ever

be

able

to

build

robots

that

have

the

power

efficiency

of

a

human

being?

How

far

away

are

we

from

this?

Is

this

like

5

years

or

this

is

like

never?

核心要点就是……

英文原文

>> The

TLDDR

is

World Labs 与空间智能

好的。非常高兴能邀请到你们两位。Fei-Fei，对于那些可能不了解背景的听众，也许你可以概述一下 World Labs 是做什么的。

英文原文

>> All

right.

Well,

it's

great

to

have

you

both

here.

So,

Feay,

for

the

listeners

that

may

not

have

the

background,

maybe

you

can

give

an

overview

of

what

World

Labs

does.

好的。World Labs 是一家成立两年的初创公司。我认为我们应该认识到它是一个前沿模型实验室。我们正在构建 AI 的下一个前沿，也就是我们所说的空间智能。空间智能关乎创造具有生成、理解、推理以及与空间进行交互能力的 AI，无论这空间是物理的还是虚拟的。当然，通往空间智能的一种手段是构建大型世界模型。而这正是 World Labs 主要关注的方向。

英文原文

>> Yeah.

Well,

World

Lab

has

is

a

two-year-old

startup.

I

I

I

think

we

should

just

recognize

it's

a

frontier

model

lab.

It's

uh

we

we

are

building

the

next

frontier

of

AI

which

is

what

we

call

spatial

intelligence.

And

uh

spatial

intelligence

is

about

um

creating

AI

that

has

the

ability

to

generate

uh

understand

reason

with

and

interact

with

spaces

whether

it's

physical

or

virtual.

And

of

course

uh

a

means

to

an

end

towards

spatial

intelligence

is

building

large

world

models.

And

that's

what

uh

World

Labs

is

mostly

focused

on.

是的。从一开始你就一直在说这一点，也就是机器感知、推理空间并在空间中行动的能力。但我过去一直以为在空间中行动是某种比较长远的未来的事情，但现在你们正在收购一家机器人公司，所以也许可以聊聊这件事的时机和意图。

英文原文

>> Yeah.

So

you've

been

saying

this

since

the

very

beginning

which

is

um

you

know

the

machine's

ability

to

perceive

and

reason

about

spaces

and

act

on

spaces

but

I

always

had

the

assumption

that

the

acting

on

spaces

was

some

like

longist

future

thing

but

now

you're

acquiring

a

robotics

company

and

so

maybe

talk

a

little

bit

about

the

timeliness

of

this

and

the

intentions.

好的。首先，要在空间内行动或交互，并不仅限于机器人，对吧？我是说，看看创意领域，不管是视觉特效、游戏还是设计。在许多用例中，你都可以在虚拟空间中进行创造和行动。World Labs 的论点一直都是：我们生活的世界可以是多元宇宙，我们创造技术来让人们、构建者、开发者能够在不同的空间内行动。

英文原文

>> Yeah.

So

first

of

all,

it

doesn't

just

take

robotics

to

act

within

spaces

or

to

interact,

right?

I

mean,

look

at

the

creative

field,

whether

it's

VFX

or

gaming

and

or

design.

Many

use

cases,

you

can

create

and

act

within

virtual

spaces.

World

Warlab's

thesis

has

always

been

that

um

the

the

the

the

world

we

live

in

can

be

multiverse

that

we

create

technology

to

allow

people

builders

developers

to

act

within

different

spaces.

话虽如此，在物理空间内行动的能力，是未来 AI 世界最令人兴奋、最极其重要的能力之一。机器人技术正是如此。因此 World Labs 一直认为机器人技术是空间智能和世界建模的一个重要应用及用例。所以，通过联合力量，邀请 Scenix 团队加入 World Labs，是我们长期愿景和使命的一部分。我们一直致力于此。太棒了。那么，Shuran，你是 Scenix 的联合创始人。也许可以向大家快速介绍一下你的背景以及 Scenix 是做什么的。

英文原文

>> Having

said

that

the

ability

to

act

within

the

physical

space

is

one

of

the

most

exciting

and

most

profoundly

important

capability

of

the

future

AI

world.

So

robotics

is

very

much

that.

So

worldlap

has

always

believed

that

robotics

is

a

important

application

as

well

as

use

case

of

uh

spatial

intelligence

and

world

modeling.

So

by

joining

force

with

uh

inviting

scen

team

to

world

labs

is

part

of

our

long-term

vision

and

uh

mission.

We

we've

always

committed

to

that.

Amazing.

So,

Yunu,

you're

the

co-founder

of

Scenix.

So,

maybe

provide

everyone

with

a

quick

um

overview

of

your

background

and

what

Scenix

does.

Scenix 与 Real-to-Sim-to-Real

好的。我是 Shuran。目前是 Scenix 的联合创始人，同时也是哥伦比亚大学的助理教授。哇。

英文原文

>> Yeah.

So,

I'm

Yunu.

So,

I'm

currently

co-founder

of

Cynics

and

also

assistant

professor

at

Columbia

University.

Wow.

我的研究始于我在麻省理工学院的博士阶段，然后是在斯坦福大学与 Fei-Fei 一起做博士后。

英文原文

>> So,

my

research

started

from

my

PhD

at

MIT

and

then

postto

with

Fay

really

at

the

Stanford

University.

太棒了。

英文原文

>> That's

great.

世界真小。

英文原文

>> The

world

is

small.

世界真小。

英文原文

>> The

world

is

small.

确实是。

英文原文

>> It

is.

在我的整个职业生涯中，我的目标一直非常简单：努力帮助机器人更好地感知物理世界并与之交互。所以我是一个非常务实的人。我希望我的机器人能够在真实的物理环境中运作。

英文原文

>> Throughout

my

career,

my

goal

has

been

very

simple.

trying

to

help

the

robots

better

perceive

and

interact

with

the

physical

world.

So

I'm

a

very

practical

person.

I

want

my

robot

to

work

in

the

real

physical

environments.

因此，对于 Scenix 来说，我们看到的独特机遇是：目前通用机器人的发展面临着很多瓶颈，特别是在训练和评估方面。所以我们正在开发所谓 real-to-sim-to-real 的管线。好的。我们希望将真实环境映射到数字世界中，并与真实环境保持最佳的一致性。我们所说的一致性，是指无论在数字世界中发生什么，也都会在真实环境中发生，

英文原文

>> So

for

cynics

the

unique

opportunity

we

see

is

that

there

has

been

a

lot

of

like

a

bottlenecks.

Right

now

we

see

faced

by

the

developments

of

general

purpose

robots

especially

around

training

and

also

around

evaluations.

So

as

we

are

developing

what

we

call

a

real

to

sim

to

real

pipeline.

Okay.

We

want

to

map

the

real

environments

into

the

digital

world

that

has

the

best

alignments

with

the

real

environments.

By

alignments

we

mean

that

whatever

happens

in

the

digital

world

is

also

going

to

happen

in

the

real

environments

从而使我们能够使用在数字世界中以可扩展方式生成的数据，来替换我们在真实环境中需要的所有数据、所有评估。这就是 Scenix 一切的起源。我们组建了一支围绕机器人、机器人学习以及模拟与渲染的极其顶尖的团队，试图构建这套 real-to-sim-to-real 技术栈来解决一些核心瓶颈。

英文原文

>> such

that

we

can

replace

all

the

data

all

the

evaluation

we

need

in

the

real

environment

by

using

the

data

that

can

generate

at

a

scalable

way

in

our

digital

world.

So

that

is

how

everything

started

in

Synex.

We

put

together

a

very

very

strong

and

best

teams

around

robotics,

robot

learning

and

also

simulation

and

rendering

trying

to

build

this

realtom

real

stack

to

solve

some

of

the

key

bottlenecks.

你们两位能走到一起合作真是太不可思议了。

英文原文

>> It's

amazing

that

you

two

work

together.

从客户到团队融合：Marble 与协同效应

是的。这里面还有一个有趣的故事，因为你可能会以为，既然我们共事过，她是我的杰出的博士后，那我们肯定是商量这个和 World Labs 的整合商量了很久。事实完全不是这样。他们最初是以客户的身份来到 World Labs 的。

英文原文

>> Yeah.

And

uh

there

is

a

funny

story

here

because

you

would

think

because

we

worked

together

he

was

my

amazing

postto

we've

been

talking

about

this

and

world

lab

um

um

integration

for

a

long

time.

It's

actually

not

true.

They

came

into

World Wars

as

a

customer.

真的吗？

英文原文

>> Really?

当我们在去年冬天大概11月、12月发布我们名为 Marble 的第一版生成模型时，

英文原文

>> When

we

when

we

released

the

first

version

of

our

generative

model

called

Marble

last

winter

around

uh

November,

December,

Scenix 就直接注册了。

英文原文

>> Cynics

just

signed

up.

没开玩笑吧。作为客户。

英文原文

>> No

kidding.

AS

A

CUSTOMER.

是的。我当时甚至都不知道那是啥。后来我意识到这是 Shuran 的公司。我给 Shuran 打电话。我说：“哇，这是你的公司啊。”你们……然后我们意识到这里有巨大的协同效应。

英文原文

>> YES.

And

I

didn't

even

know

what

it

was.

And

then

I

realized

this

is

Vindrew's

company.

I

called

Vindro.

I'm

like,

"Wow,

this

is

your

company."

and

you

guys

and

and

then

we

realized

there's

so

much

synergy.

也许 Fei-Fei 可以简要描述一下 Marble 是什么。

英文原文

>> Maybe

Faith

could

just

quickly

describe

what

Marble

is.

好的，Marble 是 World Labs 一直在训练和迭代的基础模型的代号，目前公开发布的 Marble 的核心能力是接收一个提示词——可以是一张图像、几张图像，或者是文本——并将其转化为一个几何上一致的世界，可以用 3D 几何（无论是高斯泼溅还是网格）来表示。Scenix 团队真正在做的是努力解决机器人技术中极其困难的问题，即数据的匮乏。嗯。

英文原文

>> Yeah,

Marble

is

the

code

name

for

the

base

model

that

World War

has

been

training

and

iterating

on

that

the

fundamental

capability

right

now

of

Marble

that

is

publicly

released

is

to

take

a

a

prompt.

It

can

be

an

image,

it

can

be

a

few

images

and

uh

or

a

text

and

turn

that

into

a

geometrically

consistent

world

that

can

be

represented

in

3D

geometry

whether

it's

gausian

splat

or

mesh.

Really

what

Synix

team

is

doing

is

trying

to

solve

this

extremely

difficult

problem

in

robotics

which

is

the

lack

of

data.

M

训练数据的匮乏，评估数据的匮乏。这与互联网上数据极其丰富的语言模型非常非常不同。

英文原文

>> the

lack

of

data

in

training,

the

lack

of

data

in

uh

evaluation.

This

is

very

very

different

from

language

models

where

data

is

abundant

on

the

internet.

而且我们知道，为了让机器人技术能够走通，我们必须以某种方式解锁扩展定律（scaling law）的威力。但这从何而来？这是机器人领域每个人都在与之苦战的深刻问题。聊聊这种协同效应其实挺好的，比如你组建了一支非常有才华的团队。你也组建了一支非常有才华的团队，那么重合的部分有多少？延伸补充的部分又有多少？也许可以多讲讲这方面。是的，其实就是看互补性有多强。

英文原文

>> And

we

know

that

um

in

order

for

robotics

to

work,

we

have

to

somehow

unlock

the

power

of

scaling

law.

But

where

does

that

come

from?

This

is

something

that

that

is

a

profound

problem

that

everybody's

battling

with

in

in

robotics.

It'd

actually

be

great

to

talk

about

this

energy

like

you

have

put

together

a

very

very

talented

team.

You

have

put

together

a

very

talented

team

and

so

like

to

what

extent

is

there

overlap

to

what

extent

is

this

an

extension?

Maybe

talk

a

little

bit

about

that.

Yeah,

that's

actually

like

how

complimentary

it

is.

实际上，简而言之就是高度互补且使命相同。Shuran 是三位技术联合创始人之一。另外两位是 Changxi Zheng，哥伦比亚大学的另一位教授，也是模拟领域世界级的技术专家。哇。Changxi 在视觉特效（VFX）领域也有深厚的背景。他曾在维塔（Weta）工作过，在腾讯工作过，也是个创业者。另外还有一位是工程负责人，他曾在多年前被亚马逊收购的一家初创公司工作过。因此他在亚马逊计算机视觉领域的许多不同技术栈中工作过。所以当我们在更认真地讨论时，我意识到，从人才的角度来看，Scenix 具备的几点对 World Labs 来说是极其互补的。其一显然是机器人技术领域令人难以置信的思想领导力以及纯粹的技术实力，对吧？真正涵盖了从硬件到全栈机器人技术。甚至当年他在斯坦福做我的博士后时，那时你已经拿到了教职 offer，所以你只在那待了一年。我原本想留你更久的，但他不得不去任职正式工作了。所以他是机器人领域从建模到硬件的全栈研究员。当然，Shuran 以及他在 Scenix 的学生们构成了 World Labs 此前尚未拥有的人才库。而在 Changxi 这边，则是极其强大的模拟能力，对吧？他是模拟领域非常资深的研究人员和技术专家，而 World Labs 正在做的事情很大程度上正是与模拟世界相接。因此，我认为他们显然不具备的，是在生成模型方面以及计算机视觉 3D 重建方面，而我们在这些方面也非常……

英文原文

>> It's

it's

actually

the

the

TLDDR

is

is

very

complimentary

and

with

the

shared

mission.

So

is

one

of

the

three

uh

technical

co-founders.

The

other

two

are

Changi

Jan,

another

Colombia

professor

who

has

been

a

world-class

technologist

in

simulation.

Wow.

And

Changi

has

his

background

in

also

um

VFX.

He

worked

at

Weta,

he

worked

at

Tencent,

he's

being

an

entrepreneur.

Uh

then

there's

Sunonni

who

is

a

phenomenal

engineering

leader

who

was

also

in

a

uh

startup

uh

that

was

acquired

by

Amazon

many

years

ago.

So

he

worked

in

many

different

tech

stacks

in

the

computer

vision

field

in

uh

in

Amazon.

So

when

when

we

started

talking

more

seriously

I

recognized

that

uh

a

couple

of

things

that

Phoenix

has

from

a

talent

point

of

view

is

extremely

complimentary

to

to

uh

worldaps.

one

is

obviously

incredible

um

uh

thought

leadership

and

and

just

technical

prowess

in

robotics

right

so

uh

from

really

from

hardware

full

stack

robotics

and

even

when

he

was

my

posttock

at

Stanford

at

that

time

you

already

had

your

faculty

offer

so

you

were

there

only

for

one

year

I

wanted

you

for

more

than

one

year

but

he

had

to

go

become

a

have

the

real

job

so

uh

he

was

a

full

stack

researcher

in

in

robotics

from

modeling

to

to

hardware.

Uh

and

of

course

uh

VR

and

his

student

uh

students

at

Synenix

was

that

pool

of

talent

world

hasn't

had

yet.

Then

on

the

Changi

side

is

just

incredible

simulation

um

um

capability,

right?

he's

such

a

senior

um

researcher

and

technologist

in

simulation

and

and

uh

what

world

labs

is

doing

is

very

much

um

interfacing

the

world

of

simulation.

So,

so

I

think

what

they

don't

have

um

obviously

is

on

the

generative

model

side

as

well

as

the

computer

vision

3D

reconstruction

side

we're

also

very

加入 World Labs 与技术互补

而在 World Labs 这方面实力雄厚。所以这是 Scenics 所需要的技术。于是这两方走到了一起，让整体变得更加完整。从 Fei 的动机来看，这是一种延伸与互补，借此进军机器人领域。考虑到你曾处于决定何时出售公司的处境，很想听听你是如何看待加入 World Labs、双方的契合度，以及你为何决定这么做的。

是的。最开始我们想的是就继续自己做下去，但在与 Fei 交流之后，看到在中间产生的诸多协同效应，双方力量的联合就变得完全顺理成章了。从某种意义上说，作为 Scenics，我们一直在做的就是虚实结合（real to sim to real），也就是对环境进行这种重建。我们捕捉环境的外观、环境的几何结构，以及环境的动力学特性——也就是当你施加动作时环境将如何变化。这种密集重建（tensor/dense reconstruction）目前依然偏重一些，而 World Labs 目前所做的工作涉及在稀疏重建与生成方面的许多深厚能力。因此，我们看到了很多机会，可以利用像 Marble 以及 World Labs 的其他能力，来进行非常高效的环境重建与建模。

英文原文

strong at world labs. So that's a technology that scenics needs. So together these uh these two sides come together and make it much more complete.

Fe's motivation in this as like this is an extension and a compliment to get into robotics you know having been in your situation which is deciding when to sell a company it would be great to hear from you on like how you think about joining world labs and kind of the fit there and like why you made the decision to do it.

Yeah. So at the very beginning we were deciding okay we want to just keep going but after chatting with F after seeing all the synergies that happening in the middle it just makes perfect sense for the forces to join each other. So in any sense as cynics what we have been doing is real to seem to real is to do this reconstruction of the environment. So we capture the appearance of the environment geometry of the environments and also the dynamics of the environment meaning how the environment is going to change when you apply actions. So this tensor reconstruction right now is still a little bit on the heavier side and what world labs right now has been doing involves a lot of profound capabilities around sparse reconstruction and generations. So we see a lot of opportunities of leveraging like a marble and other like capabilities as world labs in order to do very efficient reconstructions and modeling of the environments.

机器人基础模型与多模态

那么我们可以期待 World Labs 推出面向机器人的基础模型吗？

英文原文

>> So can we expect a foundation model for robotics from world labs?

World Labs 正在构建基础模型。正如你所知，Martin，我们正在构建一个底座模型，并且随着技术的演进，一些最令人兴奋的底座模型是全模态模型（omni models），对吧？它们接收多模态输入，输出多模态结果。那么什么是面向机器人的基础模型呢？它很可能会涉及动作。

英文原文

World Lab is building a foundation model. As you know, Martin, we're building a base model and uh as the technology has been evolving, some of the most exciting base models are omniodels, right? They take they take multimodal input, they have multimodal outputs. And uh what is a foundation model for robotics? Uh it's very likely going to involve actions. M

很可能会涉及在世界状态之外输出动作，我们绝对不排除这种可能性。

英文原文

>> it's very likely going to involve the output of actions in addition to the state of the world and we're definitely not ruling this out.

好的，太棒了。

英文原文

>> Yeah. Great.

例如，对于基础模型而言，它本质上需要是一个多模态模型。所以它必须将帧、文本、图像、深度以及不同种类的模态考虑在内，而动作是这些模态中非常非常重要的组成部分。如果你把帧和动作视为输入，那本质上就是一个前向模拟器，用来预测当你施加特定动作时环境将如何改变。当动作作为输出时，这本质上是一个策略模型，试图预测在给定特定目标的情况下，你在真实环境中应该采取什么动作来让你更接近该目标。因此，这种全模态模型实际上可以带来很多好处，并为机器人学界在尝试理解如何对环境进行建模、同时如何在环境中行动方面提供巨大的价值；这也可以作为一个骨干网络，供你微调到具体的机器人应用中，以确保它真正达到客户所期望的可靠性与效率。

英文原文

>> So for example for the foundation models it's essentially needs to be a multimodal model. So it has to take into account frame text image deps and different kind of modalities and action is a very very important parts of that modalities. So if you think about frame actions as inputs that is essentially a forward simulator that is going to predict how the environment is going to change when you apply a specific action. When the action is output this is essentially a policy model that is trying to predict given a specific goal like what should be the action you take in the real environment to get you closer to that goal. So this kind of omni models actually can benefit a lot and actually provide huge amount of values for the robotics communities in trying to understand how to model the environments and at the same time how to act in the environments and this can also acts as a backbone for you to fine-tune into specific robotic applications to making sure it's really leave up to the reliability and efficiency that expected by the clients.

纯视频模型与 3D 模拟路线的对比

Yun，如果你不介意一个外行投资人的提问的话，我见过很多机器人公司，目前对于找过来的机器人公司来说，一种非常流行的方法就是“我们会使用视频模型”，那是主流方法；而这里采用的是 3D 和仿真，这是一种截然不同的方法。所以或许你可以对比一下这种仅使用视频的流行方法与这里的雄心之间有何不同。

英文原文

you know if uh Yun if you don't uh if you don't mind a kind of a lay investor question I see a lot of robotics companies and a very popular approach right now for the robotics companies that come in is like we'll use a video model you know and like you know that's the the predominant method where this is you know 3D and simulation it's a very different approach and so maybe you could contrast you know this popular approach of just using video only versus kind of what the ambition here is

是的。为了创造能让机器人从中学习的世界（words/worlds），正如我提到的，关键在于捕捉问题的本质结构。而对于这些世界来说，一个非常重要且必要的条件就是一致性。这正是我看到与 Marble 存在极强协同效应的地方，因为我们正在构建的是一个一致的世界——在空间上、在时间上、在不同视点上以及在不同交互类型上都保持一致。而由 Marble 生成的世界，也提供了整个世界的基础设施与组成部分，我们认为这对于机器人学习是必不可少的。

想象一下，如果机器人向前推一个物体，该物体就凭空消失了——这正是许多现有视频预测模型存在的问题。这无法为机器人提供足够好的信号来让它知道正确的做法是什么。但显然，目前有很多关于构建越来越好、越来越强大的视频模型的研究。因此，我们确实看到了一种途径：我们构建的部分基础设施可以提供初始动量，并通过这个数据飞轮——从这种偏仿真的模型走向机器人的策略模型，策略模型在真实环境中执行并收集新数据，数据再回流进来。在这里，模型不一定非得是纯物理驱动或是纯学习驱动，而是介于两者之间，既能够捕捉问题的本质结构，同时又能够随着积累更多的数据而实现扩展并变得越来越好。

英文原文

>> yeah so in order to create words where the robot can learn the words as I mentioned is to capture the essential structure of the problem and one of the very important necessary like requirements for those words will be consistency so that is where I actually see there's very very strong synergies with marble because what we are building is a consistent world consistence both over space over time over different viewpoints and over different type of interactions and marble the generated words from marble is also provides an infrastructure a component of that entire words that we believe is necessary for the robot to learn.

Imagine if a robot push an object forwards. The object just magically disappear which has been a problem of many of the existing like video prediction models. This won't provides good enough signal for the robot to know like what is the right thing to do. But obviously right now there has been a lot of investigation on building better and better and stronger and stronger like a video models. So we actually see a way where some of the infrastructure we build can provide as a initial momentums and to going through this data flywheel of going from this like a more simulationdriven models into like a robot policy models which going to do the execution in the real environment collecting new data the data will come back in where the model doesn't necessarily have to be physics only or learning only but somewhere in the middle which be able to capture the essential structure of the problem but at the same times be able to scale and become better and better as you accumulate more data.

实用主义与真实应用场景

话说回来，我和 Fei 密切共事已经有一段时间了，你一直都有指引着你的北极星，你曾以 3D 以及其他多种方式来阐述它。我只是好奇，对你而言，是否也有类似的哲学层面的北极星，抑或你更偏向务实——就像我一样，“把系统做出来，把事情搞定”？

英文原文

>> You know, um I've worked now fay very closely for a while and and and you've always had this north star which has driven this and you know you you've articulated variously as kind of 3D and and in a number of other ways. And I'm just wondering for you is there also a similar philosophical northstar or you're more the pragmatic like I am like build the system like do the thing.

我的北极星就是让机器人在真实环境中运转起来。

英文原文

>> My northstar is to make robots work.

太棒了。

确实，在真实环境中。我是一个非常务实的人，我希望机器人能派上用场。一件很有意思的事情实际上来自我和 Fei 在我博士后期间的合作，当时我们正在构建这种基准测试。我们实际向公众发放了问卷，询问他们希望机器人为他们做什么。

英文原文

>> Amazing.

Yeah.

in the real environment. I'm a very practical person. I want the robot to work. One interesting thing that's actually coming from my collaborations with FIP during my postto, we are building this kind of benchmark. We actually send out surveys asking the general public what they want their robots to do for them.

是的。

英文原文

>> Yeah.

在我们收集的上千项任务中，有三分之一的任务都是关于清洁的。

英文原文

>> Among the thousand tasks we collected, onethird of the tasks are about clean.

人们就是不喜欢做那些又脏又累的活。而这些正是我们非常希望能确保拥有机器人解决方案来应对的场景。

英文原文

>> People just don't like to do those like a d and dirty tasks. And those are the scenarios we really want to making sure we have robotic solutions to deal with.

我非常喜欢 Scenics 的一点，Martin，尤其是接着你的问题来说，现在有很多机器人公司在构建模型等等。我真正喜欢 Scenics 的一点，在于 Renu 和他的联合创始人对机器人技术有着极其务实的方法。特别是，他们来自学术界，对吧——Sunny 不是，但 Vindra 和 Chani 来自学术界——但他们的第一本能是与真实工业界的工业设计合作伙伴和客户合作，无论是在工业实验室、仓库，还是电子装配领域。

英文原文

>> One thing I really like about Synenix uh Martin especially uh uh continuing your question there's a lot of robotics companies building models and all that. One thing I truly like about Synenix is is Renu and his co-founders have such an incredibly pragmatic approach to robotics. They especially they come from academia right Sunny doesn't but Vindra and Chani come from academia but their first instinct is work with design partners and customers in real industry whether it's is uh labs at um industry labs or or warehouses or um um

电子设备。

英文原文

>> electronics

电子设备组装。这种涉足机器人的方式实在令人耳目一新，这也确实让我对与他们合作感到非常兴奋。

英文原文

>> electronics you know uh assembly that is such a refreshing actually a refreshing way of approaching robotics and that that really tr made me very excited to work with them.

精确度与模型在机器人中的角色

也许这个问题是提给你们的，不过这纯属个人好奇：在我看来，对于机器人技术，你必须做到相当精确。我的意思是，不需要完美，但要相当接近。然而对于创意用例——World Labs 做了很多这类工作——你似乎并不需要那么精确，因为即使有时出错，也可以是一种风格化、特意为之或者怎样。那么从技术角度来看，调和这两者面临的挑战是什么？还是说它们永远无法调和，就像设计空间中永远存在的两个点？

当然，长远来看它们会被调和。而且对环境的建模并不需要做到完美。在机器人技术中，模型并不要求是完美的。

英文原文

>> Maybe this is for you and but I'll just be this is this is personal curiosity which is it seems to me that for robotics you have to be pretty exact. I mean not perfect but pretty close. But for the creative use cases which worldaps has done a lot of you kind of don't need to because you know I mean you know even sometimes like being wrong is stylistic or intentional or whatever. And so from a technical perspective what is the challenge here for reconciling these two things or do they never get reconciled like there always be two points in the design space?

So they will be like reconciled in the in the long terms of course and um modeling of the environments um doesn't have to be perfect. The model doesn't have to be perfect in robotics.

顺便问一句，这纯属好奇，有没有更正式一点的说法来解释“不要求完美”是什么意思？它必须相当接近。

英文原文

>> And by the way is there again this is pure curiosity but is there like um a bit more formal way to say that like what does that mean not to be perfect? It has to be pretty close.

那么让我这样来说吧。例如在所有各类机器人应用的发展过程中，模型一直都是非常重要的基石。是的，如果你纵观现有的所有机器人应用，比如飞机、无人机、Roomba 扫地机器人，甚至四足机器人、双足机器人，模型一直都是让它们真正运转起来并能够从仿真迁移到真实环境的方式。

英文原文

>> So so let me putting this way for example models over the developments of all different kind of robotic applications has been a very important cornerstones. Yeah. If you look at all the existing robotic applications like plane, drones, Roomba or even for quadripad robots, bipedal robots, model has been the way for them to actually work and be able to transfer from simulation to the real environment.

我明白了。

英文原文

>> I see.

但如果你看看那些移动机器人，比如四足机器人，

英文原文

>> But if you look at those locomotion robots like quadriped robots,

双足机器人，它们可以在雪地上行走，可以在灌木丛中行走。但你并不需要拥有一个能够极其精确地模拟所有灌木丛和雪地的模拟器。你需要的是一个能够捕捉问题核心结构、并在数字环境中进行各种不同随机化处理的模拟。这就是我们所追求的目标。因此，基本上通过 Synenix 并与 word labaps 合作，我们正试图探究：除了机器人本身之外，我们究竟需要多高保真度的水平来对这庞大世界建模，才能将模拟环境与数字世界中训练出的机器人系统迁移回真实场景中。

英文原文

bipedal robots, they can walking on snows, they can walking on bushes. But you don't need to have a simulator. They can simulate all the bushes and snows like very precisely. you needs to have a simulation that capture the essential structure of the problem and do a whole different kind of randomizations inside the digital environment. So that is what we're aiming for. So basically with Synenix and together with word labaps we're trying to investigate like what is the level of fidelity we needs to model the mass massive worlds besides the robots I said we'll be able to transfer the robotic systems training the simulated environment digital worlds back into the real scenarios.

作为一名投资者，我听过其他研究人员（比如 Sergey Lavine）说过，模拟最终总是会偏离物理世界，而真实世界的数据收集是绝对关键的。所以，能否请你谈谈以模拟为基石的这种方法（相对于其他某些方法而言）的可行性？

英文原文

As an investor, I've heard other researchers say like Sergey Lavine say simulation will always eventually deviate from the physical world and real world data collection is absolutely critical. And so maybe talk a little bit about like the viability of this approach where simulation is a cornerstone as opposed to some other approach.

模拟与现实世界数据的互补

所以它们之间并不矛盾。如果你思考一下模拟，模拟本质上是在试图预测当你施加动作时环境将如何发生改变，这本质上就是一个世界模型。它不一定非要是纯粹的物理，它可以是物理与学习两者的结合。我们正在收集真实世界的数据，我们也会使用这些真实世界的数据。这只是处于类似数据飞轮的不同阶段。也许在最一开始，我们更加强调、拥有更多的物理规律，以确保我们具备正确的连贯性和正确的结构，来供我们学习这个世界，来供我们训练机器人策略。但随着我们通过数据收集以及与客户的合作积累越来越多样的数据，我们将拥有推动环境建模更加偏向基于学习的数据。因此，这种转变以及这种数据流，实际上正是促成因素，既能汲取物理、几何与连贯性的最佳优势，又能获得来自数据和算力的全部力量与魔力。

英文原文

>> So they don't contradict with each other. So if you think about the simulation, simulation is essentially trying to predict how the environment is going going to change when you apply the actions and this is essentially a model of the worlds. It doesn't necessarily have to be pure physics. It can be a combination between both physics and also learning. We are collecting real world data. We will be using those real world data. It just at different stages of this like a data fly. Maybe at the very beginning we have stronger emphasize on we have more physics to making sure we have the right consistency and right structure for us to learn the uh the the worlds for us to train the robot policies. But as we accumulate more and more data both through data collection and also through the collaboration with our clients we'll have the data that will be moving towards more towards more learning based like modeling of the environments. So this kind of transition and also this kind of data flow is really an enabling factors of both getting the best of both physics and the and and geometry and consistency as well as all the power and magics from the data and compute.

我想在此基础上作些补充，并探讨得稍微带点哲学意味：在有模拟或没有模拟之间，从来不是非此即彼的二元选择。所有这些结合在一起，才能让机器人技术运转起来。想想人类的智能吧，我们在脑海里做着大量的模拟。你知道为什么吗？模拟扮演着一个真实世界数据无法发挥的极其重要的作用，那就是反事实推理——去推演那些尚未发生、不可能发生、或者你在真实世界中缺乏足够数据来促成其发生的事件。而当你推演它时，你便学会了如何在其中行动。人类一直在这么做。我们可能不会，你知道，我们只是，我知道你去了世界杯。

英文原文

>> I want to add to this and be slightly philosophical here is there isn't a a binary choice between simulation or no simulation. All this come um in together um to to make robotics work. Think about human intelligence. We do a lot of simulation in our head. You know why? There's a very important role simulation plays that real world data doesn't play which is counterfactual reasoning is that you play out events that c hasn't happened or cannot happen or you don't have enough data to make it happen in real world. And while you play it out, you learn how to act in it. Humans do this all the time. We probably don't, you know, we just I know you were at World Cups.

我当时在世界。

英文原文

>> I was at the World.

祝贺西班牙夺冠。我确信在每场比赛的规划中都存在模拟，无论它是数字形式的，还是在白板上，亦或其他形式，那种模拟、模拟所扮演的角色就是反事实推理。这在机器人领域非常重要，因为对于这些情况，我们根本不可能拥有足够的真实世界数据。这里有一个现实生活中的例子，自动驾驶汽车行业。Whimo 官方曾表示，他们使用了数十亿小时的模拟。

英文原文

>> Congratulations to Spain winning. I'm sure in the planning of every game there is simulation, whether it's digital or on the on the whiteboard or whatever, that simulation, the role simulation plays is counterfactual reasoning. And that's really important in robotics because we just do not have cannot possibly have enough real world data for that. Here's a real life uh example the the industry of self-driving cars. Whimo has officially said they use billions of hours of simulation

而且实际上 Whimo 更加侧重模拟，而不仅仅是重度依赖真实世界数据。所以这些都是真实的例子，而且正如你所知，Martin，Andrew 也是，汽车是最简单的一种机器人。

英文原文

>> and and actually Whimo is more simulationheavy than just real world data heavy. So these are real examples and and as you know Martin Andrew too cars are the simplest kind of robots.

是的。

英文原文

>> Yeah.

是的。因此很明显，模拟在机器人学习中发挥着巨大作用。

英文原文

>> Yeah. So so clearly simulation plays a huge role in robotic learning.

模拟的价值：可靠性与效率

我也想对此作些补充。具体来说，模拟可以带来两个层面的收益：第一是可靠性，第二是效率。

对于可靠性而言，如果你考虑让机器人系统在真实环境中可靠运行，你需要数据对机器人可能遇到的所有状态空间和变异提供系统性的覆盖。这就是你如何能够学习到具备鲁棒性的表现。借助模拟，你可以对光照、摩擦力、几何形状、物体类型以及所有不同种类的物理参数进行系统性的随机化与控制，以确保你对状态空间有充分的覆盖。因此，这就是赋予机器人系统可靠性的因素。

其次是关于效率。现在很多人都在做遥操作，如果观察许多遥操作设备，想象你正在使用的所有外骨骼，你收集数据的速度实际上比人类真正执行任务的速度还要慢。

英文原文

>> I also want to add to that. So like there are if you put things more specific simulation can provide two levels of benefits. The first one is reliability and the second one is efficiency. So for reliability, if you're thinking about a robotic system working reliable in the real environments, you need data to provide systematic coverage of all the state space and the variations that robots might encounter. That's how you can learn of how that is robust. So with simulation, you can do systematic randomizations and control and the variations of lighting, frictions, geometries, object types and also all different kind of physical parameters to making sure you have sufficient coverage of the state space. So this is what can give the robotic systems reliability and second is about efficiency. So right now many people are doing tele operation and if you look at many of the television device imagining all the actual skeletons you are using you're actually collecting the data at a speed that is actually slower than human actually doing the task.

但对我们的许多客户来说，人类的速度对他们而言还不够快。他们想要比人类更快的速度。是的。

英文原文

>> But for many of our clients human speed to them is not good enough. They want faster than human speeds. Yeah.

所以对于机器人来说，要移动得更快，并不像直接加速驱动机器人那么简单，因为重力并不会改变。

英文原文

>> So for the robot to like move faster, it's not as simple as just drive the robot faster because the gravity doesn't change.

但在模拟中，你可以对机器人的行为进行系统性的加速，来训练机器人，使其能够把环境的所有动力学变化都考虑在内。这就是能为我们的客户带来效率的所在。因此，无论在可靠性还是效率方面，模拟都能够提供一些非常独特的价值。

你谈到了这项技术以及该平台的作用。或许可以谈谈人们使用它的具体应用场景。

英文原文

>> But in simulation, you can do systematic speed up of the robots behaviors to train the robots such that it considers all the dynamics changes of the environments. So this is what's can give like our clients for them efficiency. So both for the reliability and efficiency though there are some kind of like a very unique like values where simulation can provide. You've talked about the technology and the platform, what it does. Maybe talk about the specific use cases people use it for.

具体应用场景：评估与训练

基本上有两个具体的使用场景，特别是围绕训练和评估这两个方面。

英文原文

>> There are essential like two specific use cases especially around both training and also around evaluations.

好的。

英文原文

>> Okay.

嗯，先从评估谈起。

英文原文

>> Um starting from the evaluations.

评估是人们在机器人领域常常忽略的事情。

英文原文

>> So evaluation is something like people often overlooked in the robotics

但是如果你在微调机器人模型，你必须知道它运行得有多好，而这是你进行迭代的唯一信息来源。

英文原文

>> but if you are tuning like robotic models you have to know how well it works and that is the only source of information for you to iterate.

是的。顺便说一下，很多很多每一个 AI 从业者都非常理解什么是评测并且一直在使用它。对于非 AI 人士来说，它通常意味着稍微不同的东西。所以也许值得具体描述一下你所说的评估到底是指什么。

英文原文

>> Yeah. By the way, a lot of a lot every every AI person really understands what eval are and uses it all the time. NonAI people, it often means something a little different. So maybe it's even worth just describing specifically what you mean by evaluation.

好的。我所说的评估是指：你能够了解针对这特定的检查点，它的表现究竟如何？它的表现是比如在 95% 的时间里成功，还是在 99.9% 的时间里成功？而工业界使用的关键标准是：需要花费多长时间？你需要花费多长时间的实际挂钟工时，才能区分出一个 90% 的检查点和一个 90、92 点的检查点？如果你只在真实环境中做这件事，要做出区分就要耗费非常漫长的时间。如果你再仔细想想目前人们在真实环境中进行的机器人评估，其迭代速度要比那些语言模型的迭代慢上数个数量级。

英文原文

>> Okay. So what I mean by evaluation is you'll be able to understand for this specific checkpoints how well does it perform? Does it perform for example 95% of the time or 99.9% of the time? And the key criteria people use in industry is how long does it take? How long in walk work walk time does it take for you to distinguish between a checkpoints that is 90% from a checkpoint that is 90 92 points. And if you only do that in the real environment that's just takes so long for you to do the distinguishment. And if you really think about also the robotic evaluations right now people are doing in the real environments the iteration speeds is multiple orders of magnitude slower than iterations of those language models.

是的。

英文原文

>> Yeah.

不仅是因为机器人任务非常多变、极其多样。

英文原文

>> So not only is like the robotic tasks very varied very diverse.

噢是的，因为你必须实际去执行那件事，对吧，原子必须在空间中移动。

英文原文

>> Oh yeah because like you actually have to do the thing right atoms have to move through space.

没错。但不仅存在

英文原文

>> Exactly. But only there are

物理定律必须起作用。

英文原文

>> the laws of physics have to do.

你看过那些机器人视频吗？每个视频都是大概 10 倍速、8 倍速，因为它的动作太慢了。

英文原文

>> Have you watched those robotics videos? Every video has like 10 x 8x because it moves so slowly.

没错。所以它不仅慢，而且危险、代价高昂，但与此同时速度也慢了数个数量级。因此，我们的一些客户实际上需要这种能够用来评估其机器人系统的数字环境，并且由于我们的数字环境已被证明与真实世界具有对齐性。这意味着在模拟中发生的任何事情，在真实环境中也很有可能会发生。正如我们在博客文章中所讨论的那样，如果一个检查点在模拟中运行得更好，它在真实环境中也非常有可能运行得更好。因此，这实际上为我们的客户提供了极大的信心，去实际使用来自数字环境的数据与信号，对其机器人系统开展可扩展、安全且快速得多的评估。

太好了。以上是关于评估的部分，接下来是关于训练的部分。在训练方面，基本上就像我之前提到的，关键在于可控性。你希望控制所有可能的状态变异、参数、光照、摩擦力、物理参数，甚至包括物体几何形状、物体类型。你希望确保对所有不同种类的场景具有充分的覆盖，从而能够生成类似

英文原文

>> Exactly. So not only it's slow, it's dangerous, it's costly, but at the same time the speeds is also like multiple orders of magnitudes like slower. So some of our clients actually needs this digital environment that's can be used to evaluate their robotic like systems and because our digital environment has proven alignments with the real world. So meaning whatever happens in the sim is also likely to happen in the real environment. If a checkpoint is working better in the simulation is also highly likely to also work better in the real environments as we have also been discussed in the blog post. So that actually give our clients very strong confidence in actually using the data using the signal from the digital environment to do scalable safe and much faster evaluations of their robotic systems. Great. So that is on the evaluation then on the training. So on the training sides, so basically like I also mentioned it's about controllability. So you want to control all the different possible variations of states, parameters, lighting, frictions, physical parameters like even object geometry, object types. So you want to making sure you have sufficient coverage of all different kind of scenarios such that you'll be able to generate like a

仿真环境与机器人训练

为了让你的机器人具备鲁棒性，需要信息量充足的数据。正如我们之前讨论的那样，仅在真实环境中做到这一点将是非常困难的。如果你采用遥操作，收集数据的速度会很慢。你还会受到诸如拥有多少台机器人、拥有多少台遥操作设备的限制。围绕这一切的数据运营也面临着各种各样不同的挑战。但在仿真中，一切都可以是可控的，一切都可以是系统化的，一切都可以被理解到这样一个层次：你能够确切知晓并断言自己究竟覆盖了什么分布，从而建立起在此分布范围内机器人必然能够正常工作的信心。因此，这种信心、效率和可扩展性，也是我们的客户所看重、并使用我们的数字化世界来训练机器人系统的地方。

英文原文

informative data for your robots to be robust. And this is just going to be so hard to do just in the real environments like we discussed. If you do tele operation, the speed at which you're collecting data is slow. You're also limited by like how many robots you have, how many tele operation device you have. There's like a whole different kind of like challenges around all the data operations around it. But in simulation everything can be controllable, everything can be systematic and everything can be understand at a level where you know exactly and making claims about exactly what distribution you have covered to develop confidence about within that distribution. we know the robot will work. So those kind of confidence and efficiency and scalability is something that our clients also value to use our like a digital words for the training of robotic systems.

疯狂的是，甚至在 Synix 和我们洽谈之前，我们在 Marble 的入站客户就已经看到了这种需求。我们只是无法服务这些客户。但我们已经接到了很多电话，来自从开发自己模型的早期机器人公司，一直到非常务实的下游用例，我们都看到了这些需求。当人们听说你要进入机器人领域时，他们脑海中浮现的是你搬出一台 3D 打印机，开始制造硬件，然后为机器人的大脑编程并装入机器人体内，这样你就得到了一台机器人。而我认为你们在这里谈论的并不是这个。所以也许可以谈谈，你知道，这在制造机器人的生命周期中处于什么位置，以及你们的边界在哪里、生态系统的其余部分又将在哪里开始。

英文原文

>> Here's the crazy thing even before uh Synix and we are talking our inbound customers for Marble were already seeing this kind of demands. we just cannot serve these customers. But we are already getting a lot of phone calls from from robotics early stage robotics companies who are developing their models all the way to downstream very pragmatic use cases and uh we're seeing these uh these needs. When people hear you're going into robotics, what they're going to envision is you're pulling out a 3D printer and you're going to be making hardware and then you're going to be programming the brain of a robot and sticking it in the robot and then you've got a robot. And I don't think that's what you guys are talking about here. So maybe talk about what you know where this fits in the life cycle of creating a robot and like where you will end and where the rest of the ecosystem will will begin.

我们一直在构建的，你可以想象成是一套基础设施，以及围绕这套基础设施的软件，供人们构建世界，以便机器人进行学习和评估；而且这种基础设施天然是模型无关和形态无关的。

英文原文

So what we have been building you can imagine is a infrastructure like with the softwares around this infrastructures for people to for them build words such that robot can learn and evaluate and this infrastructures is naturally model agnostic and embodiment agnostic.

所以我只是想讲得非常清楚，因为这对你来说可能显而易见，但实际上是一个非常微妙的观点，也就是说，从你所说的来看，那不是在制造机器人，而是在构建一个环境，让另一家公司可以把它们的机器人大脑放进去——

英文原文

So I just want to be very clear just because this is actually a very subtle for you it's obvious but it's a very subtle point which is um from what you said that's not building a robot it's building an environment which another company can place their robot brain

没错。

英文原文

>> exactly

进而进行导航和学习。

英文原文

>> to navigate and to learn.

是的。目前对我们的客户而言，他们拥有各种不同类型的机器人。有些使用的是单机械臂，有些使用的是双臂，有些使用的是固定机械臂。有些使用的是移动操作臂。有些使用夹爪，有些则使用更精巧复杂的末端执行器版本。因此，我们的平台目前天然就是形态无关的。我们可以非常轻松地整合不同种类的机器人形态，能够将它们置于我们生成的数字化世界中，从而让我们能够为这些单个机器人赋予在真实环境中以恰当的可靠性和效率水平执行正确任务的能力。

英文原文

>> Yeah. So for our customers right now they have all different kind of robots. Some are using for them single robot arm some are using bio some are using a fixed arm. Some are using like a mobile manipulators. Some using grippers, some are using some more elaborate versions of the end factors. So our platform right now is just naturally embodiment agnostic. We can very easily integrate different kind of robotic embodiment be able to put them into the works we generated with digitalized such as we will be able to give those individual robots capabilities of doing the right tasks and at the right levels of reliability and efficiency in the real environments.

对。

英文原文

>> Yeah.

而且举例来说，我们也是模型无关的。因此，我们仅利用我们世界生成的数据来训练不同的模型，无论是从零开始训练，还是对现有的基座模型进行后训练，比如视觉-语言-动作模型或世界-动作模型。所以对我们来说这无所谓，我们只是想确保我们拥有基础设施，拥有所有的世界，从而让机器人可以在真实环境中可靠地工作。

英文原文

>> And we are also for example model agnostic. So we can just using the data generated by our words to train different models either from scratch or doing post training of existing foundation models like vision language action models or word action models. So to us it doesn't matter we just want to making sure we have the infrastructure we have all the words such as the robot can work reliably in the real environment.

从结构化到非结构化环境

你知道，你曾告诉我，你认为很多围绕人形机器人的预测都有点过于激进了，我们更有可能看到在更受限的场景中展开部署，比如仓库之类的地方。你能稍微谈谈这一点吗，以及这如何影响你们在 World Labs 这里将要解决的问题？

英文原文

you know, you you you have uh told me um that you think uh a lot of the predictions around humanoids were a little bit aggressive and we're likely to see more constrained rollouts like warehouses or whatever. Can you talk a little bit about that and like how that impacts what you're going to be tackling here at uh the like world labs?

这是一个非常好的问题。如果你看一下真实环境中机器人应用的所有发展演进，它一直遵循着从完全结构化环境走向半结构化环境，进而走向非结构化环境的趋势。

英文原文

>> So that's a very good question. So if you look at for example all the uh progressions of robotic applications in the real environments it has always followed the trend from going from fully structured environments into semiructured environments and then into unstructured environments.

对于完全结构化的环境，我们的意思是你对环境内的所有构型都具备认知和控制能力——

英文原文

>> For fully structured environments what do we mean that you have knowledge and control over all the configurations within the environments

比如工厂。

英文原文

>> like factories

比如工厂，或者汽车制造生产线，那些几十年来一直处于自动化状态。

英文原文

>> like factories or for car manufacturing mice those has been automated for decades.

是的，对，没错。

英文原文

>> Yeah. Yeah. Yeah.

然后你还有例如半结构化环境，你对环境有某种程度的控制，例如像亚马逊的仓库，或者比如餐馆、酒店，在这些地方你对环境有一定的控制，好让机器人的任务变得更容易，但显然还有许多其他物体，或者比如衣物，这些是你无法控制的物体。而到了非结构化环境，就像你家和我家——

英文原文

>> And then you have for example semiructured environments which you have certain controls over the environments for example like the Amazon warehouses or for example like restaurants hotels where you have certain control over the environment to just make the task easier for your robots but there are obviously many other like objects or for example clothes those are the object you don't have control and then for the unstructured environments it's like your home and my host

那些我认为是最大的挑战。尤其特别是我的房子，相信我。三只狗，

英文原文

>> those is I would say the the grand challenge Especially Especially my house, trust me. Three dogs,

一个五岁的小孩。

英文原文

>> five-year-old.

是的，狗。

英文原文

>> Yes, dogs.

确实如此。

英文原文

>> Exactly.

如果你思考鲁棒性来自何处，鲁棒性来自于对机器人可能遭遇的场景的充分覆盖。因此，至少在目前，先聚焦于半结构化环境，然后再转向完全非结构化环境，要容易得多、也切实可行得多。因此，我们会朝着那个方向前进。只是我们希望采取一种更可持续、更务实的方式去实现它。

英文原文

>> If you're thinking about where does the robustness coming from, robustness coming from a sufficient coverage of the scenarios that robots might encounter. So, it's so much easier and more approachable at least like right now to focus more on the semiructured environments before we move on to fully unstructured environments. So, we will move into that direction. It's just we want to uh take a more sustainable and more realistic approach towards it.

我想你这里的观点是，人形机器人模仿人体，而进化为了非结构化环境对人体进行了优化。

英文原文

>> I think your point here is that humanoids mimics human body and evolution has optimized human body for unstructured environment.

因此，我们的手指、我们的双腿并不是只做一件事的最佳器具。例如，如果我们作为一个物种唯一的目的就是爬树，我们未必会拥有现在的身体，对吧？我们会有不同类型的手指。但人类最终拥有并进化成的是这样一种体型：它可以非常通用，但未必在每件事上都是最优秀的。那是为了在非结构化环境中生存。但从商业角度、从务实的技术角度来看，这种非结构化环境和通用化的身体实际上是最难解决的问题。甚至未必是解决问题的正确途径。我们的方式是专门化。因此，我们采用更专门化的形体去解决更狭窄的问题。但对 Scenics 来说，挑战在于要做到更加形态无关，从而使其基础设施能够服务不同的形体和不同的半结构化环境。

英文原文

And so our fingers, our legs are not the best apparatus to do one thing. For example, if if if our only goal as a species is to climb trees, we will not have this body necessarily, right? So, we'll have different kind of fingers. But what humans end up having are involve evolved into is this this body shape that can be very general but not necessarily best at everything. And that is for the survival of unstructured environment. But but but from a business point of view, from a pragmatic um technology point of view that this unstructured environment and a generalized body is actually the hardest problem to solve. It's not necessarily even the right way to solve the problem. It's we specialize. So we take more more specialized body to solve a narrower problem. But the challenge for scenics is is that to be more body agnostic so that their infrastructure can serve different bodies and different semistructured environments.

能效与机器人发展周期

你知道，看待完全相同问题的常见视角是经济视角，对吧，也就是说——

英文原文

>> You know a common lens to look at exactly this question is an economic lens right which is um

好吧，你可以把它与生成式大语言模型相比较，后者创作散文或代码的速度比人类快一万倍，并且比人类便宜得多。所以这种经济效益是讲得通的，因为我们的大脑在这方面并不是非常高效。然而，我们的大脑和我们的身体在三维导航方面是非常高效的，对吧？你知道，就像在世界上穿行或捡起物品。所以这只是一个预测性问题，但你是否相信，至少在可预见的未来，我们能够制造出在从事杂活琐事时拥有与人类相同能效的机器人？比方说基本上相当于最低工资之类的水平，我们距离这个目标还有多远？这是像 5 年，还是像永远不可能？我认为这需要很长的时间。所以如果你仔细思考现实环境中的机器人，归根结底它始终是一个系统。因此，在现实环境中工作的每台机器人都是一项系统工程。你必须非常注意并深思熟虑系统是如何组合在一起的。硬件、软件、大脑，甚至细致到比如你手指的摩擦系数是多少。因此，要让这些东西成为现实，你必须考虑很多事情，并且这需要不断的迭代。但我感到兴奋的是，我一直身处机器人学习的最前沿，并且也在努力推动这项前沿向前发展。是的。

英文原文

>> well you compare it to like generative LLMs where they can create uh pros or code 10,000 times faster than a human being a bunch cheaper than a human being. So the economic case makes sense because our brains aren't very efficient at that. However, our brains and our bodies are very efficient at 3D navigation, right? You know, like moving through the world or picking things up. And so this is just a prediction question, but do you believe we'll ever be able to build robots, at least in the foreseeable future, that have the power efficiency of a human being when it comes to to menial tasks? So let's say just basically you know minimum wage or something like that like how far away are we from this? Is this like 5 years or this is like never I think it's going to take a very long time. So if you really think about like a robots in the real environment in the end it it will always be a system. So every working robot in the real environment is a system work. It's you need to be very mindful and thoughtful about how the system are coming together. the hardware, the software, the brain, even like to the details of for example what's the friction coefficients of your of your fingers. So there's a lot of things you have to consider to make these things uh a reality and it will take iterations. But what I am excited about is that I have always been at the state of the arts of robot learning and also trying to push the state of the art forward. Yeah.

但最前沿的发展总是比我预期的要快。因此，我目前所关注和试图探究的东西，与比如我刚开始读博时相比，已经大不相同了。所以这说明了整个生态系统的演进速度有多快，以及构建这些机器人系统的所有动态组件都开始汇聚在一起，但我们——

英文原文

>> But the state-ofthe-arts always moving faster than I expected. So what's I'm focusing on and trying to investigate right now is very different from for them when I started my PhD. So this is a speak to how fast the whole ecosystem has been evolving and all the moving pieces started coming together or building this robotic systems but we

衡量乐观与具身智能的挑战

也必须对我们的预测进行校准。所以我们会看到很大的进展，但要达到例如人类水平的效率和能力，还需要更长的时间。Martin，在当今的 AI 领域，最难的事情就是保持恰当、审慎的乐观，对吧？

英文原文

also have to be like calibrated about our predictions. So we will see a lot of like a progress but to achieve for example human level efficiency and capabilities it will take uh longer. Martin, the hardest thing in today's AI is to have the right measured optimism,

对吧？

英文原文

>> right?

没错。这完全属实。是的。

英文原文

>> That's right. It's totally true. Yeah.

我的意思是，连 LLM 都没有人脑的效率。人脑是以 30 瓦的功率运行的。

英文原文

>> I mean, even LLM does not have human brain efficiency. Human brain operates on 30 watts.

是的，确实如此。

英文原文

>> Yeah, that's true.

确实是，所以我们离那还很远。所以，

英文原文

>> That's so we are far from that. So,

但那——但我的意思是，能效比在狭窄任务上可能很接近，对吧？比如软件——

英文原文

>> but that but I mean performance to power it may be close, right? in narrow task like software

比如生成图像或者软件工程，那确实是。是的，我认为是的。我不认为在机器人领域我们有任何接近。这是否改变了你对团队在战略层面上可以追求的雄心水平的看法？我的意思是，这改变了这一点，还是它仍然非常符合你刚开始时的预期？

英文原文

>> like generating an image or or software engineering than it is right. Yeah, I think so. I don't think we're anywhere close when it comes to robotics. Does this change how you think about your um like strategically the level of ambition that your team can go after? I mean does it is it changed that or is it still very much in line with what you expected to do when you started?

数字世界仿真与团队整合策略

这绝对以非常深刻的方式改变了轨迹。我们看到在能够进行这整个过程、进行环境建模方面有了很大的突破，是的，以高效得多且更具可扩展性的方式，特别是在与 word labaps 合作的情况下。我还想补充一点，如果你想想例如语言模型的当前状态——

英文原文

>> It definitely changed the trajectories in a very profound manners. So we see a lot of unlock in be able to do this whole process do the modeling of the environments yeah much more efficient and much more scalable manners especially in partner together with word labaps and I also want to add to f if you think about for example the current states of the language models

那些是具有令人难以置信的能力的模型——

英文原文

>> so those are models that's with incredible capabilities

但你仍然不会仅仅盲目信任它来订机票或预订酒店，你仍然希望仍然有人在阅读那些语言模型的输出。

英文原文

>> but still you don't just blind trust it to book your flight tickets or make your hotel reservations you still hopefully there's still a person who reading the output from those language models.

是的。

英文原文

>> Yeah.

但这与人们以及我们使用机器人模型的方式非常不同，因为对于机器人模型，开箱即用的机器人必须在真实环境中可靠地工作。

英文原文

>> But that is very different from how people and we'll be using for them robotic models because for robotic models out of the box the robots has to work reliably in the real environment.

所以我们甚至没有数据，我们甚至没有围绕这些所需的所有基础设施，能让机器人直接开箱即用、在真实环境中可靠运行。因此，能够创建这些数字世界，这些可扩展的数字世界，让机器人可以在其中学习并在其中进行评估，将释放出更多的潜力，能够用从这些世界生成的数据来取代现实环境中所有昂贵且不安全的数据，以便机器人能够进行可扩展的学习和评估。

英文原文

>> So and we don't even have the data we don't even have all the necessary infrastructures around those for the robots to just out of box work reliable in the real environments. So for that reasons be able to create this digital worlds this scalable digital worlds where the robot can learn in and evaluates within that is going to unlock so much more potentials for be able to replace all the like a costly and unsafe data in the real environments with the data generated from the the the the words for the robots to be able to do scalable learning and evaluations.

你知道，我见过很多这类整合。在有这么多契合度的阶段，它们实际上运作得非常好，这很棒。但总会有这样的问题：你是现在就整合到当前正在进行的事情中，还是保持相对独立，并提供一种会在一年左右的时间范围内实现的长期轨迹？费，你是怎么看这个问题的？这是马上整合的事情，还是某种独立的、更长期的方向？

英文原文

>> You know I've I've seen uh I've seen many of kind of these kind of integrations. they actually work very well at this stage when they have this much alignment which is great. Um but there's always like this question of do you integrate now into what's happening now or do you keep things quite separate and provide kind of like a long-term trajectory that will you know be realized you know in in in the year time frame. How are you thinking about this fee? Is this something that integrates right away or is this kind of a separate longer term?

这是一个很好的问题。我认为在这一点上，你知道 Changi、Sunonny、Justin、Ben 和我都一直在讨论这个，在这一点上我们要深思熟虑地进行，我们不会急于整合从代码库到团队的所有内容，因为我认为 Synex 确实有一个非常周密考量的、我不会称它完全独立、但也相当自成一体的技术栈，以及他们的客户，以及他们正在构建的产品类型。我们会循序渐进。我们肯定会——我们已经在仿真端，以及潜在的基座模型、动作条件模型端，我们已经开始交流了，而且他们也作为内部客户在试用 Marvel。所以我们会进行整合，但我们并不急于把团队混合成一整碗沙拉。

英文原文

>> This is a great question. I think at this point you know Changi Sunonny Justin Ben and have been talking about this at this point we are going to take it thoughtfully we're not rushing to integrate everything from codebase to teams because I think Synex does have a very uh um wellthought and I wouldn't call it standalone completely but fairly uh contained um tech stack as well as their customers as well as the the the kind of products they're building. We're going to take time. We definitely will we already on the simulation side as well as the the potential base model um action condition model side. We already are starting to talk and also they are using Marvel as a internal customer. So we we will be integrating but uh we're not rushing to blend the team as like a full salad bowl.

是的。

英文原文

>> Yeah.

地域布局与客户合作

随着这次 scenics 的变动，在地域方面你是怎么考虑的？会留在同一个地方吗？

英文原文

>> How are you thinking about geographies with this with scenics move? Is it going to stay in the same place?

我们要——Vundra 要搬迁。

英文原文

>> We're going to Vundra is going to move.

哦，欢迎来这里。搬到旧金山。是的，佛罗伦萨与文艺复兴。太棒了。

英文原文

>> Oh well, welcome here. Moving to San Francisco. Yeah, Florence and the Renaissance. Perfect.

我认为我们 World Labs 正正式成为一家横跨东西海岸的双海岸公司，总部位于旧金山。你知道，我住在帕洛阿尔托，感觉就像在另一个州。我们——但我其实很兴奋我们将在纽约设立办公室，这可以帮助我们吸引东海岸的人才；而且我们也一直在讨论确保在两个办公室都布置机器人，这样我们基本上就可以测试并完善我们的工程栈，从而能够远程操作机器人，因为无论如何我们都必须为我们的客户做到这一点。

英文原文

>> I think we our labs is officially becoming a by coastal company where the headquarter is in San Francisco. I've, you know, I live in Palo Alto. I feel like I'm in a different state. We we but uh we but um I'm actually excited that we're going to have an office in New York that can help us to attract talent on East Coast and also we have been talking about making sure that in both offices we set up the robots so that we get to basically test out and and and uh mature our engineering stack so that we can work with robots remotely because we have to do that for for our customers. Anyway,

那么也许为了非常具体起见，费，也许让我们描绘一下，在两年内最完美的成功案例是怎样的？比如你们有什么产品？谁在与其互动？他们如何使用它？就是简要明确地说明是怎样的？

英文原文

>> so may maybe just to be very concrete, Fay, maybe let's just pencil out like what is the the perfect success case in two years? Like what product do you have? Who's engaging with it? How do they use it? Just just the crisp like what?

我们会非常高兴 Synix 团队和 World Apps 团队在少数几个重要的垂直用例中验证客户，在这些用例中，我们的系统、我们的基础设施被证明对他们的自动化需求真正有益，并且这些客户成为了我们拓展业务的灯塔典范。

英文原文

>> We're very happy that uh Synix team and World Apps team will have validated um customers in in a small number of important vertical use cases where our system, our infrastructure has proven to be truly beneficial to their automation needs And these customers became our lighthouse examples to scale our business.

那么有多早——假设收听这期节目的人正在经营一家机器人公司，他们应该在什么阶段与 World Labs 接洽？是在非常早期？还是在中间某个阶段？

英文原文

>> And how how early let's say someone listening to this uh is running a robotics company? How at what stage do they engage with world labs? Is it really early on? Is it somewhere in the middle?

目前对于我们的客户，因为我们正在构建这种 real-to-sim-to-real 的流程流水线，其中仿真本质上是我们提供训练和评估场地的世界。

英文原文

>> So right now um for our customers because we are building this kind of real to sim to real pipelines like where the simulation is essentially the words we're going to provide the training evaluation grounds.

有些客户只需要 real-to-sim 部分，

英文原文

>> Some customers they needs only the real to sim part

他们希望将他们关心的任务数字化，并能够对他们的机器人系统进行评估。

英文原文

>> they want to digitalize the task they care about and be able to do the evaluations of their robotic systems.

有些客户则需要这整套 real-to-sim-to 完整流程，例如他们将能够让策略在他们的硬件上运行。因此，我们的平台设计方式也很灵活，取决于我们客户的需求；与此同时，与我们合作的客户实际上已经非常接近部署阶段。因此，他们基本上是在处理非常非常实际的任务。那些任务在被替换时、当我们有机器人解决方案就位时，就能立即创造价值；而且他们正在考虑针对至少数十种或数百种此类场景进行自动化。因此，与 word labaps 携手合作，我们将能够为这些场景开发可靠的解决方案，正如我们已经展示的那样，我们已经在博客文章中实例化了许多场景，并且我们将能够进一步深入研究，了解它们如何切实解决现实世界部署所面临的关键要求和限制。

英文原文

>> Some customers they need this real to sim to this entire pipeline such as they will be able to like have policies like running on their hardwares. So our platform is also designed in a way that is flexible depending on what our clients needs and at the same times uh the clients we are working with are actually pretty close to the deployments like a stage. So basically they are working on very very practical tasks. those tasks when replaced when we have a robotic solutions that are there can just create value immediately and they have like at least like uh tons or hundreds of like this kind of situations they are thinking about to do the automations for. So as like together with word labaps we'll be able to develop reliable solutions for those scenarios as we have already showed we have a number of scenarios already instantiated in our blog post and we'll be able to further our investigation to see how they can actually solve the key like like requirements and and and also constraints faced by the real world deployments.

太好了。我想针对这一点问得非常具体。如果是机器人公司，联系 World Labs 会存在太晚或太早的情况吗？

英文原文

>> Great. I want to be very specific about this. Is it ever too late or too early to call World Labs if you're a robotics company?

不会。

英文原文

>> No.

我们希望每个人都来联系我们。我们想了解你们的用例。

英文原文

>> We we want everybody to call us. We want to learn about your use case.

太棒了。如果你在听，如果你正在收听这期播客，且你与任何机器人项目或机器人公司沾边，请关注 World Labs。

英文原文

>> Wonderful. If you listen, if you're listening to this and you're anywhere close to a robotics project or robotics company, uh please track World Labs.

是的。谢谢。

英文原文

>> Yes. Thank you.

绝对欢迎业务合作。

英文原文

>> Definitely open for business.

我们欢迎业务合作。好的。

英文原文

>> We are open for business. All right.

为时不早。

英文原文

>> Not too early.

好的。如果你在做机器人，请联系 World Labs。非常感谢两位的光临。

英文原文

>> All right. If you're doing robotics, call World Labs. Thank you both very much for coming.

谢谢。

英文原文

>> Thank you.