---
author: AI Engineer
date: '2026-10-19'
guest: ''
layout: post.njk
source: https://www.youtube.com/watch?v=L4dUFimVDKU
speaker: AI Engineer
tags:
- agentic-search
- backtesting
- version-control
- overfitting
title: Exa 智能体网络搜索与快照回测应用介绍
summary: Exa 工程师 Ryan 介绍了其智能体网络搜索能力及其在代码与金融等领域的应用，重点阐述了通过快照功能对历史非结构化网络数据进行版本控制与模型回测，并结合现场比赛强调了防范过拟合与设置保留测试集的重要性。
insight: ''
draft: true
series: ''
category: ai-application
area: tech-engineering
project: []
people: []
companies_orgs:
- Exa
products_models: []
media_books: []
status: evergreen
---
### Exa 与智能体网络搜索介绍
>> 大家好。首先，我是 Ryan。我是 Exa 的全栈工程师。非常兴奋能带大家了解这项非常酷的能力。我们基本上很快就会为 AI 智能体带来完美的搜索，但至少现在大家已经可以对它进行版本控制了。今天我要带大家了解几件事。首先，先简单介绍一下什么是 Exa，什么是智能体网络搜索（agentic web search），为什么大家需要关注这个东西？
Original English
[music]
>> Hello everyone. First of all, I'm Ryan.
I'm a full stack engineer here at Exa.
Very excited to walk you all through
this very cool capability. Um we
basically have perfect search for AI
agents coming soon, but at least now you
can version control it. Um what I'm
going to take you through today is a few
things. So, first off, just a little bit
like what is Exa, what is agentic web
search, why do I even care about this
thing?
Original English
At the end of point one, you're going to
be like, I need agentic web search. I
need it in my thing, regardless of
you're doing anything financial or not.
Number two, okay, I know I want this
thing, but why do I care about version
control? At the end of number two,
you'll know why you care. And it'll be
very important cuz on number three,
there'll be a little hands-on exercise
where you may or may not win a prize.
Hopefully that motivates you to pay
attention through points one and two.
Original English
So, um I'm going to start off a little
bit with like a show of hands just to
situate. So, who here has heard of Exa
before?
Original English
Okay, we have a good few people, but a
few people that haven't heard, which is
perfect. How many of you have used like
a search engine like a Google to do
desktop research in a
prior prior role?
Like using Google just to search things
around, at home, at work, whatever.
Hopefully like most people, sweet. Then
the last piece is going to be, hey, how
many times have you now stopped using
Google and instead use ChatGPT or Claude
or some other agent and say, hey, can
you find this thing out for me? Raise
your hands if like now you do that.
Original English
Sweet, great. Okay, perfect. We're going
to tie all of those three things
together because um the exact same thing
that so many people just put their hands
up, that's what we're seeing right now.
Original English
So, AI is actually searching a lot more
than humans do nowadays. Why? Cuz it can
handle so much like back and forth,
right? It can go out, it can do multiple
searches, it can find what you're really
looking for, can search through multiple
pages, and work really hard to get you
the answer you need. And we're starting
seeing way more people are using AI to
search and AI searches more and it
searches more efficiently than humans.
Humans are kind of lazy. You'll be like,
all right, I'm looking for this thing.
I'll put in a few keywords. Hopefully
it's on page one, otherwise I don't
care. Whereas AI systems, they really
want to satisfy. These are the tend to
be quite agreeable. I don't know if you
found that out. And so they'll continue
to searching until they find the thing.
It's like all of these things mean
there's a lot of search.
Original English
And all of those tools, when you ask it
a question, it's like,
I need to look outside my knowledge
window. I need to consult the internet.
It will use a tool like Exa.
And that's what we provide. So we
actually provide search for some of the
largest
like coding agents and agentic
applications in the world. So whenever
you're using Cursor, and Cursor's like,
gosh, if only I knew what the docs were,
I could really help. It will use Exa to
go out and find that doc
and bring it back to its orchestration
agent so it can answer the question. Or
you're using Gamma and you're like, gee,
if only I knew like what the financial
returns were of like I don't know, Bank
of America in 2019. This slide would
pop. It's able to use Exa to go out and
find that information. So that's kind of
where we sit. I'm also just going to
like do like a quick thing on finance
cuz this is like Wall Street meets AI.
So hopefully you're either in finance or
you're interested in this. But we work
across a lot of these areas. It's a
particular area of interest to me. I
work with some hedge funds and sovereign
wealth funds and asset managers and VCs
across these topics because they want
information. Either they're like, you
know, hedge fund and they're looking for
alpha to to make financial decisions
against. They're an asset manager,
they're trying to forecast risk. They're
a VC and they're like, oh, I want the
hottest new startup in my pipeline and I
got to find them before like The Verge
or someone like in um um like puts up an
article about it.
They all leverage Exa.
Original English
Um
so
hopefully by the end of this you're
like, okay, yes.
Um this is a really, really interesting
space. But like kind of like why Exa?
Like what do we do?
Original English
Where we're a little bit different is
that like we we of see ourselves as a
lab and doing fundamental research to
really push this forward. So, I'm going
to touch on like the core triangle of
like quality, cost, and latency really
quick just so you know that you really
want to put Exa in your application, and
then we'll talk about versioning, um
which would be like, "Ah, now I can
actually see how and where this would be
used to back test my application." So,
Original English
first, quality. You want your results to
be good. We care about that so much that
we built our own regenerative benchmark.
Why do you want to do that? Because
search moves very quickly. You publish a
benchmark and all of the search
um providers and eventually the labs
would be like, "We've got to train on
that and climb the hill." So, what you
really really care about, if you're a
search engine that really believes in
your stuff, is you want to create the uh
you want to create the benchmarks that
people can make for themselves against
their use case and keep it up-to-date so
that your search engine is working for
you because you want to return the
latest information. That's what the
whole uh that's the whole reason why
you're using a search provider and not
just asking uh ChatGPT or OpenAI and
answering from its weights. But, you're
Original English
like, "Okay, I want quality, but I want
it the lowest cost." We care about that
not just at like the search time, but
also, you know, search is a lot of
tokens and tokens goes into a context
window. And what does that mean? You're
paying input costs. So, we developed
technologies to try and return just the
tokens that you need, not just across
one result, but because all the results,
so that your coding agent or whatever
application can answer the question
without having to pay a lot on input
token costs.
Original English
Lastly, latency. Really quickly, you can
go very fast, so we have like sub-300
millisecond search all the way up to
like, "Hey, work for like
um I think I did one with like, 'Find me
all the video game publishers' cuz I was
trying to do some eval." It worked for
20 minutes and it found 3,212
of them and the developers that they
work with. So, across the full spectrum,
we've got you, no matter what point
you're trying to implement it into your
architecture. And now you're saying,
Original English
"Ryan, okay, that's great. I really like
Exa, but I want to version control it."
Or, "Why do I want to version control
it? Um and that's going to be the uh
kind of focus of this discussion today
to talk about this new capability we
released called Exa Snapshot, which
allows you to take our search as it is
today, um but actually see what it would
look like if you searched at a point in
the past.
Original English
I'm going to motivate like firstly what
this is and then why you care about it.
So, firstly, what is this? So,
uh we're a search engine, so we spend a
lot of time looking over the web. Uh and
whenever we find this content, what
we've been doing has been putting it
into our index. And when we find like a
new version of the page, or even when we
just discover the same page at a date in
the past, we store a version of that.
Um and that means that like every URL
across our full index, which is an index
of the entire web with um everything
around 100 billion documents tracking
trillions of URLs, uh
you can now go back in time. Um and
Original English
there's two ways you can do that. The
first one is like I know a URL, but I
want to see what was that like in the
past. Um and there's content versioning
for that. So, you can go in, you can
provide a date, and we will show you the
content as of that date or the nearest
date that we had before. So, you can
guarantee that you have a version of
that page that we would have served at
that time.
More broadly, you can use version
search, which is not just like finding a
URL, but executing a search
Original English
as you would have done now, but in the past.
Original English
Let's like dive into just like two very very small examples around this. Um and then like can kind of give like a vibe of some use cases. So, first one, let's say you're uh interested in financial markets. Who would be interested in financial financial markets around here? Um and one thing you might be trying to do is figure out like, "Hey, how can I understand the hottest and latest startups?" Maybe you guys would have heard of OpenAI. And there was some interesting news happening recently where um previously Axios and a few other people were reporting around uh OpenAI's uh annualized um like revenue and there's ideas like oh it might be up near like sort of 70 billion. And then more recently there's a lot of news about is it 70 billion or 50 billion? Turns out to be some like you know accounting management about whether it's gross revenue or net revenue, whether you're passing through certain cloud providers costs to you, do you recognize that revenue or not? But if you're a person who is in the business of understanding these things, it would be really helpful if you can be like hey I want to make sure that I'm designing a system that is able to look at these things, right? That is able to like at the point where you would have done that search, when you would have had that information, like is it tuned in such a way that it would reveal this like accounting footnote that could actually be very valuable to trade against.
Original English
Let's say you're trying to do something arguably even more or less important, which is to understand the adverse media for like very important folks or maybe less important folks. And so like Ed Sheeran he had some like adverse media against him relatively recently. But if you look back a little bit before that event, you can see like hey what are a few other things that existed or you know he was he was not really good at insurance, maybe you wouldn't want to sponsor him for some insurance product. But the main thing here is that like at the point now where you're like hey I'm maybe I'm an insurance provider and I wanted to do some like counterparty due diligence or KYB or KYC or you're working like any business and you're trying to do partnerships and part of your flow is understanding another party. It would be good to be able to go back and be like hey I've designed this system to go out and do a bunch of web search and figure out what this person is, what this business is, how they could be classified, what whether we would insure against them or not. And now you're able to design that system and then with Hexo go back and test it. Um
Original English
So that just gives like a few examples of sort of how and where uh, like snapshot can be useful in order to to kind of like backtest these systems. So, um, here's just like an an example of this. Um, as a note, this is important to take in some some awareness of because it may help with uh, you achieving some some some great excellence later. Um, but let's say you're like trying to understand something around like acquiring companies. So, you may be like, "Hey, I'm trying to design my system. If you do a search right now, I could see that this company is already acquired, but I might want to go back in the past and be like, "Hey, what about 1 day before the acquisition, 1 week before the acquisition, a month before the acquisition? Would I be able to find the information to predict that?" So, any live search necessarily is kept up to date. That's what you're kind of paying for, right? That if you did a search now, you'd have the latest information.
Original English
But, um, that also leaks kind of the future knowledge of that into an evaluation of the past. If you're creating some predictive system, and all it does is goes like, "Okay, I look at the news and it says that this thing was acquired, so tick, it was acquired. 100% successful true positive rate." It's assuming the search is good enough, which you hope it would be. Um, what's what's helpful with with snapshot and and kind of version version control search, you can constrain like what particular information is exposed at which time.
Original English
So, here in this little example, I might set the snapshot date to like just before the outcome was announced. So, I know that the only thing that's going to be informing this system is uh, information that um, doesn't actually leak the exact confirmation of that thing, right? And then you could tune it to do different things. Like, maybe I have a feature that I really care about hiring. I want to make sure I find the right hiring signals. Maybe I have a feature that cares about like analyzing earning calls or or transcripts or or podcast interviews with the CEO. Whatever it is that's important to you, um, you're able to kind of tune that system and see, "Hey, would that have actually improved my prediction or would it not have improved my prediction?"
Original English
I'm going to share just like and I'll spend like a tiny bit of time here but just talking about like a couple of patterns that we've seen in the backtesting space recently. So, the first one is around this idea of like a traditional model. So, the idea is here is you're just going to gather data that you need to backtest the model that you already have. You can kind of think of this is like, "Hey, maybe I have some system and it operates in the world. I don't need I don't need all that agentic web search. I love my system. It's amazing." Right? But maybe one thing that you can't do is actually go back and gather the features that you had in the past today. Right? Could be that they're based on web pages and those web pages just don't store a version that you can go back to.
Original English
It could be that you're trying to have some sort of system that like takes like newspaper pages or some other articles and you can't really trust the publish date. Very small digression here but like publish dates of articles actually update all of the time. Both for like hey, like we've legitimately updated it all the way down to like hey, like SEO really likes up-to-date articles. So, maybe we just like update it one more time.
Original English
So, that means that like you can't really trust that like publish date feature feature and is really important especially if you're making quite like substantial decisions around this that you're able to trust what that feature is. So, you may just decide, "Hey, I'm going to use snapshot. I know some URLs or some websites that I care about. I'm going to like jump back and just gather the data that I need and then put it in whatever feature I need, some array of numbers etc." X is not even involved in the prediction but this is like a perfectly valid and like relatively straightforward way that you might already be using some kinds of version data to backtest your system.
Original English
Okay, that's the that's the existing space.
Original English
Um what about the future? The future of perfect search. Okay, so there's two sort of patterns here.
Original English
The first one is like I'm going to add an agentic feature to a traditional model. It's kind of similar to what we were talking about before, but the idea is in the live model, it's actually going to be using X or to like X to go out, search the web, turn all of that unstructured information into a structured signal, and then trade against it. So you could think about something like the adverse media scan like against our friend Ed Sheeran before. Um you you might be designing your system to be able to do some like counterparty due diligence, um and in the live production system, that might just be like one arm, right? It's uh the system itself may not have any agency, it's just going to collect a bunch of features, and then maybe you've like trained a model um um just like a traditional ML model to say like yes, we're going to invest, or no, we're not going to invest. You can uh go back and back test those agentic features one by one with X or and be like hey, uh against like a true positive match where something um like
Original English
>> [clears throat]
Original English
>> either like I don't know, the the the the thing that you care about happened, either like that business succeeded or it didn't succeed, you can test and be like were we able to correctly predict that, um or did we not? And most
Original English
importantly, you can then tune that system. Because the way that with all of these non-deterministic systems, the way that you tune them is like really really important. So if you tune it to search a certain way, maybe it succeeds more often. If you choose to search another way, succeeds less often. And that is you can control that experiment and then add it into your system like one feature at a time.
Original English
Um then like uh on the far would be right hand side, the idea is like there's this like future where there's some like predictions, startups, and other folks operating where potentially like the full system is agentic. Um so the system itself may or may not choose to search the web through some tool call.
Original English
Um it could be uh combining internal knowledge. It could be combining like structured external knowledge. And it can kind of do that at run time. The idea is be like, "Hey, this is kind of what humans do when they go out and do some prediction. They have some agency. They're going to search for this or search for that." And essentially you can ground that uh like interface for your model into the outside world into the outside world. And then you can uh ensure that when that system was operating, it was able to search um only the information that it had at that time.
Original English
So one like practical example of this is like let's say you're trying to improve your coding model. And hey, one part of that coding model would be uh whether it can solve a PR or not. You could be like, "Hey, I'm going to freeze right before that PR was solved uh in an in an external repo and then let the system run. And then hopefully, if it's able to resolve the task, you can have more confidence that that's a change in the model weights or whatever component in the system that you're actually paying to improve through your training process, then it is just like future information sort of like leaking into the current.
Original English
Okay. So that's just like a little bit of like an overview of like snapshot and a few of the ways that it's being used to backtest. I'm just going to like uh recap what we've seen so far. So first, we really like Exa. Why do we like Exa? Because it's very good search. And search is very important for models because they need to know about uh the current uh and it needs to be high quality. It needs to be like uh cost effective. And it needs to be at the latency that makes sense for my system.
Original English
Okay, we've all learned that. And then we're like, "Hey, I really want that. And a big part about putting that into my model is being able to version control it so I can prove that it adds value, not just like vibes and hope that it adds value. Um what we're going to do now is like the fun part of this workshop, which is we're going to give you all time to kind of go hands-on with this tool. So, there's a little QR code on the screen, you can scan it with a mobile. You can also just like enter this [clears throat] little URL into your computer and the uh interface on your laptop will probably be uh more amenable.
Original English
Um and then I'm just going to give you like a little bit of an overview about what we're trying to do here. So, the idea is, if you think back to the example that we had before, um you're going to be trying to uh predict like what are company acquisitions right before they happen. Um to tell like a little bit of the story here. So, let's assume that you're working in some financial institution. Uh your boss's boss's boss's boss like went to some AI engineer workshop and they're like, "Hey, I heard this like really, really cool web search feature. I think we should present it to our portfolio manager." And portfolio manager's like, "Okay, I want to see it. You've got 20 minutes. Um try and see if you can make it good."
Original English
Um and the most confounding variable here is you don't have like a coding agent. So, you have to do it by hand because your usage just ran out or something. Um so, that's the kind of context of where we're sitting ourselves at.
Original English
Your task is going to be to find the best two features that like accurately predict a company acquisition. Uh and then Access Snapshot is going to use that to go back in time uh 1 day, 7 days, and 30 days before the acquisition and test your feature. Um or a random date if they weren't acquired, so there's a few confounding ones in there.
Original English
Um feature is just like a yes/no question um that you believe would predict an acquisition. You can test as many as you like, there's a few examples in there, and I'm going to be coming around just to talk with you all um and kind of like help you along the way. Um and you can test as many as you like.
Original English
Scoring, you get plus one point if you uh uh if your feature declares that this company would be an acquired uh on 170 30 days before it did. You get minus one if you say that it got acquired, uh but it didn't. And then, no impact for misses or passes cuz we're very friendly. Best score automatically captured, you can try as many as you want. I'm going to set us up for let's say like 30 minutes.
Original English
Um so uh we'll kick that off now. And you have 30 minutes to try and put in the absolute best feature that you want. I'll be coming around. Maybe we can put on like a little bit of soft music that kind of gives people the vibe of working in [music] a portfolio manager company uh and listening to like low-fi beats. Uh and I'll come around and talk with you all.
Original English
Okay, I'm just going to do like one like real quick example just to kind of show people how it works. If you're already going away and and and winning, keep feel free to keep grinding. Um but I'm just going to put one example in here. So, you'll see this interface in front of you. You have a bunch of um it'll ask you to put in a new feature. You can pick one of the examples or type whatever you want and hit run.
Original English
Um and what it'll do is it'll go out, it'll use excess snapshot and search a bunch of them. This one was fast because it was cached, so uh that's a performance improvement to show you that we did test this beforehand. Um but it'll go down and it'll run that against this set of 10 companies. So, within this set of companies, some of them were acquired. Some of them were not acquired. They're confounding variables.
Original English
Um and the idea is that like we're going to take this particular query, so has this company lost its CEO or CFO or appointed interim leadership. And for all of this, we're going to go back and do that version search. So, Delivery Hero, we can see, "Hey, um there was an executive leadership turnover. Yes." And the company did end up being acquired. So, hey, maybe this is predictive.
Original English
Except, Quagen, I don't know if I'm pronouncing that correctly, they also had like executive leadership turnover, but they didn't end up being acquired. Oh, no, that's a confounding variable. So, your task is to try and be like, "Hey, what are the one or two features that I can add to this that will be the most predictive?" So, here I'm going to like add one more.
Original English
So, this one is like, "Hey, has it been reported to be exploring a sale?" Does adding that improve it or make it worse? And in this case, you can see, "Hey, we've picked up a lot of the um uh we've picked up like a lot of the acquisitions, so we're doing pretty well in terms of finding stuff, but also like, "Hey, we're picking up like a lot of false alarms here." So, we might need to be more uh constraining in the way that we're combining these two features.
Original English
So, here I've put a very simple like combined thing. You can either choose or, which is if either of your two features trigger yes, then we will count that whole thing as being like, "Yeah, you're predicting that it is going to be acquired as of that day." You can go and, and if I see if I go and here, the score improves a little bit, which means both of these two features need to be true.
Original English
And if both of those two features are true for that company on that day, then you'll trigger yes. Um that's just like a PSA. I'll go back to the kind of like leadership board there, but feel free to chuck up your hands with any questions as we come through.
Original English
Okay, 10 seconds left. If you haven't already hit run, it may
Original English
be too late.
Original English
Feel free to nobble over the combined and or to try and get your last few results.
Original English
Phew. Okay, congratulations. So, Zane, John, and Ree, could you come and join me on stage if you won a prize? If not, we'll go further down in the list.
Original English
Can we have a big round of applause?
Original English
Come up, don't be shy. Don't be shy.
Original English
Brilliant goods and mystery boxes await you.
Original English
Hey, congratulations.
Original English
Congratulations, John. Very valuable land.
Original English
Hey, congratulations.
Original English
Um so, yeah, let me hand these out.
Original English
>> Thank
Original English
You're going to line up brilliantly so we can do a photo.
Original English
Okay, we'll take a still out of that video later. So, let's just like poke around and be nosy and see like what did our number one person did. So, Zane
Original English
Is this company a publicly listed company? Very interesting feature.
Original English
Number two, is this company a company other than Qiagen, Vitris, Bank of Cryptos, and American Electric Power?
Original English
Congratulations.
Original English
You are the reason why I wanted to have a holdout
Original English
>> [laughter]
Original English
>> I removed it because I wanted it to be simple, but this is a good example of why you need to do backtesting and why you need to do backtesting with a holdout set. Um so, I'm going to take this as a feature, not a bug,
Original English
>> [laughter]
Original English
>> of not including a holdout set and why overfitting is very dangerous because I do not think that this particular algorithm would be successful in the future, but who knows?
Original English
>> [laughter]
Original English
>> Number two, Ree. So, okay, he stole Zane. [laughter] Maybe it would have been seated nearby. And lastly, with John. So, within the past 12 months, has an active investor or significant shareholder publicly called for a supported sale of companies as a whole or is company's current share price at least 60% below its high price during the past 5 years or is it named Dominion? Also, a little bit of overfitting there, but I would argue it probably performed better.
Original English
Um so, this gives like an example of just like hey, how and why it's important to do backtesting, right? Because, you know, this is like a little constrained example. I I feel like I'm regretting removing my holdout set, although I do feel like people would have been way more confused if I did do that.
Original English
Um
Original English
but
Original English
this gives you an idea of like why it's valuable, right? Because if you know what the existing set is, you can you can overfit it, but the idea is like because you have this capability, you can check for that, right? You can go back and be like, "Hey, I'm going to go back in my book and be like, hey, I'm going to constrain like these 10 companies that I know something had happened to them, 20 companies that didn't, and train and iterate on my model, right?" And make sure that it continues to perform beyond just like the publicly visible sets.
Original English
Um so, let's like recap the highlights. I'm also going to
Original English
Oh my god, there's a lot of people in here. I have to scroll very far. I'm going to put you all on a uh thank you page. I should already be on the thank you page. On the thank you page, you've got a link to our docs. We love documentation. You have a link to our blog. It's more readable than the docs
Original English
>> [snorts]
Original English
>> and is more like human-facing. Uh and lastly, there's a chance to send me an email. I really regret that now because there are 100 people in the room, so I'm expecting a lot of emails. Um but this is an enterprise feature as well. We have some public usage, but if you're interested in having access to this for your team, send me an email and I'll connect you to our enterprise team. Um let's just like recap the highlights. Number one, we love Exa. Why? Because it's great agentic web search. It's good value. It's very cost-effective. It's at whatever latency I need. I want it in my model. Number two, snapshot. It lets me backtest my model. It lets me see just how good Exa is at improving your model. And number three, um consider really adding the holdout set to your competition because otherwise you may overfit. Thank you, everyone.