Thanks a lot for your time. Really appreciate you dropping by, and it's always a great honor to speak at the World's Fair. So, I'll do my best to give you guys some valuable insights and hopefully make it worth your time.

My name is Maximillian Piras, and today I'll be talking about mouse power—this is a talk about measuring agents through mental models. But before I get into talking about measuring agents, I'm going to talk through a bit about how I use them every day. It might seem familiar to you, but just to level set, we'll go through it.

I tend to background them, like I'm sure a lot of you people are doing as well. While my active attention is focusing on one thing—like perhaps giving this talk to you—I still want to make some progress on peripheral tasks. So I'll keep my attention focused on giving this talk while my agents help me explore some designs in the background, because I think my slides need a bit of work. I've got my design system already set up, and I've given some guidance to my agents. I'll kick off an agent to explore different directions on typography and layout, trying to get as many explorations as possible.

Of course, one agent is never enough, so I like to kick off a bunch in parallel. I've got a lot of slides to get through, so I need all my agents running and exploring in different directions. Hopefully, I can get some interesting ideas to make my slides better before our deadline.

This is generally how I work, and it's probably familiar to many of you: kicking off agents in parallel as much as possible because it feels like there is always more research to do and more design directions to explore. Why not maximize your time by running agents concurrently while focusing on your main task? It is a lot of fun, until you get the bill. Then you start to wonder: was it all worth it? Did you vibe code too hard? Were you token maxing too much? Could you have been more efficient in how you sequenced your agents? That brings us to our core question: how do we value token costs, and specifically, how do we help our customers value them?

For the past year and a half, I've had the pleasure of working as the founding designer at a company called Yutori, where we focus on computer use models. These are models that learn to use a computer just like a human would. The primary use case arises when you cannot get information through an API or an MCP—why not send an agent out to use the computer like a human, extracting and manipulating data that wasn't previously accessible? Obviously, this is less efficient than native APIs and MCPs, but as a last resort, having an agent operate a computer gets the job done.

Here is an image of the Yutori agent using the Yutori website, checking out its own benchmark—admiring itself, in a way. A significant part of my role as founding designer is talking to customers to understand how we can make agents as intuitive as possible, and figuring out the mental models they use to evaluate the use cases they assign to agents.

So far, many customers seem quite confused. People are excited about agents, but a phrase that comes up repeatedly is that they feel they are "just scratching the surface." It is not yet intuitive how best to utilize them. In my customer discussions, it consistently comes down to questions like: What is the best way to use agents? What are the ideal use cases? How should we evaluate the trade-offs between token costs and actual value? We are still building this organizational muscle.

This brings me to the core thesis of this talk: agents have a fundamental measurement problem. Here is a photo of me at work attempting to measure agents; a coworker took it and remarked that I looked like I was trying to solve the Pepe Silvia conspiracy. It is not an easy task.

Some of you might think: "Hold on, what is this guy talking about? I have a fleet of agents working for me right now, building our next million-dollar app, and I have no problem measuring them." While that may be true for early adopters in this room, remember the mandatory Upton Sinclair quote: we are extremely biased early adopters. Our excitement does not represent the broader audience we ultimately need to help adopt this technology.

Whether directly or indirectly, we are all selling tokens. Is our personal token usage representative of people who have never touched an agent? Many people are still manually copying and pasting into ChatGPT. I might even be married to one of them—despite my efforts, she hasn't let me set her up with agents yet. When thinking about scaling adoption across the globe, we must remember this emotional and cognitive gap.

It really boils down to the age-old problem of introducing a new technology. There is plenty of history we can study to see how people solved this in the past. We have this exciting new capability, but we haven't yet figured out the right framework to communicate it.

For this talk, let's go back to the 1700s and take notes from when James Watt was trying to sell steam engines. At the time, he decided that a prime use case for his engines was replacing the horse gin—the standard power source for mills and breweries used to grind barley and run machinery. The standard setup was simple: you hitched a horse to a rotary arm, the horse walked in circles, and that generated your power. It might seem primitive today, but it was ubiquitous back then.

Watt recognized that an efficient machine was vastly superior to a horse. However, he correctly understood that a major adoption barrier was cognitive dissonance. How do you convince people who think entirely in terms of horses to adopt an intimidating, unfamiliar machine that claims to solve all their problems when they can't visualize the benefits? Furthermore, regardless of efficiency, horses had great "vibes"—it is hard to compete with emotional attachment.

Watt knew he had to overcome emotion with tangible ROI calculations. His solution was to meet people within their existing mental model by creating a metric that established a baseline of relative efficiency. He studied horse gins, estimated the average mechanics and performance of draft horses, and coined the term "horsepower."

He used horsepower to quantify the general output horses were generating, providing a comparative baseline to demonstrate the exact efficiency multiplier a steam engine delivered. The metric was not strictly scientific or perfectly accurate at the time, but its primary function was communicating value. It allowed horse-reliant business owners to calibrate potential efficiency gains and gave them the confidence to try a steam engine. The crucial lesson is: if we cannot provide a tangible ROI for our customers, communicating real value becomes nearly impossible.

We only need to look at our own industry to see countless examples where technology teams struggle to calculate sound ROIs. In this room, we might assume this is a solved problem. But look at brilliant engineers who aren't deeply immersed in AI: they end up blowing through their entire annual token budget in a single quarter or obsessing over token leaderboards without clear business alignment.

We find ourselves in a doom loop of "overspending and underusing"—a term borrowed from a great blog post by Ramp. It is a vicious cycle where teams token-max themselves into austerity, pull back, and then re-enter the loop once FOMO strikes again. We must break this loop through better metrics that clearly communicate value.

Some organizations are heading in the right direction. The CEO of Coinbase recently shared a chart on X illustrating how changing internal routing defaults—reserving frontier models exclusively for the hardest tasks—allowed their AI value creation to diverge from raw token spend. Ramp highlighted similar findings.

However, the prevailing problem is an excessive focus on tokens. Tokens are a useful metric for internal system observability, but at the end of the day, tokens are merely an output. Tokens must be cleanly mapped to business outcomes: How many bugs were squashed with that token spend? How many customer support tickets were closed? How much measurable progress was made against core objectives? Without a tight coupling between spend and business outcomes, sustainable ROI remains elusive.

This dynamic is not just an abstract industry problem; many of us experience it firsthand. While we all enjoy coding alongside agents and feel a noticeable boost in personal velocity, teams are increasingly suffering from "death by a thousand pull requests." Even Anthropic, where some team members have claimed to solve code generation, openly admits that code review remains unsolved.

The bottleneck has shifted decisively to human verification. Efficiency gains from coding agents stall because engineers spend the majority of their time reviewing generated code, having not yet figured out how to scale verification in tandem with generation. When verification cannot match the speed of creation, measuring quality and calculating true ROI at scale becomes impossible. We generate vast amounts of code, but cannot efficiently verify whether enough of it is production-grade to justify the token expense.

Code review may have always had systemic flaws, but agents are exposing the cracks. As Noah Hein noted in a post about solving code review, the underlying assumptions of code review must be fundamentally revisited. We need to examine our priors to establish a new operational baseline for reviewing code in the agentic era.

Why does code review feel solvable? Because engineering culture has achieved strong convergence around shared assumptions and structured rubrics. Our immediate challenge is adapting those assumptions for the agentic age. If we succeed, we can transition from execution at the speed of compute to measurement at the speed of compute. Those measurements must align directly with the mental models of users, reinforcing the rule: if you build an agent for a task, you must equally build the methodology to verify its output.

This brings me to the core concept of "mouse power"—the potential equivalent of horsepower for the agentic age. Just as James Watt demonstrated efficiency gains relative to the horse gins of his era, we must establish a baseline for how people use computers today and demonstrate the performance multiplier an agent delivers across specific vectors.

Of course, this is not as simple as tracking cursor coordinates across a screen. I actually joked with this idea and had Claude vibe-code a cursor movement measurement tool, thinking I could quantify mouse velocity to derive a clean metric. That is obviously a fool's errand because the information space of computer use is far too high-dimensional. Mouse power is not a literal physical metric; it is a mental model. The core principle is: if you sell someone an agent, you must also provide the rubric to verify that the agent is performing valuable work, proving that the consumed tokens are justified.

To operationalize this, I turn to information theory—specifically Claude Shannon's concept of entropy, representing uncertainty within a probability distribution, which underpins modern training techniques like cross-entropy loss. Entropy provides a compelling lens for evaluating not just agent capability, but the nature of the tasks we assign to them.

I mapped this into a two-axis matrix:

• The X-axis represents the uncertainty in task steps. Booking a flight has very low step uncertainty: you need a departure, destination, date, and seat selection; these structural requirements are invariant. In contrast, painting a masterpiece has near-infinite step uncertainty with no predictable operational pathway. When step uncertainty is extremely low, do not waste tokens—write a deterministic script. When step uncertainty is excessively high, the task is likely out-of-distribution for pre-training and suffers from sparse rewards in reinforcement learning, making it unsuitable for reliable agent modeling. The sweet spot lies in the middle.
• The Y-axis represents the uncertainty in acceptance criteria. When acceptance criteria uncertainty is high, verification becomes indistinguishable from execution—a person must essentially redo the entire task to confirm its validity, destroying any token leverage.

The ideal deployment zone consists of tasks with moderate operational uncertainty paired with low acceptance uncertainty. These mirror NP-style problems: tasks that are significantly easier to verify than to execute. When verification follows a reliable, repeatable pattern, you can deploy agents to execute the work and deploy secondary verifier agents to validate the results. As you build your next generation of agents, ensure you simultaneously design their verification mechanics and quantify their mouse power.