AI Agent 能跑通，但企业敢信任吗？规模化落地的价值与治理框架

[Speaker 0]: 没有有没有没有异据 goal。 [Speaker 1]: life proves that a system exists, but it doesn't prove the people behind the work will actually use it or that it will ever deliver a single dollar from the deal model. [Speaker 1]: i am michellebrand creator of leading at scale. [Speaker 1]: and the question every enterprise is confronting right now, the the gyt works of anyone trusted. [Speaker 1]: i has spent over twenty years in enterprise transformation and more than a decade leading emony integrations across media tech and consumer boods. [Speaker 1]: on those deals trust was not something we crossed our fingers in hope for after launch. [Speaker 1]: we engineered it before day one through early change management and risk management. [Speaker 1]: so when the platfam went live, the teams on the ground actually relied on it. [Speaker 1]: ai agents demand the exact same muscle and agent earns trust when it delivers the value the business budget ted for while stainks lee within the companies approved risk appetite policies and regulatory guardills. [Speaker 1]: in my experience, leadership gets there by answering three straight forward questions. [Speaker 1]: what value doesn't create who is on the hook for it, and what evidence gives it permission to run with greater autonomy. [Speaker 1]: the moment in agent steps out of a standbox and into a cord isn't workflow. [Speaker 1]: the entire test changes, a pilot test whether the software functions, but daily operations is where we test whether it creates value within the controls set by leadership, established AR risk frameworks. [Speaker 1]: give you strong principles. [Speaker 1]: what they do not give you is how to apply these principles to each egyt for every agent leadership has to decide four things. [Speaker 1]: what is it permitted to do, who owns it, what evidence it records and what triggers in escalation that work is not an engineering task. [Speaker 1]: we find it is an executive operating mandate. [Speaker 1]: and today, i'll show you how to meet it.

[Speaker 1]: what is ground this in a real world scenario? [Speaker 1]: picpicture, a fifteen billion dollar opposition targeting four hundred million in nets energies. [Speaker 1]: the deal has already close and the integration efforts are now under way. [Speaker 1]: the integration management office, which real also referred to have the im o today is driving the nergy ed roadmap across forty workstrings, covering everything from raalizing render contracts to consolidating ERP systems, and every single initiative has a business sywner accountable for its bottom line savings. [Speaker 1]: every week, analysts in the IMO are going to reconcile the numbers and against the general leger as they track down status from all forty leads. [Speaker 1]: they're also comparing actuals against the plan, then finally assembling the executive deck for the sterian committee, which is going to include the cfo, the chief risk officer business unit heads and the integration lead the engineering team, builds an agent to automate that for a dropped, the agent, then in just the weekly updates reconcelthe the leger runs variance map against the plan and flagged the initiatives that are falling behind. [Speaker 1]: now thing looks good. [Speaker 1]: and they decide to run a for wek pilot, the agets stress match the analyst reports line for line. [Speaker 1]: so we can say that the tech works. [Speaker 1]: now the question is, what that value does that staring committee need before before they trust st it to run the business? [Speaker 1]: first, the staring committee need in written value case agreed upon before the agent ever touches production data in our example that comes down to three specific matrix. [Speaker 1]: first, let's look at cycle time. [Speaker 1]: if that weekly report, it's five business days to compile right now. [Speaker 1]: you're the lonest of of hard ararget, such as two days, your second metric will be analysts capacity. [Speaker 1]: if four analysts spend half of their week building the stuack, that is eighty hours a week. [Speaker 1]: the agent gives back that time counts as value only when the integration lead assigns it to the workstreams that are behind on their syergy targets need help getting back back track that you only calt that value. [Speaker 1]: after nedding out the compute licensing and human review costs of unning ning dagent, your third metra is the most important one detection time. [Speaker 1]: that is how long it takes from the first sign of a slip in the data to the moment, the stearing committee see ve it cutting the report from five days to two days does not guarantee that the committee sees that slipped any soon. [Speaker 1]: the egypt pass to catch it. [Speaker 1]: and the valvalent case measures that directly every single metric requires a predeployment baseline. [Speaker 1]: if the IMO cannot stay the baseball today, do not deploy. [Speaker 1]: if you can't measure where you started, you can never prove the agent move the needle next.

[Speaker 1]: 嗯, the staring committee needs to know who owns the risk. [Speaker 1]: the agent creates tes wegoing going to apply the three line model to the agent in the first line. [Speaker 1]: ririement owns the controls, the integration ment lead. [Speaker 1]: if the accountable executive she owned agents output exactly the way she owned the manual debt in the fact line risk and complaints oversee the agent and challenge how to run it. [Speaker 1]: how did they go depends on the agents exposure an internal draft roped by a senior analyance warrants, a very different reviews ntered than at an autonomist agent publishing figures straight to the board in the bird line. [Speaker 1]: internal audit provides assurance deciding based on material risk when and how the egypt enters the annual audit plan. [Speaker 1]: eeent also needs one and play assigned to oversee it every day in our example, that is the senior analyst. [Speaker 1]: he refused the draft before it reaches the community. [Speaker 1]: how role requires for specific capabilities. [Speaker 1]: she has the first understand the operational boundaries. [Speaker 1]: she has to interpret its output with context. [Speaker 1]: she must have the authority to override or reverse, and she needs clear, kill switch to shut it down to a leave bus bucket. [Speaker 1]: it behits unexpectedly, accountability also means defining where the agents authority spots i document this in a one page policy called the human work standard for every fork in the process. [Speaker 1]: it matps the decision the human role authorized to make it the required evidence and the escalation path the decisior exist at every tear. [Speaker 1]: the senior analyst decides whether of arians is an encounting timing difference or a real cash shortfall. [Speaker 1]: the worextreme lead decides how to remediate that shshortfall and the integration leaade is going to devide, whether to normally flag and initiative as at risk to the syarring committee, the agent prepares the analysis behind every one of those decisions. [Speaker 1]: and it stops there, the person whom makes the decision owns the conincipants. [Speaker 1]: so keeping these decisions with your team does not slow the agent down, it can go into use, while each role keeps its dirty. [Speaker 1]: the one reason to the light delay, dedeployment is a decision with no assigned owner. [Speaker 1]: this means if relationship has not agreed who declres an initiative off trp, the agents flags lead to no action.

[Speaker 1]: the third question is where the engineers in this room come in the evidence for greater autonomy comes from with how the agent is built. [Speaker 1]: i'd look for three artifects. [Speaker 1]: the first is an auditable decision log for every workstream. [Speaker 1]: the agent evaluates it records the road data injusted. [Speaker 1]: the tool cross executed the prompt and the model versions output generated, along with the human sign of the two ooineing choices, make that loloair tight first. [Speaker 1]: first, your core map and reconciliation rules must run in deterministic finance approved code, not generated as conversational prose from an LM and d treat the models written summary as a helpful explanation. [Speaker 1]: not proof, your real proof lives in the teleetary. [Speaker 1]: what records were pull, what deterministic tools ran and who signed the review? [Speaker 1]: if reported savings do not match, the general ledger agent does not touch the number it flags, discrepancy and roots it up. [Speaker 1]: the second artifact is the model inventory record. [Speaker 1]: this captures the accountable executive data liniendependenccies across source systems, validation results, escalation count and the chill switch run book. [Speaker 1]: most importantly, it documents your regression triggers, which are any upbeits to a prompt base model or source api that require full revalidation before running in protection. [Speaker 1]: again, your third artifact. [Speaker 1]: this is going to be the autononomies or card. [Speaker 1]: the steering committee uses it to grant autonomy in four stages stage. [Speaker 1]: one is shadow moen. [Speaker 1]: the analysts built the report as they do today. [Speaker 1]: the agent runs right along side them, and the team compares to two stage two is the supervised draft, the enger compitiles. [Speaker 1]: the first draft and the senior analysts reviews every single line item 没有 stage. [Speaker 1]: three is the exception and samle, where the analyst reviews land initiatives plus a random sampll of the cliims. [Speaker 1]: then we have stage for where exexceptions only the analyst revieonly only ririst anonyales, supported by periodic independent sampling pickeles attention to stage for. [Speaker 1]: however, because what's humans only at exceptions, your biggest vulnnerability is what the agent failed to flag. [Speaker 1]: so i advise tracking mss material issues through independent sampling, monitoring arroseverity and watching your override rate carefully. [Speaker 1]: a falling override rate does not automatically mean the agent is getting smarter. [Speaker 1]: it can also mean the dept of review has lits. [Speaker 1]: so independent stmbling tells you which one of these you are seen the square card, puts those control metrics side by side with your value metrics cycle time, analyst hours deployed at your detection time. [Speaker 1]: and with that single page view, this staring committee can make clean efficient data back decisions to either promote the agent, hold it where it is, stepped back or down to supervised review stage. [Speaker 1]: so let's come back to the three questions. [Speaker 1]: ask some of every agent you build front or govern, what value does it create? [Speaker 1]: cool is accountable for it and what evidence permits it operate with autonomy in our emeninexvambel. [Speaker 1]: today. [Speaker 1]: the agent did not magically creep before hundred million and thergies procurement, supply chain and operating leaders capture that value through the greedy work of integration, but what the agent delivers is clear. [Speaker 1]: verified evidence weeks, i had a schedule giving leadership the runway to intervene, while the deals cenergies can still be captured. [Speaker 1]: building in the engine. [Speaker 1]: that works is just the first step. [Speaker 1]: answering those three questions is what earns it the trust to run at scale. [Speaker 1]: thank you.