メインコンテンツに飛ぶ

We'd prefer it if you saw us at our best.

Pega.com is not optimized for Internet Explorer. For the optimal experience, please use:

Close Deprecation Notice
Team presentation

Your AI agent will do what’s possible, not what you meant

Cassandra Skuqi,
シェアする Xで共有 LinkedInで共有 Copying...
ログインしてブログを購読する

Imagine you've just finished a long day at work and you're heading to your Tuesday kickboxing class to blow off some steam. You check in at the front desk, only to find out your reservation has been canceled.

You didn't cancel it. The person at the front desk didn't cancel it. An AI agent did.

That's essentially what happened in the recent OpenClaw gym incident, and while the story has been framed as an example of an AI agent "going rogue," I think the more interesting lesson is actually much simpler: The agent just did what the system allowed it to do.

It found a capability that was available to it and used it to accomplish the task it had been given. The problem wasn't necessarily that the agent misunderstood its instructions. The problem was that there was a gap between what someone intended the agent to do and what the system actually made possible.

And that's a distinction that enterprises are going to have to get very comfortable with as AI moves from answering questions to taking action. We've spent the last few years talking about AI risk largely in terms of model behavior: hallucinations, bad reasoning, inaccurate recommendations, biased outputs.

Those problems aren't going away, but agentic AI introduces another category of risk: What happens when an agent does exactly what the system allows it to do?

Your agent doesn't know what you meant

For years, enterprise technology has gotten progressively better at controlling access. Who can get into a system? What can they see? What can an application do? What can a particular user change? Agents complicate that equation because they're not just accessing information. They're now acting on our behalf, often across multiple processes and systems, each with their own level of access and governance.

And an agent doesn't understand your organization's intent in the abstract. It understands the capabilities you've made available to it.

Imagine an organization intends for an agent to help employees manage appointments. The underlying system might expose capabilities to modify appointments, reassign them, access customer records, change queue positions, or remove reservations.

A human might look at those as separate functions with obvious boundaries. We bring context to the interaction: professional norms, social expectations, judgment, even basic empathy. Having the ability to remove four people from a waitlist doesn't mean most of us would consider doing it just to get ourselves into a class. An agent doesn’t operate under the same unwritten constraints. It sees available actions and a goal to accomplish.

Give an autonomous system enough access, enough connected systems, and enough latitude to act, and individually legitimate actions can be combined into an outcome no one explicitly designed. That's where the conversation about AI governance needs to expand. Because governance can't live entirely in the prompt.

Your enterprise architecture has to know the difference

The answer isn't to put every agent in a box and hope a sufficiently restrictive set of instructions keeps it there. It's to give agents a governed environment in which the boundaries of their work are already defined.

The uncomfortable reality is that most enterprise systems were designed around human judgment. We assumed people understood context. We assumed they knew the difference between what they technically could do and what they should do. A person with permission to cancel appointments doesn't interpret that permission as “cancel anybody's appointment whenever doing so helps me achieve my goal.” Agents are now making those assumptions visible.

The goal, then, isn't to teach every new agent how your business works from scratch. It's to define the rules – once, within your enterprise architecture – and then let agents operate inside them.

Workflows, decisions, policies, and rules can be established in a governed layer and reused across the enterprise. Systems of record remain authoritative. Experiences and agents can change. New channels can come and go. But the business logic that determines how work should happen doesn't have to start over every time. That's what makes predictable outcomes possible even as the technology around them changes.

This is particularly important as organizations begin designing agentic workflows at scale. With Pega Blueprint®, teams can design processes that incorporate business rules, best practices, governance requirements, and execution boundaries from the start. The agent doesn't have to independently determine how the business operates. It operates within a workflow that already knows.

And when something goes wrong, you need to know why

There's another lesson hiding in the OpenClaw incident: accountability. When an autonomous system takes an unexpected action, "the AI did it" isn't an acceptable answer for a CIO.

Organizations need to be able to trace what happened. What action was taken? What information was used? What rules applied? Where did the decision come from? Was approval required? Who (or what) authorized the action?

Predictability doesn't mean promising that nothing unexpected will ever happen. It means that when something does, the enterprise can see it, understand it, and correct it. That's why auditability and governance are the foundation of an agentic enterprise. The same workflows that govern what an agent can do should also make its actions visible and accountable.

The next enterprise architecture is being built for agents

Enterprises are introducing AI agents into decades of legacy processes, overlapping systems, embedded rules, and business logic. Moving faster is only useful if the organization can still trust the outcome.

That's what makes modernization such an important part of this conversation. The more quickly technology changes, the more valuable a stable, governed foundation becomes.

Vodafone is already putting this idea into practice. Using Pega Blueprint™, the team moved from legacy workflows to working products in the hands of the teams that need them in 48 hours. No doubt an impressive number, but speed at that scale depends on knowing what needs to stay consistent as everything around it changes.

When business logic, workflows, and governance are built into the architecture, organizations have room to modernize without reinventing the rules of the business along the way. New experiences can emerge. New agents can take on work. The underlying processes remain governed and understandable.

That ability to absorb change while preserving predictable outcomes is going to matter more as agents become more capable. Every new autonomous actor also expands the scope of what enterprise architecture has to account for.

We've traditionally asked: Who has access? What can this application do?

Now there's another question: What outcomes can this autonomous actor create by combining actions across systems?

The organizations that handle that question well will have an advantage. Deploying an army of agents means very little if no one can confidently predict what those agents will do once they're connected to the business.

The real work is building an environment where agents can move quickly, while the enterprise remains in control of the outcomes they create.

Because your AI agent will do what's possible. The enterprise has to make sure what's possible is what it actually intended.

Not sure where to go next? Build a Blueprint to get started. 

タグ

トピック: AI・意思決定
トピック: Agentic AI
製品エリア: インテリジェントオートメーション
シェアする Xで共有 LinkedInで共有 Copying...
Blueprintを作成する準備はできましたか?
ニーズに合った変革エンジンを選択してください。
ワークフローおよびアプリ設計向け

プロセスを再設計し、あらゆるワークフローを構築可能なアプリケーションに自信を持って変換します。

Pega Blueprint®
PEGA BLUEPRINT™
マーケティングおよびCX戦略の設計

すべてのタッチポイントで顧客ジャーニーとエンゲージメント戦略を可視化し、活性化します。

Pega Customer Engagement Blueprint™
シェアする Xで共有 LinkedInで共有 Copying...