An LLM in Your App Is Not Agency

We have stretched the word "agentic" so far that it now includes almost anything with an LLM, a tool call, and a diagram.

Add a model to an application. Call a retrieval endpoint. Let the model choose between two tools. Wrap the flow in LangChain or LangGraph. Draw a few boxes. Label one of them "planner."

Now suddenly a normal software system is being described as agentic.

That is too loose.

This is not about using AI to build software. This is not about whether developers are using Cursor, Claude Code, Copilot, or any other AI-assisted development tool.

This is about software that includes an LLM as part of its runtime behavior.

And the central point is simple: an application can include an LLM without becoming agentic.

The model may summarize, classify, extract, draft, retrieve, transform, route, or recommend. Those are useful capabilities. They can create real product value. They can change the shape of the user experience. They can make software feel more adaptive than it used to feel.

But the presence of an LLM does not create agency.

Agency begins when decision rights move into a feedback loop that can revise how it decides.

The industry is converging on a useful distinction between workflows and agents.

Anthropic describes workflows as systems where LLMs and tools follow predefined code paths, while agents dynamically direct their own process and tool use.

I am not trying to throw that distinction away. I am trying to show where it stops.
The question is not: Does this system use an LLM?
The question is: Where does the agency live?

An LLM feature is not agency

The most common pattern today is the LLM-powered feature.

A user clicks a button. The application gathers some context. The application sends that context to a model. The model returns an output. The application displays it, stores it, or passes it to the next step.

That is useful.
It is also not agency.

The application still owns the flow. The user still owns the goal. The model owns a response.

That distinction matters.

A support product that summarizes a ticket includes an LLM.
A marketing tool that drafts campaign copy includes an LLM.
A document platform that extracts fields from a PDF includes an LLM.
A reporting interface that turns natural language into SQL includes an LLM.

None of those are automatically agentic.
They are applications with smarter components.

That is not an insult. Many valuable systems belong in this category. A well-designed LLM feature can save time, reduce friction, improve search, make workflows more accessible, and collapse tedious steps into a better interaction.

But it is still a feature.

The application asks. The model answers. The system continues along a path that was mostly determined before the model was called.

An LLM in your application is not agency.
It is a capability.

But it is a step toward agency

This is the part that gets missed.

Saying an LLM feature is not agency does not mean LLM features are unimportant.

They matter.

Including an LLM in an application is often the first real step toward agency because it forces the system to expose the ingredients that agency will eventually need.

The application has to gather context. It has to decide what information matters. It has to shape user intent into something the model can work with. It has to manage prompts, outputs, permissions, errors, and trust.

It may need retrieval, tool access, memory, evaluation, logging, human review, rollback, and audit trails.

Those are not small things.

A basic LLM feature may not be agentic, but it starts building the muscle. It gives the application a reasoning surface.

That is a big shift.

Traditional software mostly follows paths we define ahead of time. LLM-enabled software can interpret messy input, work with ambiguity, and produce useful output without every path being hand-coded in advance.

That does not make it agentic.
But it does create the surface where agency can begin to emerge.
The mistake is treating that first step as the destination.

An LLM feature gives the system a reasoning surface. An agentic workflow gives that reasoning surface room to choose. A system with agency gives it a durable objective, feedback, memory, and authority to improve its path over time.

That is the north star.
Not simply an application with an LLM inside it.

An application that can pursue an objective through feedback, signal detection, hill climbing, and controlled action.

The goal is not just better responses.
The goal is systems that can improve their path through the problem.

Orchestration is not agency either

The next level up is orchestration.

This is where many systems start to look more sophisticated. The application no longer makes one model call. It chains multiple steps together.

Retrieve context.
Classify intent.
Select a prompt.
Call a tool.
Generate a result.
Validate the result.
Ask for human approval.
Write the result somewhere.

This is where tools like LangChain and LangGraph often enter the conversation.

They are useful tools. They can help structure complex LLM systems. They can make state, routing, tool use, retries, and human checkpoints more explicit.

But using an orchestration framework does not make a system agentic by default.

A graph is not agency.
A graph is choreography.

You can build a deterministic workflow in LangGraph.
You can build a bounded agentic loop in LangGraph.
You can also build a mess in LangGraph.

The tool does not decide the architecture.
The decision rights inside the system do.

A workflow can include an LLM at every step and still not be agentic in any meaningful sense. If the path is predefined, if the system cannot inspect outcomes and change course, if the model is only filling in blanks inside an application-owned sequence, then the system is still mostly a workflow.

Again, that may be exactly what you want.
Many enterprise systems should be workflows.

Determinism is underrated.

Predictable paths, explicit states, and boring control flow are not weaknesses. They are often the reason a system can be trusted.

The problem is not orchestration.
The problem is calling orchestration agency.

There is another trap here: the flat loop.
A system can loop without becoming agentic.

Retry three times. Poll until complete. Regenerate until the validator passes. Search again when no results are found.

Those are loops in shape, but workflows in decision rights. Every iteration runs the same fixed policy. Nothing the system observes changes how it decides. It repeats, but it does not revise.

Imagine a support automation system for a large e-commerce company. The "agent" searches documentation, drafts a reply, validates it, and retries on failure. It has a router that chooses between three search tools. Then it hits a class of warranty refund requests where the documentation is incomplete. The validator keeps failing, the router keeps choosing the same search, and the loop just burns tokens. It can escalate to a human, but it never figures out that the right move is to ask the customer for one missing piece of information. It is not an agent. It is a brittle workflow with a nicer UI.

That mislabeling matters. It leads teams to expect adaptation from a system that only has repetition. They underinvest in instrumentation, outcome signals, policy boundaries, and escalation design because they think the "agent" will figure it out. Then production exposes the truth: the system does not fail like an agent. It fails like a brittle workflow with a more expensive model in the middle.

That is not an argument against workflows.
It is an argument for naming them honestly.

Agency lives in the loop

Agency is not magic.
It does not require consciousness, desire, personality, or pretending the model "wants" anything.

This is closer to the old cybernetics answer, reapplied to LLM systems: agency is a control problem.

Norbert Wiener's cybernetics framed control and communication as feedback problems across animals and machines. That lineage matters here because LLM systems do not escape control theory just because the control surface is now language.

A system becomes more agentic when it can pursue a goal by selecting among strategies, observing outcomes, and revising how it decides within a boundary of authority.

That last part matters.
A feedback loop alone is not enough.

A thermostat has a goal, an observation, and an action. Nobody calls it an agent. The reason matters: its policy is fixed. It maps temperature to switch position the same way forever. It regulates. It does not pursue.

Agency requires the second loop, the one that revises the first.

The system does not just act on feedback. It uses feedback to change how it decides. It can select among strategies, abandon one that is not working, and carry what it learned into the next attempt.

A loop that reacts is control.
A loop that revises is agency.

That is where the distinction begins to appear.

A chain completes steps.
A flat loop repeats them.
An agentic loop changes course.

The discriminator is not the shape of the diagram. The discriminator is whether the system can revise its strategy based on what it observes.

A workflow says: first do this, then do that.
A flat loop says: keep doing this until the condition passes.

An agentic loop says: given the goal, the current state, the available tools, and the observed result, what strategy should I try next?

That shift is the beginning of agency.
Not full autonomy.
Not free-roaming intelligence.
Not a digital employee wandering through the enterprise.

Just bounded decision-making inside a control loop where the system can change how it chooses.

The model may have room to move, but the harness owns the walls.

The agency boundary

When evaluating whether a system is agentic, the useful question is not whether it includes modern AI components.

The useful question is where decision rights move.

Who owns the goal?
Who decides the next step?
Who determines whether the result is good enough?
Who decides whether to retry, continue, escalate, or stop?
Who can use tools?
Who can change external state?
Who owns memory?
Who owns the consequence?

That is the agency boundary.

In a basic LLM feature, the agency boundary sits almost entirely outside the model. The application controls the flow. The model transforms input into output.

In an LLM workflow, the agency boundary is still mostly in the workflow. The model may make local judgments, but the path is largely predefined.

In an agentic workflow, some decision rights move into the model-driven loop. The system can choose tools, inspect intermediate results, recover from failure, ask clarifying questions, and adapt its path inside a bounded goal.

In a system with agency, the system can pursue a durable objective over time. It can monitor, decide, act, evaluate, and adapt across multiple runs without waiting for a fresh human prompt at every step.

That is a much bigger claim.

Most systems called agentic today are not systems with agency.
They are LLM workflows with pockets of model discretion.

That is not a failure. It is probably the correct intermediate architecture for many production systems.

The mistake is pretending the architecture is more autonomous than it is.

The Workflow to Agentic Workflow litmus test

The boundary between a standard workflow and agentic workflow is easy to blur, so here is the litmus test:
Can the model choose an action the developer did not explicitly enumerate?

If the answer is no, the system is probably still a workflow.

A router that chooses between predefined branches is not enough. A retry loop is not enough. A validator that sends bad output back to the model is not enough.

Those are useful control structures, but they are still choreography.

Agentic Workflow begins when the system can revise the strategy inside a bounded goal. It can decide that the current approach is not working, select a different path, gather different evidence, reformulate the task, ask for clarification, escalate for a specific reason, or change the plan based on what it learned during the run.

This distinction matters because a lot of systems look agentic from the outside. They have loops. They have tools. They have model calls. They may even have a node called planner.

But the real question is whether the system can change course in a way that was not simply prewired by the developer.

If not, it is still valuable.
It is still useful.
It may still be the right architecture.
But it is not yet agency.

A maturity model for agency

Here is the simplest way I think about it.

The progression matters.
An LLM feature is not agency, but it is likely to be the first layer of an agentic system.
A workflow is not agency, but it may provide the structure agency needs.

By the industry's current definition, Agentic Workflows are already an agent.

Anthropic's distinction is useful here: workflows follow predefined code paths, while agents dynamically direct their own process and tool use. By that definition, an agentic workflow qualifies. The system has moved past fixed choreography. It can choose tools, revise a plan, and adapt its path inside a bounded goal.

I agree with that definition.
I just think it stops one level short.

Systems with Agency - Level 4 is my claim, not the industry's baseline definition. Most of the industry would already call Agentic Workflows an agent, and that is fine. A system that can dynamically direct its own process and tool use inside a bounded goal deserves the label.

But if we stop there, we miss the bigger architectural shift.

Agentic Workflows revises the plan.

Systems with Agency revises the policy.

That is agency plus optimization.

It needs durable memory, reliable signal, policy revision, and control boundaries that survive beyond a single task. This is also the least solved part of the stack today. Cross-run adaptation is where systems drift, overfit, forget constraints, optimize against bad signals, or quietly expand their blast radius.

So Level 4 is not "more autonomous agent."
It is a different control problem.

The difference is not the presence of an LLM.
The difference is where decision rights move.
And that movement has consequences.

The more decision rights the system has, the more the architecture needs constraints, observability, permissions, rollback, evaluation, escalation, and auditability.

You do not get agency for free.
You inherit responsibility for everything the system is now allowed to decide.

How we get there

The path from LLM feature to system agency is not a jump.

It is a progression of control.

First, the application includes an LLM as a capability. The model can summarize, classify, extract, draft, retrieve, or transform. The system becomes more flexible, but the application still owns the path.

Second, the application starts to orchestrate work around the model. It chains steps together. It retrieves context. It validates output. It routes requests. It adds tools. It creates checkpoints.

This is where many teams stop and call the system agentic.

Third, the system starts to close the loop. It can inspect the result of an action and decide what to do next. It can notice missing evidence. It can retry with a different strategy. It can ask for clarification. It can escalate. It can compare possible paths. It can adapt within a bounded goal.

That is where agency starts to become real.

The first decision right to hand over should be small.
Not "go solve the problem."
Something narrower.

Choose which evidence to gather next.
Choose whether the current result is sufficient.
Choose whether to ask a clarification question.
Choose whether to retry with a different strategy.
Choose whether the system should stop and escalate.

These are bounded decisions with observable consequences. They let the system gain room to move without pretending it has full autonomy.

But you should not hand those decisions over blind.

Before the model gets decision rights, the system needs instrumentation. It needs to record what goal it was pursuing, what options were available, what it chose, what evidence it used, what happened after the choice, and whether the result improved the state of the task.

Without that, you cannot tell the difference between adaptation and noise.
That closes the loop within a run.

Fourth, the system begins to hill climb across runs.

This is the move from revising a plan to revising a policy. The system is no longer just asking, "What should I try next in this task?" It is asking, "Which strategies keep working, which keep failing, and how should that change future decisions?"

That requires signal.

Did this action improve the outcome?
Did the user accept the result?
Did the workflow complete faster?
Did the generated patch pass tests?
Did the customer issue resolve?
Did the sales outreach get a response?
Did the recommendation reduce risk?

Without signal, the system cannot adapt.
It can only move.

With signal, the system can compare paths, learn which actions work, and adjust its future behavior.

That is why full agency is a control system problem, not just a model capability problem.

The model provides reasoning.
The tools provide reach.
The memory provides continuity.
The signal provides direction.
The harness provides constraint.

The control loop turns all of it into agency.

A fully agentic application needs to observe the environment, interpret signal, choose actions, evaluate outcomes, update state, and continue moving toward the objective within defined authority.

That is the architecture!

Not a chatbot.
Not a prompt chain.
Not a graph by itself.

A control system with an LLM inside the loop.

Bounded agency is the enterprise sweet spot

This is where I think many serious enterprise systems are heading.

Not toward fully autonomous agents running wild through corporate systems.
Not toward every application simply having a chat box bolted onto it.
The useful middle is bounded agency.

A system that can reason, but only inside a defined scope.
A system that can use tools, but only with explicit permissions.
A system that can adapt, but only within a visible control loop.
A system that can act, but only within an acceptable blast radius.
A system that knows when to escalate.

This is especially important in regulated, high-consequence, or operationally complex environments.

In those settings, "maximum autonomy" is not the goal.

Useful agency inside accountable boundaries is the goal.

Reliability is not the same axis as agency. Reliability is the harness that determines whether any amount of agency is safe to give the system. A system can take action without being agentic, and an agentic system can still be dangerous if its blast radius is not bounded.

That means the harness matters as much as the model.

The harness defines what the system can see, what it can touch, what it can change, when it must ask, when it must stop, and how its work can be inspected.

The model provides reasoning capacity.
The harness provides authority boundaries.
Without the model, the system may not be adaptive.
Without the harness, the system may not be safe.

Production-grade agentic systems need both.

A system with agency manages an objective

The real shift happens when the system no longer just completes a request.

It manages an objective.
That is a higher bar.

A support system with an LLM might summarize a ticket.

An LLM workflow might classify the ticket, retrieve similar issues, draft a response, and route it to the right team.

An agentic workflow might inspect the ticket, search multiple systems, ask for clarification, identify missing evidence, retry failed searches, draft a response, and escalate when confidence is low.

A system with agency might monitor incoming tickets over time, identify incident patterns, group related failures, open internal tasks, notify the right teams, track resolution progress, update customers within policy, and learn which interventions reduce time to resolution.

That is a different kind of system.
The same distinction applies to software delivery.

An LLM feature might explain a failing test.
An LLM workflow might generate a patch, run tests, and open a pull request.

An agentic workflow might inspect the failure, form a hypothesis, modify code, run tests, evaluate the result, change approach after failure, and request review when appropriate.

A system with agency might monitor builds, detect recurring failure patterns, prioritize fixes, create patches, observe review feedback, update its strategy, and manage the health of a codebase over time within defined engineering policies.

The distinction is persistence.
A workflow completes a request.
An agentic system manages an objective.

The accountability problem

As agency increases, accountability becomes more important, not less.

This is where a lot of the hype gets the story backward.

The human does not disappear as the system becomes more agentic.

The human moves to a different layer.

Less direct execution.
More goal definition.
More constraint design.
More policy setting.
More evaluation.
More oversight of the control system itself.

The work shifts from approving every output to designing the conditions under which the system is allowed to act.

That is a very different skill.
It is also a more important one.

Because once a system can act with agency, the main question is no longer whether the model can produce a good answer.

The main question is whether the system should have been allowed to take that action in the first place.

That is architecture.
That is governance.
That is product design.
That is operations.
That is engineering.

The test

Before calling a system agentic, ask these questions: Who owns the goal?

Can the system choose the next step?
Can it observe the result of its action?
Can it change course based on that result?
Can it revise its strategy?
Can it choose an action the developer did not explicitly enumerate?
Can it use tools with consequence?

Does it maintain state beyond a single request?
Can it operate against a durable objective over time?
Can its decision policy end the run different from how it started?
Does it know when to stop?
Does it know when to ask?
Can a human inspect what happened?
Is the blast radius bounded?
Is someone accountable for the outcome?

If the answer to most of those is no, the system may still be useful. It may still be AI-powered. It may still be a strong product.

But it is probably not a system with agency.
It is an application with an LLM in it.
Or it is an LLM workflow.

Those are legitimate architectures.
They are just not the same thing.

Where does the agency live?

The future is not every application getting an LLM bolted onto it.
The future is also not every workflow becoming a free-roaming autonomous agent.

The future is agency as an explicit design boundary.

Where can the system decide?
Where must it ask?
Where can it act?
Where must it stop?
What does it remember?
What can it change?
What signal tells it whether it is getting better?
How does it hill climb without running away?

Who owns the consequence?
That is the real architecture question.

Not: does this use an LLM?
Not: does this use LangGraph?

The question is simpler and harder: Where does the agency live?

And once we answer that, the engineering work changes.

The future of agentic engineering is not writing every path.
It is building the infrastructure that lets systems find better paths safely.

We stop engineering the route and start engineering the terrain.