Chapter 15 of 36 · ~1 min

The Agentic Application Architecture

Inside a real application an agent is one component among many. There is the model; the prompts that shape it; the tools it may call; the state of the current task; memory across tasks; retrieval; orchestration that decides which step runs next; ordinary application logic; the APIs it talks to; and the interface a person sees. Most of these existed long before language models did.

Drawing this diagram for your own application is the fastest way to see where the uncertainty actually lives. Usually it is in two or three boxes. The rest is software you already know how to build and test.

Inside an agentic applicationInterfacewhat a person seesApplication logicordinary codeOrchestrationwhich step runs nextAPIssystems it talks toStatethe current taskPromptsshape the modelModelthe uncertain partTool decisionsthe model choosesMemoryacross tasksRetrievalfetch, then readinvolves the model: here the uncertainty livessoftware you already know how to build and test
An agent inside a real application. Orange boxes involve the model; blue ones are ordinary software.

Exercise

Do it yourself

Take a small task and place each of its steps into one of the boxes above. Count how many boxes involve a model.

Big question

How much of an "AI application" is AI?