A model that acts
Three architectures, three bills, and the question to ask before writing a single loop
A customer service desk receives a question: "Has my order 4417 shipped?" Three implementations answer it. The first sends the question and the order record in a single call to the model, then returns its answer: 900 tokens, 400 ms, and an identical result on every run. The second runs a sequence written in advance: extract the number, call the tracking API, write the reply. That's three calls, a single path, and one test per step. The third gives the model four tools and lets it decide: anywhere from 2 to 30 calls, a different trajectory on every run, and a bill that no one on the team can predict in advance.
All three answer the simple case correctly. The difference only shows up at the edges: the order doesn't exist, the tracking API returns a 503, the customer mentions two orders in the same sentence. There, the single call makes things up, the fixed chain breaks at the anticipated step and stops, and the agent tries something else. That ability to try something else is the only thing you buy by moving to an agent, and it's expensive: in tokens, in latency, in attack surface, and in hours of debugging. This entire lesson is about deciding whether you need it, because the honest answer is no more often than people claim.
Comments
Comments are powered by GitHub Discussions
Sign in with GitHub to join the discussion