Case study
A natural-language agent that drives food ordering, grocery shopping and restaurant table booking end to end. Orders are cash on delivery and cannot be cancelled, so the interesting engineering was never fluency. It was building a system that fails visibly instead of confidently.
Most agent demos stop at the first tool call. The hard part of a real ordering flow is the chain: find an address, search restaurants, open a menu, build a cart, apply a coupon, place the order, and track it, with each step depending on the last. Any link can fail, and the model will happily invent a plausible result for a step that did not actually happen.
That is tolerable when the output is text. It is not tolerable when the output is a real order that cannot be cancelled and will be paid for in cash at the door. So the design question was not "how do I get the model to do this" but "what must the model be structurally unable to do".
I built it against a published Model Context Protocol manifest from Swiggy, an Indian food delivery and quick commerce platform, covering three separate surfaces: restaurant food, grocery delivery, and dining out reservations. 28 tools in total across the three.
Chat UI (streaming SSE, Zustand timeline store)
|
/api/chat -> agent loop -> provider seam (one OpenAI-compatible client)
| NVIDIA NIM (default) | Groq
|
guard layer <- the only path to the backend
|
MCP client router -> food | groceries | dineout
|
+-- in-process mock MCP server (default)
+-- StreamableHTTP adapter (same Client interface)
Three seams let behaviour swap without touching callers: the LLM provider, the backend transport, and the guard layer. Every call to the backend goes through one function, which is what makes the guarantees below enforceable rather than aspirational.
There is a hard Rs 1,000 ceiling on food carts. It is not an instruction the model is asked to follow. It is a pre-call check that reads the live cart, compares the total, and returns a failed tool result before the order tool is ever reached. A model cannot talk its way past a conditional. Prompt-level limits are a suggestion; this is a wall.
When a guard refuses, it returns the same shape a normal tool error returns. Nothing synthetic is spliced into the message history. The model reads the refusal, understands the constraint, and adapts, usually by telling the user to remove an item. The conversation stays coherent and the history stays clean, which matters because the history is replayed on every later turn.
Order placement and checkout are not idempotent. If the network drops the response, a naive retry places a second order. So those calls are wrapped: on a failure, the agent queries the order list, and if an order appeared it returns that one instead of retrying. The retry only fires when the system can prove nothing happened.
The same wrapper would fit table booking, but there is no bookings list endpoint to verify against. Without a way to check, a retry could double book a real restaurant table. I left it un-retried and wrote down why. An unverifiable retry is worse than a visible failure, and a documented gap is worth more than a silent guess.
A tool name arriving off the wire is sanitised for control tokens and namespace prefixes, then validated against the live tool list before anything is routed. An unknown name returns a plain error result rather than reaching the backend. After repeated unknowns the loop stops offering tools at all, so a confused model ends its turn in text instead of burning every round on the same failure.
Tool results are replayed into the model's context on later turns. A cart from three turns ago is not a fact, it is a stale photograph, and a model handed both will cheerfully merge them into a cart that never existed. Tools that mutate or report the cart are marked as snapshots so a later call supersedes an earlier one in history.
Dining out works in latitude and longitude. Food and groceries work in an address id. Passing one where the other is expected does not raise an error, it returns a confident wrong answer, which is the worst failure mode available. The types keep them apart at the boundary.
Model selection was benchmarked against the real 28-tool schema set rather than a toy example. Four models tied on single-shot tool routing, so the tiebreak was a multi-round test. The chosen model chained addresses to search to menu; models five times faster stopped after one hop. The entire value of this agent is the chain, so the default is the reliable one and the fast one is a single environment variable away.
The live endpoints are invite only, so I wrote an in-process mock server mirroring the published manifest and put it behind the same client interface the real transport uses. Development never waited on access, and the real adapter is a URL and a token away rather than a rewrite.
Caching the tool inventory for the process lifetime is correct in production and wrong in development. Editing the mock registry hot reloads that module but leaves the cache holding the previous tool set, while the system prompt reloads. The prompt then steers the model at a tool the list no longer offers, the loop rejects it as unknown, and after two strikes gives up and reports a missing capability. The bug reads as "that whole surface is broken" when nothing but the cache is stale. It cost an evening, and it is why the caching is now conditional on environment.
Every request also ships all enabled tool schemas, roughly 7,100 tokens for all three surfaces. On a free tier the daily token cap binds well before the request rate does, so the number of enabled surfaces became a deliberate lever rather than an afterthought.
Ten architecture decision records cover the choices above and the reasoning behind each. The source is private at the moment. I am glad to walk through it, or share access, on request.