A Model That Cannot Write a Sentence: Routing WhatsApp ERP Queries with a Decision Model

A model that cannot write a sentence just became the most interesting thing in my chat agent stack.

Here is why.

I build WhatsApp assistants that answer questions straight from a factory ERP. Sales, purchase orders, invoices, outstanding. The people asking run factories. They type in Hindi, Gujarati or English, mostly in Roman script. A typical message looks like this:

Reliance ka pending PO?

Four words, two languages, no punctuation, and a party name that matches four different rows in the party master. The assistant has to turn that into one correct number, or admit it is not sure. I wrote about the first generation of these bots in Building AI Chatbots That Actually Serve Real Customers. This post is about what changes when you stop asking a chat model to do the routing.

Every assistant over company data has three jobs

1. Decide which question was asked. Pending purchase orders, outstanding receivables, last invoice, monthly sales. One of a fixed list.
2. Decide which record is meant. Which "Reliance". Which period. Which PO number.
3. Write the reply.

Job 3 is where hallucination lives. In a numbers domain I do not let a model anywhere near it. The reply is a template. Every figure in it comes from a query result set. The model never emits a digit, never writes SQL, never sees a row.

That leaves jobs 1 and 2. Both are closed-set choices. Pick one of 25 capabilities. Pick one of 4 parties named Reliance. There is no open-ended text to generate. There is a list, and the right answer is on it, or it is "none of these".

Until now I used a general chat model for those two picks, because that was what was available. It works. It also returns a string, which means parsing, validating and retrying whenever the string is not quite one of the options.

Walking through the pipeline

The diagram above is the whole agent. Grey boxes are plain code. Blue and orange boxes are the two decision points. Green is the template.

Menu tap or free text. Most messages are menu taps. A menu tap needs no model at all, because it already carries the capability id. Only free text continues down the pipeline.

Job 1: which question. The decision model gets the typed sentence and the list of 25 capability ids, each with a one-line condition describing when it applies, plus a 26th option: "none of these". It returns one id and a calibrated probability for each option. For "Reliance ka pending PO?" that is the pending-PO-by-party capability at 0.91.

The confidence gate. This is code, not a model. If confidence is 0.75 or higher, proceed. Below 0.75, send the menu and let the user tap. The bot never guesses.

Job 2: which record. The capability declares which slots it needs. Here that is a party and a period. Code fills the candidate lists. A trigram search on the party master turns "Reliance" into four rows. A regex finds PO numbers and dates. The decision model is then asked a second closed-set question: which of these four candidates, or none? Low confidence here means the bot asks the user to tap the right party. Again, no guessing.

The capability registry. Twenty-five human-written, reviewed, parameterised SQL queries. Each one maps to a capability id. The model never writes SQL. It never sees the schema beyond the catalogue of ids and conditions.

Job 3: the template. The query runs, the rows come back, and a template renders them:

PO 4521 · Reliance Industries. Ordered 12,000 kg · received 7,400. Short 4,600 kg · as of 14:32.

Every number is from the result set. The timestamp is there so nobody treats an old answer as live.

What the decision model is sent:

• The typed sentence
• The list of capability ids with one-line conditions
• Four candidate party names from the trigram search

What it is never sent:

• Any row from the ERP
• Amounts, quantities, invoice numbers
• Credentials, or schema beyond the catalogue

What a decision model is

A decision model is a model that does not generate text. You give it a question and a list of options. It returns one value from that list, plus a calibrated probability for every option.

TypeSafe Jev is the one I have been mapping against this pipeline. It is structurally incapable of returning option 26. There is no string to parse, nothing to validate, nothing to retry. The output type is the list you gave it.

That is a much narrower tool than a chat model. For jobs 1 and 2, narrower is the point.

What it costs

I priced it against the two models already in my pipeline. One routing decision is a WhatsApp message plus 25 capability options with conditions, roughly 1,200 input tokens and 60 output tokens. List prices, checked on 9 October 2026 on each vendor's own pricing page.

Per 1,000 routing decisions:

• Claude Haiku 4.5: $1.50 list, $0.60 with prompt caching
• GPT-6 Luna: $0.15 list, $0.06 with prompt caching
• TypeSafe Jev 1.13: $0.05, output tokens are free, nothing to cache

Honest take: at a few hundred free-text questions a month, all three bills are pocket change. Less than a cup of tea. Cost is not the reason to look at this.

The reason is the confidence number

Ask a chat model how confident it is and it guesses. The number is self-reported, and it is not calibrated against anything. A 0.9 from a chat model does not mean it is right 90 percent of the time. It means the model produced the token "0.9".

Jev is trained so that 0.9 means right about 90 percent of the time. That is what calibrated means. And a calibrated number lets you write one rule and defend it in front of a director:

Below 0.75, do not answer. Show the menu.

Factory owners do not want a bot that always has an answer. They want one that says "not sure, pick one" and is right when it does answer. A threshold on a calibrated probability is how you get that behaviour without a prompt-engineering ritual every time the capability list changes.

What comes back, side by side

Output shape. Haiku and Luna return any string. You parse, validate and retry. Jev returns a typed value only. It cannot be malformed.

Confidence. Haiku and Luna: self-reported, not calibrated. Jev: a calibrated probability per option.

Response time. Haiku and Luna: roughly 0.5 to 1.5 seconds for a short structured call. That is a typical range, not a vendor claim. Jev: 70 to 500 ms, vendor stated.

Writes prose. Haiku and Luna: yes. Jev: no. Not what it is for.

Non-English. Haiku and Luna: strong. Jev: English first, test on your own data.

Hosting. Haiku and Luna: US. Jev: US West Coast, early access.

What it cannot do

From the docs, not from marketing:

• Pull a name out of the sentence. It does not extract. Code generates the candidates, Jev picks one.
• Hindi and Gujarati. English first. I am still testing on my own data, and romanised Gujarati is the hard case.
• Prose. Keep a template for that. In a numbers domain you should anyway.
• Early access only, US hosted. If a client's data residency rules say India, that is a conversation to have before any test. Though note what it is sent: a sentence and a list of names, never a row.

Where I am with it

I have not switched a client to it. I have mapped it onto the pipeline, priced it, and queued an offline test on my 50-question set. Those are questions in the mix of languages and scripts users actually type, each labelled with the capability that was meant.

The test is simple. Run every question through Jev. Plot confidence against correctness. If the confidence scores separate right from wrong on romanised Gujarati, it becomes the router in the next build. If they do not, it does not, and the chat model stays.

Either way, the architecture does not change. Jobs 1 and 2 are closed-set choices. Job 3 is a template. The only question is which model makes the picks.

Frequently asked questions

What is a decision model? A model that returns one value from a list you supply, with a calibrated probability for each option, instead of generating free text. It is built for classification and routing, not for writing.

Why not let the LLM write the SQL? Because the person reading the reply is deciding whether to chase a payment or ship a truck. A wrong number costs real money. Twenty-five reviewed, parameterised queries are safer than generated SQL, and they are easier to audit.

Does the model ever see ERP data? No. It sees the typed sentence, the capability catalogue, and a short list of candidate names. Never a row, an amount, an invoice number or a credential.

What happens when confidence is low? The bot does not answer. It sends the menu, or asks the user to tap the right party. A menu is cheaper than a wrong answer.

Is a decision model cheaper than an LLM for routing? On list prices, yes. Per 1,000 decisions, TypeSafe Jev 1.13 is $0.05 against $0.06 to $1.50 for the chat models. At a few hundred questions a month, none of the bills matter. The typed output and the calibrated confidence are what matter.

---

Prices are list prices checked on 9 October 2026 on each vendor's own page: platform.claude.com pricing, developers.openai.com pricing, and the docs.typesafe.ai models page. Response times for the chat models are typical ranges for a short structured call, not vendor claims.