How you present an option changes what a decision model tells you

jevchat generates text out of a typed decision model by asking it one question per symbol: given the question and the reply so far, which symbol comes next. The options are an alphabet plus a stop option, the model returns a probability for each, and a sampler draws from the normalised distribution. The author is blunt about the vehicle, writing that "the idea is for fun, the cost is somewhat impractical, and the results are hilarious," and that the implementation was written by an agent from his description of the sampling strategies. Two of its findings are not about chatbots at all. The repository reports that presenting each option as the resulting text, rather than as the bare symbol a reader would have to append in their head, is "the single largest improvement in the project": on character alphabets it claims roughly triple the top-1 rate and double the probability mass landing on the right symbol, for fewer input tokens. Separately, the mode that skips re-ordering the options is labelled the worst one, because the re-ordering is what cancels the model's position bias. It ships a bench command that compares every mode against the API, and 158 offline tests, but the comparison table those figures point at did not load for us, so both are the repository's claims and not numbers we read.

Why it matters: If you call a typed decision endpoint anywhere in a harness, to route work or to grade an output, both findings transfer whatever you are deciding. Hand the model the finished option instead of the fragment it would have to assemble, and shuffle the option order before you ask. An unshuffled option list means part of what you are measuring is your own ordering.