decidealot: Stop Making a Chatbot Write an Essay When You Needed a Fucking Yes or No

Written by

TypeSafe dropped Jev and my whole feed lost its shit over it. They call it a System One model: you hand it messy state plus a few typed questions, and it hands back a choice, a score or a yes/no with an actual probability on it. No essay, no “Sure! Here’s the JSON you asked for:”, no parsing prose back into the one word you wanted. Credit where it’s due, that is the right fucking idea.

Then you read the fine print. Hosted API. Early access. Waitlist. No weights. And the thing you’d want judged is by definition the sensitive thing, “this agent is about to drop the customers table”, so it has to leave your network and sit in somebody else’s queue before you get your yes or no. Nope.

So I did what I already did with talkies for speech, flickies for video and predictalot for forecasting. I went through the open models that do this job, kept the two best, Laya and Von, and nailed them into one Docker image behind one API. That’s decidealot, the offline Jev: the same System One request and response shape, MCP on the same port, your hardware, no cloud bill, no waitlist, and the thing you’re asking about never leaves the box. Then I shoved it into aigate right next to its siblings, so if you already run that stack it’s one env var away.

Asking a Text Generator for a Decision Is Fucking Stupid

Jev blew up because the status quo is dumb as shit:

  • LLMs as classifiers. A model built to generate text, generating text, which you then parse back into the one word you wanted. You beg for “valid JSON ONLY, no explanation” and get it wrapped in markdown fences with a helpful little note about its reasoning, plus the one time in fifty it invents a fourth category you never listed. “Respond only with JSON” is not a contract, it’s a prayer, and structured-output modes just move the parsing into somebody else’s sampler. You are still paying for token-by-token decoding to produce a label.
  • Self-reported confidence. Ask a chatbot how sure it is and it says “85%”, a number it pulled out of its ass with a straight face because a number looked good in that spot. That number has never been anywhere near a softmax. You can’t threshold on it, you can’t calibrate it, and you can’t put it in an audit log and defend it later.
  • Latency spent on nothing. Every “Certainly! Based on the context provided” is wall clock sitting between your event and your decision.
  • The open-weight models. The best two that do this job locally are Laya from NandhaKishorM and Von from wfzyx, weights under Apache 2.0. Great. Each one ships its own server with its own opinions about what a request looks like, its own install and its own Torch runtime, and neither one takes the official TypeSafe request as-is. Want both and you’re running two servers, two installs and two Torch runtimes, and writing the glue yourself.

I wanted one box I start once and point everything at. The hosted contract, so code written against it doesn’t have to learn a second API. Both local models behind it. MCP for the agents. And not sitting on a loaded model and a whole Torch runtime in RAM while nothing is asking it shit.

Three Question Types, One Request

The contract is small. You send a model, a state, and a map of named questions. The state is whatever you want judged: a string, a JSON object or an array. The question names are yours and come back as the keys under answers. Every question is one of three types:

  • choice picks one label from the keys of criteria. Those keys are the only answers it can give. It can’t invent a fourth bucket, because it isn’t writing a goddamn thing. You get the choice, a confidence, and a probability for every label.
  • score takes an ordered array of criteria where the position is the score, starting at 0. It returns an expected score, so 1.9 on a three-level rubric is a real answer meaning “blocking, with a sliver of soon”, plus per-level probabilities and a legend mapping positions back to your own wording.
  • noul is yes or no. One field, noul, the probability that the statement is true. No separate confidence field, because that number already is the confidence.

Put all three in one request and they get answered against the same state:

curl --fail http://127.0.0.1:8080/v1/systemone \
  --header 'Content-Type: application/json' \
  --data '{
    "model": "laya",
    "state": "You billed me twice for March. Refund the duplicate today or I am cancelling.",
    "questions": {
      "department": {
        "type": "choice",
        "instructions": "Which team should handle this?",
        "criteria": {
          "billing": "Invoices, payments, refunds.",
          "technical": "Bugs, outages, errors.",
          "other": "Everything else."
        }
      },
      "urgency": {
        "type": "score",
        "criteria": ["not urgent", "soon", "blocking"]
      },
      "churn_risk": {
        "type": "noul",
        "instructions": "Does the customer threaten to leave?"
      }
    }
  }'

That’s not a made-up example. This is what Laya actually sent back, running on the CUDA image behind my own aigate box:

{
  "model": "laya",
  "answers": {
    "department": {
      "type": "choice",
      "choice": "billing",
      "confidence": 0.8055,
      "probabilities": { "billing": 0.9543, "technical": 0.0317, "other": 0.014 }
    },
    "urgency": {
      "type": "score",
      "score": 1.8986,
      "confidence": 0.7053,
      "legend": { "0": "not urgent", "1": "soon", "2": "blocking" },
      "probabilities": { "0": 0.0222, "1": 0.0571, "2": 0.9208 }
    },
    "churn_risk": {
      "type": "noul",
      "noul": 0.2673
    }
  },
  "usage": { "input_tokens": 156, "output_tokens": 0 }
}

Look at output_tokens. Zero. Laya didn’t write a single token, so nothing can come back as “Sure! Here’s”. The same request to Von reports three, which is still nowhere near a paragraph. Either way the answer shape is fixed by the schema, not by whether the model gave a shit about your instructions today.

Now look at churn_risk. The customer literally wrote “or I am cancelling” and Laya put the threat at 0.27. I sent Von the same request as a second opinion and it said 0.23. Billing and blocking, both dead on. The churn threat, both shrugged. That’s not decidealot mangling anything, that’s what the models answered, and it’s the whole reason the next paragraph exists. Test your questions on your own cases before you wire a threshold to them. Rewording the question is cheap. Finding out in production is not.

decidealot hands you the numbers and stops there. It doesn’t act. Your code owns the threshold: allow when allow clears 0.95, queue everything else for a human, store the whole response next to the action record. Same stance as predictalot, just one layer over. Numbers in, and the decision about what to do with them stays yours.

Two Models, Ten Selectors, One in Memory

Every request names a model. There’s no default, so your decisions don’t silently switch brains because some asshole edited an env var. GET /v1/models returns the catalog, and it’s these ten:

  • laya, laya-auto, laya-latest: Laya with automatic checkpoint routing. It looks at the script and the language of the state and picks the English or the multilingual checkpoint on its own.
  • laya-english: the English checkpoint, for English in Latin script.
  • laya-multilingual: the multilingual checkpoint, for everything else, including short Latin-script text that isn’t clearly English.
  • laya-typed-decisions: the checkpoint tuned for repeated structured workflow calls, like policy, routing, triage and approvals. Test it on your own cases before you trust it.
  • von, von-latest, von-1.1, von-1.1.0: Von, an independent English-only model for short, well-posed decisions. Useful on its own, and useful as a second opinion before you standardize a workflow on Laya.

Laya is one model family with three checkpoints. Von is a different model from different people. Both take the same request and return the same answer types, which is the whole point of putting them behind one contract: swap the model string and compare.

Unload Means the Process Dies

This is the part I actually care about. There are three Python virtualenvs in the image. /opt/app-venv is the gateway: FastAPI, httpx, the MCP SDK, pydantic, uvicorn. No Torch, not a single import of it anywhere in the gateway’s source. /opt/laya-venv and /opt/von-venv each hold one model’s stack. Right now both happen to pin the same torch and transformers, so this isn’t a workaround for a fight already happening. It means a Laya upgrade can never reach into Von’s environment, and the process answering your HTTP requests never has a model in it.

Each model runs as a child process of the gateway, started by a supervisor from a fixed command with no shell, listening on a fixed loopback port. Only one is resident at a time. Ask for Von while Laya is loaded and the supervisor waits for every in-flight Laya request to finish, kills Laya, starts Von, and polls Von’s /health every quarter second until it answers. Moving between the Laya selectors stays inside the Laya process, since they’re checkpoints of the same thing. Trimmed down, the gate looks like this:

async with self._provider_switch_condition:
    spec = self._require_spec(provider_name)
    while self._has_active_other_provider(provider_name):
        await self._provider_switch_condition.wait()
    await self._unload_other_idle_providers_locked(provider_name)
    await self._start_provider_locked(spec)
    self._active_requests[provider_name] += 1
try:
    yield
finally:
    async with self._provider_switch_condition:
        self._active_requests[provider_name] -= 1
        self._last_used_at[provider_name] = asyncio.get_running_loop().time()
        self._provider_switch_condition.notify_all()

Why a whole process instead of del model and a prayer to the garbage collector? Because that doesn’t give the memory back. PyTorch’s caching allocator hangs on to what it grabbed, and the CUDA context stays put for as long as the process lives. Killing the process is the only unload that is actually an unload, so that’s what unload does: terminate, ten seconds of grace, then kill if it’s being a little bitch about it. Weights, Torch allocations, worker threads and the CUDA context all go with it. The next request starts it again.

That start isn’t free, and I’m not going to pretend it is. On my GPU box, through aigate, the first Laya request after a cold start took about 61 seconds, because the process had to come up and load the model. The next identical request came back in 55 milliseconds end to end, network included, with byte-for-byte the same answer. Switching to Von took about two minutes, since Laya had to go down first and Von had to come up. Warm Von answered in about 115 milliseconds. So the idle timeout is a real trade: set it long enough that your traffic doesn’t keep paying the cold start, and short enough that the GPU isn’t babysitting a model nobody’s using.

Two things trigger that. An idle reaper unloads a provider that has sat unused for DECIDEALOT_PROVIDER_IDLE_UNLOAD_SECONDS, 600 by default, checking every tenth of that window, clamped between 10 milliseconds and 30 seconds. Set it to 0 and only the timer turns off. And POST /v1/models/unload does it on demand. That one is all or nothing: if any provider is in the middle of a request you get a 409 PROVIDER_BUSY and nothing gets released. It never yanks a model out from under a caller.

Download Once, Then Cut the Cord

You mount one host directory at /models, and decidealot manages /models/laya and /models/von inside it. On first start the supervisor runs a prepare step for each model inside that model’s own venv, which pulls the Hugging Face snapshot, convaiinnovations/laya and wfzyx/von, each pinned to an exact commit revision. It then checks that every file the runtime needs is actually there: four files in each of Laya’s three checkpoint directories, six for Von. None of that imports Torch. /health answers 503 until both bundles are ready, which is about 5.3 GB and a few minutes the first time. After that the files are already there and nothing gets downloaded.

Then, before a model loads, it sets HF_HUB_OFFLINE=1 and TRANSFORMERS_OFFLINE=1. Once the bundle is verified, no model gets to wander back to the Hub halfway through a run and fetch some shit you didn’t pin.

Making Two Upstream Servers Speak One Contract

The official schema says instructions is optional and can be a nested JSON value, and so can the criterion values. Laya wants an instructions field on every single question. Von wants it to be a string. Send either of them a request the official API happily accepts and you get rejected over a shape difference nobody asked for.

So decidealot validates your request against the official request models first, then builds each model’s native body from it: a missing instructions becomes an empty string, and nested values get rendered as compact JSON text. Your choice labels, your score order and your state go through untouched. On the way back the provider’s answer is validated against the official response schema. Provider-only junk like Laya’s routing block gets dropped, and model gets set to the public name of whatever actually answered. If a provider hands back something that doesn’t fit the schema, you get a clean 503 instead of garbage dressed up as a decision. If a provider rejects a request with a bare message, it gets restated as the official {"detail": [...]} validation envelope, so your error handling only ever sees one shape.

Then there’s Von’s server, which finds its backend through a process-wide singleton whose public constructor has no way to be told where the checkpoint lives. So decidealot builds the engine itself and jams it into the slot the server reads from:

engine = engine_type(
    backend_name=os.environ.get(_von_backend_env, _default_von_backend),
    device=os.environ.get(_von_device_env),
)
engine.backend = option_marker_backend_type(checkpoint_dir=str(model_dir), device=engine.device)
# Von's server resolves its backend from this singleton, whose public constructor
# has no checkpoint-directory argument.
with engine_type._lock:
    engine_type._instance = engine

Yes, that’s reaching into a private singleton. It’s ugly as shit, and it’s exactly as ugly as it has to be to point Von at a directory you control.

MCP on the Same Port

The same container serves MCP Streamable HTTP at /mcp, with three tools: system_one, list_models and unload_models. They go through the same decision service as the REST routes, which means the same validation, the same supervisor, the same body limit and the same bearer token. system_one takes exactly the model, state and questions you’d POST, and returns the same structured result, so an agent can read the probabilities before it decides what to do next. A validation failure comes back as isError: true with the TypeSafe detail body in the text, so the agent sees what it fucked up instead of a bare “error”.

No tool takes a URL, a filesystem path, an executable, a model directory or a runtime option. The model router maps every alias to a fixed loopback endpoint, and a caller never gets to pick a network target. The most power an agent has here is choosing a name out of the catalog.

Putting it behind a reverse proxy or a tunnel doesn’t mean turning off DNS-rebinding protection. You allowlist the exact public Host in DECIDEALOT_MCP_ALLOWED_HOSTS and, for browser clients, the exact origin in DECIDEALOT_MCP_ALLOWED_ORIGINS. The checks run in a fixed order: a missing or wrong bearer gets 401 first, then an unknown host gets 421, then an unknown browser origin gets 403. Don’t wildcard it on an internet-facing box. Set DECIDEALOT_API_KEY to a real secret before it goes anywhere near a public address.

The Boring Shit That Makes It Safe to Leave Running

  • A container that can’t do much. The documented run is a read-only root filesystem, --cap-drop ALL, no-new-privileges, noexec tmpfs for /tmp and /var/run, a pids limit, a memory cap, and a port published on loopback only.
  • Your UID, not the image’s. The container runs as whatever --user you pass it, so the model directory you just mkdir‘d is writable with no chown dance. The image falls back to non-root 1000:1000 only when you don’t say.
  • One exec hole for CUDA, and only one. Triton compiles small CUDA helpers at runtime and has to load them from somewhere, so the CUDA run adds a single exec tmpfs at /var/cache. Everything else stays read-only and noexec.
  • Bearer auth that doesn’t leak timing. Optional, off unless DECIDEALOT_API_KEY is set, compared with hmac.compare_digest on encoded bytes. /health stays open for your liveness probe.
  • A body cap. 1 MiB by default via DECIDEALOT_MAX_REQUEST_BYTES, anything declared bigger gets a 413 before a model ever sees it.
  • Request IDs you can grep. Send a UUID or ULID in X-Request-Id and it follows the request through the logs. Send garbage or nothing and it mints a UUID. Either way it comes back on the response.
  • A supply-chain gate the models were too young for. The gateway’s dependencies sit behind a uv exclude-newer age gate, and the model stacks install with --require-hashes from hash-locked files. Both model packages are younger than the gate allows. Laya’s pinned release isn’t even on PyPI, so it’s installed from the upstream commit tarball with the SHA-256 in the lock. Each one carries a written, owner-approved exception. The gate did its job and I signed the permission slip.
  • Tests with a floor. Branch coverage with a hard 90% minimum, plus real HTTP runs against the downloaded CPU and CUDA weights.

Two images. The CPU one is about half a gig compressed on Docker Hub and is the right default. The CUDA one is about 9 GB, built on CUDA 12.6, amd64 only, and needs the NVIDIA Container Toolkit and --gpus all. Only reach for it when speed and model memory actually justify the GPU setup.

Run It

model_directory="$HOME/.local/share/decidealot/models"
mkdir --parents "$model_directory"
docker run --detach --name decidealot --init --restart unless-stopped \
  --user "$(id -u):$(id -g)" \
  --read-only --cap-drop ALL --security-opt no-new-privileges:true \
  --pids-limit 512 --memory 8g --cpus 4 \
  --tmpfs /tmp:rw,noexec,nosuid,size=128m \
  --tmpfs /var/run:rw,noexec,nosuid,size=8m \
  --mount type=bind,source="$model_directory",target=/models \
  --publish 127.0.0.1:8080:8080 \
  psyb0t/decidealot:latest
curl --fail http://127.0.0.1:8080/health

Wait for /health to go green, then send it the request from above. For the GPU, add --gpus all, the /var/cache tmpfs, and use psyb0t/decidealot:latest-cuda. The deployment doc in the repo has the full CUDA recipe.

If you already run aigate, skip all of that shit. Set DECIDEALOT=1 for the CPU service at /decidealot/ or DECIDEALOT_CUDA=1 for the GPU one at /decidealot-cuda/, with MCP under each, and the tools also join aigate’s aggregated /mcp/ next to everything else your agents already talk to. The two can run side by side and share one model directory, so the 5.3 GB lands on disk once.

Installing It Into Your Agent

An agent that’s about to act on a probability should at least know what the probability means. The skill tells it how to deploy the container, pick Laya or Von, send a typed decision, read the probabilities without treating them as permission, and use MCP directly. Everything under .agents/ is catalogued in one marketplace, so it’s two commands:

claude plugin marketplace add psyb0t/agents
claude plugin install decidealot@psyb0t

Codex uses the same marketplace with a different verb, codex plugin add decidealot@psyb0t, because there is no codex plugin install. OpenClaw gets the skill, plus an optional stdio bridge for clients that can only talk to a local stdio MCP server, which forwards to the container you already run:

openclaw skills install @psyb0t/decidealot
openclaw plugins install clawhub:@psyb0t/decidealot

It Decides. You Act.

Jev had the right idea and the wrong address. decidealot is the same idea at yours: a closed set of answers, a real probability on each one, and zero chance of getting an essay back. It doesn’t know what your threshold should be, and it doesn’t pretend to. You pick the cutoff based on what being wrong costs you, and the model never gets a vote on that.

Grab it at github.com/psyb0t/decidealot or pull psyb0t/decidealot off Docker Hub. The code is WTFPL, so do what the fuck you want with it. Laya, Von, PyTorch, Transformers and the downloaded weights all keep their own licenses, so read those before you bolt a model into something you sell. Now go stop asking chatbots for yes or no.