decidealot: Stop Making a Chatbot Write an Essay When You Needed a Fucking Yes or No

Written by

TypeSafe dropped Jev and my whole feed lost its shit over it. They call it a System One model: you hand it messy state plus a few typed questions, and it hands back a choice, a score or a yes/no with an actual probability on it. No essay, no “Sure! Here’s the JSON you asked for:”, no parsing prose back into the one word you wanted. Credit where it’s due, that is the right fucking idea.

Then you read the fine print. Hosted API. Early access. Waitlist. No weights. And the thing you’d want judged is by definition the sensitive thing, “this agent is about to drop the customers table”, so it has to leave your network and sit in somebody else’s queue before you get your yes or no. Nope.

So I did what I already did with talkies for speech, flickies for video and predictalot for forecasting. I went through the open models that do this job, kept the best ones, Laya and Von, later added CLM as a third, and nailed all of them into one Docker image behind one API. That’s decidealot, the offline Jev: the same System One request and response shape, MCP on the same port, your hardware, no cloud bill, no waitlist, and on the local models the thing you’re asking about never leaves the box. Hand it a TypeSafe key and it’ll front the real hosted Jev too, which is the one road where your state does leave. Then I shoved it into aigate right next to its siblings, so if you already run that stack it’s one env var away.

Asking a Text Generator for a Decision Is Fucking Stupid

Jev blew up because the status quo is dumb as shit:

  • LLMs as classifiers. A model built to generate text, generating text, which you then parse back into the one word you wanted. You beg for “valid JSON ONLY, no explanation” and get it wrapped in markdown fences with a helpful little note about its reasoning, plus the one time in fifty it invents a fourth category you never listed. “Respond only with JSON” is not a contract, it’s a prayer, and structured-output modes just move the parsing into somebody else’s sampler. You are still paying for token-by-token decoding to produce a label.
  • Self-reported confidence. Ask a chatbot how sure it is and it says “85%”, a number it pulled out of its ass with a straight face because a number looked good in that spot. That number has never been anywhere near a softmax. You can’t threshold on it, you can’t calibrate it, and you can’t put it in an audit log and defend it later.
  • Latency spent on nothing. Every “Certainly! Based on the context provided” is wall clock sitting between your event and your decision.
  • The open-weight models. The best two that do this job locally are Laya from NandhaKishorM and Von from wfzyx, weights under Apache 2.0. Great. Each one ships its own server with its own opinions about what a request looks like, its own install and its own Torch runtime, and neither one takes the official TypeSafe request as-is. Want both and you’re running two servers, two installs and two Torch runtimes, and writing the glue yourself.

I wanted one box I start once and point everything at. The hosted contract, so code written against it doesn’t have to learn a second API. Every model behind it, each one switchable. MCP for the agents. And not sitting on a loaded model and a whole Torch runtime in RAM while nothing is asking it shit.

Three Question Types, One Request

The contract is small. You send a model, a state, and a map of named questions. The state is whatever you want judged: a string, a JSON object or an array. The question names are yours and come back as the keys under answers. Every question is one of three types:

  • choice picks one label from the keys of criteria. Those keys are the only answers it can give. It can’t invent a fourth bucket, because it isn’t writing a goddamn thing. You get the choice, a confidence, and a probability for every label.
  • score takes an ordered array of criteria where the position is the score, starting at 0. It returns an expected score, so 1.9 on a three-level rubric is a real answer meaning “blocking, with a sliver of soon”, plus per-level probabilities and a legend mapping positions back to your own wording.
  • noul is yes or no. One field, noul, the probability that the statement is true. No separate confidence field, because that number already is the confidence.

Put all three in one request and they get answered against the same state:

curl --fail http://127.0.0.1:8080/v1/systemone \
  --header 'Content-Type: application/json' \
  --data '{
    "model": "laya",
    "state": "You billed me twice for March. Refund the duplicate today or I am cancelling.",
    "questions": {
      "department": {
        "type": "choice",
        "instructions": "Which team should handle this?",
        "criteria": {
          "billing": "Invoices, payments, refunds.",
          "technical": "Bugs, outages, errors.",
          "other": "Everything else."
        }
      },
      "urgency": {
        "type": "score",
        "criteria": ["not urgent", "soon", "blocking"]
      },
      "churn_risk": {
        "type": "noul",
        "instructions": "Does the customer threaten to leave?"
      }
    }
  }'

That’s not a made-up example. This is what Laya actually sent back, running on the CUDA image behind my own aigate box:

{
  "model": "laya",
  "answers": {
    "department": {
      "type": "choice",
      "choice": "billing",
      "confidence": 0.8055,
      "probabilities": { "billing": 0.9543, "technical": 0.0317, "other": 0.014 }
    },
    "urgency": {
      "type": "score",
      "score": 1.8986,
      "confidence": 0.7053,
      "legend": { "0": "not urgent", "1": "soon", "2": "blocking" },
      "probabilities": { "0": 0.0222, "1": 0.0571, "2": 0.9208 }
    },
    "churn_risk": {
      "type": "noul",
      "noul": 0.2673
    }
  },
  "usage": { "input_tokens": 156, "output_tokens": 0 }
}

Look at output_tokens. Zero. Laya didn’t write a single token, so nothing can come back as “Sure! Here’s”. The same request to Von reports three, which is still nowhere near a paragraph. Either way the answer shape is fixed by the schema, not by whether the model gave a shit about your instructions today.

Now look at churn_risk. The customer literally wrote “or I am cancelling” and Laya put the threat at 0.27. I sent Von the same request as a second opinion and it said 0.23. Billing and blocking, both dead on. The churn threat, both shrugged. That’s not decidealot mangling anything, that’s what the models answered, and it’s the whole reason the next paragraph exists. Test your questions on your own cases before you wire a threshold to them. Rewording the question is cheap. Finding out in production is not.

decidealot hands you the numbers and stops there. It doesn’t act. Your code owns the threshold: allow when allow clears 0.95, queue everything else for a human, store the whole response next to the action record. Same stance as predictalot, just one layer over. Numbers in, and the decision about what to do with them stays yours.

Three Local Models Plus Hosted Jev, Each With Its Own Off Switch

Every request names a model. There’s no default, so your decisions don’t silently switch brains because some asshole edited an env var. GET /v1/models returns the catalog of whatever this deployment has switched on. With all three local providers on, that’s all fourteen below, plus whatever names TypeSafe hands back if you also switch on hosted Jev:

  • laya, laya-auto, laya-latest: Laya with automatic checkpoint routing. It looks at the script and the language of the state and picks the English or the multilingual checkpoint on its own.
  • laya-english: the English checkpoint, for English in Latin script.
  • laya-multilingual: the multilingual checkpoint, for everything else, including short Latin-script text that isn’t clearly English.
  • laya-typed-decisions: the checkpoint tuned for repeated structured workflow calls, like policy, routing, triage and approvals. Test it on your own cases before you trust it.
  • von, von-latest, von-1.1, von-1.1.0: Von, an independent English-only model for short, well-posed decisions. Useful on its own, and useful as a second opinion before you standardize a workflow on Laya.
  • clm, clm-latest, clm-0.1, clm-0.1-8b: CLM v0.1, built nothing like the other two. More on that right below.

Laya is one model family with three checkpoints. Von is a different model from different people. CLM, which I bolted on later, is a different animal again: a 75 MB projection head from Contrastive-LM that can’t read a single word on its own. CLM sends your rendered state and criteria to one OpenAI-compatible /v1/embeddings endpoint, which has to hand back Qwen3-8B last-token vectors exactly 4096 floats wide, throws out any vector that isn’t exactly that wide, normalizes the rest before the head even runs, and scores every answer against the state from those vectors. Answer texts it has already embedded come out of a bounded cache with a TTL instead of hitting the endpoint again, but your raw state never gets cached and gets embedded fresh every time. All three take the same request and return the same answer types, which is the whole point of putting them behind one contract: swap the model string and compare.

Each of the three local ones has its own switch: DECIDEALOT_LAYA_ENABLED, DECIDEALOT_VON_ENABLED and DECIDEALOT_CLM_ENABLED. Run all three, any two or just one, but at least one provider has to be on, because a box with zero providers refuses to start instead of sitting there answering nothing. Laya and Von default to on. CLM defaults to auto, which turns it on the moment you set DECIDEALOT_CLM_EMBEDDINGS_URL. Set it to true and a missing URL kills startup, set it to false and CLM stays off even with a URL. The URL is the part to think about: your state goes to that endpoint, so CLM is exactly as private as whatever sits behind it. Point it at something you run yourself, or at least something you trust. A provider you switch off doesn’t get downloaded, doesn’t start, doesn’t show up in GET /v1/models or in MCP’s model list, and its names get rejected. A CLM-only box downloads the one CLM checkpoint and nothing else. And yes, there’s a fourth switch now, for the hosted Jev this whole post started out flipping off. DECIDEALOT_JEV_ENABLED defaults to auto too, which turns it on the moment you set DECIDEALOT_TYPESAFE_API_KEY. Its model names come straight from TypeSafe’s own catalog, cached for 60 seconds and fetched again on the next request after that, with no local jev alias, and every Jev decision ships your whole state and questions to TypeSafe. That’s the exact opposite of offline, so if that’s not what you want, leave the key unset and Jev doesn’t exist. With the key set you can even switch all three local ones off and run a Jev-only box that downloads nothing.

Unload Means the Process Dies

This is the part I actually care about. There are two Python virtualenvs in the image. /opt/app-venv is the gateway: FastAPI, httpx, the MCP SDK, pydantic, uvicorn. No Torch in there, and the gateway never imports it. /opt/torch-venv is the model side, shared by Laya, Von and CLM: the model packages go in from one hash-locked file on top of my Torchbase images, which exist so the fat Python, Torch and CUDA layers get built once and reused instead of rebuilt every time I touch a line of app code. Sharing a venv isn’t sharing a process, though. The process answering your HTTP requests never has a model in it.

Each model runs as a child process of the gateway, started by a supervisor from a fixed command with no shell, listening on a fixed loopback port. By default only one is resident at a time, and DECIDEALOT_MAX_RESIDENT_LOCAL_PROVIDERS raises that to two or three if you’ve got the RAM or VRAM for it. At the default, ask for Von while Laya is loaded and the supervisor waits for every in-flight Laya request to finish, kills Laya, starts Von, and polls Von’s /health every quarter second until it answers. With more slots it only kills something once they’re all taken, and then it’s the least recently used idle one. Hosted Jev never touches any of this, it’s just an HTTPS call. Moving between the Laya selectors stays inside the Laya process, since they’re checkpoints of the same thing. Trimmed down, the gate looks like this:

async with self._provider_switch_condition:
    spec = self._require_spec(provider_name)
    while (
        provider_name not in self._processes
        and len(self._processes) >= self._settings.max_resident_local_providers
    ):
        idle_candidates = [
            name for name in self._processes if self._active_requests[name] == 0
        ]
        if idle_candidates:
            oldest = min(idle_candidates, key=lambda name: self._last_used_at.get(name, 0))
            await self._stop_provider_locked(oldest, reason="model_switch")
            continue
        await self._provider_switch_condition.wait()
    await self._start_provider_locked(spec)
    self._active_requests[provider_name] += 1
try:
    yield
finally:
    async with self._provider_switch_condition:
        self._active_requests[provider_name] -= 1
        self._last_used_at[provider_name] = asyncio.get_running_loop().time()
        self._provider_switch_condition.notify_all()

Why a whole process instead of del model and a prayer to the garbage collector? Because that doesn’t give the memory back. PyTorch’s caching allocator hangs on to what it grabbed, and the CUDA context stays put for as long as the process lives. Killing the process is the only unload that is actually an unload, so that’s what unload does: terminate, ten seconds of grace, then kill if it’s being a little bitch about it. Weights, Torch allocations, worker threads and the CUDA context all go with it. The next request starts it again.

That start isn’t free, and I’m not going to pretend it is. On my GPU box, through aigate, the first Laya request after a cold start took about 61 seconds, because the process had to come up and load the model. The next identical request came back in 55 milliseconds end to end, network included, with byte-for-byte the same answer. Switching to Von took about two minutes, since Laya had to go down first and Von had to come up. Warm Von answered in about 115 milliseconds. So the idle timeout is a real trade: set it long enough that your traffic doesn’t keep paying the cold start, and short enough that the GPU isn’t babysitting a model nobody’s using.

Two things trigger that. An idle reaper unloads a provider that has sat unused for DECIDEALOT_PROVIDER_IDLE_UNLOAD_SECONDS, 600 by default, checking every tenth of that window, clamped between 10 milliseconds and 30 seconds. Set it to 0 and only the timer turns off. And POST /v1/models/unload does it on demand. That one is all or nothing: if any local provider is in the middle of a request you get a 409 PROVIDER_BUSY and nothing gets released. It never yanks a model out from under a caller.

Download Once, Then Cut the Cord

You mount one host directory at /models, and decidealot manages /models/laya, /models/von and, with CLM on, /models/clm inside it. On first start the supervisor runs a prepare step for each enabled model inside the Torch venv, which pulls the Hugging Face snapshot, convaiinnovations/laya and wfzyx/von, each pinned to an exact commit revision. It then checks that every file the runtime needs is actually there: four files in each of Laya’s three checkpoint directories, six for Von. CLM pulls exactly one file from Contrastive-LM/CLM-v0.1-8B, also at a pinned revision, and won’t use it unless its SHA-256 matches the pinned one. None of that imports Torch. /health answers 503 until every enabled bundle is ready. With the default Laya and Von that’s about 5.3 GB and a few minutes the first time, and CLM adds its 75 MB on top. After that the files are already there and nothing gets downloaded.

Then, before a model loads, it sets HF_HUB_OFFLINE=1 and TRANSFORMERS_OFFLINE=1. Once the bundle is verified, no model gets to wander back to the Hub halfway through a run and fetch some shit you didn’t pin.

Making Two Upstream Servers Speak One Contract

The official schema says instructions is optional and can be a nested JSON value, and so can the criterion values. Laya wants an instructions field on every single question. Von wants it to be a string. Send either of them a request the official API happily accepts and you get rejected over a shape difference nobody asked for.

So decidealot validates your request against the official request models first, then builds Laya’s and Von’s native bodies from it: a missing instructions becomes an empty string, and nested values get rendered as compact JSON text. Your choice labels, your score order and your state go through untouched. On the way back the provider’s answer is validated against the official response schema. Provider-only junk like Laya’s routing block gets dropped, and model gets set to the public name of whatever actually answered. If a provider hands back something that doesn’t fit the schema, you get a clean 503 instead of garbage dressed up as a decision. If a provider rejects a request with a bare message, it gets restated as the official {"detail": [...]} validation envelope, so your error handling only ever sees one shape.

Then there’s Von’s server, which finds its backend through a process-wide singleton whose public constructor has no way to be told where the checkpoint lives. So decidealot builds the engine itself and jams it into the slot the server reads from:

engine = engine_type(
    backend_name=os.environ.get(_von_backend_env, _default_von_backend),
    device=os.environ.get(_von_device_env),
)
engine.backend = option_marker_backend_type(checkpoint_dir=str(model_dir), device=engine.device)
# Von's server resolves its backend from this singleton, whose public constructor
# has no checkpoint-directory argument.
with engine_type._lock:
    engine_type._instance = engine

Yes, that’s reaching into a private singleton. It’s ugly as shit, and it’s exactly as ugly as it has to be to point Von at a directory you control.

MCP on the Same Port

The same container serves MCP Streamable HTTP at /mcp, with four tools: system_one, system_one_batch, list_models and unload_models. They go through the same decision service as the REST routes, which means the same validation, the same supervisor, the same body limit and the same bearer token. system_one takes exactly the model, state, questions and optional config you’d POST, and returns the same structured result, so an agent can read the probabilities before it decides what to do next. system_one_batch is the MCP face of POST /v1/systemone/batch: a list of independent requests, each with its own model, state and questions, answered in the order you sent them with each result naming the model that actually answered. Requests for the same model run one after another, DECIDEALOT_MAX_BATCH_CONCURRENCY caps how many provider calls overlap, and one failed item fails the whole batch with no partial results. A validation failure comes back as isError: true with the TypeSafe detail body in the text, so the agent sees what it fucked up instead of a bare “error”.

No tool takes a URL, a filesystem path, an executable, a model directory or an arbitrary runtime option. The one knob a request gets is config, and right now only CLM takes anything in it: a validated temperature. Everything else answers a non-empty one with a 422. The model router maps every local alias to a fixed loopback endpoint and every Jev name to TypeSafe’s one fixed HTTPS API, and a caller never gets to pick a network target. The most power an agent has here is choosing a name out of the catalog. CLM’s embeddings URL and the TypeSafe key get set once by whoever deploys the box, never by a request.

Putting it behind a reverse proxy or a tunnel doesn’t mean turning off DNS-rebinding protection. You allowlist the exact public Host in DECIDEALOT_MCP_ALLOWED_HOSTS and, for browser clients, the exact origin in DECIDEALOT_MCP_ALLOWED_ORIGINS. The checks run in a fixed order: a missing or wrong bearer gets 401 first, then an unknown host gets 421, then an unknown browser origin gets 403. Don’t wildcard it on an internet-facing box. Set DECIDEALOT_API_KEY to a real secret before it goes anywhere near a public address.

The Boring Shit That Makes It Safe to Leave Running

  • A container that can’t do much. The documented run is a read-only root filesystem, --cap-drop ALL, no-new-privileges, noexec tmpfs for /tmp and /var/run, a pids limit, a memory cap, and a port published on loopback only.
  • Your UID, not the image’s. The container runs as whatever --user you pass it, so the model directory you just mkdir‘d is writable with no chown dance. The image falls back to non-root 1000:1000 only when you don’t say.
  • One exec hole for CUDA, and only one. Triton compiles small CUDA helpers at runtime and has to load them from somewhere, so the CUDA run adds a single exec tmpfs at /var/cache. Everything else stays read-only and noexec.
  • Bearer auth that doesn’t leak timing. Optional, off unless DECIDEALOT_API_KEY is set, compared with hmac.compare_digest on encoded bytes. /health stays open for your liveness probe.
  • A body cap. 1 MiB by default via DECIDEALOT_MAX_REQUEST_BYTES, anything declared bigger gets a 413 before a model ever sees it.
  • Request IDs you can grep. Send a UUID or ULID in X-Request-Id and it follows the request through the logs. Send garbage or nothing and it mints a UUID. Either way it comes back on the response.
  • A supply-chain gate the models were too young for. The gateway’s dependencies sit behind a uv exclude-newer age gate, and the model stack installs into the Torchbase venv with --require-hashes from a hash-locked file. Both model packages are younger than the gate allows. Laya’s pinned release isn’t even on PyPI, so it’s installed from the upstream commit tarball with the SHA-256 in the lock. Each one carries a written, owner-approved exception. The gate did its job and I signed the permission slip.
  • Tests with a floor. Branch coverage with a hard 90% minimum, plus real HTTP runs against the downloaded CPU and CUDA weights.

Two images, each on the matching Torchbase image. The CPU one is about 300 MB compressed on Docker Hub and is the right default. The CUDA one is about 5.4 GB, built on CUDA 12.6, amd64 only, and needs the NVIDIA Container Toolkit and --gpus all. Only reach for it when speed and model memory actually justify the GPU setup.

Run It

model_directory="$HOME/.local/share/decidealot/models"
mkdir --parents "$model_directory"
docker run --detach --name decidealot --init --restart unless-stopped \
  --user "$(id -u):$(id -g)" \
  --read-only --cap-drop ALL --security-opt no-new-privileges:true \
  --pids-limit 512 --memory 8g --cpus 4 \
  --tmpfs /tmp:rw,noexec,nosuid,size=128m \
  --tmpfs /var/run:rw,noexec,nosuid,size=8m \
  --mount type=bind,source="$model_directory",target=/models \
  --publish 127.0.0.1:8080:8080 \
  psyb0t/decidealot:latest
curl --fail http://127.0.0.1:8080/health

Wait for /health to go green, then send it the request from above. For the GPU, add --gpus all, the /var/cache tmpfs, and use psyb0t/decidealot:latest-cuda. The deployment doc in the repo has the full CUDA recipe.

If you already run aigate, skip all of that shit. Set DECIDEALOT=1 for the CPU service at /decidealot/ or DECIDEALOT_CUDA=1 for the GPU one at /decidealot-cuda/, with MCP under each, and the tools also join aigate’s aggregated /mcp/ next to everything else your agents already talk to. The two can run side by side and share one model directory, so the 5.3 GB lands on disk once. CLM is on by default there: make run-bg also brings up aigate’s own llama.cpp CUDA route, local-llamacpp-cuda-qwen3-8b, which serves Qwen3 8B embeddings at exactly the 4096 dimensions CLM wants, over the internal network. So even the state CLM sends out never leaves your stack. That route needs an NVIDIA GPU even next to the CPU service, so on a box without one set DECIDEALOT_CLM_ENABLED=false. Hosted Jev only shows up if you put DECIDEALOT_TYPESAFE_API_KEY in aigate’s .env, and that state does leave. One catch: aigate pins decidealot v0.6.0, so the CLM temperature knob from v0.7.0 isn’t there yet.

aigate also babysits the memory for you. Before any other local model that aigate routes through LiteLLM (Ollama, sd.cpp, talkies, vLLM, llama.cpp) runs on the same hardware, its resource manager tells decidealot to unload. Laya and Von decisions take the same hardware lock as everything else, whether they come in over REST, MCP or a batch, and on CUDA aigate unloads its own Qwen encoder before Laya or Von loads. CLM leaves the lock to its nested embeddings call. POST /v1/unload/cuda or /v1/unload/cpu clears decidealot along with the rest of the stack. A decision that’s mid-flight answers 409 and keeps its model until the idle timer, so nothing gets yanked out from under a caller. The hole that’s left: a request sent straight to /decidealot/ only kicks out that encoder, not every other model squatting on the GPU, so an unrelated resident model can still eat your VRAM.

Installing It Into Your Agent

An agent that’s about to act on a probability should at least know what the probability means. The skill tells it how to deploy the container, pick Laya, Von, CLM or hosted Jev, send a typed decision, read the probabilities without treating them as permission, and use MCP directly. Everything under .agents/ is catalogued in one marketplace, so it’s two commands:

claude plugin marketplace add psyb0t/agents
claude plugin install decidealot@psyb0t

Codex uses the same marketplace with a different verb, codex plugin add decidealot@psyb0t, because there is no codex plugin install. OpenClaw gets the skill, plus an optional stdio bridge for clients that can only talk to a local stdio MCP server, which forwards to the container you already run:

openclaw skills install @psyb0t/decidealot
openclaw plugins install clawhub:@psyb0t/decidealot

It Decides. You Act.

Jev had the right idea and the wrong address. decidealot is the same idea at yours: a closed set of answers, a real probability on each one, and zero chance of getting an essay back. It doesn’t know what your threshold should be, and it doesn’t pretend to. You pick the cutoff based on what being wrong costs you, and the model never gets a vote on that.

Grab it at github.com/psyb0t/decidealot or pull psyb0t/decidealot off Docker Hub. The code is WTFPL, so do what the fuck you want with it. Laya, Von, CLM, PyTorch, Transformers and the downloaded weights all keep their own licenses, so read those before you bolt a model into something you sell. Now go stop asking chatbots for yes or no.