About The Decision Bench
Decision-first AI, tested in the real world.
Why this site exists
Most AI writing starts with a demo and ends with a promise. A model does something impressive on stage, someone writes that it "changes everything," and you're left with one question nobody answered:
What exactly did the system decide?
The Decision Bench exists to answer that question. We look at AI systems that classify, route, score, select, and act — systems that return a decision instead of a paragraph. Then we test them the way they'd actually be used: with real inputs, real latency budgets, real ambiguity, and real failure conditions.
No breakthroughs. No universal solutions. Just inputs, outputs, probabilities, timings, and honest notes on where things break.
What "decision-first" means here
A decision-first model doesn't write you an essay about whether an email is spam. It returns spam: 0.94 in 8 milliseconds and stops. That difference — structured output versus generated text — changes everything about how you build, measure, and deploy the system around it.
The Decision Bench treats that as the interesting part:
What did the model receive?
What did it decide?
What probability or confidence came back with it?
How fast was the response, on what hardware?
What happened when the input was ambiguous or adversarial?
Is the result useful in a product, or only impressive in a demo?
Those six questions are our testing framework. Every post on this site runs through them.
Who's behind it
Isaac Bell — independent AI systems developer, Pittsburgh, Pennsylvania.
I spent years as a machine learning engineer in industrial automation, where a model that was "usually right" was a liability. Predictable outputs, low latency, and clean failure handling mattered more than anything you could put in a launch video. I carried that mindset into independent work, where I now study small decision-oriented models and build reproducible experiments around classification, routing, guardrails, and real-time action selection.
My approach runs in three layers: hands-on development first, model reasoning second, product experimentation third. That means you'll see code, logs, and failed tests on the table — not just conclusions.
I work mostly in Python and JavaScript, with Hugging Face, Transformers.js, ONNX, FastAPI, and local inference. I test latency, confidence, calibration, browser performance, and deployment friction, because those are the things that decide whether a model ships or dies in a notebook.
Laya's non-autoregressive decision model is the main case study on this site — not because it's the answer to everything, but because it's a clean example of what decision-first inference actually looks like when you open it up.
I try to keep observed evidence separate from my interpretation. When the evidence isn't there, I say so.
What you'll find here
Decision Under the Hood — mechanisms, non-autoregressive inference, structured outputs, probabilities, classification, routing.
Bench Tests — latency, accuracy, calibration, memory use, hardware comparisons, browser performance.
Deploy Notes — Transformers.js, ONNX, FastAPI, Hugging Face, local and mobile execution.
Field Tests / Scene Tests — games, email triage, content safety, agent tool selection, RAG filtering.
Failure Log — abnormal inputs, adversarial examples, out-of-distribution data, confidence errors, and demos that looked useful until they weren't.
What you won't find here
A new model called a breakthrough every week
Confidence scores treated as proof
A demo presented as production-ready
Safety claims that go further than the evidence
Generic AI news commentary with no experiment behind it
A polished interface mistaken for a reliable system
If a post doesn't have an experiment, a log, or a reproducible result attached, it doesn't belong here.
How I write
Like an engineer explaining a difficult system at a workbench — with the code, logs, and failed tests still visible on the table.
That means you'll see me say things like:
"Here is what the model actually received."
"Here is what it returned."
"Here is what surprised me."
"Here is where the demo stops being evidence."
"Would I ship this? Not yet — and here is why."
The goal is to be rigorous without being cold, and accessible without stripping out the technical detail that makes the result meaningful.
The brand promise
Understand the mechanism. Measure the behavior. Deploy the system. Test the scene. Log the failure.
That's the whole method, in one line. Everything on this site is an attempt to follow it.
Outside the workbench
I'm married to Claire and father to Nora, age 6. I repair vintage arcade cabinets and retro game consoles, hike around Pittsburgh with Nora on weekends, and brew coffee at home with a notebook full of grind sizes, water temperatures, and extraction times — the same notebook where I log unusual model failures. The two lists have more in common than you'd think.
Get in touch
Found a result you can't reproduce? Have a decision-first system worth testing? Disagree with a conclusion?
Bring the logs. I'll bring mine.
Show me the decision, not just the demo.