A demo becomes evidence only when the inputs, the decisions, the confidence scores, and the latency are recorded under conditions that can be repeated. Without that record, a smooth interface remains a demonstration rather than a measurement.
The official Laya demo is a convenient starting point because it exposes a decision-first interface: short input in, structured choice and score out. The value of running it on The Decision Bench is not the demo itself. The value is the discipline of capturing exactly what the model received, what it returned, how confident it claimed to be, and how long the response took under the hardware and runtime in use.
This post describes the recording method I use. It does not claim any particular accuracy number or latency figure. Those numbers belong in later measurement posts once the logging pipeline is stable and the conditions are stated.
Why Recording Matters More Than the Demo Interface
Most public demos are optimized for first impression. The interface is clean, the examples are well-chosen, and the response appears quickly. That is useful for orientation. It is not sufficient for evaluation.
When the goal is to understand a decision-first model, the interesting artifacts are the ones that can be inspected later:
The exact text or feature set that was sent
The label, route, or action that came back
The probability or confidence attached to that choice
The wall-clock time from request to response under known conditions
Any alternative scores the model also returned
If those five pieces are missing, later discussion of calibration, latency tails, or failure cases rests on memory and screenshots. Memory and screenshots are poor substitutes for a structured log.
In earlier industrial automation work the same rule applied. Any classification that reached a control decision had to leave a trace: input, output, score, timestamp, and model version. Without the trace, debugging a high-confidence error months later became guesswork. The habit transferred directly to decision-first models running in the browser or on local runtimes.
The Minimum Logging Checklist
Every run I keep follows a short, fixed checklist.
Model and version identifier
Runtime and hardware description (browser, OS, device class, or local inference stack)
Exact input string or structured fields, copied without editing
Full structured output returned by the model
Primary confidence or probability score
Any secondary scores or alternative rankings that appear
Wall-clock latency measured from the moment the request is sent to the moment the structured result is available
Timestamp and a short note on anything unusual about the run
The checklist is deliberately plain. It does not require a heavy experiment-tracking platform. A text file, a simple JSON line, or a notebook entry is enough at the beginning. The important property is that every field is present and that the input is recorded exactly as it was submitted.
I keep the raw input unedited. Small changes in whitespace, punctuation, or casing can move a decision boundary. If the log contains a cleaned version instead of the original, later attempts to reproduce the same decision will fail for reasons that have nothing to do with the model.
Capturing Latency Without Self-Deception
Latency is easy to mis-measure.
Browser developer tools, console timers, and external stopwatches each introduce their own offsets. The practical approach is to pick one consistent method and state it clearly. For browser-based demos I usually instrument the call site so that the timer starts immediately before the inference request and stops immediately after the structured result is parsed. The number that is logged is therefore the end-to-end time visible to the calling code, not an internal model kernel time that the user never experiences.
I also record whether the measurement is a cold start or a warm run. First-load cost in the browser (model download, initialization, WASM or ONNX runtime setup) is often dominated by factors other than the forward pass. Mixing cold and warm numbers without labeling them produces a latency distribution that no one can interpret.
When the same input is run multiple times, I keep the individual measurements rather than only the average. A tight cluster is more informative than a single mean. A long tail is a signal that needs investigation, not an inconvenience to be averaged away.
Handling the Structured Output
Laya-style decision models return a structured result rather than a paragraph. That property makes logging easier and more reliable. The chosen label and the confidence score can be written directly into the log without a secondary extraction step.
I store the full structure when it is available. Secondary scores matter. A top score of 0.81 paired with a second score of 0.17 is a different situation from a top score of 0.81 paired with a second score of 0.02. The margin between the first and second choice is often more useful for threshold setting than the absolute value of the top score alone.
When the demo interface does not expose secondary scores, that limitation is noted in the log. The absence of information is itself information. Later comparison with a local or API deployment that does return the full distribution becomes possible only if the original demo runs recorded what was and was not visible.
Reproducibility Conditions
A log entry is only as useful as the conditions attached to it.
I record the browser version or local runtime version, the approximate device class (for example, mid-range laptop, desktop with discrete GPU, or phone), and whether any extensions or performance throttling were active. I do not claim that these notes make the measurement perfectly reproducible across every machine. They do make it possible to notice when a later run differs because the environment changed rather than because the model changed.
Model version is equally important. Public demos can be updated without notice. A log that lacks a version identifier becomes ambiguous the moment the underlying weights or post-processing change. When the demo provides no explicit version string, I record the date and a short description of the interface state so that later readers can at least place the measurement in time.
What This Method Does Not Claim
Recording inputs, outputs, and latency does not by itself prove that a model is accurate, calibrated, or ready for production. It only makes those later claims testable. A clean log of a hundred well-chosen examples is still only a hundred examples. Distribution shift, adversarial inputs, and long-tail latency under concurrent load remain separate measurements.
The method also does not replace proper evaluation sets. It is a lightweight discipline that sits in front of more formal testing. Its purpose is to stop the common pattern in which a demo is declared “fast and accurate” on the basis of a few manual trials whose inputs and timings were never written down.
How the Record Feeds the Rest of the Site
Bench Tests posts will use this logging style for latency, memory, calibration, and hardware comparisons. Deploy Notes will reuse the same fields when the model moves from the official demo to ONNX, Transformers.js, FastAPI, or mobile runtimes. Field Tests will apply the identical checklist inside concrete scenes such as email triage or content-safety filtering. Failure Log entries will begin from the same raw records when a high-confidence decision turns out to be wrong.
The common thread is visibility. When the decision, the score, and the timing are written down under stated conditions, later discussion can refer to evidence instead of recollection. That is the standard this site applies to every decision-first model it examines, starting with the official Laya demo and continuing through the measurements that follow.
