Running a decision-first model inside a browser tab changes the constraints. Memory is shared with the rest of the page and with other tabs. The first load must fetch and initialize weights. Inference speed is limited by the device’s CPU or GPU access through WebAssembly, WebGPU, or similar runtimes. A desktop demo that feels instant can become unusable once those browser realities appear.
Laya is a useful case for this measurement because its primary output is a structured decision rather than a long generated sequence. The absence of token-by-token generation removes one source of variable cost, but it does not remove the costs of model download, runtime initialization, and peak memory during the forward pass. This post describes how those quantities are measured on The Decision Bench. It does not report specific numbers from any single device; those belong in later posts once the method is fixed and the conditions are stated.
Why Browser Measurements Differ from Local or Server Runs
On a local Python process or a dedicated inference server the model usually owns the memory and the compute device. In the browser the model shares the tab’s heap with the page’s JavaScript, DOM, and any other libraries. The browser itself may pause or throttle background tabs. Mobile browsers add further limits on memory and on how long a script may run before the system intervenes.
These differences make three quantities especially important:
Peak memory attributed to the model and its runtime
Time from page load or explicit model request until the model is ready for the first inference
Wall-clock time for a single structured decision once the model is warm
A measurement that ignores any of the three can produce a misleading picture. A model that reports low inference latency after a long warm-up may still be impractical if the first-load cost drives users away or if the memory footprint causes the tab to be discarded on mid-range phones.
In earlier work with industrial systems we treated resource limits as first-class constraints. The same posture applies here. The browser is simply another constrained environment that must be measured rather than assumed.
Measuring Load Time
Load time is the interval between the moment the application requests the model and the moment the runtime reports that inference can begin. It usually includes network transfer of weights, compilation or instantiation of the runtime (WASM, ONNX Runtime Web, Transformers.js pipeline, or similar), and any one-time initialization the library performs.
I separate cold-load measurements from subsequent warm loads. A cold load starts with an empty cache and no pre-initialized runtime. A warm load benefits from browser HTTP cache, already-compiled WASM, or an in-memory model that has not been released. Mixing the two without labels produces a distribution that cannot be interpreted.
The timer starts when the load request is issued and stops when the library’s ready callback or equivalent promise resolves. I also note whether the measurement was taken on a desktop or mobile browser, and whether the network was throttled. Those conditions are written into the same log entry as the timing number so that later comparisons remain meaningful.
Measuring Inference Speed
Once the model is loaded, inference speed is the wall-clock time from the moment a concrete input is submitted until the structured decision and confidence score are available to the calling JavaScript.
For decision-first models this interval is usually dominated by the forward pass and by any post-processing that produces the final label and score. Because there is no autoregressive loop, the cost does not grow with output length. That property makes repeated measurements more stable than they would be for a generative model, yet stability still requires care.
I run the same input multiple times after a warm-up pass and keep the individual timings. The mean is useful; the spread and the upper percentiles are more useful. A tight cluster suggests predictable behavior under the current device and browser. A long tail suggests contention, garbage collection, or thermal throttling that will appear in real use.
The input used for timing is recorded exactly. Different input lengths or different token counts can change the cost even for a non-generative model, because the encoder still processes the full sequence. Logging the input alongside the timing prevents later confusion when two “average” numbers were produced by different input distributions.
Observing Memory Use
Browser memory measurement is inherently approximate. The JavaScript heap can be inspected through performance APIs, and Chrome’s task manager or memory profiler can show tab-level figures. Native memory used by WASM or WebGPU is harder to attribute precisely from inside the page.
The practical approach is to record several complementary signals:
JavaScript heap size before model load, after load, and after a series of inferences
Tab-level memory reported by the browser’s task manager or equivalent at the same points
Any explicit memory-related warnings or tab discards observed during the session
I take the snapshots at consistent moments: empty page, model loaded but idle, after N inferences, and after the model is explicitly released if the library supports disposal. The differences between those snapshots give a usable picture of the incremental cost of the decision model under the tested runtime.
On mobile devices the same snapshots are more important. Mobile browsers are quicker to discard tabs that grow too large, and users notice when a page becomes sluggish or reloads unexpectedly. A desktop measurement that looks comfortable can still be unacceptable once the same model is opened on a mid-range phone.
Conditions That Must Be Stated
Every browser measurement is conditional. The log entry therefore includes:
Browser name and version
Device class (desktop, laptop, phone) and approximate hardware tier
Whether the tab was foreground or background
Whether other heavy tabs were open
Runtime library and version (Transformers.js, ONNX Runtime Web, etc.)
Model identifier and any quantization or optimization flags
Without those fields a latency or memory number cannot be compared to a later run. The Decision Bench treats the conditions as part of the measurement rather than as optional metadata.
What These Measurements Do Not Claim
Recording load time, inference speed, and memory snapshots does not prove that a model is suitable for production browser use. It only makes suitability testable. A model that loads quickly and runs with modest memory on a high-end laptop may still exceed practical limits on the devices that matter to the product. Conversely, a model that looks heavy in an unoptimized demo may become acceptable after quantization, caching, or deferred loading.
The measurements also do not replace accuracy or calibration checks. A fast, memory-efficient decision that is poorly calibrated remains a fast, memory-efficient source of errors. Later Bench Tests posts examine those dimensions under the same logging discipline.
How Browser Figures Feed the Rest of the Site
Deploy Notes will reuse the same measurement points when the model moves between Transformers.js, ONNX, and other browser-capable runtimes. Field Tests will note whether the observed browser cost remains acceptable inside concrete workflows such as email triage or content-safety filtering. Failure Log entries will record cases where memory pressure or long load times caused the decision path to be skipped or deferred.
The common requirement is visibility. When load time, inference latency, and memory cost are written down under stated browser conditions, later discussion can refer to evidence instead of impression. That is the standard applied to Laya and to any other decision-first model examined in the browser on this site.
