Benchmarks

Every figure on this page was produced by a named command whose raw output is kept with the measurement. A cell without a measurement reads "not measured"; every cell of the 8 GB laptop column reads "not measured on 8 GB". Numbers from the Apple-silicon development machine are placeholders until the reference 8 GB Windows laptop without a GPU has been measured.

A figure appears here only when a named command produced it, with the command and raw output saved; the row's evidence names that output. Apple-silicon numbers are placeholders until the 8 GB Windows no-GPU laptop has been measured. The 8 GB laptop column reads not measured on 8 GB in every row; every other cell without a figure reads not measured.

BenchmarkRepoCommandApple-silicon development machine (placeholder)8 GB Windows no-GPU laptopNotes
700 MB asset download (wall time, MB/s, peak JS heap)runtimenode test/browser/range-server.mjs --asset big=734003200; Playwright demo ensure('big')not measurednot measured on 8 GBSprint 1 done-when test
Resume after reload at 40 percent (one Range request)runtimePlaywright reload-at-40 specnot measurednot measured on 8 GB
Eviction under a 1.5 GiB origin quotaruntimeCDP Storage.overrideQuotaForOrigin; 700 + 400 then 500 MBnot measurednot measured on 8 GB
detectCapabilities() latencyruntimePlaywright capability specnot measurednot measured on 8 GBmust resolve within 2 s
Same prompt, WebGPU versus WASM, SmolLM2-135M q4 test model (load, prefill, decode, tokens per second)runtimeRUNTIME_MODEL=1 RUNTIME_REQUIRE_WEBGPU=1 npm run test:modelWebGPU: load 2,667 ms, prefill 66 ms, decode 51 ms, 155.9 tok/s. WASM: load 2,592 ms, prefill 368 ms, decode 732 ms, 10.9 tok/s. Same answer on both.not measured on 8 GBSprint 2 done-when; test model, not the Hindi production model; filled only from this command's output Evidence: evidence/sprint-2/D1/model-run.json.
runtime model-load timing per phase (acquire, verify, hand-off, session, first token): SmolLM2-135M, WebAssembly engine in a workerruntimeRUNTIME_MODEL=1 npm run test:devtools (headed)Apple silicon, Chrome 157.0.8103.79, WebAssembly engine in a worker (WebGPU was forced off in these runs; WebGPU not measured here). D1 run: acquire 1,260.9 ms, verify 663 ms, hand-off 7.6 ms, session create 481.2 ms, first token 655.1 ms. T1 run: acquire 1,254 ms, verify 664.9 ms, hand-off 7.4 ms, session create 479.7 ms, first token 655.5 ms.not measured on 8 GBSprint 3 slot, filled in Sprint 4 from the saved Airplane Mode DevTools-panel output of two headed runs (D1, then T1 independently). The figures are the panel's own phase durations; 'acquire' is the panel's 'Acquire (download, 0 when cached)' phase. SmolLM2-135M is the test model, not the production model. Apple-silicon figures are a placeholder until the 8 GB laptop is measured; not measured on 8 GB. Evidence: evidence/sprint-3/D1/devtools/devtools-figures.json and evidence/sprint-3/T1/r3/devtools/devtools-figures-t1.json.
runtime memory at model load (page JS heap where Chrome exposes it; worker where exposed)runtimeRUNTIME_MODEL=1 npm run test:devtools (headed)Apple silicon, Chrome 157.0.8103.79, page JS heap used (performance.memory, coarse). D1 run: 2,747,591 B before load, 79,554,898 B after session create, 79,611,170 B after first token. T1 run: 3,587,929 B, 33,865,195 B and 33,954,615 B at the same three points. The worker heap and WebAssembly memory were not exposed (source none), so there is no figure for them.not measured on 8 GBSprint 3 slot, filled in Sprint 4 from the saved DevTools-panel output of the same two headed runs; the figures are read from the browser, never estimated. The two runs differ after session create and the difference is not explained here. Apple-silicon figures are a placeholder until the 8 GB laptop is measured; not measured on 8 GB. Evidence: evidence/sprint-3/D1/devtools/devtools-figures.json and evidence/sprint-3/T1/r3/devtools/devtools-figures-t1.json.
pagetriage page report (ms per page, native vs wasm)pagetriagecargo bench; wasm demonot measurednot measured on 8 GB
glyphsift validation of about 1 MiB of synthetic Devanagari (12,888 lines)glyphsifttarget/release/glyphsift check corpus-1mib.txt (five runs); WASM validate() and score_ppm() in Node, best of fivenative CLI: 0.05 to 0.06 s (one cold run 0.17 s); WASM validate(): 232 ms; WASM score_ppm(): 75 msnot measured on 8 GBBrowser figures not measured. Evidence: evidence/sprint-2/D3/throughput.log.
glyphsift score versus noise (mixed-family mean and per-family curves, seeds 1, 2, 3)glyphsiftcargo run -p glyphsift-cli --release -- bench --corpus bench/corpus/clean.txt --seeds 1,2,3 --csv docs/score-vs-noise.csv --svg docs/score-vs-noise.svgMixed-family mean, ppm, at p = 0, 0.01, 0.02, 0.05, 0.1, 0.2, 0.4: 1,000,000; 998,476; 996,985; 991,696; 983,977; 967,538; 933,440. Falls at every step.not measured on 8 GBSprint 2 done-when; regenerated in Sprint 3 after two rule changes (the Sprint 2 write-up keeps the Sprint 2 figures); filled only from this command's output. The score is deterministic (integer arithmetic), so it does not depend on the machine; the laptop column stays not measured until it is run there. Evidence: evidence/sprint-3/D3/bench-s3.log and docs/score-vs-noise.csv.
localdata set/get throughput, OPFS vs IndexedDBlocaldataPlaywright behavioural suite timingnot measurednot measured on 8 GB
localdata flat index, 100,000 x 384 + 256-byte synthetic chunks, OPFS, cold open + first query (median / worst of 5)localdatanpm run bench:vectorsApple M5 Pro, 48 GB: median 186 ms / worst 195 ms of 5 cold runs; browser-restart run 182 msnot measured on 8 GBSprint 4 done-when row: cold open plus the first query (k = 10) must stay under 2,000 ms on an 8 GB laptop; this machine has more memory, so the figure is not an 8 GB figure. Measured on Apple M5 Pro, 48 GB (machine column), uncommitted working tree based on 0390d65 (commit column: 0390d65-dirty), by npm run bench:vectors in localdata; match is true on every row. JS heap was read with precise memory info enabled. The row is not measured on 8 GB: the 8 GB Windows laptop is the owner's step. Evidence: localdata/docs/benchmarks/vector-index.csv.
localdata flat index, 100,000 x 384 + 2,048-byte synthetic chunks, OPFS, cold open + first query (median / worst of 5)localdatanpm run bench:vectorsApple M5 Pro, 48 GB: median 181 ms / worst 183 ms of 5 cold runs; browser-restart run 182 msnot measured on 8 GBSprint 4 done-when row: cold open plus the first query (k = 10) must stay under 2,000 ms on an 8 GB laptop; this machine has more memory, so the figure is not an 8 GB figure. Measured on Apple M5 Pro, 48 GB (machine column), uncommitted working tree based on 0390d65 (commit column: 0390d65-dirty), by npm run bench:vectors in localdata; match is true on every row. JS heap was read with precise memory info enabled. The row is not measured on 8 GB: the 8 GB Windows laptop is the owner's step. Evidence: localdata/docs/benchmarks/vector-index.csv.
dockit cold start to interactive, offlinedockitPlaywright pwa.spec.ts timingnot measurednot measured on 8 GBskeleton only

dockit: an answer over a PDF, in the browser, offline

dockit A PDF goes through pagetriage (what is on each page), text extraction, the glyphsift gate, localdata chunks and vectors, and a small local model, in installed Chrome with the network blocked after the assets are cached. The command for every row is npm run bench:e2e; the raw rows are in dockit's docs/benchmarks/e2e.csv (sha256 1bbe066f768889ca) and docs/benchmarks/e2e-questions.csv (sha256 502b8aa50903c1de), taken at commit 0282d19da192. Each time is the median and the worst of the runs in its row; no run was left out.

MachineRun kindRunsTime to first token (median / worst)Total, first question (median / worst)Peak memory: Chrome process tree, summed resident size (median / worst)JS heap peak, main thread (median / worst)WebAssembly memory peak, workers (median / worst)Storage used (median / worst)Requests after offlineModel and backendCommandCommit, Chrome8 GB laptop
Apple M5 Pro 48 GBone Chrome session, model cached57,731 ms / 7,861 ms9,280 ms / 9,435 ms6,283 MiB / 6,377 MiB11 MiB / 31 MiB2,574 MiB / 2,574 MiB762 MiB / 762 MiB0Qwen3-0.6B q8 + multilingual-e5-small q8; transformers-js wasm (single-threaded)npm run bench:e2e0282d19da192, Chrome/157.0.8103.79not measured on 8 GB
Apple M5 Pro 48 GBafter a full Chrome restart, same profile17,861 ms / 7,861 ms9,440 ms / 9,440 ms6,147 MiB / 6,147 MiB33 MiB / 33 MiB2,574 MiB / 2,574 MiB762 MiB / 762 MiB0Qwen3-0.6B q8 + multilingual-e5-small q8; transformers-js wasm (single-threaded)npm run bench:e2e0282d19da192, Chrome/157.0.8103.79not measured on 8 GB

Peak memory is the peak summed resident size of the test Chrome's processes (browser, GPU, utility and renderer); a page shared between processes is counted in each. The JS heap and the workers' WebAssembly memory are different figures and are shown under their own names.

Requests after offline counts application requests: those made by the page, its workers and its service worker, as the largest count of network requests attempted after the network was blocked. The browser itself may still check /dockit/sw.js for an update and fetch the favicon; this count does not include those two requests, and dockit's code makes neither of them.

Question set

Seven fixed questions over a synthetic three-page policy, six with an answer on a known page and one with no answer in the document. Correct means the answer holds the expected text and a source the answer cites names the page that holds it. First run: 4 of 6 correct with the correct source page. Over all 6 runs, counting every run of every question with none selected: 24 of 36. The question with no answer was refused in 6 of 6 runs.

Two page columns are kept apart. A cited source on the right page was shown in 24 of 36 answerable rows: a refused answer cites nothing, so it never counts there. The top-ranked retrieved passage was on the right page in 36 of 36 (retrieval quality, reported on its own).

QuestionRunsAnswer correctCited source page correctTop-ranked retrieved passage on the right pageTime to first token (median / worst)8 GB laptop
q166667,732 ms / 7,861 msnot measured on 8 GB
q260067,819 ms / 7,833 msnot measured on 8 GB
q366667,736 ms / 7,743 msnot measured on 8 GB
q466667,823 ms / 7,827 msnot measured on 8 GB
q566667,803 ms / 7,815 msnot measured on 8 GB
q660067,840 ms / 7,852 msnot measured on 8 GB

Done-when: build done on Apple M5 Pro 48 GB; 8 GB Windows done-when NOT met (owner runbook step 5).

Limitations

Reference machine

The reference machine is an 8 GB Windows laptop with no discrete GPU, Wi-Fi off, running the tools in Chrome. It has not been measured yet; the column stays empty until it has.