Sprint 2: a validator that scores corruption, and one prompt on two backends
Sprint 2 (planned 23 November to 6 December 2026, started early on 9 November) had two measurable goals: the glyphsift validator's score falls steadily as synthetic OCR noise rises, and the same prompt is answered on WebGPU and on WebAssembly. Both were met. Every figure below names the command that produced it and the saved output it was copied from.
Any figure that lands here was measured on an Apple-silicon development machine and is a placeholder; the reference 8 GB Windows laptop without a GPU has not been measured yet.
Correction, 14 November 2026. The "What is next" section below says Sprint 3 is
the runtime alpha, including the WebLLM backend. The WebLLM backend moved to Sprint 4; Sprint 3
delivered the runtime alpha without it, and WebLLM was then cut from Sprint 4 (TD-08, not yet). The score-versus-noise figures below are the Sprint 2
ones; glyphsift's docs/score-vs-noise.csv was regenerated in Sprint 3 after two rule
changes, so the file in the repository no longer reproduces this table. The benchmarks page
carries the regenerated figures. The text below is otherwise left as it was written.
The done-when, stated honestly
Same prompt answered on WebGPU and on WebAssembly with the test model SmolLM2-135M-Instruct (q4), in Chrome on an Apple-silicon Mac only. This is not the Hindi production model; Windows and the 8 GB no-GPU laptop are not measured.
glyphsift: score versus noise
Command:
cargo run -p glyphsift-cli --release -- bench --corpus bench/corpus/clean.txt --seeds 1,2,3 --csv docs/score-vs-noise.csv --svg docs/score-vs-noise.svg
(output in docs/score-vs-noise.csv in the glyphsift repository and
evidence/sprint-2/D3/bench-run-1.log).
| Noise level p | Mixed-family mean score (ppm, seeds 1 to 3) |
|---|---|
| 0 | 1,000,000 |
| 0.01 | 998,454 |
| 0.02 | 996,931 |
| 0.05 | 991,588 |
| 0.1 | 983,739 |
| 0.2 | 967,065 |
| 0.4 | 932,405 |
- The mixed-family mean falls strictly at every step, and the bench command exits with an error if it ever does not.
- Mixed is non-increasing for each seed, and so is every family that triggers a rule. Two runs gave byte-identical CSV and chart files. Native, Python and WebAssembly give byte-identical reports.
- The score is deterministic: integer arithmetic, weights in a readable TOML file, no model. It is the same on any machine.
- What it cannot see: seven noise families, among them a dropped or swapped vowel sign, a lost nukta and a deleted virama, produce clusters that are still structurally valid. They change meaning, not structure, so the validator does not flag them. The README says so in its Limitations section.
runtime: one prompt, two backends
Command: RUNTIME_MODEL=1 RUNTIME_REQUIRE_WEBGPU=1 npm run test:model
(output in evidence/sprint-2/D1/model-run.json), prompt "What is the capital of
France? Answer in one word.", greedy decoding.
| Leg | Load | Prefill | Decode | Speed | Answer |
|---|---|---|---|---|---|
| WebGPU | 2,667 ms | 66 ms | 51 ms | 155.9 tokens per second | The capital of France is Paris. |
| WebAssembly | 2,592 ms | 368 ms | 732 ms | 10.9 tokens per second | The capital of France is Paris. |
- Every model file is checked against its SHA-256 in a signed manifest before the engine sees it, and the engine has no network access of its own. A tampered file is caught before it loads.
- The browser tester repeated both legs with an independent harness and logged every request: none left the page origin, worker threads included.
- After the push, both legs also answered on the live demo page, served by GitHub Pages.
The model decision
Planned production model: Qwen3-1.7B (Apache-2.0), WebGPU q4f16 1.43 GB, WASM q8 1.74 GB. Provisional until the Sprint 5 measurement on an 8 GB laptop; contingency Qwen3-0.6B.
The comparison table, the licence conditions and the Hindi evidence for each candidate are in
the runtime repository's docs/model-choice.md. Loaded once in a browser: a local, unscored WebGPU smoke run of five synthetic Hindi prompts on an Apple-silicon Mac, which needed ONNX Runtime graph optimisation turned off (evidence/sprint-2/D1/hindi-smoke.json). The decision stays provisional.
What else landed
- Contracts: spec release 0.2.0 (draft) adds the model descriptor and backend adapter contracts and the new backend and model events. localdata, dockit and control-plane re-vendored them.
- pagetriage: the determinism gate (native, Python and WebAssembly give identical reports) now runs on Windows as well as Linux and macOS, and passes on all three.
- Demos: all six were refreshed to show this sprint's work. The browser tester built each one the way its Pages workflow builds it and found zero console errors and only same-origin requests.
- Security: no critical, high or medium finding is open. Making the six public repositories and their demos public was approved with conditions, and the conditions are met.
What did not pass
- The first CI run on Windows found three bugs that only show up there. Two test scripts assumed forward slashes in file paths, and one Python script could not print Devanagari to a Windows console. All three are fixed.
- The live backend demo first refused to load any model: GitHub Pages compresses JSON on the fly, which tripped a file-size check. That check now allows for compression, and the SHA-256 check still decides.
- A WebGPU failure blocks the WebAssembly fallback in the same page, and WebAssembly generation runs on the main thread, so the page freezes while it generates. Both are documented, and both are Sprint 3 work.
- Hindi quality is not measured. The smoke run above is unscored.
What is next
Sprint 3 is the runtime alpha, including the WebLLM backend. Sprint 5 is the measurement on the 8 GB Windows laptop and the real Hindi pipeline in dockit. Until then every figure on this site stays labelled as a placeholder.
Prose on this page is CC BY 4.0; attribute "Airplane Mode Labs" with a link.