| | Card 2026-09-07 (evening): two-box pipelined prefill in the served path; prefill defaults landed; DFlash2 p512/gen64 diagnostic corrected to 59.5 tok/s | avlp12 | Sep 7, 2026 |
| | Card 2026-09-07: multi-turn session reuse row (99.3% cached, 0.17 s TTFT), branch head a6634a75; earlier sections unchanged | avlp12 |
| | Card update 2026-09-07: 5-bit-KDA build served locally (repo files unchanged), new speculative headline, thinking budget with speculation, disk prefix vault; earlier sections unchanged | avlp12 |
| | Card: date-label the historical sections (as of 2026-09-05) so they are not read as current; numbers unchanged | avlp12 |
| | Card update 2026-09-06: new measurements section first (decode/speculative/sampled/structured/prefix-cache/prefill/pipeline-parallel/ceilings), corrections, negative results; earlier content unchanged | avlp12 |
| | Refresh chat_template.jinja to upstream 690b7052; card note on what changed (image markers, clear_thinking default, tool-result guards) | avlp12 |
| | Update model card with serving measurements and cache caveats | avlp12 |
| | card: condition-labelled decode/speculative rows; withdraw identical-row B=8 aggregate (paired +49% kept), unprovenanced 1.64x TP-spec headline, and the context-flat claim; add speculative-prefill limit (32k) and non-identity caveats; label Metal buffer settings; #2105 status | avlp12 |
| | recipe table: KDA attention projections/gates are 8-bit (DSA q_a/q_b/o 6-bit; kv_a/indexer 8-bit) -- corrected from a full safetensors header scan | avlp12 |
| | prefill@131k 295 tok/s (1.33-1.35x paired x2 loads); spec 50.8 code with adaptive width + refit constants | avlp12 |
| | prefill@65k 314 tok/s: 2^31 query-chunk boundary fix + gate retune; buffer env superseded by fused kernels | avlp12 |
| | TP+spec re-validated: 49.8 tok/s (1.44x on top of TP, prose 1.32x); 60.8 retired | avlp12 |
| | speed section speculative bullet corrected to natural-prompt figures | avlp12 |
| | speculative rows corrected: 45.9 (1.51x) natural code replaces PAD-flattered 54.7; TP+spec marked re-validation pending | avlp12 |
| | TP setup: note FAST_SYNCH not-guaranteed status per mlx#4438 maintainer reply; recommend per-fleet soak validation | avlp12 |
| | buffer env caveat: it is a B=1 lever, regresses batched serving (-7.5% at B=16) | avlp12 |
| | TP cells closed: B=8 162 and prefill 631/910 re-validated post-fix; B=1 both-accountings noted | avlp12 |
| | TP numbers corrected after indexer-reduction fix: B=1 36.2 (+12%), TP+spec 60.8 (1.88x); B=8/prefill re-validation pending | avlp12 |
| | gather gate: PR 2087 now defaults to 32768 upstream (adopted); note config-dependence | avlp12 |
| | remove reference to private conversion-pipeline repo | avlp12 |