[Sync] Lua Hyprland chain, licence cleanup

This commit is contained in:
Coja committed 2026-11-09 18:25:35 +01:00
1 parent 93b4f695ce
commit 47166d877f
183 files changed
+5618 -5839

No files matched your search

+32 -9
View File
@@ -70,9 +70,9 @@ starves the desktop and freezes the PC (reboot). Hard-won rules:
| `GLM-4.7-Flash-UD-Q4_K_XL` | **21.5** @moe22 | 24k | reasoning | quality coder (opencode subagents); KV fix + moe22 ✓ verified clean 08-08 |
| `GLM-4.7-Flash-UD-Q3_K_XL` | **26.0** @moe12 (2.0 daytime baseline; ~28 clean) | 24k | reasoning | **fast coder** — re-added 09-21 because Q4's 21.5 dragged in agent loops; 3-bit, evening-safe at moe12 (13.4 used / 3.8 free @2.0); Q4 stays for hard reviews |
| `Qwen3-Coder-Next-UD-IQ3_XXS` | 16.0 | **128k** | 80B-A3B | long-session coder (128k ctx) |
| `Qwen3.8-27B-UD-IQ3_XXS` | **15.5** (10.9 w/o MTP) | 24k | vision, MTP, hybrid reasoner @ **xhigh** | ⚠ **clean-desktop specialty** (kept 09-10 over the ≥22 gate) — dense-27B quality, slow: its Gated-DeltaNet layers are Vulkan-bound (prefill ~35). 2.5 GB free @1.7 baseline @24k (✓ 10-02 on b11146, `-n 512`, acceptance 0.84). The *felt* slowness is the **xhigh thinking preamble** (17.5 s before the first answer token on a trivial prompt; `reasoning_effort=low` measured 6.45 s at the same decode on 10-02 — siblings benched then removed by decision, the kwarg is one line if ever wanted); **never on a busy evening desktop** (~1 GB free = GTT freeze) |
| `Qwen3.8-27B-UD-IQ3_XXS` | **36–37** on b11429 (10-22 ×2; was 15.5 on b11146, 10.9 base) | 24k | vision, MTP, hybrid reasoner @ **xhigh** | ⚠ **clean-desktop specialty** (kept 09-10 over the ≥22 gate — **the speed half is now passed 1.7× over**, since llama.cpp 0.6.0 halved the per-step MTP cost; role re-rank pending the user) — dense-27B quality; prefill still GDN-bound (~20–35). 2.7 GB free @1.3 baseline @24k (✓ 10-22 on b11429, `-n 512`, acceptance 0.89, len 2.78). The *felt* slowness is the **xhigh thinking preamble** (17.5 s before the first answer token on a trivial prompt; `reasoning_effort=low` measured 6.45 s at the same decode on 10-02 — siblings benched then removed by decision, the kwarg is one line if ever wanted); **never on a busy evening desktop** (~1 GB free = GTT freeze) |
| `Qwen3-Embedding-0.6B` | 33 (CPU) | 8k | CPU-only, `/v1/embeddings` | RAG/search embedder (not a chat model) |
| `Ornith-1.5-35B-A3B-IQ3_XXS` | **32.1** @moe16 | 24k | MTP, vision, thinking | **agentic coder** (added 09-21) — Qwen3.5-MoE derivative claiming +15 Terminal-Bench / +5.6 SWE-bench over Qwen3.6-35B; the only coder with MTP; 13.8 used / 3.3 free @2.0 |
| `Ornith-1.5-35B-A3B-IQ3_XXS` | **32.1** @moe16 | **32k** (10-22, unbenched) | MTP, vision, thinking | **agentic coder** (added 09-21) — Qwen3.5-MoE derivative claiming +15 Terminal-Bench / +5.6 SWE-bench over Qwen3.6-35B; the only coder with MTP; 13.8 used / 3.3 free @2.0 |
Expected run-to-run spread: MTP models swing ±15% with draft **acceptance rate** (content-
dependent); CPU-heavy presets (Coder-Next, embedder) dip under daytime CPU contention.
@@ -94,9 +94,9 @@ Treat clean night runs as the reference; don't retune on daytime deltas.
- **Hard reasoning**: `35B-Thinking` (quality) or `gpt-oss-20b` (speed + tool use).
Their multi-second TTFT is the reasoning phase streaming first — not a slow load.
For quick interactive gpt-oss answers use `gpt-oss-20b-low` (reasoning effort low).
- **Best single-model quality, speed no object** (quiet desktop only): `Qwen3.8-27B` — dense
27B hybrid reasoner + vision at ~14.5 t/s; it cannot go faster on this card (GDN layers on
Vulkan), and a busy evening desktop pushes it into GTT-freeze territory — check `rocm-smi` first.
- **Best single-model quality** (quiet desktop only): `Qwen3.8-27B` — dense 27B hybrid reasoner +
vision at **37 t/s since llama.cpp 0.6.0** (was ~15 on 0.5.0; prefill still GDN-bound on Vulkan),
but a busy evening desktop pushes it into GTT-freeze territory — check `rocm-smi` first.
- **Images**: `gemma-4-26B` for quality, `gemma-4-E4B` or `Qwen3.5-9B` for speed,
`Qwen3.6-35B` when you want the daily driver to see the screenshot.
- **Long one-shot documents**: `gemma-4-E4B` — 185 t/s prefill eats 64k in ~6 min
@@ -150,8 +150,13 @@ the router, but chats/presets saved against the old id need re-picking.
2026-09-10: kept as ⚠ clean-desktop specialty after the gate bench finally ran on the v3 gguf
(14.5 t/s MTP / 10.9 base, 2.8 GB free @1.5); server ctx restored to 24576 = client ctx, no client edits.
2026-09-21: `GLM-4.7-Flash-UD-Q3_K_XL` (⚡ fast coder) added to all six client files next to the Q4.
2026-10-22: Qwen3.8-27B quant race opened (ISTA GSQ-RCO IQ3_XXS-mtp + DFlash2 sibling) — bench-only,
NO client edits until a candidate wins; see the watch list. Same night: fl found on llama.cpp 0.6.0
(b11429) and the Qwen3.8 incumbent re-measured 37.1 t/s (15.5 on b11146) — every MTP/hybrid preset
on the roster needs a re-sweep on this build; ledger numbers above it are from 0.5.0 unless dated 10-22.
2026-09-21: `Ornith-1.5-35B-A3B-IQ3_XXS` (agentic coder, thinking + vision) added to all six client files after passing its gate (32.1 t/s @moe16).
2026-09-21: default model → `Qwen3.6-35B-A3B-Thinking` everywhere (opencode global + plan/build/auto, pi defaultModel + thinking level medium, aichat common/lw/wm) — gemma-4-26B refused routine sysadmin work. Thinking preset gets `reasoning-budget 8192`. wm aichat caught up (GLM Q3, Ornith).
2026-10-22: Ornith in pi: `Compacting in OVERFLOW recovery` right after a truncated reply is pi's *truncated-reply* recovery, not a full window — pi clamps each request's `max_tokens` to `contextWindow − estimate − 4096`, which on 24k/4096 starts exactly at the compaction threshold, so the first request after a compaction (estimate gains the system prompt + tool schemas) can leave a thinking reply ~1024 answer tokens → `length` → compact + retry. `compaction-watch.ts` now names the case with numbers; `common/.pi/CLAUDE.md` > Compaction tuning has the arithmetic. Applied the same day, UNBENCHED: Ornith `ctx-size` 24576 → 32768 + the seven client files (pi/opencode/aichat, common+lw+wm; predicted ≈ 0.1 GB of KV on this hybrid arch) — restart llama.service, then `./bench.py -m Ornith-1.5-35B-A3B-IQ3_XXS`; gate and the revert path are in the preset comment.
2026-10-12: Ornith in pi: "Response was truncated before completion." = `finish_reason: length` — the `<think>` block counts against pi's 4096 `max_tokens` and the preset has no reasoning-budget. Fixed client-side: pi sends `thinking_budget_tokens` per thinking level on Ornith (`compat.thinkingTokenBudgetField` + `settings.json > thinkingBudgets`, common + lw). The preset deliberately keeps NO CLI reasoning-budget — it would override the per-request field (see the preset comment).
## Tuning cheat-sheet
@@ -200,9 +205,27 @@ reasoning-effort siblings; for stable MTP decode numbers use `-n 512`.
## Watch list
- **GLM MTP in llama.cpp** — the single biggest pending win (primary coder ~22 → ~30+).
- **DFlash2 for Qwen3.8-27B** — `z-lab/Qwen3.8-27B-DFlash2-GGUF` drafter, upstream 1.81× vs the
MTP head's 1.33× here (b11146 has the support). Blockers: #27408 (spec + mmproj → image requests
500, so only as a text-only sibling preset) and #28158 (Vulkan OOB draft token, open). Trial =
`spec-type draft-dflash`, `spec-draft-n-max 3`, no mmproj; n-max 7 does not fit the VRAM envelope.
- **Qwen3.8-27B quant race (opened 2026-10-22, bench-only, not client-wired)** — two candidate
sections in `config.ini` after the incumbent: `Qwen3.8-27B-GSQ-RCO-IQ3_XXS` (ISTA-DASLab GSQ-RCO
file, 10.44 GB with MTP head = −0.49 GB vs unsloth v3) and its `-dflash` text-only sibling
(`z-lab/Qwen3.8-27B-DFlash2-GGUF` Q4_K_M drafter, `spec-draft-n-max 3`). Expected: +4 % decode and
+0.5 GB free from the file; +5–15 % over MTP from DFlash2 on RDNA3/Vulkan (not the paper's 1.8×).
Quality is unmeasured against our file — the user's own prompts decide. First run 10-22 03:14: both
candidates HTTP 400 = fl's config.ini not pulled yet (router: "model not found"); the incumbent in
the same run read **37.1 t/s on b11429** (see its row). Re-run 03:25, one session: incumbent 36.2 /
2.7 free, **GSQ 37.5 / 3.2 free** (+3.6 %, +0.5 GB — speed+VRAM passed, **quality call open**),
**DFlash2 15.4 / 1.9 free → dropped** (tombstoned in `config.ini`). Ruled out by the same
research: ByteShape IQ4_XS (13.1 GB), ISTA IQ3_S (12.1 GB with MTP, under the floor), the
adaptive KV-cache-streaming fork (CUDA-only, PCIe 5.0). Still open: #27408 (spec + mmproj → 500),
#28158 (Vulkan OOB draft token). Research + verdicts + download lines: the config block.
- **llama.cpp 0.7.0 (b11669) — HOLD fl at 0.6.0**: #30688, Vulkan draft-mtp decode 4× / prefill 6×
slower (target compute buffer lands in GTT = this box's freeze precursor). Every MTP preset on the
roster is exposed. When `pacman -Si llama-cpp` on fl shows 0.7.0, `IgnorePkg` it until fixed.
The 0.5.0 → 0.6.0 jump itself (Qwen3.8 15.5 → 37) is llama-side, best explained by #29856 (hybrid
graph reserve; the old 10.9 base was probably GTT-starved, not kernel-bound — see the preset block).
**ggml 0.26.0** (next `-Syu`): #30039 reports a −45 % prefill regression on a GDN hybrid — bench
Qwen3.8 before and after; downgrade ggml/ggml-vulkan from the pacman cache if prefill tanks.
- **Coder-Next 256k**: if the load log's `n_ctx_train` says 262144, ctx can likely double
again for ~1 GB (KV measured nearly flat).
> **vLLM on this box** (bench rig, docker/ROCm, never resident together with the router): see `../vllm/README.md`.