From 885a62520db4bd09cb5c5163e64fd6c82e923701 Mon Sep 17 00:00:00 2001 From: coja Date: Thu, 13 Aug 2026 00:04:37 +0200 Subject: [PATCH] [Docs] sanitize locators in the published infra guides --- fl/.config/llamacpp/README.md | 7 ++++--- fl/.config/whisper/README.md | 24 ++++++++++++------------ 2 files changed, 16 insertions(+), 15 deletions(-) diff --git a/fl/.config/llamacpp/README.md b/fl/.config/llamacpp/README.md index fd1f9c2..90218ce 100644 --- a/fl/.config/llamacpp/README.md +++ b/fl/.config/llamacpp/README.md @@ -6,14 +6,15 @@ Last full clean sweep: **2026-08-08 03:22** — all 11 presets loaded and ran gr - **Box**: AMD RX 7600 XT 16 GB (ROCm, ~288 GB/s) + Ryzen 5600X (6 cores) + 48 GB RAM. ⚠ The GPU **also drives the display** — see [VRAM safety](#vram-safety-the-golden-rules). -- **Endpoints**: LAN `http://192.168.0.204:11343/v1` · remote `https://llm.duskadiy.com/api/v1` - (own reverse proxy to the same server, no key). +- **Endpoints**: LAN `http://$SERVER:11343/v1` · remote `https:///api/v1` + (own reverse proxy to the same server; keyless — put auth in front before exposing it). + `$SERVER` = the box's LAN address. - **Files here**: `config.ini` (model presets — the section names ARE the API model ids), `bench.py` (safety-first benchmark), `bench-results.md` / `bench-history.md` (ledgers, generated). ## How it runs -systemd unit **`llama.service`** (on the server, user `anon`) runs a script wrapping: +systemd unit **`llama.service`** (on the server, under a dedicated service user) runs a script wrapping: ``` llama-server --port 11343 --host 0.0.0.0 --models-max 1 --models-preset ~/.config/llamacpp/config.ini diff --git a/fl/.config/whisper/README.md b/fl/.config/whisper/README.md index 46f7b3e..87a3dcc 100644 --- a/fl/.config/whisper/README.md +++ b/fl/.config/whisper/README.md @@ -9,19 +9,19 @@ mic button works against local hardware. Added **2026-08-08**. | instance | endpoint | model | speed / use | |---|---|---|---| -| **`whisper@turbo`** | `http://192.168.0.204:11345/v1` | `ggml-large-v3-turbo-q5_0` (574 MB) | **default** — best accuracy, multilingual, ~15 s per request. This is what OWUI points at. | -| `whisper@small` | `http://192.168.0.204:11346/v1` | `ggml-small.en-q5_1` (181 MB) | English-only, ~5x lighter. Point a client here when latency beats accuracy. | +| **`whisper@turbo`** | `http://$SERVER:11345/v1` | `ggml-large-v3-turbo-q5_0` (574 MB) | **default** — best accuracy, multilingual, ~15 s per request. This is what OWUI points at. | +| `whisper@small` | `http://$SERVER:11346/v1` | `ggml-small.en-q5_1` (181 MB) | English-only, ~5x lighter. Point a client here when latency beats accuracy. | Both run at once — together they're well under 1 GB of RAM and **zero VRAM**, so there is no reason to stop one to use the other. Adding a third is a new `.env` file plus `systemctl enable --now whisper@`. ## How it runs -Template unit **`whisper@.service`** (user `anon`), from `/usr/bin/whisper-server` (Arch `extra/whisper-cpp`). +Template unit **`whisper@.service`** (a dedicated service user), from `/usr/bin/whisper-server` (Arch `extra/whisper-cpp`). Each instance reads `~/.config/whisper/.env`: ``` -WHISPER_MODEL=/home/anon/software/models/ggml-large-v3-turbo-q5_0.bin +WHISPER_MODEL=/home//software/models/ggml-large-v3-turbo-q5_0.bin WHISPER_PORT=11345 WHISPER_LANG=auto ``` @@ -101,7 +101,7 @@ transcribing it, and the `gemma-4-E4B` preset samples at temp 1.0. Whisper is pu ``` curl -F file=@clip.wav -F model=gemma-4-E4B-it-UD-Q8_K_XL \ - http://192.168.0.204:11343/v1/audio/transcriptions + http://$SERVER:11343/v1/audio/transcriptions ``` ## Open WebUI wiring @@ -111,7 +111,7 @@ Admin Settings → **Audio** → Speech-to-Text: | field | value | |---|---| | STT Engine | `OpenAI` | -| Base URL | `http://192.168.0.204:11345/v1` — no trailing path, OWUI appends `/audio/transcriptions`. Swap `11345`→`11346` for the fast instance. | +| Base URL | `http://$SERVER:11345/v1` — no trailing path, OWUI appends `/audio/transcriptions`. Swap `11345`→`11346` for the fast instance. | | API Key | any non-empty string (`none`) — ignored by whisper-server, but OWUI needs it set | | STT Model | `whisper-1` — ignored by whisper-server; must not be blank | | Request format | leave default (`multipart`) | @@ -126,14 +126,14 @@ with line breaks mid-sentence. systemctl status whisper@turbo whisper@small rocm-smi --showmeminfo vram # must be UNCHANGED vs before the services started -curl -F file=@sample.wav http://192.168.0.204:11345/v1/audio/transcriptions # → {"text": "…"} +curl -F file=@sample.wav http://$SERVER:11345/v1/audio/transcriptions # → {"text": "…"} ffmpeg -i sample.wav -c:a libopus sample.webm # the case that actually matters: -curl -F file=@sample.webm http://192.168.0.204:11345/v1/audio/transcriptions # proves --convert +curl -F file=@sample.webm http://$SERVER:11345/v1/audio/transcriptions # proves --convert curl -F file=@sample.webm -F model=whisper-1 -F language=en \ - http://192.168.0.204:11346/v1/audio/transcriptions # OWUI's exact shape + http://$SERVER:11346/v1/audio/transcriptions # OWUI's exact shape ``` -Then: with a chat model loaded, transcribe and confirm `http://192.168.0.204:11343/v1/models` still +Then: with a chat model loaded, transcribe and confirm `http://$SERVER:11343/v1/models` still shows it `ready`/`sleeping` — **not** `unloaded`. That is the whole point of the separate service. **Measured 2026-08-08** (6 threads, CPU, 11 s JFK sample, `--convert` in the path): @@ -168,7 +168,7 @@ time whisper-cli -m ~/software/models/ggml-base.en-q5_1.bin -f jfk.wav -t 6 -nt ## Troubleshooting - **Every request returns `{"error":"FFmpeg conversion failed."}`, wav included** → the temp dir is not - writable by `anon`. Check `--tmp-dir /run/whisper-` is in ExecStart and that + writable by the service user. Check `--tmp-dir /run/whisper-` is in ExecStart and that `RuntimeDirectory=` created it (`ls -ld /run/whisper-turbo`). ffmpeg itself is fine — the startup log says `ffmpeg is available.` either way. - **`Failed to load environment files`** → the instance has no `.env` (`whisper@foo` needs `foo.env`), @@ -182,7 +182,7 @@ time whisper-cli -m ~/software/models/ggml-base.en-q5_1.bin -f jfk.wav -t 6 -nt - **Still too slow** → next step down is `ggml-base.en-q5_1.bin` (56 MB, ~4x lighter again) at a real cost in accuracy on names and jargon. New `.env`, or edit `small.env`. - **OWUI: "Server Connection Error"** → base URL has a trailing `/audio/transcriptions` (it appends that - itself), or the port is not reachable from `192.168.0.72`. + itself), or the port is not reachable from the client host. ## Watch list