coja/dots
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[Docs] sanitize locators in the published infra guides

Browse Source
This commit is contained in:
coja
2026-08-13 00:04:37 +02:00
parent a099091b6c
commit 885a62520d
2 changed files with 16 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files
fl/.config/whisper/README.md
+12 -12
View File
@@ -9,19 +9,19 @@ mic button works against local hardware. Added **2026-08-08**.
| instance | endpoint | model | speed / use |
|---|---|---|---|
| **`whisper@turbo`** | `http://192.168.0.204:11345/v1` | `ggml-large-v3-turbo-q5_0` (574 MB) | **default** — best accuracy, multilingual, ~15 s per request. This is what OWUI points at. |
| `whisper@small` | `http://192.168.0.204:11346/v1` | `ggml-small.en-q5_1` (181 MB) | English-only, ~5x lighter. Point a client here when latency beats accuracy. |
| **`whisper@turbo`** | `http://$SERVER:11345/v1` | `ggml-large-v3-turbo-q5_0` (574 MB) | **default** — best accuracy, multilingual, ~15 s per request. This is what OWUI points at. |
| `whisper@small` | `http://$SERVER:11346/v1` | `ggml-small.en-q5_1` (181 MB) | English-only, ~5x lighter. Point a client here when latency beats accuracy. |
Both run at once — together they're well under 1 GB of RAM and **zero VRAM**, so there is no reason to
stop one to use the other. Adding a third is a new `.env` file plus `systemctl enable --now whisper@<name>`.
## How it runs
Template unit **`whisper@.service`** (user `anon`), from `/usr/bin/whisper-server` (Arch `extra/whisper-cpp`).
Template unit **`whisper@.service`** (a dedicated service user), from `/usr/bin/whisper-server` (Arch `extra/whisper-cpp`).
Each instance reads `~/.config/whisper/<instance>.env`:
```
WHISPER_MODEL=/home/anon/software/models/ggml-large-v3-turbo-q5_0.bin
WHISPER_MODEL=/home/<user>/software/models/ggml-large-v3-turbo-q5_0.bin
WHISPER_PORT=11345
WHISPER_LANG=auto
```
@@ -101,7 +101,7 @@ transcribing it, and the `gemma-4-E4B` preset samples at temp 1.0. Whisper is pu
```
curl -F file=@clip.wav -F model=gemma-4-E4B-it-UD-Q8_K_XL \
http://192.168.0.204:11343/v1/audio/transcriptions
http://$SERVER:11343/v1/audio/transcriptions
```
## Open WebUI wiring
@@ -111,7 +111,7 @@ Admin Settings → **Audio** → Speech-to-Text:
| field | value |
|---|---|
| STT Engine | `OpenAI` |
| Base URL | `http://192.168.0.204:11345/v1` — no trailing path, OWUI appends `/audio/transcriptions`. Swap `11345`→`11346` for the fast instance. |
| Base URL | `http://$SERVER:11345/v1` — no trailing path, OWUI appends `/audio/transcriptions`. Swap `11345`→`11346` for the fast instance. |
| API Key | any non-empty string (`none`) — ignored by whisper-server, but OWUI needs it set |
| STT Model | `whisper-1` — ignored by whisper-server; must not be blank |
| Request format | leave default (`multipart`) |
@@ -126,14 +126,14 @@ with line breaks mid-sentence.
systemctl status whisper@turbo whisper@small
rocm-smi --showmeminfo vram # must be UNCHANGED vs before the services started
curl -F file=@sample.wav http://192.168.0.204:11345/v1/audio/transcriptions # → {"text": "…"}
curl -F file=@sample.wav http://$SERVER:11345/v1/audio/transcriptions # → {"text": "…"}
ffmpeg -i sample.wav -c:a libopus sample.webm # the case that actually matters:
curl -F file=@sample.webm http://192.168.0.204:11345/v1/audio/transcriptions # proves --convert
curl -F file=@sample.webm http://$SERVER:11345/v1/audio/transcriptions # proves --convert
curl -F file=@sample.webm -F model=whisper-1 -F language=en \
http://192.168.0.204:11346/v1/audio/transcriptions # OWUI's exact shape
http://$SERVER:11346/v1/audio/transcriptions # OWUI's exact shape
```
Then: with a chat model loaded, transcribe and confirm `http://192.168.0.204:11343/v1/models` still
Then: with a chat model loaded, transcribe and confirm `http://$SERVER:11343/v1/models` still
shows it `ready`/`sleeping` — **not** `unloaded`. That is the whole point of the separate service.
**Measured 2026-08-08** (6 threads, CPU, 11 s JFK sample, `--convert` in the path):
@@ -168,7 +168,7 @@ time whisper-cli -m ~/software/models/ggml-base.en-q5_1.bin -f jfk.wav -t 6 -nt
## Troubleshooting
- **Every request returns `{"error":"FFmpeg conversion failed."}`, wav included** → the temp dir is not
writable by `anon`. Check `--tmp-dir /run/whisper-<instance>` is in ExecStart and that
writable by the service user. Check `--tmp-dir /run/whisper-<instance>` is in ExecStart and that
`RuntimeDirectory=` created it (`ls -ld /run/whisper-turbo`). ffmpeg itself is fine — the startup log
says `ffmpeg is available.` either way.
- **`Failed to load environment files`** → the instance has no `.env` (`whisper@foo` needs `foo.env`),
@@ -182,7 +182,7 @@ time whisper-cli -m ~/software/models/ggml-base.en-q5_1.bin -f jfk.wav -t 6 -nt
- **Still too slow** → next step down is `ggml-base.en-q5_1.bin` (56 MB, ~4x lighter again) at a real
cost in accuracy on names and jargon. New `.env`, or edit `small.env`.
- **OWUI: "Server Connection Error"** → base URL has a trailing `/audio/transcriptions` (it appends that
itself), or the port is not reachable from `192.168.0.72`.
itself), or the port is not reachable from the client host.
## Watch list