kohen: the watchdog — /kodesh/health watched from outside, a Gitea issue on failure

2026-10-15: the site was 503 for 13.5 h across a day boundary and the
Doorpost outage was found by losing Claude; 24 h of database refusals
were found the same way. Nothing watched.

scripts/bikoret-watchdog (installed at ~/.local/bin, cron every 5 min
on the dev VM) probes /kodesh/health. Two consecutive failures open ONE
Gitea issue in this repo naming the failed checks and what to read
first; while down it comments only when the failed set changes; on
recovery it comments the duration and closes the issue. Transitions go
to ~/shorashim-operator-notes/watchdog.log; silence is health.

Proven: healthy → exit 0, no log; dead URL ×3 → FAIL #1, #2, one ALERT
(not two), #3 no second issue; recovery → comment + close. The real
Gitea call was refused 403: the clone token in ~/.git-credentials has
only the code scopes. The script prefers
~/.config/bikoret/gitea_watchdog_token (write:issue), which the kohen
issues; until then it logs and tracks recovery but cannot open the
issue. KOHEN.md "Every morning" carries it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

KOHEN.md

@@ -488,6 +488,29 @@ curl https://bikoret.ch/kodesh/status | jq .halt_status
 curl https://bikoret.ch/kodesh/status | jq '.refuge_components'
 ```
 
+**The watchdog — *shomer*.** `scripts/bikoret-watchdog` (installed at
+`~/.local/bin/`, cron every 5 minutes on the dev VM) probes
+`/kodesh/health` from outside the cluster. Two consecutive failures —
+ten minutes, so a rollout's own blip does not alert — open ONE Gitea
+issue in this repo titled `WATCHDOG: bikoret.ch down since …`, naming
+the failed checks and what to read first; while down it comments only
+when the failed set changes; on recovery it comments the duration and
+closes the issue. Every transition goes to
+`~/shorashim-operator-notes/watchdog.log`. It needs a Gitea token with
+`write:issue` at `~/.config/bikoret/gitea_watchdog_token` (0600) — the
+clone token in `~/.git-credentials` has only the code scopes and Gitea
+answers 403 on an issue. The kohen issues it (Gitea → Settings →
+Applications → token, scopes `read:issue` + `write:issue`); the builder
+does not write credentials. Until it exists the watchdog still logs and
+tracks recovery; it cannot open the issue. 2026-10-15: the site was 503
+for 13.5 h and the Doorpost outage was found by losing Claude — nothing
+watched.
+
+```bash
+tail ~/shorashim-operator-notes/watchdog.log        # transitions only; silence is health
+BIKORET_WATCHDOG_DRY=1 bikoret-watchdog              # a probe that opens nothing
+```
+
 ### Issuing a key — *Mishmeret HaSha'ar*
 
 The courtyard and the chatzer open only to a key you issued. Keys live