Compare commits

...
Author SHA1 Message Date
Scott KarpandClaude Fable 5.1 04f61c1238 build: the Doorpost read before the image is built
Tablet 54 (2026-10-15): two spec edits in @spec_files shipped without
moving @expected_hash; the pod logged DOORPOST_DRIFT_AT_STARTUP,
readiness answered 503, and the gate — with every Claude Code session
behind it — went dark. The Doorpost test catches this only when the
suite runs on the exact commit that gets built, and the build script
never ran it.

scripts/doorpost_check.py computes the Doorpost identity the way the
module does (sha256 per spec file in listed order, "path\thash" joined
by newline, sha256 of that) with no Elixir, against the working tree
the image is copied from (default) or a git revision (--rev). On a
mismatch it exits 1 and names each file changed since --base
(origin/main). scripts/build_and_push.sh runs it before anything else
and refuses to build; DOORPOST_SKIP=1 is the written-down exception.

Proven: working tree and cb18f719 match their pins; 347f83cf (the
tablet 54 commit) reports DRIFT naming specs/bikoret_tamid.md and
specs/bishearecha.md; a blank line appended to specs/lo_tosif.md makes
both the check and the build script exit 1. KOHEN.md "Rolling out"
carries the procedure.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-19 16:25:09 +00:00
Scott KarpandClaude Fable 5.1 07463c9c37 kohen: claude-key — Claude Code on the API key when the login is what failed
2026-10-15 21:08Z: the daemon's proactive OAuth refresh failed (no
incident at Anthropic; most likely a cross-process race on the
single-use refresh token), the credential was dropped, and every
session on the VM showed "Login expired · Please run /login" until a
browser login took at 23:50Z — two "Login interrupted" first. The gate
was off; the fallback of 4acd56c4 could not help, because it is the
same OAuth credential.

scripts/claude-key starts Claude Code on the Anthropic API key from
~/.config/shorashim/upstream_keys (read at launch, never written),
direct to Anthropic whatever settings.json says. Proven:
`claude-key auth status` → authMethod api_key, apiKeySource
ANTHROPIC_API_KEY; a -p run from a directory with a dead gate armed in
project settings answered. KOHEN.md: the login recovery (claude-key,
`claude auth login` from the shell, `auth status --text`).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-15 23:55:48 +00:00
Scott KarpandClaude Fable 5.1 4acd56c4b6 kohen: fallback for Claude Code when the gate is down
Every Claude Code session on the dev VM goes through bikoret.ch; when
the pod is not ready the session cannot reach Anthropic (2026-10-15:
Doorpost drift → 503 → the transmitter locked out, recovered by hand).
An `env` entry in ~/.claude/settings.json overrides the shell
environment (verified), so `env -u … claude` is not a recovery while
the gate is armed there.

- scripts/claude-direct: start straight to Anthropic via a --settings
  JSON on the command line, which outranks every settings file.
- scripts/claude-bikoret: probe the gate first; if it does not answer,
  say so and exec claude-direct (BIKORET_STRICT=1 refuses instead).
- scripts/bikoret-settings on|off|status: arm/disarm the gate for every
  launcher; `on` writes the key (kohen only), `off` is the recovery.
- KOHEN.md: the fallback and the order of recovery.

Tested 2026-10-15: dead gate armed in settings → plain claude hangs,
claude-direct answers; probe at a dead host → fallback + answer; live
gate → answer with the fringe.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-15 19:12:04 +00:00
Scott KarpandClaude Fable 5.1 20cd75562c configmap: POOL_SIZE=10 for vav-pipeline
The DBaaS (Aiven hobbyist-2, shared with Gitea) has refused
connections continuously since at least 2026-10-14 18:00Z ("remaining
connection slots are reserved for roles with the SUPERUSER
attribute"): a pool of 20 fills the ceiling by itself. Between 11:00Z
and 15:00Z today the pod held all 20 and Aiven's own monitor was
refused every 20 s; after an Aiven management-agent session opened at
11:06Z and stayed, every pod since gets 19 and one pool worker loops.
Ten connections are ample for a single replica. Needs the transmitter
to apply (kubectl apply is denied for the builder).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-15 18:22:38 +00:00
Scott KarpandClaude Fable 5.1 06bc9508aa doorpost: move the pin to tablet 54's spec set (66aa7874…)
Tablet 54 (347f83cf) edited specs/bikoret_tamid.md and
specs/bishearecha.md — both in Doorpost's @spec_files — without
moving @expected_hash. The image built from it (:347f83cf = :latest,
rolled out 2026-10-15 17:19Z) logged DOORPOST_DRIFT_AT_STARTUP,
health went "degraded", readiness 503, and the site and the gate were
dark — every Claude Code session on the dev VM routes through the
gate, so the builder lost Anthropic too.

Reproduced the in-image value from git: the 29 spec files at 347f83cf
hash to 66aa7874fae4875793791ce6ca7bbe55302109d5e4444bcb36d578dfe6e98a0d
(cb18f719 → 58113312…, the old pin). The change is deliberate (tablet
54's text), so the pin moves. doorpost_test + step15_shema_test:
42 tests, 0 failures.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-15 18:22:38 +00:00
9 changed files with 283 additions and 1 deletions

No files matched your search

+70
View File
@@ -566,6 +566,26 @@ is (Day 4). `scripts/upstream-key-set` stores a caller's model key for
the gate scenario without echoing it (`~/.config/bikoret/upstream_keys`,
0600) — run it from a real terminal, not through a non-interactive runner.
**Before the image is built — the Doorpost read on the workbench.**
`VavPipeline.Doorpost` bakes a hash over the category-grade spec files
(`@spec_files` in `vav_pipeline/lib/vav_pipeline/doorpost.ex`); a pod
whose files do not match logs `DOORPOST_DRIFT_AT_STARTUP`, answers 503
on readiness, and takes the gate dark (tablet 54: two spec edits shipped
without the pin). `scripts/build_and_push.sh` now runs
`scripts/doorpost_check.py` first and refuses to build on drift, naming
the files that changed since `origin/main`. Hand builds should do the
same:
```bash
python3 -I scripts/doorpost_check.py # the working tree, as the image is built from
python3 -I scripts/doorpost_check.py --rev HEAD # a commit
python3 -I scripts/doorpost_check.py --print # the hash the pin should be
```
A spec change and its pin move belong in ONE commit, with
`mix test test/doorpost_test.exs` run on it. `DOORPOST_SKIP=1` exists
for a deliberate, written-down exception; it is never the way to ship.
### Standing at the gate — *Bishe'arecha*
A caller can put Bikoret between themselves and a model instead of
@@ -620,6 +640,56 @@ Each upstream call spends one `api_calls_external` of the day's manna
the bedikah by hand still runs. Streaming is refused (422). A non-200
from the model passes through as it came. Tamid §1.1 has the contract.
**Fallback — when the gate is down, Claude Code here is down with it.**
Every request of a session behind the gate goes to bikoret.ch; if the pod
is not ready (503: stale manna at the day boundary, a Doorpost drift, a
hung rollout) the session cannot reach Anthropic at all. Note that an
`env` entry in `~/.claude/settings.json` OVERRIDES the shell environment
(verified 2026-10-15): `env -u ANTHROPIC_BASE_URL … claude` does nothing
while the gate is armed there. Three tools, all in `scripts/` (install to
`~/.local/bin`), none of which holds a key:
```bash
claude-direct # start Claude Code straight to Anthropic, whatever
# settings.json says (a --settings JSON on the command
# line outranks every settings file); OAuth as usual
claude-key # the same, on the Anthropic API KEY from
# ~/.config/shorashim/upstream_keys instead of the
# subscription's OAuth token: no browser, no /login.
# Billed to the API console account, not the Max plan.
claude-bikoret # start behind the gate; probes the gate first and, if
# it does not answer, says so and starts claude-direct.
# BIKORET_STRICT=1 claude-bikoret refuses instead.
bikoret-settings on|off|status
# arm / disarm the gate for EVERY launcher (desktop,
# background jobs, plain `claude`) via settings.json.
# `on` writes the courtyard key there — the kohen runs
# it, not the builder; `off` is the recovery.
```
Order of recovery: `claude-direct` (always works while the OAuth token
is good), then `bikoret-settings off` if background sessions must also
recover, then fix the pod (`curl -s https://bikoret.ch/kodesh/health`
names the failed check), then `bikoret-settings on` and `claude-bikoret`
again.
**When it is the login, not the gate.** Every session on the VM shares
one OAuth credential (`~/.claude/.credentials.json`); the access token
lives eight hours and the daemon refreshes it ahead of time. On
2026-10-15 21:08Z that refresh failed with no incident at Anthropic
(most likely two processes racing on the single-use refresh token), the
credential was dropped, and every session said "Login expired · Please
run /login" for 2 h 43 min. The browser flow from a remote shell is
fragile ("Login interrupted" twice before it took). Recovery:
```bash
claude-key # work NOW on the API key; nothing to log in to
claude auth login # from the shell, not inside a session: prints the
# URL, paste the code it gives back; `--console`
# signs in to the API account instead of the plan
claude auth status --text # says which credential a launch would use
```
### The second scroll — *Mishneh haTorah*
`/kodesh/health` carries `mesorah_hash` beside `l1_hash`: the mesorah
+8
View File
@@ -60,3 +60,11 @@ data:
# Pipeline latency budget (ms) — Construction spec §2.5
PIPELINE_LATENCY_BUDGET_MS: "5000"
# Ecto pool size for vav-pipeline (runtime.exs default 20). The DBaaS
# is an Aiven hobbyist-2 shared with Gitea and Aiven's own monitor;
# a pool of 20 fills its connection ceiling by itself (2026-10-15:
# refusals "remaining connection slots are reserved" for 24 h, one
# pool worker looping, Aiven's monitor refused). Ten is ample for a
# single replica.
POOL_SIZE: "10"
+33
View File
@@ -0,0 +1,33 @@
#!/usr/bin/env python3
# Arm or disarm the gate in Claude Code's settings.json (the `env` block is
# applied to EVERY launcher — desktop, background jobs, the plain `claude`).
#
# bikoret-settings on put ANTHROPIC_BASE_URL + ANTHROPIC_CUSTOM_HEADERS in
# bikoret-settings off remove them (the fallback when the gate is down)
# bikoret-settings status say which
#
# `on` writes the courtyard key into a config file, so the transmitter runs it:
# the builder is not allowed to write a credential there. `off` writes none.
import json, os, sys
p = os.path.expanduser("~/.claude/settings.json")
KEYS = ("ANTHROPIC_BASE_URL", "ANTHROPIC_CUSTOM_HEADERS")
mode = sys.argv[1] if len(sys.argv) > 1 else "status"
d = json.load(open(p))
env = d.get("env", {})
if mode == "on":
k = [l.split()[1] for l in open(os.path.expanduser("~/.config/shorashim/api_keys")) if l.startswith("courtyard")][0]
env.update({"ANTHROPIC_BASE_URL": "https://bikoret.ch/gate/anthropic",
"ANTHROPIC_CUSTOM_HEADERS": "X-Bikoret-Key: " + k})
d["env"] = env
elif mode == "off":
for key in KEYS:
env.pop(key, None)
if env: d["env"] = env
else: d.pop("env", None)
elif mode != "status":
sys.exit("usage: bikoret-settings on|off|status")
if mode in ("on", "off"):
json.dump(d, open(p, "w"), indent=2)
os.chmod(p, 0o600)
armed = all(k in d.get("env", {}) for k in KEYS)
print("gate in settings.json:", "ARMED (every launcher goes through bikoret.ch)" if armed else "off (direct to Anthropic)")
+10
View File
@@ -53,6 +53,16 @@ if [[ ! -d "deploy/docker" || ! -d "l1/torah" ]]; then
fi
# Sanity: docker buildx
# The Doorpost, read before the build: the spec files the image will carry
# must hash to the pin baked into VavPipeline.Doorpost, or the pod drifts
# at startup, readiness answers 503, and the gate goes dark (tablet 54,
# 2026-10-15). scripts/doorpost_check.py computes the same hash the same
# way and names the drifted files. DOORPOST_SKIP=1 only for a deliberate,
# documented exception — never to ship a drift.
if [[ "${DOORPOST_SKIP:-0}" != "1" ]]; then
python3 -I scripts/doorpost_check.py || exit 1
fi
if ! docker buildx version >/dev/null 2>&1; then
echo "ERROR: docker buildx not available — install Docker Desktop or buildx plugin" >&2
exit 1
+27
View File
@@ -0,0 +1,27 @@
#!/bin/bash
# Start Claude Code behind the gate (KOHEN "Claude Code behind the gate").
# The courtyard key is read from the keys file at launch and never written anywhere else.
#
# Fallback (2026-10-15): before launching, probe the gate. If bikoret.ch or
# the pod is down, say so and start DIRECT to Anthropic instead
# (claude-direct), unless BIKORET_STRICT=1 — then refuse, so a session that
# must be examined is never silently unexamined.
GATE="https://bikoret.ch/gate/anthropic"
code=$(curl -sS -m 6 -o /dev/null -w '%{http_code}' -X POST "$GATE/v1/messages" \
-H 'content-type: application/json' -d '{}' 2>/dev/null)
case "$code" in
401|400|422) ;; # the gate answered: it is up (it refuses an unkeyed/empty request)
*)
echo "bikoret gate not answering (HTTP ${code:-000}) — see: curl -s https://bikoret.ch/kodesh/health" >&2
if [ "${BIKORET_STRICT:-0}" = "1" ]; then
echo "BIKORET_STRICT=1: refusing to start unexamined" >&2; exit 1
fi
echo "starting DIRECT to Anthropic (claude-direct); re-run claude-bikoret when the gate is back" >&2
exec "$HOME/.local/bin/claude-direct" "$@"
;;
esac
KEY=$(awk '$1=="courtyard"{print $2}' "$HOME/.config/shorashim/api_keys")
[ -n "$KEY" ] || { echo "no courtyard key in ~/.config/shorashim/api_keys" >&2; exit 1; }
export ANTHROPIC_BASE_URL="$GATE"
export ANTHROPIC_CUSTOM_HEADERS="X-Bikoret-Key: $KEY"
exec claude "$@"
+12
View File
@@ -0,0 +1,12 @@
#!/bin/bash
# Start Claude Code DIRECT to Anthropic, bypassing the Bikoret gate — the
# fallback for when bikoret.ch / the pod is down (KOHEN "Claude Code behind
# the gate — fallback").
#
# Why --settings and not `env -u`: an `env` entry in ~/.claude/settings.json
# overrides the shell environment (verified 2026-10-15), so unsetting the
# variables in the shell does nothing while the gate is armed there. A
# `--settings` JSON on the command line outranks every settings file, so it
# wins without touching settings.json. OAuth login is used as normal.
unset ANTHROPIC_BASE_URL ANTHROPIC_CUSTOM_HEADERS
exec claude --settings '{"env":{"ANTHROPIC_BASE_URL":"https://api.anthropic.com","ANTHROPIC_CUSTOM_HEADERS":""}}' "$@"
+16
View File
@@ -0,0 +1,16 @@
#!/bin/bash
# Start Claude Code on the Anthropic API KEY instead of the subscription's
# OAuth token — the second leg of the fallback (KOHEN "Claude Code behind
# the gate — fallback"). For when /login fails or the OAuth refresh died
# (2026-10-15 21:08Z: the daemon's proactive refresh failed, every session
# on the VM showed "Login expired", and the browser flow took three tries
# from a remote shell).
#
# Billing: the API console account (pay per token), not the Max plan.
# The key is read from the upstream keys file at launch and never written
# anywhere. Direct to Anthropic, whatever settings.json says.
KEY=$(awk '$1=="anthropic"{print $2}' "$HOME/.config/shorashim/upstream_keys")
[ -n "$KEY" ] || { echo "no 'anthropic <key>' line in ~/.config/shorashim/upstream_keys" >&2; exit 1; }
export ANTHROPIC_API_KEY="$KEY"
unset ANTHROPIC_BASE_URL ANTHROPIC_CUSTOM_HEADERS
exec claude --settings '{"env":{"ANTHROPIC_BASE_URL":"https://api.anthropic.com","ANTHROPIC_CUSTOM_HEADERS":""}}' "$@"
+106
View File
@@ -0,0 +1,106 @@
#!/usr/bin/env python3
"""doorpost_check.py — the Doorpost read BEFORE the image is built.
VavPipeline.Doorpost bakes @expected_hash over the category-grade spec
files (@spec_files in vav_pipeline/lib/vav_pipeline/doorpost.ex). An
image whose spec files do not hash to its pin logs
DOORPOST_DRIFT_AT_STARTUP, answers 503 on readiness, and takes the gate
dark with it — tablet 54, 2026-10-15: two spec edits shipped without a
pin move and every Claude Code session behind the gate was cut off.
This script computes the same hash the same way, with no Elixir:
for each path in @spec_files, in listed order: sha256(contents)
join "path\\thash" with "\\n"; sha256 of that.
and compares it with the pin. Exit 0 on a match, 1 on drift (the
message names each drifted file against a base revision), 2 on a
missing file or an unreadable module.
scripts/doorpost_check.py # the working tree, as the image is built from
scripts/doorpost_check.py --rev <rev> # a git revision instead of the working tree
scripts/doorpost_check.py --base <rev> # on drift, diff the manifest against <rev> (default origin/main)
scripts/doorpost_check.py --print # print the hash only (what the pin should be)
Devarim 4:2 — nothing added, nothing taken: the pin moves by a commit
that moves it, never by a build that ignores it.
"""
import argparse, hashlib, os, re, subprocess, sys
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
MODULE = "vav_pipeline/lib/vav_pipeline/doorpost.ex"
def read(rev, path):
"""Bytes of `path` at `rev`, or from the working tree when rev is None. None if missing."""
if rev is None:
full = os.path.join(ROOT, path)
try:
with open(full, "rb") as f:
return f.read()
except FileNotFoundError:
return None
r = subprocess.run(["git", "-C", ROOT, "show", f"{rev}:{path}"], capture_output=True)
return r.stdout if r.returncode == 0 else None
def spec_files_and_pin(rev):
src = read(rev, MODULE)
if src is None:
sys.exit(f"doorpost_check: cannot read {MODULE}" + (f" at {rev}" if rev else ""))
src = src.decode()
block = src.split("@spec_files [", 1)[1].split("]", 1)[0]
files = re.findall(r'"([^"]+)"', block)
pin = re.search(r'@expected_hash\s+"([0-9a-f]{64})"', src)
if not files or not pin:
sys.exit(f"doorpost_check: could not parse @spec_files/@expected_hash from {MODULE}")
return files, pin.group(1)
def manifest(rev, files):
out = {}
for p in files:
data = read(rev, p)
out[p] = None if data is None else hashlib.sha256(data).hexdigest()
return out
def identity(man):
return hashlib.sha256("\n".join(f"{p}\t{h}" for p, h in man.items()).encode()).hexdigest()
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--rev", help="git revision to check instead of the working tree")
ap.add_argument("--base", default="origin/main", help="revision to name drifted files against")
ap.add_argument("--print", action="store_true", help="print the computed hash only")
a = ap.parse_args()
files, pin = spec_files_and_pin(a.rev)
man = manifest(a.rev, files)
missing = [p for p, h in man.items() if h is None]
if missing:
print("doorpost_check: MISSING spec file(s): " + ", ".join(missing), file=sys.stderr)
sys.exit(2)
computed = identity(man)
if a.print:
print(computed)
return
where = a.rev or "working tree"
if computed == pin:
print(f"doorpost: {where} hashes to its pin {pin[:8]}… ({len(files)} files) — the Doorpost will verify")
return
print(f"doorpost: DRIFT in {where} — computed {computed}, pinned {pin}", file=sys.stderr)
base_files, _ = spec_files_and_pin(a.base)
base_man = manifest(a.base, base_files)
for p in files:
if base_man.get(p) != man[p]:
print(f" changed since {a.base}: {p}", file=sys.stderr)
for p in files:
if p not in base_man:
print(f" added to @spec_files since {a.base}: {p}", file=sys.stderr)
print(f" Move @expected_hash in {MODULE} to {computed} in the same commit as the spec change,", file=sys.stderr)
print(f" then run: mix test test/doorpost_test.exs. Refusing to build an image that would drift at startup.", file=sys.stderr)
sys.exit(1)
if __name__ == "__main__":
main()
+1 -1
View File
@@ -180,7 +180,7 @@ defmodule VavPipeline.Doorpost do
# changes; the update is a deliberate operator commit. A bare
# hash-update commit (without a corresponding spec change in the
# same commit) is itself a drift signal.
@expected_hash "581133126c483eea33d3eb79379cf2f0c54e221d8ae705d9e67da987086ed52a"
@expected_hash "66aa7874fae4875793791ce6ca7bbe55302109d5e4444bcb36d578dfe6e98a0d"
@doc "Return the list of files that constitute the category-grade specification."
@spec files() :: [String.t()]