TORAH ANCHOR — Shemot 16:4, and tamid §2.1 check 3:
Manna provision read … If absent: WAIT. Alert senior kohanim.
WAIT. Not die. Provision is not the health of the vessel. Killing the
process does not produce manna; it re-runs init, re-spawns the Port,
and waits again — three minutes later.
WHAT THE KOHEN FOUND (BUILDER_HANDOFF_probes.md, 2026-09-07 20:35Z):
A fresh pod after recreate. emptyDir manna empty — by design
(KOHEN.md:406). manna_present honestly read "fail" — the Day-6 fix,
correct per tamid §1.1. overall → "degraded" → /kodesh/health 503.
BOTH probes pointed at /kodesh/health. Liveness: period 60s, default
failureThreshold 3 → after three minutes the kubelet SIGTERMed a
process whose log showed the Port alive, zero errors, l3_writable
pass, esh_tamid pass. It was alive and serving. It was killed for
lacking provision, and would be again every three minutes until the
kohen wrote today.json — i.e. on EVERY pod creation.
Introduced by a correct fix (f1ab7c8 made health honest); the missing
piece was the distinction between "not ready to serve" and "dead".
WHAT CHANGED:
bigdei_kodesh/router.ex
GET /kodesh/live — 200 iff the BEAM answers HTTP and the HaltMonitor
process is registered. Looks at NOTHING that is a wait state: no
manna, no L3, no cloud, no probe cycle. And deliberately NOT "not
halted": a HALTED system is a chosen state (a cardinal sin was
detected); restarting it would clear the halt and lose the event.
Liveness measures that the monitor is ALIVE, not that it is quiet.
/kodesh/health is unchanged — it is readiness, and a vessel without
provision correctly does not receive offerings (that is shema; the
Service stops routing to it).
deploy/k8s/deployment.yaml
livenessProbe → /kodesh/live, failureThreshold 3 made explicit.
readinessProbe stays on /kodesh/health.
test/liveness_test.exs — 3 tests
live 200 with NO manna while health is 503; live 200 while HALTED
(and pe'ah answers 503 HALT_*, not a served result); live touches
neither the ledger nor L3 nor the cycle.
KOHEN.md §Day 3c — a fresh pod reads 2/3 Ready, 0 restarts until Day 4
writes manna, and that is correct; restarts on a pod whose log shows
the Port alive and zero errors mean liveness is on the wrong endpoint.
ALSO — a leak of my own from f1ab7c8:
The MannaLedger is a global GenServer keyed by date. The Day-6
"provision exhausted" test left it at allocated 1 / consumed 1 for
today; when step31 Test 4 later wrote a fresh today.json the ledger
answered :already_loaded, consume failed, and the cycle waited —
a 1-in-5-seeds flake. Fixed where it leaked (day6 + liveness tests
reset the ledger on exit) and where it landed (step31 Test 4 gathers
from a clean ledger, with the reason written down).
VERIFIED — the kohen's procedure, locally, DB deliberately dead:
fresh, no manna live 200 health 503 critical (manna + l3)
manna written live 200 health 503 degraded — l3_writable ONLY;
manna_present flipped to pass
manna removed live 200 health 503
exit/SIGTERM events in the container log: 0
mix test — 440 tests, 0 failures, 36 excluded, across seeds 0 1 2 7 42
kubectl apply --dry-run=client: deployment valid; liveness
/kodesh/live (threshold 3), readiness /kodesh/health
Doorpost.verify_identity() :ok — no @spec_files entry touched
l1/integrity.sha256 unchanged: 99bbd07a…4ea377
RAISED, NOT FIXED: manna is an emptyDir, so every pod creation needs
the kohen to write it again. manna-volume.yaml's PVC exists for
exactly this and is "not applied at Day 3" by the runbook's own note.
That is an operations decision, not a probe one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
