deployment: manna on a persistent volume — the jar survives the restart

Manna was an emptyDir: every pod creation emptied it and the vessel sat
unready until the kohen wrote the day's portion again. On 2026-10-04
that happened seven times in one day of rollouts. deploy/ARCHITECTURE.md
has always specified a block-storage volume for it and manna-volume.yaml
has always defined the claim; it was "not applied at Day 3" and stayed
that way.

- manna-volume.yaml: storageClassName exoscale-sbs (the cluster's only
  class is not marked default — without it the claim stays Pending),
  1Gi. Applied; bound in 6 s.
- deployment.yaml: the manna volume is the manna-pvc claim. `kubectl
  diff` against the live deployment showed this as the only change.
- KOHEN.md: apply order (the claim before the pod), and the daily-ops
  note — write once a day, not once per restart.

Verified on the live pod: manna written once onto the volume; then a
rollout restart with NO provision written by anyone — the new pod was
Ready and serving 49 s later, health 7/7, /peah/ask 200.

What this does not change: the portion still expires by its own date
and expires_at (Shemot 16:19-20). And the ledger is in memory — a
restarted pod counts consumption from zero against the same day's
allocation, as it always did; that is now reachable without the kohen's
hand, and is recorded as open.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

KOHEN.md

@@ -223,6 +223,7 @@ If it is absent, create it with the command in `deploy/k8s/secrets.yaml` (it rea
 ```bash
 kubectl apply -f deploy/k8s/namespace.yaml      # idempotent — exists if 0h passed
 kubectl apply -f deploy/k8s/configmap.yaml      # idempotent — content-addressed
+kubectl apply -f deploy/k8s/manna-volume.yaml   # the claim must exist before the pod mounts it
 kubectl apply -f deploy/k8s/deployment.yaml
 kubectl apply -f deploy/k8s/service.yaml
 ```
@@ -231,7 +232,7 @@ kubectl apply -f deploy/k8s/service.yaml
 
 **Do not** `kubectl apply -f deploy/k8s/secrets.yaml`. It is a template with placeholder values and would overwrite both live secrets.
 
-Note: `manna-volume.yaml` defines a PVC but `deployment.yaml` mounts manna as `emptyDir` (lost on pod restart — kohen re-writes today's provision per the daily-ops section). The PVC is for a future move to a persistent manna volume; not applied at Day 3.
+Note: manna lives on a persistent volume (`manna-volume.yaml`, claim `manna-pvc`, 1 GiB on `exoscale-sbs`; applied 2026-10-04). Apply `manna-volume.yaml` before `deployment.yaml` on a fresh cluster. Today's provision survives a pod restart; it still expires by its own `date` and `expires_at`, so the kohen writes it once a day, not once per restart.
 
 #### 3c. Watch the pod come up
 
@@ -507,7 +508,7 @@ kubectl logs -n shorashim deploy/shorashim -c vav-pipeline | grep HALT
 
 ### If a pod restarts
 
-The system has no in-memory state required for continuity (cycle boundaries reset, signal history lost). Re-write today's manna provision after any pod restart.
+The system has no in-memory state required for continuity (cycle boundaries reset, signal history lost). Today's manna provision is on the `manna-pvc` volume and survives a pod restart — do not re-write it. (The in-memory ledger does reset: a restarted pod counts consumption from zero against the same day's allocation.)
 
 ---
 

deploy/k8s/deployment.yaml

@@ -23,7 +23,8 @@ spec:
   # (maxSurge 25% → 1 extra pod) asks for 450m CPU the node does not
   # have beside the running pod, and the rollout hangs Pending while the
   # old image keeps serving (tablet 31, 2026-10-04). Replace in place.
-  # The new pod is not Ready until the kohen writes manna (emptyDir).
+  # The new pod reads the day's manna from the manna-pvc volume; it is
+  # Ready again without the kohen, as long as today's portion was written.
   strategy:
     type: RollingUpdate
     rollingUpdate:
@@ -60,8 +61,11 @@ spec:
       # whose records land on a disk no one reads has not disposed of
       # anything; it has only stopped looking.
       volumes:
+        # The day's portion, kept through a pod restart (manna-volume.yaml).
+        # ReadWriteOnce is enough: one replica, replaced in place.
         - name: manna
-          emptyDir: {}
+          persistentVolumeClaim:
+            claimName: manna-pvc
         # Shared /tmp for every container that speaks over it:
         # the Talmudic Unix socket (/tmp/shorashim/talmudic.sock) and
         # the azazel file bridge (/tmp/shorashim/azazel/pending_l3).

deploy/k8s/manna-volume.yaml

@@ -2,6 +2,16 @@
 # The kohen/treasurer writes today.json to this volume.
 # The vav_pipeline reads it. Provision is external to the system.
 # Shemot 16:4 — d'var yom b'yomo.
+#
+# Applied 2026-10-04. Until then the deployment mounted manna as an
+# emptyDir, so every pod creation emptied the jar and the vessel stayed
+# unready until the kohen wrote the day's portion again — seven times
+# in one day of rollouts. The portion still expires by its own date and
+# `expires_at` (Shemot 16:19-20); the volume keeps it through a restart,
+# it does not keep it past its day.
+#
+# storageClassName is explicit: the cluster's only class (exoscale-sbs)
+# is not marked default, and a claim without one stays Pending forever.
 ---
 apiVersion: v1
 kind: PersistentVolumeClaim
@@ -9,8 +19,9 @@ metadata:
   name: manna-pvc
   namespace: shorashim
 spec:
+  storageClassName: exoscale-sbs
   accessModes:
     - ReadWriteOnce
   resources:
     requests:
-      storage: 100Mi
+      storage: 1Gi