manna: the provision falls at the boundary — 00:00Z — and readiness is carried across it
The watchdog's first night (2026-10-21): 503 at 00:00:01Z with manna_present and esh_tamid failing, recovered 00:05Z. Every night since the CronJob was applied the site has been dark one to five minutes: the provision is stale from 00:00:00, the job ran at 00:01, and the readiness probe (10 s period, default threshold 3) fell in the gap. Not changed: the rule. A provision is for its day; Friday's double is the Shabbat's; nothing else crosses the boundary (Shemot 16:19–26, manna_shabbat_test.exs). Changed: the timing. - manna-cronjob.yaml: schedule "0 0 * * 0-5" — the manna falls at the boundary, not a minute after it. - deployment.yaml: readinessProbe.failureThreshold 6 — 60 s of failures before the Service stops routing. The job writes within seconds of 00:00; the pod is carried across. A real outage still shows within a minute; the watchdog's own threshold is ten. - KOHEN.md: the night's finding and the new timing. Both manifests validate (kubectl apply --dry-run=client). Proof: the watchdog log on the next boundary should show no FAIL at 00:00. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
1 parent
489cf11416
commit
50b08cba4b
3 files changed
+22
-6
No files matched your search
@@ -753,14 +753,19 @@ run on Saturday.
|
|||||||
### The manna, each morning — *Dvar yom b'yomo*
|
### The manna, each morning — *Dvar yom b'yomo*
|
||||||
|
|
||||||
Since 2026-10-15 the day's provision is written by the system's own hand:
|
Since 2026-10-15 the day's provision is written by the system's own hand:
|
||||||
CronJob `manna` (`deploy/k8s/manna-cronjob.yaml`) runs at 00:01Z Sunday
|
CronJob `manna` (`deploy/k8s/manna-cronjob.yaml`) runs at 00:00Z Sunday
|
||||||
to Friday, mounts the manna PVC and writes `today.json` with the amounts
|
to Friday, mounts the manna PVC and writes `today.json` with the amounts
|
||||||
from `shorashim-config` (`MANNA_COMPUTE_UNITS` …). Before it, every night
|
from `shorashim-config` (`MANNA_COMPUTE_UNITS` …). Before it, every night
|
||||||
at 00:00Z the old provision went stale and the readiness probe failed
|
at 00:00Z the old provision went stale and the readiness probe failed
|
||||||
until someone woke (13.5 hours on 2026-10-15). The kohen's acts now:
|
until someone woke (13.5 hours on 2026-10-15). With the job at 00:01Z the
|
||||||
|
site was still 503 for one to five minutes every night (the watchdog saw
|
||||||
|
it, 2026-10-21): the provision is stale from 00:00:00 and readiness fell
|
||||||
|
before the job wrote. Now the job runs at 00:00Z and readiness waits 60 s
|
||||||
|
(`failureThreshold: 6`) before the Service stops routing — the manna
|
||||||
|
falls at the boundary, and the pod is carried across it. The kohen's acts now:
|
||||||
|
|
||||||
- set the amounts in the ConfigMap (`kubectl apply -f deploy/k8s/configmap.yaml`);
|
- set the amounts in the ConfigMap (`kubectl apply -f deploy/k8s/configmap.yaml`);
|
||||||
- read `kubectl -n shorashim get jobs` after 00:01Z — a failed job is the
|
- read `kubectl -n shorashim get jobs` after 00:00Z — a failed job is the
|
||||||
Day-4 alarm; write by hand with `kubectl cp` (Day 4) if it failed;
|
Day-4 alarm; write by hand with `kubectl cp` (Day 4) if it failed;
|
||||||
- run it at once when needed: `kubectl -n shorashim create job --from=cronjob/manna manna-now`.
|
- run it at once when needed: `kubectl -n shorashim create job --from=cronjob/manna manna-now`.
|
||||||
|
|
||||||
|
|||||||
@@ -266,6 +266,13 @@ spec:
|
|||||||
port: 4000
|
port: 4000
|
||||||
initialDelaySeconds: 10
|
initialDelaySeconds: 10
|
||||||
periodSeconds: 10
|
periodSeconds: 10
|
||||||
|
# 60 s of failures before the Service stops routing here (the
|
||||||
|
# default is 30 s). The manna goes stale at 00:00:00 and the
|
||||||
|
# CronJob writes the new day's within seconds; the cushion
|
||||||
|
# carries the pod across that boundary without a 503. A real
|
||||||
|
# outage still shows within a minute, and the watchdog's own
|
||||||
|
# threshold is ten.
|
||||||
|
failureThreshold: 6
|
||||||
|
|
||||||
# === SHORESH ENGINE ===
|
# === SHORESH ENGINE ===
|
||||||
# Not a container. It runs as an Erlang Port inside
|
# Not a container. It runs as an Erlang Port inside
|
||||||
|
|||||||
@@ -3,7 +3,7 @@
|
|||||||
# Until 2026-10-15 the day's provision was written by the kohen with
|
# Until 2026-10-15 the day's provision was written by the kohen with
|
||||||
# `kubectl cp`, and every night at 00:00Z the old one went stale and the
|
# `kubectl cp`, and every night at 00:00Z the old one went stale and the
|
||||||
# readiness probe failed until someone woke (13.5 hours on 2026-10-15).
|
# readiness probe failed until someone woke (13.5 hours on 2026-10-15).
|
||||||
# This CronJob writes today.json onto the manna PVC at 00:01Z, Sunday to
|
# This CronJob writes today.json onto the manna PVC at 00:00Z, Sunday to
|
||||||
# Friday — a double portion on Friday that is valid through Shabbat, and
|
# Friday — a double portion on Friday that is valid through Shabbat, and
|
||||||
# nothing on Shabbat (Shemot 16:22–26: "on the sixth day they gathered
|
# nothing on Shabbat (Shemot 16:22–26: "on the sixth day they gathered
|
||||||
# double … on the seventh day there shall be none"). The kohen still sets
|
# double … on the seventh day there shall be none"). The kohen still sets
|
||||||
@@ -23,10 +23,14 @@ metadata:
|
|||||||
app.kubernetes.io/name: shorashim
|
app.kubernetes.io/name: shorashim
|
||||||
app.kubernetes.io/component: manna
|
app.kubernetes.io/component: manna
|
||||||
spec:
|
spec:
|
||||||
schedule: "1 0 * * 0-5" # 00:01Z, Sunday–Friday; none on Shabbat
|
schedule: "0 0 * * 0-5" # 00:00Z, Sunday–Friday; none on Shabbat. At the boundary, not after it:
|
||||||
|
# the provision goes stale at 00:00:00 and readiness has a 60 s cushion
|
||||||
|
# (deployment.yaml failureThreshold 6) — a job that writes within it never
|
||||||
|
# flips the pod. At 00:01 the site was 503 for 1–5 min every night
|
||||||
|
# (watchdog, 2026-10-21).
|
||||||
timeZone: "Etc/UTC"
|
timeZone: "Etc/UTC"
|
||||||
concurrencyPolicy: Forbid
|
concurrencyPolicy: Forbid
|
||||||
startingDeadlineSeconds: 3600 # a missed 00:01 still runs within the hour
|
startingDeadlineSeconds: 3600 # a missed 00:00 still runs within the hour
|
||||||
successfulJobsHistoryLimit: 3
|
successfulJobsHistoryLimit: 3
|
||||||
failedJobsHistoryLimit: 3
|
failedJobsHistoryLimit: 3
|
||||||
jobTemplate:
|
jobTemplate:
|
||||||
|
|||||||
Reference in new issue
Block a user