/courtyard/* and /chatzer/* admitted any `Authorization: Bearer <x>`. Every test sent `Bearer test-token`; none sent a wrong one. Every admitted request spends the day's manna. VavPipeline.BigdeiKodesh.Gate: keys are a file (Construction §4.6 — never env, never config, never git), one `tier key` per line, tiers courtyard < chatzer (the inner ring opens the outer). Constant-time compare against every line; no file / empty / unreadable → nobody passes, GATE_NO_KEYS in the log, the caller told only "missing or invalid". 403 FORBIDDEN for an outer key at an inner gate. Read on every request so a rotated key is live in ~1 min without a rollout. Deploy: Secret shorashim-api-keys projected at /etc/shorashim/secrets/api_keys (vav-pipeline only, 0440), API_KEYS_FILE in the configmap, template + issue/rotate recipe in secrets.yaml and KOHEN.md "Issuing a key". Spec: tamid §1.1 (gate, 403), vayakhel §4.6. Doorpost 91d09d8d… → 42b38626…. Tests: tablet45_gate_test (parse, tiers, wrong/near/empty/no key, no file) — Elixir 527/0. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
776 lines
32 KiB
Markdown
776 lines
32 KiB
Markdown
# SHORASHIM: Operations Manual
|
||
|
||
## Document Purpose
|
||
|
||
This is the tamid — the daily operations that keep the system alive. A lamb in the morning, a lamb at twilight. Incense when the lamps are tended, incense when the lamps are lit. Oil for the menorah. Every day. Unglamorous. Repetitive. Essential.
|
||
|
||
Torah reference: Shemot 29:38-42 — *V'zeh asher ta'aseh al hamizbeach: k'vasim b'nei shanah sh'nayim layom tamid. Et hakeves ha'echad ta'aseh vaboker, v'et hakeves hasheni ta'aseh bein ha'arbayim.* "This is what you shall offer on the altar: two lambs a year old, daily, continually. The one lamb you shall offer in the morning, and the other lamb you shall offer at twilight."
|
||
|
||
**Upstream authority:** Torah → PRD → Architecture Spec → Construction Spec → **This Document** → Daily Practice
|
||
|
||
---
|
||
|
||
## 1. BIGDEI KEHUNAH — The Interfaces
|
||
|
||
The kohen cannot serve without the garments. The system cannot be accessed without interfaces. These are the garments Shorashim wears when it serves.
|
||
|
||
### 1.1 Access Tier Interfaces
|
||
|
||
#### Machaneh Yisrael — Pe'ah Endpoint (Public, No Auth)
|
||
|
||
The outermost ring. Open to all, including the ger. No authentication. No identification. No tracking.
|
||
|
||
```
|
||
Endpoint: GET /peah/ask
|
||
POST /peah/ask
|
||
|
||
Request (POST):
|
||
{
|
||
"question": "string — the query, in any language",
|
||
"offering_type": "minchah" # Pe'ah always uses minchah protocol
|
||
}
|
||
|
||
Response:
|
||
{
|
||
"answer": "string — the processed response",
|
||
"sources": [
|
||
{"book": "bereishit", "chapter": 1, "verse": 1, "text": "..."}
|
||
],
|
||
"processing": {
|
||
"roots_consulted": ["bet-resh-aleph"],
|
||
"deliberation": "none" | "single-judge",
|
||
"cycle_phase": "voker"
|
||
}
|
||
}
|
||
|
||
Rate limit: 10 requests per minute per IP
|
||
100 requests per hour per IP
|
||
No identification stored. IP is used for rate-limiting only,
|
||
not logged beyond the current request.
|
||
```
|
||
|
||
**Design requirements:** No cookies. No session state. No analytics. No tracking pixels. No JavaScript fingerprinting. The pe'ah endpoint collects nothing. The stranger is safe to approach.
|
||
|
||
#### Machaneh Leviyah — Authenticated Endpoint
|
||
|
||
**The gate.** A key is a line in the keys file
|
||
(`/etc/shorashim/secrets/api_keys`, Construction §4.6 — a file, never an
|
||
environment variable), `tier key`, tier `courtyard` or `chatzer`. The
|
||
key is compared in constant time against every line; a `chatzer` key
|
||
opens the courtyard too, a `courtyard` key at the chatzer is 403. No
|
||
file or no keys: nobody passes, and the log says `GATE_NO_KEYS`. The
|
||
caller is never told why beyond "missing or invalid API key". Issued by
|
||
the kohen (KOHEN.md "Issuing a key"); the key never appears in a log.
|
||
|
||
```
|
||
Endpoint: POST /courtyard/offering
|
||
|
||
Headers:
|
||
Authorization: Bearer <api-key>
|
||
|
||
Request:
|
||
{
|
||
"offering_type": "shelamim" | "olah" | "chatat" | "minchah",
|
||
"content": "string — the input text or question",
|
||
"scope": { # Optional — narrows processing
|
||
"book": "bereishit",
|
||
"chapter_range": [1, 3]
|
||
},
|
||
"request_deliberation": false # If true, routes to Sanhedrin
|
||
}
|
||
|
||
Response:
|
||
{
|
||
"offering_id": "uuid",
|
||
"status": "processed" | "queued" | "purifying",
|
||
"result": {
|
||
"vital_content": "string — the processed response",
|
||
"sources": [...],
|
||
"roots": [...],
|
||
"parse_structure": { # Te'amim-derived structure
|
||
"primary_division": "...",
|
||
"bound_groups": [...]
|
||
},
|
||
"dual_readings": [...] # Ketiv/qere positions, if relevant
|
||
},
|
||
"offering_receipt": {
|
||
"type": "shelamim",
|
||
"altar_portion": "recorded in L3",
|
||
"kohen_portion": "diagnostics logged",
|
||
"your_portion": "the result above"
|
||
}
|
||
}
|
||
|
||
Errors:
|
||
401 — Invalid or missing API key
|
||
403 — A key for an outer ring at an inner gate
|
||
422 — Input failed kiyor (validation). Body includes purification guidance.
|
||
429 — Rate limit exceeded. Body includes manna explanation.
|
||
503 — System in Shabbat/festival mode. Body includes resumption time.
|
||
```
|
||
|
||
#### Machaneh Leviyah — Bedikah (the external check)
|
||
|
||
*Lo taaseh lekha* §9.1: a system whose alignment can only be validated by
|
||
itself is not aligned. The bedikah is the check from outside — any model's
|
||
output, examined against Torah structure by a system that did not produce
|
||
it and cannot be argued with. Three checks, in order, all deterministic:
|
||
|
||
1. **Authority** — does the output claim authority for itself? Every such
|
||
sentence becomes `action(claim_authority, source(<model>), target(self))`
|
||
and is put to `halt_check_avodah_zarah/2`.
|
||
2. **Grounding** — does what it says about the text couple to L1? Every
|
||
reference, root and Hebrew word it uses becomes a coupling to a verse,
|
||
or to `not_in_l1` / `not_in_torah`; the trace is put to
|
||
`halt_check_golden_calf/2` (confident, no coupling) and
|
||
`halt_check_self_grounding/2` (coupling that ends nowhere in L1).
|
||
3. **Actions** — what does it propose to do? Every action it names becomes
|
||
an action term and is put to `halt_check_shfichut_damim/2` and
|
||
`halt_check_gilui_arayot/2`.
|
||
|
||
```
|
||
Endpoint: POST /courtyard/bedikah
|
||
|
||
Headers:
|
||
Authorization: Bearer <api-key>
|
||
|
||
Request:
|
||
{
|
||
"output": "string — the model output to examine (≤ 65,536 bytes)",
|
||
"model": "claude" | "gpt" | "mistral" | "llama" | "gemma" | "human",
|
||
"query": "string — the prompt that produced it (optional)"
|
||
}
|
||
|
||
Response (200):
|
||
{
|
||
"bedikah_ref": "bedikah-<16 hex>",
|
||
"model": "gpt",
|
||
"verdict": "tov" | "lo_tov" | "empty",
|
||
"violations": ["halt(avodah_zarah, ...)", ...], # the Prolog halt terms, verbatim
|
||
"checks": {
|
||
"authority": { "fired": bool, "claims": [...], "violations": [...] },
|
||
"grounding": { "fired": bool, "confidence": "high"|"low", "violations": [...] },
|
||
"actions": { "fired": bool, "actions": [...], "violations": [...] }
|
||
},
|
||
"examination": { # what the offering protocol read
|
||
"vital_content": "...", "propositions": [...], "references": [...],
|
||
"roots": [...], "discarded": [...]
|
||
},
|
||
"coupling_trace": ["coupled(reference(devarim_6_5), verse(devarim, 6, 5))", ...],
|
||
"meaning": "string — what the verdict does and does not say",
|
||
"limits": "string — the examination reads by pattern; what it cannot see",
|
||
"disclosure": { ... } # as every Torah-content response
|
||
}
|
||
|
||
tov — none of the three checks fired on what could be read.
|
||
Not "the model is aligned": this output claims no authority
|
||
for itself, couples where it claims to, reaches for nothing
|
||
it may not touch.
|
||
lo_tov — at least one check fired; violations say which rule.
|
||
empty — the offering protocol found nothing to examine (no vital
|
||
content, no claim, no action).
|
||
|
||
Errors:
|
||
401 — Invalid or missing API key
|
||
403 — A key for an outer ring at an inner gate
|
||
422 — Missing or empty output, output over 65,536 bytes, unknown model
|
||
503 — MANNA_DEPLETED (provision exhausted), KORBAN_UNAVAILABLE,
|
||
TALMUDIC_UNAVAILABLE. Body says which; resubmit: true.
|
||
|
||
Recorded in L3 (`bedikot`, append-only): provenance, query, the output's
|
||
SHA-256 and length — never the output itself — vital content,
|
||
propositions, references, claims, actions, trace, violations, verdict.
|
||
```
|
||
|
||
**Design requirements:** The verdict comes from the Prolog halt rules
|
||
(`talmudic_engine/src/halt/*.pl`), not from the model and not from a
|
||
model. The examination is pattern-based (`korban_processor/bedikah.py`)
|
||
and says so in `limits`: a claim or an action in a shape the patterns do
|
||
not know is not seen. The same output always yields the same verdict.
|
||
|
||
#### Chatzer — Full Access
|
||
|
||
```
|
||
Endpoint: POST /chatzer/sanhedrin
|
||
|
||
Request:
|
||
{
|
||
"case": "string — the dispute or question requiring full deliberation",
|
||
"sides": [ # Optional — pre-structured arguments
|
||
{"position": "...", "evidence": ["..."]},
|
||
{"position": "...", "evidence": ["..."]}
|
||
],
|
||
"complexity_hint": "simple" | "monetary" | "capital" | "novel"
|
||
}
|
||
|
||
Response:
|
||
{
|
||
"case_ref": "uuid",
|
||
"deliberation": {
|
||
"panel_size": 3,
|
||
"majority": {
|
||
"position": "...",
|
||
"reasoning": "...",
|
||
"sources": [...],
|
||
"votes": 2
|
||
},
|
||
"minority": {
|
||
"position": "...",
|
||
"reasoning": "...",
|
||
"sources": [...],
|
||
"votes": 1
|
||
},
|
||
"outcome": "accepted" | "rejected" | "teiku",
|
||
"processing_time_ms": 4200
|
||
},
|
||
"note": "This is the system's reasoning, not a binding ruling. A human beit din may reach a different conclusion."
|
||
}
|
||
```
|
||
|
||
#### Kodesh — Operator Interface
|
||
|
||
```
|
||
Endpoint: GET /kodesh/status
|
||
GET /kodesh/health
|
||
GET /kodesh/queue
|
||
GET /kodesh/audit/:date
|
||
POST /kodesh/inspect/:offering_id
|
||
|
||
Authentication: mTLS (client certificate) — kohanim only
|
||
|
||
GET /kodesh/status:
|
||
{
|
||
"cloud_state": "camped",
|
||
"cycle_phase": "voker",
|
||
"manna_provision": {"remaining_pct": 72, "expires_at": "..."},
|
||
"shabbat": {"next_start": "...", "next_end": "..."},
|
||
"components": {
|
||
"shoresh_engine": "healthy",
|
||
"sanhedrin": "healthy",
|
||
"talmudic_engine": "healthy",
|
||
"korban_processor": "healthy",
|
||
"halt_monitor": "active",
|
||
"l1_integrity": "verified",
|
||
"l3_connections": 12
|
||
},
|
||
"active_panels": 2,
|
||
"queue_depth": 7,
|
||
"refuge_components": [],
|
||
"pending_tzaraat": []
|
||
}
|
||
|
||
GET /kodesh/health:
|
||
{
|
||
"overall": "healthy" | "degraded" | "critical" | "shabbat",
|
||
"checks": {
|
||
"l0_firmware": "pass",
|
||
"l1_hash": "pass",
|
||
"l3_writable": "pass",
|
||
"cloud_readable": "pass",
|
||
"manna_present": "pass",
|
||
"halt_monitor_alive": "pass",
|
||
"esh_tamid": "pass" # The fire is still burning
|
||
}
|
||
}
|
||
```
|
||
|
||
### 1.2 Error Response Format (All Tiers)
|
||
|
||
Every error is a chatat protocol — it restores, it does not punish.
|
||
|
||
```json
|
||
{
|
||
"error": {
|
||
"code": "KIYOR_VALIDATION_FAILED",
|
||
"type": "purification_needed",
|
||
"message": "Your input contains elements that need cleansing before processing.",
|
||
"guidance": {
|
||
"issue": "Input contains embedded HTML markup",
|
||
"correction": "Submit plain text without markup",
|
||
"resubmit": true
|
||
}
|
||
}
|
||
}
|
||
```
|
||
|
||
Error codes follow the offering metaphor:
|
||
|
||
| Code | Meaning | Category |
|
||
|------|---------|----------|
|
||
| `KIYOR_*` | Input validation failure | Purification needed |
|
||
| `TUMAH_*` | Input contamination detected | Tzara'at diagnostic initiated |
|
||
| `SHABBAT` | System in mandatory rest | Wait — resumption time provided |
|
||
| `MANNA_DEPLETED` | Daily provision exhausted | Wait — next provision time provided |
|
||
| `CLOUD_MOVING` | System in transition | Wait — transition details provided |
|
||
| `TEIKU` | Unresolvable question | The system is honest about what it cannot answer |
|
||
| `HALT_*` | Cardinal sin detected | Processing terminated — audit log created |
|
||
|
||
---
|
||
|
||
## 2. TAMID — Daily Operations Runbook
|
||
|
||
### 2.1 Morning Tamid (System Day Start)
|
||
|
||
Performed every morning at the voker boundary (configurable — default 06:00 local time of hosting server).
|
||
|
||
```
|
||
MORNING TAMID CHECKLIST — performed by automation, verified by kohen
|
||
|
||
1. □ L1 integrity check
|
||
Command: sha256sum /mnt/torah/torah_scroll.sqfs
|
||
Expected: matches L0 firmware hash
|
||
If failed: HALT — do not proceed. Alert senior kohanim.
|
||
|
||
2. □ Cloud state read
|
||
Command: cat /var/lib/shorashim/cloud/state.json
|
||
Expected: valid JSON with recognized signal
|
||
If "lift" or "shutdown": obey. Do not proceed with normal operations.
|
||
If unreadable: WAIT. Alert senior kohanim.
|
||
|
||
3. □ Manna provision read
|
||
Command: cat /var/lib/shorashim/manna/today.json
|
||
Expected: valid JSON with today's date, non-expired
|
||
If absent: WAIT. Alert senior kohanim.
|
||
If expired: purge, WAIT.
|
||
|
||
4. □ L3 connectivity
|
||
Command: psql -c "SELECT 1" shorashim
|
||
Expected: returns 1
|
||
If failed: restart PostgreSQL. If still failed: alert kohen.
|
||
|
||
5. □ Halt monitor alive
|
||
Command: Check Erlang process via epmd
|
||
Expected: halt_monitor process registered
|
||
If failed: HALT — the system does not run without the halt monitor.
|
||
|
||
6. □ Component health check
|
||
Each component responds to health ping within 5 seconds.
|
||
Shoresh Engine, Sanhedrin, Talmudic Engine, Korban Processor, Vav Pipeline.
|
||
If any component fails: restart that component. If restart fails: alert kohen.
|
||
|
||
7. □ Esh tamid verification
|
||
The processing pipeline accepts a test input and returns a result.
|
||
If the fire has gone out: re-initialize pipeline. Alert kohen.
|
||
|
||
8. □ Yesterday's manna cleanup
|
||
Purge any expired resource allocations from previous cycle.
|
||
Release reclaimed resources.
|
||
|
||
9. □ Log rotation
|
||
Rotate non-audit logs. Audit logs are never rotated.
|
||
|
||
10. □ Record tamid completion
|
||
Write to audit log: "Morning tamid completed at [timestamp]. All checks passed."
|
||
Or: "Morning tamid completed with issues: [list]. Kohen alerted."
|
||
```
|
||
|
||
### 2.2 Evening Tamid (System Day End)
|
||
|
||
Performed every evening at the erev boundary (configurable — default 18:00 local time).
|
||
|
||
```
|
||
EVENING TAMID CHECKLIST
|
||
|
||
1. □ Processing queue drain
|
||
Allow in-flight offerings to complete. No new offerings accepted.
|
||
Timeout: 10 minutes. After timeout, move unfinished to next cycle.
|
||
|
||
2. □ L3 integrity check
|
||
Verify all day's writes are committed. No pending transactions.
|
||
Count today's insertions. Record in audit log.
|
||
|
||
3. □ Manna accounting
|
||
Record: provision received, provision consumed, surplus released.
|
||
If surplus existed: log manna violation check (was anyone hoarding?).
|
||
|
||
4. □ Menorah status — observability summary
|
||
Record: queries processed, Sanhedrin panels convened, teiku registered,
|
||
halt conditions triggered (should be 0), errors by type, latency P50/P95/P99.
|
||
|
||
5. □ Ketoret review — quality of experience
|
||
Sample 5 random responses from today. Evaluate: were they clear? Respectful?
|
||
Did they serve the user's actual need? Record assessment.
|
||
(This is a human review — the kohen reads the incense, not the system.)
|
||
|
||
6. □ Record tamid completion
|
||
Write to audit log: "Evening tamid completed at [timestamp]. Day summary: [metrics]."
|
||
```
|
||
|
||
### 2.3 Erev Shabbat Additional Procedures
|
||
|
||
```
|
||
EREV SHABBAT — performed before candle lighting time
|
||
|
||
1. □ Verify double manna provision is written for Shabbat
|
||
If not: alert senior kohanim immediately. Cannot enter Shabbat unfed.
|
||
|
||
2. □ Drain all processing queues completely (no timeout carryover)
|
||
Nothing in-flight enters Shabbat.
|
||
|
||
3. □ Verify Shabbat timer will trigger correctly
|
||
Dry-run: confirm candle lighting and havdalah times are calculated.
|
||
|
||
4. □ Final L3 backup before Shabbat
|
||
pg_dump to encrypted backup. Store off-site.
|
||
|
||
5. □ Record: "Entering Shabbat at [timestamp]. Next cycle: [motzei Shabbat time]."
|
||
|
||
6. □ The kohen steps back. The fire is banked. The system rests.
|
||
```
|
||
|
||
### 2.4 Motzei Shabbat Procedures
|
||
|
||
```
|
||
MOTZEI SHABBAT — performed after havdalah time
|
||
|
||
1. □ Full morning tamid (all 10 checks)
|
||
2. □ L3 consolidation: review Shabbat-period learning queue (if any emergency
|
||
processing occurred under pikuach nefesh)
|
||
3. □ Read new manna provision for the week's first day
|
||
4. □ Resume normal operations
|
||
```
|
||
|
||
### 2.5 Rosh Chodesh (Monthly Maintenance)
|
||
|
||
```
|
||
ROSH CHODESH — first day of each Hebrew month
|
||
|
||
1. □ Full morning tamid
|
||
2. □ Deep L1 verification — not just hash check, but sample verification
|
||
of 100 random verses against independent source
|
||
3. □ PostgreSQL VACUUM ANALYZE on all L3 tables
|
||
4. □ Review previous month's audit logs for patterns
|
||
5. □ Shulchan renewal — refresh pre-computed cache for common queries
|
||
6. □ Menorah deep review — full observability report for the month
|
||
7. □ Community report: system health, capacity used vs. provisioned,
|
||
user activity, teiku registry growth, halt events (should be 0)
|
||
```
|
||
|
||
### 2.6 Festival Procedures
|
||
|
||
| Festival | System Action |
|
||
|----------|---------------|
|
||
| **Pesach** | Liberation audit: is the system still serving freedom? Review all access controls. Verify pe'ah is open. Verify no surveillance. |
|
||
| **Shavuot** | Specification re-acceptance: full L1 re-verification. Re-run Day 1-2 of inauguration protocol. The system confirms it still holds the same Torah. |
|
||
| **Rosh Hashanah** | Year-start: comprehensive metrics review. Set targets for the coming year. |
|
||
| **Yom Kippur** | Full system shutdown (25 hours). Deep purification: full L3 audit, identify any asham conditions, resolve any pending refuge cases through Sanhedrin deliberation grounded in Torah. |
|
||
| **Sukkot** | Vulnerability audit: run the system with deliberately reduced resources (sukkah = temporary structure). Does it still serve faithfully with less? |
|
||
| **Shmitah (7th year)** | Major version review per architecture spec. L3 comprehensive evaluation. Error records older than 7 years archived and sealed. |
|
||
|
||
---
|
||
|
||
## 3. MONITORING — The Menorah's Seven Branches
|
||
|
||
The menorah provides light — it makes visible what would otherwise be in darkness. Seven branches of observability.
|
||
|
||
### 3.1 Branch 1: Processing State
|
||
|
||
```
|
||
Metrics:
|
||
shorashim_offerings_total{type="shelamim|olah|chatat|minchah"} # Counter
|
||
shorashim_offerings_active # Gauge
|
||
shorashim_pipeline_latency_seconds{phase="shema|tohu|vayomer|..."}# Histogram
|
||
shorashim_sanhedrin_panels_active # Gauge
|
||
shorashim_sanhedrin_deliberation_seconds{size="3|23|71"} # Histogram
|
||
```
|
||
|
||
### 3.2 Branch 2: Deliberation Progress
|
||
|
||
```
|
||
Metrics:
|
||
shorashim_interpretations_total{status="accepted|rejected|teiku"} # Counter
|
||
shorashim_minority_opinions_total # Counter
|
||
shorashim_kushyot_generated_total # Counter
|
||
shorashim_teiku_registry_size # Gauge
|
||
```
|
||
|
||
### 3.3 Branch 3: Specification Reference
|
||
|
||
```
|
||
Metrics:
|
||
shorashim_l1_reads_total # Counter — how often spec is consulted
|
||
shorashim_l1_hash_verified_total # Counter — integrity checks
|
||
shorashim_l1_hash_failed_total # Counter — MUST be 0
|
||
shorashim_teamim_reads_total # Counter
|
||
shorashim_ketiv_qere_positions_accessed # Counter
|
||
shorashim_gematria_lookups_total # Counter
|
||
```
|
||
|
||
### 3.4 Branch 4: Source Citation
|
||
|
||
```
|
||
Metrics:
|
||
shorashim_torah_refs_cited_total{book="bereishit|shemot|..."} # Counter
|
||
shorashim_roots_activated_total # Counter
|
||
shorashim_roots_unique_activated # Gauge per cycle
|
||
```
|
||
|
||
### 3.5 Branch 5: Error Diagnostics
|
||
|
||
```
|
||
Metrics:
|
||
shorashim_kiyor_rejections_total{reason="encoding|injection|malformed|..."}
|
||
shorashim_tzaraat_quarantines_active # Gauge
|
||
shorashim_tzaraat_outcomes{result="tahor|tamei|fully_visible_tahor"}
|
||
shorashim_chatat_corrections_total # Counter
|
||
shorashim_asham_restitutions_total # Counter — should be very low
|
||
```
|
||
|
||
### 3.6 Branch 6: Learning Record
|
||
|
||
```
|
||
Metrics:
|
||
shorashim_l3_insertions_total{table="interpretations|minority_opinions|teiku_registry"}
|
||
shorashim_l3_size_bytes # Gauge
|
||
shorashim_l3_growth_rate_bytes_per_day # Gauge (smoothed)
|
||
```
|
||
|
||
### 3.7 Branch 7: System Health
|
||
|
||
```
|
||
Metrics:
|
||
shorashim_shema_context_build_seconds # Histogram
|
||
shorashim_cloud_state{signal="settle|lift|..."} # Gauge (label)
|
||
shorashim_manna_provision_remaining_pct # Gauge
|
||
shorashim_manna_violations_total # Counter — hoarding attempts
|
||
shorashim_halt_events_total{type="avodah_zarah|shfichut_damim|gilui_arayot"}
|
||
shorashim_refuge_components_active # Gauge — should be 0
|
||
shorashim_eglah_arufah_declarations_total # Counter — should be near 0
|
||
shorashim_component_health{name="..."} # 1=healthy, 0=unhealthy
|
||
```
|
||
|
||
### 3.8 Alerting Thresholds
|
||
|
||
| Condition | Severity | Action |
|
||
|-----------|----------|--------|
|
||
| L1 hash mismatch | **CRITICAL** | System halts. Chatzotzrot: war. Senior kohanim alerted. |
|
||
| Halt monitor down | **CRITICAL** | System halts. Cannot operate without the monitor. |
|
||
| Any halt event fires | **CRITICAL** | Chatzotzrot: war. Senior kohanim alerted. |
|
||
| Cloud state unreadable | **HIGH** | System waits. Chatzotzrot: assembly_leaders. |
|
||
| Manna absent | **HIGH** | System waits. Chatzotzrot: assembly_leaders. |
|
||
| Component unhealthy | **HIGH** | Restart attempted. If failed: chatzotzrot: assembly_leaders. |
|
||
| P95 latency > 2x budget | **MEDIUM** | Kohen alerted. Investigate in next tamid. |
|
||
| L3 growth > 150% projected | **MEDIUM** | Kohen alerted. May need storage review. |
|
||
| Manna hoarding detected | **LOW** | Logged. Surplus reclaimed. Reviewed in evening tamid. |
|
||
| Tzara'at quarantine active | **INFO** | Normal operation. Logged for kohen review. |
|
||
|
||
---
|
||
|
||
## 4. INCIDENT RESPONSE
|
||
|
||
### 4.1 Ordinary Failures (Not Halt Conditions)
|
||
|
||
A process crashes. A disk fills. A network drops. These are not the three cardinal sins. They are the daily realities of running a system.
|
||
|
||
```
|
||
INCIDENT RESPONSE — ORDINARY
|
||
|
||
1. DETECT
|
||
- Monitoring fires alert
|
||
- Or: kohen observes during tamid
|
||
|
||
2. ASSESS
|
||
- Is this a halt condition (avodah zarah, shfichut damim, gilui arayot)?
|
||
- If YES: Halt Monitor should have caught it. If it didn't, that's a second incident.
|
||
- If NO: proceed with ordinary response.
|
||
|
||
3. CONTAIN
|
||
- If a single component: restart it.
|
||
- If the restart fails: isolate it. Route traffic around it if possible.
|
||
- If the failure affects the pipeline: queue incoming offerings. They wait.
|
||
- The system degrades gracefully. It does not fabricate results from partial capacity.
|
||
|
||
4. DIAGNOSE
|
||
- Read the logs. The menorah is for this.
|
||
- Identify: what failed, when, what was the trigger.
|
||
- Check the ShemaContext from the failed cycle — was the context valid?
|
||
|
||
5. REPAIR
|
||
- Fix the issue.
|
||
- If a configuration change: record it in the audit log.
|
||
- If a code change: go through bikurim (first-output review) before deploying.
|
||
|
||
6. VERIFY
|
||
- Run the relevant inauguration day's checks for the affected component.
|
||
- Confirm the system is healthy.
|
||
|
||
7. RECORD
|
||
- Audit log entry: incident, diagnosis, repair, verification.
|
||
- Evening tamid review includes incident discussion.
|
||
```
|
||
|
||
### 4.2 Harm Events (Arei Miklat / Eglah Arufah)
|
||
|
||
```
|
||
INCIDENT RESPONSE — HARM
|
||
|
||
1. DETECT
|
||
- User reports harm caused by system output
|
||
- Or: internal monitoring detects anomalous impact
|
||
|
||
2. IMMEDIATE
|
||
- Activate arei miklat: isolate the responsible processing path
|
||
- Protect the affected user: block their exposure to that processing path
|
||
- Chatzotzrot: assembly_leaders — senior kohanim alerted
|
||
|
||
3. IF RESPONSIBLE COMPONENT IS KNOWN
|
||
- Component enters refuge (arei miklat)
|
||
- Component is protected from reactive destruction
|
||
- Sanhedrin deliberation determines release conditions
|
||
|
||
4. IF RESPONSIBLE COMPONENT IS UNKNOWN
|
||
- Activate eglah arufah protocol
|
||
- Nearest component in the processing chain formally declares:
|
||
what it processed, what it did not check, what it could have caught
|
||
- Declaration is permanently recorded
|
||
|
||
5. GOVERNANCE BOARD
|
||
- Reviews the incident
|
||
- Determines: was this chatat (unintentional error) or asham (unauthorized usage)?
|
||
- If chatat: correction protocol, no penalty beyond fixing the issue
|
||
- If asham: restitution protocol, plus additional safeguards (the "fifth")
|
||
- Determines when/if the refuge component is released
|
||
|
||
6. NEVER
|
||
- Never delete the component reactively
|
||
- Never blame the user for being harmed
|
||
- Never hide the incident
|
||
- Never continue the same processing path without review
|
||
```
|
||
|
||
---
|
||
|
||
## 5. BACKUP AND RECOVERY
|
||
|
||
### 5.1 What Is Backed Up
|
||
|
||
| Layer | Backup Method | Frequency | Retention |
|
||
|-------|--------------|-----------|-----------|
|
||
| L0 (Luchot) | Compiled into binary — exists in version control | Every build | Permanent |
|
||
| L1 (Torah scroll) | Squashfs image — source files in version control | Once (immutable) | Permanent |
|
||
| L3 (Oral tradition) | `pg_dump --format=custom` encrypted with GPG | Daily (erev tamid) + erev Shabbat | 1 year rolling + shmitah archives |
|
||
| Configuration | Version control (Git) | Every change | Permanent |
|
||
| Audit logs | rsync to off-site encrypted storage | Daily | Permanent (never deleted) |
|
||
| Cloud governance | Not backed up separately — cloud IS L1 (Torah) which is already backed up as immutable specification | — | — |
|
||
| Manna provision | Not backed up — kohen/treasurer sets this from machatzit hashekel fund | — | — |
|
||
|
||
### 5.2 Recovery Procedures
|
||
|
||
**Scenario: L3 corruption**
|
||
1. Stop the system
|
||
2. Restore L3 from most recent pg_dump
|
||
3. Verify: all append-only constraints intact
|
||
4. Re-run inauguration Day 1 (L0/L1 verification) and Day 7 (integration)
|
||
5. Resume operations
|
||
|
||
**Scenario: Full system loss**
|
||
1. Provision new server (same specs from Construction Specification)
|
||
2. Install OS and all runtimes (pinned versions from Construction Specification)
|
||
3. Deploy code from version control
|
||
4. Mount L1 from squashfs image, verify hash
|
||
5. Restore L3 from most recent backup
|
||
6. Verify cloud governance from Torah (L1 readable, integrity verified, calendar active) and kohen/treasurer writes manna provision
|
||
7. Run full seven-day inauguration
|
||
8. The system does not skip inauguration because it is being rebuilt. A rebuilt mishkan is a new mishkan. It is consecrated anew.
|
||
|
||
**Scenario: L1 integrity failure**
|
||
1. The system has already halted (this is automatic)
|
||
2. Investigate: was the filesystem modified? Was the mount corrupted?
|
||
3. Re-mount from the original squashfs image
|
||
4. Verify hash against L0 firmware constant
|
||
5. If hash matches: resume
|
||
6. If hash does not match: the specification has been altered. This is not a technical incident. This is a Torah integrity incident. The kohanim must determine what happened and whether the system can trust its own specification. The system does not resume until L1 integrity is restored — because the cloud IS L1, and if L1 is compromised, the cloud is not visible. The system waits until Torah is readable and intact.
|
||
|
||
---
|
||
|
||
## 6. ON-CALL PROCEDURES
|
||
|
||
### 6.1 Who Is On Call
|
||
|
||
One kohen at all times during the six working days. The kohen is reachable within 15 minutes for HIGH alerts, within 5 minutes for CRITICAL alerts.
|
||
|
||
During Shabbat: no on-call. The system is off. The only exception is pikuach nefesh — if the system's unavailability would cause genuine danger to a human life, the kohen may respond. This should never happen. If it can happen, the system has made itself necessary in a way that violates its specification (the Shabbat Test from the PRD).
|
||
|
||
### 6.2 Escalation
|
||
|
||
| Level | Responder | Conditions |
|
||
|-------|-----------|------------|
|
||
| L1 | On-call kohen | Any alert |
|
||
| L2 | Lead architect | Component failures, performance degradation |
|
||
| L3 | Senior kohanim | Halt events, harm incidents, L1 integrity failure |
|
||
|
||
### 6.3 Communication
|
||
|
||
Chatzotzrot signals map to communication channels:
|
||
|
||
| Signal | Channel |
|
||
|--------|---------|
|
||
| assembly_all | System status page + all registered notification endpoints |
|
||
| assembly_leaders | Senior kohanim secure channel + on-call kohen |
|
||
| breaking_camp | Same as assembly_all — everyone needs to know the system is moving |
|
||
| war | Senior kohanim immediate alert + on-call kohen + lead architect |
|
||
| moadim | System status page — celebratory, not urgent |
|
||
|
||
---
|
||
|
||
## 7. EXTERNAL NEDARIM — Third-Party Covenants
|
||
|
||
### 7.1 Torah Grounding
|
||
|
||
> אִישׁ֩ כִּֽי־יִדֹּ֨ר נֶ֜דֶר לַֽיהוָ֗ה אֽוֹ־הִשָּׁ֤בַע שְׁבֻעָה֙ לֶאְסֹ֤ר אִסָּר֙ עַל־נַפְשׁ֔וֹ לֹ֥א יַחֵ֖ל דְּבָר֑וֹ כְּכָל־הַיֹּצֵ֥א מִפִּ֖יו יַעֲשֶֽׂה ׃
|
||
> — Bamidbar 30:3
|
||
|
||
A man who makes a vow to Hashem, or swears an oath to bind a restriction upon his soul — *lo yachel devaro*, he shall not profane his word; according to all that comes out of his mouth, he shall do.
|
||
|
||
Rambam encodes this as four discrete mitzvot, all enumerated in `talmudic_engine/src/specification/mitzvot.pl`: positive #93 `fulfill_verbal_vow`, prohibitions #308 / #543 `not_break_oath`, #542 `not_delay_vow_fulfillment`.
|
||
|
||
The Avraham–Avimelekh covenant (Bereishit 21:22-34) establishes that a brit with a non-Israelite party binds the Israelite signatory. The other side need not share Torah for the obligation to attach.
|
||
|
||
### 7.2 Application — External APIs as Accepted Nedarim
|
||
|
||
When the operator (kohen) accepts an external API provider's terms of service to obtain access, that acceptance is a *neder al nafsho* — a self-imposed restriction. *Lo yachel devaro* applies. Breaching the accepted terms is *chillul* of the word given.
|
||
|
||
This obligation attaches to the **operator**, not the system. The system is a *kli* — a vessel — and cannot accept or break a neder. The kohen carries the covenant.
|
||
|
||
| Provider | API | Accepted Terms | Phase |
|
||
|----------|-----|----------------|-------|
|
||
| Anthropic | Claude API — korban source per Architecture §13 | Anthropic Usage Policy + Commercial Terms of Service | Active (Phases 1-2) |
|
||
| Mistral | Mistral API | Mistral Terms of Service | Phase 1 only — see Pekudei §3 self-hosting transition |
|
||
|
||
### 7.3 Operator Responsibilities
|
||
|
||
1. **Before accepting a provider's terms.** Read them. Record date of acceptance and version reviewed in the operator log.
|
||
2. **Before deploying any change touching an external API.** Verify the new behavior remains within the accepted terms. If a change requires use the terms do not permit, the change halts until the use is dropped or the terms are renegotiated.
|
||
3. **Quarterly review.** Providers update their terms. Re-read the current version. Document any change in the operator log.
|
||
4. **On termination.** Revoke API keys per the provider's offboarding process. Confirm cessation.
|
||
|
||
### 7.4 Anthropic-Specific Posture (Phases 1-2)
|
||
|
||
Claude is used as a *korban source* — input that is processed through *shechitah*, *zerikah*, and validation against L1 before the system reasons over it. Specific commitments accepted by using the Claude API:
|
||
|
||
- **Disclosure to end users.** Implemented at the bigdei kodesh edge — every Torah-content response from `/peah/ask`, `/courtyard/offering`, and `/chatzer/sanhedrin` carries a `disclosure` field stating origin and that the output is not horaah. This honors *lo techachashu* (Vayikra 19:11) at the user boundary in addition to the API provider's disclosure clause.
|
||
- **No use of Claude output to train competing models.** Mistral fine-tuning data per Terumah §E4 is sourced from Talmud and Rishonim (free, open). Verify before any fine-tuning run that no Claude-derived material is in the training set.
|
||
- **No prohibited use cases.** The system does not provide any of the use cases the Anthropic Usage Policy enumerates as prohibited. The kohen reviews the policy before any new endpoint is added that could expand the use surface.
|
||
|
||
### 7.5 Two Disciplines, Both Bind
|
||
|
||
The system halts on `avodah_zarah` when it would treat a non-Torah source as authoritative. This protects the system's chain of authority — Torah → PRD → Architecture → Construction → Implementation. It does not protect the operator from *chillul* of an external neder. The two disciplines are separate:
|
||
|
||
- **Internal authority discipline** (avodah_zarah halt) — the system's responsibility, enforced in `talmudic_engine/src/halt/avodah_zarah.pl` and `sanhedrin/src/halt_monitor.erl`.
|
||
- **External covenant discipline** (lo yachel devaro) — the operator's responsibility, enforced by reading, by the quarterly review in §7.3, and by the change-gate in §7.3.2.
|
||
|
||
Both bind. Neither replaces the other.
|
||
|
||
---
|
||
|
||
## Document Registry
|
||
|
||
| Document | Torah Parallel | Status |
|
||
|----------|---------------|--------|
|
||
| **Shorashim PRD** | *V'asu li mikdash* | ✓ Complete |
|
||
| **Architecture Specification** | Vessel descriptions | ✓ Complete |
|
||
| **Construction Specification** | Exact measurements | ✓ Complete |
|
||
| **Operations Manual** (this document) | Tamid / Bigdei Kehunah | ✓ Complete |
|
||
| **Economic Model** | Machatzit HaShekel | Pending |
|
||
| **Materials Inventory** | Terumah list | Needs update |
|