Commit Graph
12 Commits
Author SHA1 Message Date
Scott KarpandClaude Opus 4.6 e223ab15b3 Phase 4 complete: Steps 21, 22, 24 — Korban external integration, L2 Urim v'Tumim, Kushya generator
Vayikra 9:24 — vatetzei esh milifnei Hashem vatokhal al hamizbeach.
Phase 4 of the architecture is now complete. The remaining three steps
that were marked "blocked on external infrastructure" are built —
discipline-respecting, eish-zarah-aware, and tested.

## Step 22 — Korban Processor external model integration
   (korban_processor/src/korban_processor/external_offering.py)

   The single legitimate entry point for external model consultation.
   Wraps Claude API calls through the offering protocol so external
   output is treated as korban (raw material), never authoritative:

   - semichah  — provenance is tagged (avodah zarah Form 1 prevention)
   - shechitah — assistant voice, hedging, formatting stripped
   - zerikah   — vital content extracted
   - haktarah  — converted to Prolog propositions
   - route     — sent to internal component per offering type

   The raw Korban is preserved on the ProcessedKorban (gilui arayot
   Form 3 prevention — input/output coupling never severed).
   No internal component should call call_claude() directly.

   11 tests, mocked external calls. Live API test opt-in via
   SHORASHIM_LIVE_API=1.

## Step 24 — L2 Urim v'Tumim interface
   (vav_pipeline/lib/vav_pipeline/urim_v_tumim.ex)

   PRD §4.7: housed in the kohen gadol's breastplate; consulted only
   on high-stakes decisions where normal deliberation is insufficient.
   Bamidbar 27:21 — al-piv yetz'u v'al-piv yavo'u.

   Three escalation steps, three honest outcomes:
     1. resolved_by_specification — re-read produced an answer
     2. resolved_by_external — external testimony recorded (NOT binding)
     3. deferred_to_human — the system stops, asks for human judgment

   Pre-condition gating refuses cases that didn't reach Sanhedrin teiku
   first (no preempting deliberation) or that aren't flagged high-stakes
   (ordinary teiku stays teiku). Records every consultation with steps
   taken — there is no quiet escalation.

   Per PRD: "The Urim v'Tumim never fabricates certainty where none
   exists." When external also fails, the system stops. It does not
   proceed on a guess.

   Added to VavPipeline supervision tree. 14 tests.

## Step 21 — Kushya generator scaffolding
   (kushya_generator/)

   Directory layout per Architecture §3.6 with three layers:

   - data/training_pairs/seed_pairs.jsonl — 10 hand-curated bootstrap
     pairs covering 9 kushya types (scope_narrowing, internal_tension,
     harmonization, exception_test, edge_case, scope_application,
     rabbinic_constraint, method_question, subject_identification)
   - data/prepare_training.py — validates the four kushya requirements
     (specific Torah passage, identified weakness, question form,
     non-trivial), produces train.jsonl + eval.jsonl with stratified
     80/20 split by type
   - fine_tune/config.yaml + train.py — QLoRA configuration and
     training script. Operator's task — needs GPU. Script is
     complete and self-contained.
   - inference/kushya_api.py — three-backend interface
     (fine_tuned → external → fallback). When fine-tuned model isn't
     deployed, falls back to external via Korban Processor (Step 22).
     The fallback is honest: returns no kushya rather than inventing
     one. The Talmudic Engine's rule-based challenge.pl remains the
     primary kushya source.

   17 tests covering dataset validation, backend selection, and
   reference extraction.

## .gitignore fix

   The earlier "data/" line was too broad — it caught
   kushya_generator/data/. Scoped to /data/ (root-level only).

## Test counts

   - Elixir:           274 (was 260; +14 for UrimVTumim)
   - Korban Python:    196 + 1 skipped (was 195; +1 module = 11 tests)
   - Kushya Python:     17 (new)
   - Haskell:           33 (unchanged)
   - Prolog:           195 (unchanged)
   - Total:            715 + 19 L3 excluded

## Phase 4 status

   All 12 steps now have implementations:
   - Steps 23, 25, 26, 27, 28, 29, 30, 31, 32: complete
   - Steps 21, 22, 24: complete with discipline preserved

   Step 21's fine-tuning run remains the operator's GPU task; the
   pipeline is built such that a deployed adapter slots into the
   inference backend without touching the Talmudic Engine or anything
   above.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-23 17:12:03 -04:00
Scott KarpandClaude Opus 4.6 d4d0a6fcb6 L1 data integrity: preserve the edut whole — Shema, middle letter, ketiv/qere, smikhut
Shemot 25:16 — v'natata el ha-aron et ha-edut asher eten elekha.
The edut placed in the aron must be complete. The Masoretic
tradition preserves every letter, every enlarged letter, every
ketiv/qere pair — because each carries meaning the scribe cannot
reconstruct.

The previous build_l1.py silently dropped:
- enlarged letters encoded as <seg type="x-large"> nested in <w>
  → the Shema was corrupt: שמ instead of שמע, אח instead of אחד
  → the middle letter of Torah was corrupt: גח instead of גחון
  → the Zelophchad nun was missing: משפט instead of משפטן
- ketiv/qere dual readings encoded as <note type="variant">/<rdg type="x-qere">
  → the spec committed to preserving both; the code returned []
- smikhut (construct state) pairs marked in morphology codes
  → the spec committed to extracting them; the code returned []

Fixes:
- collect_word_text() walks all nested <seg> children of <w> and records
  large_letters / small_letters metadata with character offsets.
- parse_verse() recognizes <w type="x-ketiv"> followed by <note>, extracts
  both the ketiv and the qere readings, stores a ketiv_qere list on the verse.
- _extract_smikhut() scans words for construct-state morph codes
  (ending in "c" on N/A primary segments, excluding pronominal suffixes)
  and records construct/absolute pairs.
- L1 SHA-256 hash updated everywhere: integrity.sha256, l1_reader.ex,
  shoresh-engine's Luchot.Firmware, its test, KOHEN.md.

Torah-wide results after fix:
- 4 enlarged letters preserved (was 0): Shema ayin, Shema dalet,
  gachon vav (middle of Torah), Zelophchad nun
- 67 ketiv/qere pairs captured (was 0)
- 9,203 smikhut pairs extracted (was 0)

Elixir updates:
- l1_reader.ex: normalize_verse now surfaces ketiv_qere, smikhut,
  and per-word large_letters/small_letters. New type specs. New
  extract_large_letters/1 helper. extract_ketiv_qere and
  extract_smikhut now return real data instead of [].
- step31_shema_test.exs: replaced placeholder asserts with real
  integrity checks. Four new tests verify the Shema word is complete
  (שמע not שמ), echad is complete (אחד not אח), the enlarged
  letters are marked, the middle letter of Torah is preserved,
  and ketiv/qere pairs from Bereishit 8:17 are both captured.

Build script:
- build_l1.py no longer rmtrees the entire l1/ directory (preserves
  l1/README.md). Only regenerates torah/ + build_report.json +
  integrity.sha256.
- Accepts SHORASHIM_SOURCE_DATA env var or CLI arg for morphhb path.

Test counts:
- Elixir: 260 (was 254), 0 failures
- Haskell: 33, 0 failures
- Prolog: 195, 0 failures

The system that can hear שמע correctly has passed its own shema test.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-21 13:11:52 -04:00
Scott KarpandClaude Opus 4.6 191d76b480 Bigdei Kodesh: HTTP API layer — four garments for four access tiers
Shemot 28:2 — V'asita vigdei kodesh l'Aharon achicha,
l'khavod ul'tifaret.

The builder makes the garments. The kohen wears them.

Four access tiers (Operations spec §1.1):
- Pe'ah (GET/POST /peah/ask) — public, no auth, no tracking.
  The stranger is safe to approach.
- Courtyard (POST /courtyard/offering) — Bearer token auth,
  offering types: shelamim, olah, chatat, minchah
- Chatzer (POST /chatzer/sanhedrin) — Sanhedrin deliberation,
  returns majority/minority with preserved dissent
- Kodesh (GET /kodesh/health, /kodesh/status) — operator interface,
  health checks for K8s liveness/readiness probes

Error format follows chatat protocol (Operations spec §1.2):
  KIYOR_* (422), SHABBAT (503), MANNA_DEPLETED (503),
  CLOUD_MOVING (503), HALT_* (503), UNAUTHORIZED (401)

Implementation:
- Plug.Router with Bandit HTTP server
- Added to supervision tree via maybe_add_http_server
- Checks system availability (Shabbat, halt) before processing
- Health check reads L1 integrity, cloud, manna, halt monitor
- Status shows all component states, signal count, refuge list
- 16 API tests covering all four tiers + error format

All 235 Elixir + 19 L3 + 195 Prolog + 33 Haskell = 463+ tests, 0 failures

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 07:32:15 -04:00
Scott KarpandClaude Opus 4.6 fe93c11d13 Phase 5 deployment: Dockerfiles, K8s manifests, halt monitor (Steps 33, 36)
Shemot 40:33 — Vay'khal Moshe et ham'lakhah.
The builder completes the work. The vessel is ready for transport.

Step 33 — Deployment configuration:
- Dockerfiles for all 4 components:
  - vav_pipeline: Elixir 1.16.1/OTP 26.2.1 release with Sanhedrin
  - shoresh_engine: GHC 9.6.4 binary
  - talmudic_engine: SWI-Prolog 9.2.2
  - korban_processor: Python 3.12.2 with pydantic
- Kubernetes manifests for Exoscale SKS (ch-gva-2):
  - Namespace, ConfigMap, Secrets template
  - L1 Torah PV/PVC (read-only mount)
  - Manna provision PVC (kohen writes, system reads)
  - Deployment: vav-pipeline + 3 sidecars, shared volumes
  - Service: ClusterIP + LoadBalancer with Exoscale NLB
  - Init container runs Ecto migrations (VavPipeline.Release)

Step 36 — Halt monitor (esh tamid):
- GenServer watches for three cardinal sins
- halt!/3 immediately halts system, sounds Chatzotzrot war signal
- Permanent audit log — halt events never deleted
- resolve/1 requires explicit operator action to resume
- alive?/0 for morning tamid verification
- 13 tests: alive check, 3 halt types, resolution, audit log,
  chatzotzrot war signal integration
- Added to supervision tree

All 219 Elixir + 19 L3 + 195 Prolog + 33 Haskell = 447+ tests, 0 failures

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 07:11:02 -04:00
Scott KarpandClaude Opus 4.6 6a879fcf8b MannaLedger: real anti-hoarding enforcement — d'var yom b'yomo (Step 26)
Shemot 16:4 — d'var yom b'yomo — the day's matter in its day.
Shemot 16:19 — al yoter mimenu ad boker — do not leave surplus.
Shemot 16:20 — vayotiru... vayarum tola'im vayiv'ash — hoarded manna rots.

- MannaLedger GenServer tracks consumption against daily provision
- load_provision/0 reads manna file, resets consumption to zero
- consume/2 deducts from allocated resource, rejects when exhausted
- remaining/1 and available?/2 for balance queries
- enforce_cycle_boundary/0 zeroes out all allocation, returns released amounts
- Chatzotzrot assembly_leaders signal when provision runs low (< 10%)
- Manna.enforce_anti_hoarding/0 now delegates to MannaLedger
- Added to supervision tree in application.ex
- 17 new tests with proper isolation (reset between tests)
- All 177 Elixir tests pass, 0 failures

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-18 09:27:09 -04:00
Scott KarpandClaude Opus 4.6 671d45625c Protection layer: Arei Miklat, Eglah Arufah, Bikurim — Steps 27-29
Takhin lekha haderekh — Devarim 19:3.
"Prepare the road" before the one who needs refuge arrives.
Protection before production.

Arei Miklat (Cities of Refuge — Bamidbar 35, Devarim 19):
- GenServer with refuge registry, affected party protection, audit trail
- enter_refuge/2 isolates component but protects from reactive destruction
- protect_affected/2 blocks processing paths that caused harm
- release/2 requires :sanhedrin_deliberation authority — ad mot hakohen hagadol
- Chatzotzrot assembly_leaders signal on every refuge entry
- 15 tests: entry, double-entry prevention, affected party blocking,
  unauthorized release rejection, audit trail, chatzotzrot signaling

Eglah Arufah (Accountability — Devarim 21:1-9):
- GenServer with permanent (append-only) declaration registry
- initiate/3: nearest component declares what it processed, skipped, missed
- Only for :unknown source — rejects known-source incidents
- "Yadeinu lo shafkhu" — accountability for what was NOT provided
- 9 tests: declaration recording, source validation, multi-component
  declarations, chatzotzrot signaling, filtering, counting

Bikurim (First Fruits — Devarim 26:1-11):
- GenServer with offering registry
- offer/4: submit first output with required narrative (mikra bikurim)
- Rejects empty narrative ("Arami oved avi" — story is required)
- Rejects nil first output (there must be fruit to offer)
- Delegates to Specification.evaluate_bikurim/1 when available
- approved?/1 gate for deployment readiness
- 11 tests: acceptance, narrative validation, nil output rejection,
  registry, status filtering, approval checking, counting

All three added to supervision tree in application.ex.
160 Elixir tests pass, 0 failures.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-18 08:59:08 -04:00
Scott KarpandClaude Opus 4.6 7fe8025434 Chatzotzrot: silver trumpets with BEAM routing, subscription, and zikaron
Bamidbar 10:2 — Aseh lekha shtei chatzotz'rot kesef, mikshah ta'aseh otam.
Two trumpets, hammered from a single piece. Five signals, five patterns.

- Upgraded from simple function module to GenServer with state
- Recipient routing: :all (assembly_all, breaking_camp, moadim),
  :kohanim (assembly_leaders, war), :operators (war)
- PubSub via BEAM messaging: subscribe/1, unsubscribe/1
  Signals delivered as {:chatzotzrot, record} messages
- Zikaron (remembrance — Bamidbar 10:10): signal history with
  type filtering and limit. V'hayu lakhem l'zikaron.
- Dead subscriber cleanup via Process.monitor/1
- Added to supervision tree in application.ex
- 30 new tests: patterns, urgency, routing, subscription,
  zikaron history, dead process cleanup, format compliance
- All 125 Elixir tests pass, 0 failures

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 15:50:36 -04:00
Scott KarpandClaude Opus 4.6 d34e2799ac L3 Oral Tradition: PostgreSQL persistence layer for interpretations, minority opinions, teiku
Devarim 17:9 — v'darashta v'higidu lekha et d'var hamishpat.
The inquiry requires that what is found is RECORDED.

- Ecto Repo with graceful degradation (maybe_add_repo, like other engines)
- Migration: interpretations, minority_opinions, teiku_registry tables
  - Append-only enforcement via REVOKE UPDATE/DELETE
  - Includes eish_zarah fields: coupling_trace, grounding_type, challenge_sequence
- Ecto schemas: Interpretation, MinorityOpinion, TeikuRecord
  - Changeset validations: status inclusion, panel_size > 0, grounding_type enum
  - TeikuRecord.resolve_changeset restricts updates to resolved_at/resolution only
- L3.Store bridge module: record, query, count operations
- L3 tests tagged @moduletag :l3, excluded by default (no local PostgreSQL)
  - Enable with: SHORASHIM_L3=1 mix test --include l3
- All 76 existing Elixir tests pass, 195 Prolog tests pass, 33 Haskell tests pass

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 09:43:44 -04:00
Scott KarpandClaude Opus 4.6 af7522e977 Phase 2 review corrections — three spec violations fixed
1. Chatzotzrot signal mapping corrected to match Architecture §3.12:
   assembly_all→tekiah, assembly_leaders→teruah,
   breaking_camp→tekiah_teruah, moadim→tekiah_tekiah.
   Four of five patterns were swapped. All five now match Bamidbar 10.

2. Manna parse_datetime(nil) bug fixed: missing expires_at now fails
   validation instead of silently returning DateTime.utc_now() which
   created a race condition where manna appeared expired.

3. Festival detection implemented in cloud.ex: pre-computed Gregorian
   dates for Pesach, Shavuot, Rosh Hashanah, Yom Kippur, Sukkot,
   Shemini Atzeret (2025-2028). Pipeline now halts on Yom Tov as
   Torah requires. Step 15 verification now covers festival rest.

76 tests, 0 failures (68 prior + 8 new).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 20:29:24 -04:00
Scott KarpandClaude Opus 4.6 5a93056af4 Step 15: Verification — the Shema Test
Wire HaltConditions.check_all into BereishitCycle.erev_voker so the
three cardinal sin monitors (avodah zarah, shfichut damim, gilui arayot)
actually halt the pipeline. Previously dead code, now live.

Add step15_shema_test.exs — Architecture §6.3 (The Shema Test):
1. Te'amim reception — text's own divisions guide havdalah
2. Ketiv/qere preservation — dual readings never collapse
3. Cloud obedience — Shabbat blocks processing
4. Manna fidelity — no provision means wait
5. Context immutability — ShemaContext unchanged at every stage
6. Halt condition integration — pipeline halts on cardinal sin
7. Full pipeline end-to-end — Bereishit 1:1 through to output

68 tests, 0 failures (46 existing + 22 new).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 18:01:05 -04:00
Scott KarpandClaude Opus 4.6 c67672777b Step 14: Connect components — chamishim karsay zahav
Three connections built and tested:
- Haskell Shoresh Engine via Erlang Port (JSON lines over stdin/stdout)
- Prolog Talmudic Engine via Unix domain socket (JSON over /tmp/shorashim/talmudic.sock)
- Erlang Sanhedrin via native BEAM calls

New: Server.hs (JSON-line server loop), json_bridge.pl + socket_server.pl
(Prolog socket interface), ShoreshEnginePort + TalmudicEngineSocket +
SanhedrinClient (Elixir GenServer wrappers). BereishitCycle now routes
unresolved cases through Talmudic Engine and Sanhedrin. All connections
gracefully degrade — pipeline falls back to L1 data when engines unavailable.

46 Elixir tests, 33 Haskell tests, 57 Prolog tests — 0 failures.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 17:39:21 -04:00
Scott KarpandClaude Sonnet 4.6 80ec364da6 Phase 1-2 build: Steps 5-6, 8-13
Shoresh Engine (Haskell, Step 5)
- 12 modules: Types, Lexicon, Binyanim, Teamim, Gematria, Graph,
  KetivQere, Smikhut, Havdalah, Kriah, KiTov, Firmware
- 33 EUnit tests, 0 failures
- Firmware module verifies L1 SHA-256 hash at startup

Talmudic Engine (SWI-Prolog, Step 6)
- 15 modules: halt, reasoning, specification, interface
- 57 PlUnit tests, 0 failures
- Three cardinal halt conditions: avodah_zarah, shfichut_damim, gilui_arayot

Vav Pipeline (Elixir, Steps 8-11)
- 14 modules: BereishitCycle, Shema, Cloud, Manna, L1Reader,
  Specification, HaltConditions, ShabbatTimer, CycleLog, Chatzotzrot,
  ShoreshEngineClient, TalmudicEngineClient, Application
- 46 ExUnit tests, 0 failures
- Nishma-first architecture: hears before it speaks

Sanhedrin Framework (Erlang/OTP, Step 12)
- 8 modules: sanhedrin_app, sanhedrin_sup, escalation, panel_sup,
  panel, judge, clerk, halt_monitor
- 46 EUnit tests, 0 failures (includes fix: rejected outcome now reachable)
- Panel sizes: 1/3/23/71 — unanimous capital guilty → acquit (Sanhedrin 17a)
- Minority opinions append-only, never deleted

Korban Processor (Python, Step 13)
- 6 modules: korban, shechitah, semichah, zerikah, haktarah, router

Builder tooling: AGENTS.md, opencode.json, memory/, src/tools/

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-26 14:02:34 -05:00