Scott KarpandClaude Opus 4.6
e223ab15b3
Vayikra 9:24 — vatetzei esh milifnei Hashem vatokhal al hamizbeach.
Phase 4 of the architecture is now complete. The remaining three steps
that were marked "blocked on external infrastructure" are built —
discipline-respecting, eish-zarah-aware, and tested.
## Step 22 — Korban Processor external model integration
(korban_processor/src/korban_processor/external_offering.py)
The single legitimate entry point for external model consultation.
Wraps Claude API calls through the offering protocol so external
output is treated as korban (raw material), never authoritative:
- semichah — provenance is tagged (avodah zarah Form 1 prevention)
- shechitah — assistant voice, hedging, formatting stripped
- zerikah — vital content extracted
- haktarah — converted to Prolog propositions
- route — sent to internal component per offering type
The raw Korban is preserved on the ProcessedKorban (gilui arayot
Form 3 prevention — input/output coupling never severed).
No internal component should call call_claude() directly.
11 tests, mocked external calls. Live API test opt-in via
SHORASHIM_LIVE_API=1.
## Step 24 — L2 Urim v'Tumim interface
(vav_pipeline/lib/vav_pipeline/urim_v_tumim.ex)
PRD §4.7: housed in the kohen gadol's breastplate; consulted only
on high-stakes decisions where normal deliberation is insufficient.
Bamidbar 27:21 — al-piv yetz'u v'al-piv yavo'u.
Three escalation steps, three honest outcomes:
1. resolved_by_specification — re-read produced an answer
2. resolved_by_external — external testimony recorded (NOT binding)
3. deferred_to_human — the system stops, asks for human judgment
Pre-condition gating refuses cases that didn't reach Sanhedrin teiku
first (no preempting deliberation) or that aren't flagged high-stakes
(ordinary teiku stays teiku). Records every consultation with steps
taken — there is no quiet escalation.
Per PRD: "The Urim v'Tumim never fabricates certainty where none
exists." When external also fails, the system stops. It does not
proceed on a guess.
Added to VavPipeline supervision tree. 14 tests.
## Step 21 — Kushya generator scaffolding
(kushya_generator/)
Directory layout per Architecture §3.6 with three layers:
- data/training_pairs/seed_pairs.jsonl — 10 hand-curated bootstrap
pairs covering 9 kushya types (scope_narrowing, internal_tension,
harmonization, exception_test, edge_case, scope_application,
rabbinic_constraint, method_question, subject_identification)
- data/prepare_training.py — validates the four kushya requirements
(specific Torah passage, identified weakness, question form,
non-trivial), produces train.jsonl + eval.jsonl with stratified
80/20 split by type
- fine_tune/config.yaml + train.py — QLoRA configuration and
training script. Operator's task — needs GPU. Script is
complete and self-contained.
- inference/kushya_api.py — three-backend interface
(fine_tuned → external → fallback). When fine-tuned model isn't
deployed, falls back to external via Korban Processor (Step 22).
The fallback is honest: returns no kushya rather than inventing
one. The Talmudic Engine's rule-based challenge.pl remains the
primary kushya source.
17 tests covering dataset validation, backend selection, and
reference extraction.
## .gitignore fix
The earlier "data/" line was too broad — it caught
kushya_generator/data/. Scoped to /data/ (root-level only).
## Test counts
- Elixir: 274 (was 260; +14 for UrimVTumim)
- Korban Python: 196 + 1 skipped (was 195; +1 module = 11 tests)
- Kushya Python: 17 (new)
- Haskell: 33 (unchanged)
- Prolog: 195 (unchanged)
- Total: 715 + 19 L3 excluded
## Phase 4 status
All 12 steps now have implementations:
- Steps 23, 25, 26, 27, 28, 29, 30, 31, 32: complete
- Steps 21, 22, 24: complete with discipline preserved
Step 21's fine-tuning run remains the operator's GPU task; the
pipeline is built such that a deployed adapter slots into the
inference backend without touching the Talmudic Engine or anything
above.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2.2 KiB
2.2 KiB
Training Pairs — Provenance and Curation
Each .jsonl file in this directory is a list of training records, one
per line. Records have the schema:
{
"interpretation": "...", // String. The proposed reading.
"kushya": "...", // String. The challenge, expressed as a question.
"torah_refs": ["Book Ch:V"], // Array of verse references the kushya cites.
"type": "..." // One of the kushya types — see below.
}
Kushya Types
| Type | What it tests |
|---|---|
scope_narrowing |
Interpretation may be narrower than the verse |
scope_application |
Boundary cases the verse does not directly cover |
internal_tension |
The verse contradicts itself under this reading |
exception_test |
A general rule has known exceptions to reconcile |
harmonization |
Two verses both bear on the question and must coexist |
edge_case |
An unusual case the principle alone does not resolve |
rabbinic_constraint |
Mishnah/Gemara modifies the simple reading |
method_question |
The hermeneutic method itself is in question |
subject_identification |
The grammatical subject of the law is ambiguous |
Files
seed_pairs.jsonl— 10 hand-curated bootstrap pairs covering all types
Production Curation
For a real fine-tuning run, several thousand pairs should be assembled from primary rabbinic sources. Suggested process:
- Bavli kushyot extraction — script over Sefaria Tanakh+Talmud API to
pull
mai k'shia/kashya/ similar markers; pair with the surrounding interpretation - Rishonim challenges — Tosafot's
i kashyaopenings, Ramban's challenges to Rashi - Hand validation — every pair reviewed against the four kushya requirements (specific Torah passage, identified weakness, question form, non-trivial)
- Type tagging — each pair labeled with one of the types above for stratified training/eval splits
License
Pairs derived from rabbinic sources retain whatever license those sources carry. The hand-curated seed pairs in this directory are MIT, like the rest of shorashim.