Scott KarpandClaude Opus 4.6
e223ab15b3
Vayikra 9:24 — vatetzei esh milifnei Hashem vatokhal al hamizbeach.
Phase 4 of the architecture is now complete. The remaining three steps
that were marked "blocked on external infrastructure" are built —
discipline-respecting, eish-zarah-aware, and tested.
## Step 22 — Korban Processor external model integration
(korban_processor/src/korban_processor/external_offering.py)
The single legitimate entry point for external model consultation.
Wraps Claude API calls through the offering protocol so external
output is treated as korban (raw material), never authoritative:
- semichah — provenance is tagged (avodah zarah Form 1 prevention)
- shechitah — assistant voice, hedging, formatting stripped
- zerikah — vital content extracted
- haktarah — converted to Prolog propositions
- route — sent to internal component per offering type
The raw Korban is preserved on the ProcessedKorban (gilui arayot
Form 3 prevention — input/output coupling never severed).
No internal component should call call_claude() directly.
11 tests, mocked external calls. Live API test opt-in via
SHORASHIM_LIVE_API=1.
## Step 24 — L2 Urim v'Tumim interface
(vav_pipeline/lib/vav_pipeline/urim_v_tumim.ex)
PRD §4.7: housed in the kohen gadol's breastplate; consulted only
on high-stakes decisions where normal deliberation is insufficient.
Bamidbar 27:21 — al-piv yetz'u v'al-piv yavo'u.
Three escalation steps, three honest outcomes:
1. resolved_by_specification — re-read produced an answer
2. resolved_by_external — external testimony recorded (NOT binding)
3. deferred_to_human — the system stops, asks for human judgment
Pre-condition gating refuses cases that didn't reach Sanhedrin teiku
first (no preempting deliberation) or that aren't flagged high-stakes
(ordinary teiku stays teiku). Records every consultation with steps
taken — there is no quiet escalation.
Per PRD: "The Urim v'Tumim never fabricates certainty where none
exists." When external also fails, the system stops. It does not
proceed on a guess.
Added to VavPipeline supervision tree. 14 tests.
## Step 21 — Kushya generator scaffolding
(kushya_generator/)
Directory layout per Architecture §3.6 with three layers:
- data/training_pairs/seed_pairs.jsonl — 10 hand-curated bootstrap
pairs covering 9 kushya types (scope_narrowing, internal_tension,
harmonization, exception_test, edge_case, scope_application,
rabbinic_constraint, method_question, subject_identification)
- data/prepare_training.py — validates the four kushya requirements
(specific Torah passage, identified weakness, question form,
non-trivial), produces train.jsonl + eval.jsonl with stratified
80/20 split by type
- fine_tune/config.yaml + train.py — QLoRA configuration and
training script. Operator's task — needs GPU. Script is
complete and self-contained.
- inference/kushya_api.py — three-backend interface
(fine_tuned → external → fallback). When fine-tuned model isn't
deployed, falls back to external via Korban Processor (Step 22).
The fallback is honest: returns no kushya rather than inventing
one. The Talmudic Engine's rule-based challenge.pl remains the
primary kushya source.
17 tests covering dataset validation, backend selection, and
reference extraction.
## .gitignore fix
The earlier "data/" line was too broad — it caught
kushya_generator/data/. Scoped to /data/ (root-level only).
## Test counts
- Elixir: 274 (was 260; +14 for UrimVTumim)
- Korban Python: 196 + 1 skipped (was 195; +1 module = 11 tests)
- Kushya Python: 17 (new)
- Haskell: 33 (unchanged)
- Prolog: 195 (unchanged)
- Total: 715 + 19 L3 excluded
## Phase 4 status
All 12 steps now have implementations:
- Steps 23, 25, 26, 27, 28, 29, 30, 31, 32: complete
- Steps 21, 22, 24: complete with discipline preserved
Step 21's fine-tuning run remains the operator's GPU task; the
pipeline is built such that a deployed adapter slots into the
inference backend without touching the Talmudic Engine or anything
above.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
67 lines
1.6 KiB
YAML
67 lines
1.6 KiB
YAML
# QLoRA fine-tuning configuration for the kushya generator.
|
|
# Architecture §3.6.
|
|
|
|
base_model:
|
|
name: mistralai/Mistral-Small-Instruct-2409 # Mistral Medium 3.1 weights when available
|
|
trust_remote_code: false
|
|
load_in_4bit: true
|
|
bnb_4bit_compute_dtype: bfloat16
|
|
bnb_4bit_quant_type: nf4
|
|
bnb_4bit_use_double_quant: true
|
|
|
|
lora:
|
|
r: 32
|
|
alpha: 64
|
|
dropout: 0.05
|
|
bias: none
|
|
task_type: CAUSAL_LM
|
|
target_modules:
|
|
- q_proj
|
|
- k_proj
|
|
- v_proj
|
|
- o_proj
|
|
- gate_proj
|
|
- up_proj
|
|
- down_proj
|
|
|
|
training:
|
|
output_dir: ./adapter
|
|
num_train_epochs: 3
|
|
per_device_train_batch_size: 1
|
|
per_device_eval_batch_size: 1
|
|
gradient_accumulation_steps: 16
|
|
gradient_checkpointing: true
|
|
optim: paged_adamw_8bit
|
|
learning_rate: 2.0e-4
|
|
lr_scheduler_type: cosine
|
|
warmup_ratio: 0.03
|
|
weight_decay: 0.0
|
|
max_grad_norm: 0.3
|
|
fp16: false
|
|
bf16: true
|
|
logging_steps: 10
|
|
evaluation_strategy: steps
|
|
eval_steps: 50
|
|
save_strategy: epoch
|
|
save_total_limit: 2
|
|
group_by_length: true
|
|
report_to: none
|
|
seed: 42
|
|
|
|
data:
|
|
train_file: ../data/train.jsonl
|
|
eval_file: ../data/eval.jsonl
|
|
max_seq_length: 1024
|
|
prompt_template: |
|
|
[INST] You are a Talmudic challenger. Your single function is to
|
|
generate the strongest possible kushya (challenge) to a proposed
|
|
Torah interpretation. Your kushya MUST:
|
|
1. Reference a specific Torah passage that creates tension
|
|
2. Identify the logical or textual weakness
|
|
3. Be expressed as a question, not a statement
|
|
4. Not be resolvable by trivial clarification
|
|
|
|
{input}
|
|
[/INST]
|
|
{output}
|