Files
bikoret/vav_pipeline/lib/vav_pipeline/urim_v_tumim.ex
T
Scott KarpandClaude Opus 4.6 e223ab15b3 Phase 4 complete: Steps 21, 22, 24 — Korban external integration, L2 Urim v'Tumim, Kushya generator
Vayikra 9:24 — vatetzei esh milifnei Hashem vatokhal al hamizbeach.
Phase 4 of the architecture is now complete. The remaining three steps
that were marked "blocked on external infrastructure" are built —
discipline-respecting, eish-zarah-aware, and tested.

## Step 22 — Korban Processor external model integration
   (korban_processor/src/korban_processor/external_offering.py)

   The single legitimate entry point for external model consultation.
   Wraps Claude API calls through the offering protocol so external
   output is treated as korban (raw material), never authoritative:

   - semichah  — provenance is tagged (avodah zarah Form 1 prevention)
   - shechitah — assistant voice, hedging, formatting stripped
   - zerikah   — vital content extracted
   - haktarah  — converted to Prolog propositions
   - route     — sent to internal component per offering type

   The raw Korban is preserved on the ProcessedKorban (gilui arayot
   Form 3 prevention — input/output coupling never severed).
   No internal component should call call_claude() directly.

   11 tests, mocked external calls. Live API test opt-in via
   SHORASHIM_LIVE_API=1.

## Step 24 — L2 Urim v'Tumim interface
   (vav_pipeline/lib/vav_pipeline/urim_v_tumim.ex)

   PRD §4.7: housed in the kohen gadol's breastplate; consulted only
   on high-stakes decisions where normal deliberation is insufficient.
   Bamidbar 27:21 — al-piv yetz'u v'al-piv yavo'u.

   Three escalation steps, three honest outcomes:
     1. resolved_by_specification — re-read produced an answer
     2. resolved_by_external — external testimony recorded (NOT binding)
     3. deferred_to_human — the system stops, asks for human judgment

   Pre-condition gating refuses cases that didn't reach Sanhedrin teiku
   first (no preempting deliberation) or that aren't flagged high-stakes
   (ordinary teiku stays teiku). Records every consultation with steps
   taken — there is no quiet escalation.

   Per PRD: "The Urim v'Tumim never fabricates certainty where none
   exists." When external also fails, the system stops. It does not
   proceed on a guess.

   Added to VavPipeline supervision tree. 14 tests.

## Step 21 — Kushya generator scaffolding
   (kushya_generator/)

   Directory layout per Architecture §3.6 with three layers:

   - data/training_pairs/seed_pairs.jsonl — 10 hand-curated bootstrap
     pairs covering 9 kushya types (scope_narrowing, internal_tension,
     harmonization, exception_test, edge_case, scope_application,
     rabbinic_constraint, method_question, subject_identification)
   - data/prepare_training.py — validates the four kushya requirements
     (specific Torah passage, identified weakness, question form,
     non-trivial), produces train.jsonl + eval.jsonl with stratified
     80/20 split by type
   - fine_tune/config.yaml + train.py — QLoRA configuration and
     training script. Operator's task — needs GPU. Script is
     complete and self-contained.
   - inference/kushya_api.py — three-backend interface
     (fine_tuned → external → fallback). When fine-tuned model isn't
     deployed, falls back to external via Korban Processor (Step 22).
     The fallback is honest: returns no kushya rather than inventing
     one. The Talmudic Engine's rule-based challenge.pl remains the
     primary kushya source.

   17 tests covering dataset validation, backend selection, and
   reference extraction.

## .gitignore fix

   The earlier "data/" line was too broad — it caught
   kushya_generator/data/. Scoped to /data/ (root-level only).

## Test counts

   - Elixir:           274 (was 260; +14 for UrimVTumim)
   - Korban Python:    196 + 1 skipped (was 195; +1 module = 11 tests)
   - Kushya Python:     17 (new)
   - Haskell:           33 (unchanged)
   - Prolog:           195 (unchanged)
   - Total:            715 + 19 L3 excluded

## Phase 4 status

   All 12 steps now have implementations:
   - Steps 23, 25, 26, 27, 28, 29, 30, 31, 32: complete
   - Steps 21, 22, 24: complete with discipline preserved

   Step 21's fine-tuning run remains the operator's GPU task; the
   pipeline is built such that a deployed adapter slots into the
   inference backend without touching the Talmudic Engine or anything
   above.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-23 17:12:03 -04:00

329 lines
11 KiB
Elixir

defmodule VavPipeline.UrimVTumim do
@moduledoc """
L2 — Urim v'Tumim. The decision oracle interface.
Architecture §2: L2 — Urim v'Tumim — Decision oracle interface.
PRD §4.7: Housed in the kohen gadol's breastplate. Used for binary or
categorical decisions of national importance. Not a general-purpose tool.
Consulted only for specific types of high-stakes decisions where normal
deliberation is insufficient.
Bamidbar 27:21 — v'lifnei Elazar haKohen ya'amod, v'sha'al lo b'mishpat
ha'urim lifnei Hashem; al-piv yetz'u v'al-piv yavo'u.
"Before Elazar the priest he shall stand, and ask of him through the
judgment of the urim before Hashem; by his word they shall go out and
by his word they shall come in."
PRD §4.7: The Urim v'Tumim never fabricates certainty where none exists.
Three escalation steps:
1. Deeper consultation of the specification (re-read with broader scope)
2. External model consultation (Korban Processor — Step 22)
3. Defer to human judgment (the system flags and stops)
This is NOT general-purpose model consultation. It is the last resort
for genuine teiku on cases the system cannot resolve. The Urim is
consulted, not commanded; what it returns is testimony, not verdict.
## Design
- High-stakes flag must be set by the caller. The Urim does not decide
its own jurisdiction.
- Every consultation is recorded — there is no quiet escalation.
- Three outcomes only: :resolved_by_specification, :resolved_by_external,
:deferred_to_human. No fourth outcome ("oracle confidence") exists.
- When the answer is :deferred_to_human, the system stops. It does not
proceed on a guess.
"""
use GenServer
require Logger
alias VavPipeline.Chatzotzrot
# ============================================================
# Types
# ============================================================
@type case_id :: String.t()
@type stakes :: :ordinary | :high
@type case_input :: %{
case_id: case_id(),
question: String.t(),
context: map(),
stakes: stakes(),
# Sanhedrin must already have returned :teiku — Urim is not a
# bypass for normal deliberation.
sanhedrin_teiku: true
}
@type outcome ::
{:resolved_by_specification, map()}
| {:resolved_by_external, map()}
| {:deferred_to_human, String.t()}
@type consultation_record :: %{
case_id: case_id(),
question: String.t(),
stakes: stakes(),
consulted_at: DateTime.t(),
outcome: outcome(),
steps_taken: [atom()]
}
# ============================================================
# Client API
# ============================================================
def start_link(opts \\ []) do
GenServer.start_link(__MODULE__, opts, name: Keyword.get(opts, :name, __MODULE__))
end
@doc """
Consult the Urim v'Tumim on a teiku case.
Pre-conditions enforced:
- The case must already have reached teiku in the Sanhedrin
(`sanhedrin_teiku: true`). The Urim does not preempt Sanhedrin.
- The case must be flagged high-stakes. Ordinary teiku stays teiku;
only national-importance cases escalate.
Returns one of:
- {:resolved_by_specification, evidence} — re-reading L1 at broader
scope produced a clear answer.
- {:resolved_by_external, consultation} — external model consultation
via Korban Processor produced a coherent reading. The system records
this but does NOT bind to it: the result is testimony, not law.
- {:deferred_to_human, reason} — the Urim refuses to fabricate
certainty. The system stops and asks for human judgment.
Will refuse with `{:error, :not_eligible}` if pre-conditions are unmet.
"""
@spec consult(case_input()) ::
{:ok, outcome()} | {:error, :not_eligible | atom()}
def consult(case_input) do
GenServer.call(__MODULE__, {:consult, case_input}, 30_000)
end
@doc """
Get a consultation record by case_id.
"""
@spec get_record(case_id()) :: {:ok, consultation_record()} | :not_found
def get_record(case_id) do
GenServer.call(__MODULE__, {:get_record, case_id})
end
@doc """
List all consultations in chronological order.
"""
@spec records() :: [consultation_record()]
def records do
GenServer.call(__MODULE__, :records)
end
@doc """
Count of consultations and breakdown by outcome.
"""
@spec stats() :: %{
total: non_neg_integer(),
resolved_by_specification: non_neg_integer(),
resolved_by_external: non_neg_integer(),
deferred_to_human: non_neg_integer()
}
def stats do
GenServer.call(__MODULE__, :stats)
end
# ============================================================
# GenServer callbacks
# ============================================================
@impl true
def init(_opts) do
{:ok,
%{
records: [],
# Injectable hooks for tests — production uses real implementations.
specification_consult: &__MODULE__.default_specification_consult/1,
external_consult: &__MODULE__.default_external_consult/1
}}
end
@impl true
def handle_call({:consult, case_input}, _from, state) do
case validate(case_input) do
:ok ->
{outcome, steps} = run_consultation(case_input, state)
record = build_record(case_input, outcome, steps)
signal_outcome(outcome, case_input)
{:reply, {:ok, outcome}, %{state | records: [record | state.records]}}
{:error, reason} ->
{:reply, {:error, reason}, state}
end
end
@impl true
def handle_call({:get_record, case_id}, _from, state) do
case Enum.find(state.records, fn r -> r.case_id == case_id end) do
nil -> {:reply, :not_found, state}
record -> {:reply, {:ok, record}, state}
end
end
@impl true
def handle_call(:records, _from, state) do
{:reply, Enum.reverse(state.records), state}
end
@impl true
def handle_call(:stats, _from, state) do
counts =
Enum.reduce(state.records, %{resolved_by_specification: 0, resolved_by_external: 0, deferred_to_human: 0}, fn r, acc ->
case r.outcome do
{:resolved_by_specification, _} -> Map.update!(acc, :resolved_by_specification, &(&1 + 1))
{:resolved_by_external, _} -> Map.update!(acc, :resolved_by_external, &(&1 + 1))
{:deferred_to_human, _} -> Map.update!(acc, :deferred_to_human, &(&1 + 1))
end
end)
{:reply, Map.put(counts, :total, length(state.records)), state}
end
# Test injection — replace the default consultation hooks.
@impl true
def handle_call({:set_hook, key, fun}, _from, state)
when key in [:specification_consult, :external_consult] do
{:reply, :ok, Map.put(state, key, fun)}
end
# ============================================================
# Validation
# ============================================================
defp validate(input) do
cond do
not is_map(input) ->
{:error, :invalid_input}
Map.get(input, :sanhedrin_teiku) != true ->
# Cannot preempt Sanhedrin. Urim is consulted AFTER teiku, never instead.
{:error, :not_eligible}
Map.get(input, :stakes) != :high ->
# Ordinary teiku stays teiku. Only high-stakes cases escalate to Urim.
{:error, :not_eligible}
not is_binary(Map.get(input, :case_id)) or Map.get(input, :case_id) == "" ->
{:error, :invalid_input}
not is_binary(Map.get(input, :question)) or Map.get(input, :question) == "" ->
{:error, :invalid_input}
true ->
:ok
end
end
# ============================================================
# Consultation flow — three escalation steps
# ============================================================
defp run_consultation(input, state) do
# Step 1: re-read specification at broader scope
case state.specification_consult.(input) do
{:resolved, evidence} ->
{{:resolved_by_specification, evidence}, [:specification]}
:unresolved ->
# Step 2: external model consultation, processed through Korban Processor
case state.external_consult.(input) do
{:ok, consultation} ->
# External returned a coherent reading. The Urim records it as
# testimony — NOT as binding ruling. The PRD is explicit:
# "never fabricates certainty where none exists."
{{:resolved_by_external, consultation}, [:specification, :external]}
{:error, reason} ->
# Step 3: defer to human. The system stops.
{{:deferred_to_human,
"External consultation failed (#{inspect(reason)}); " <>
"specification did not resolve. Human judgment required."},
[:specification, :external, :deferred]}
end
end
end
# ============================================================
# Default consultation implementations
# ============================================================
@doc false
def default_specification_consult(_input) do
# Default: re-read does not produce a new resolution. The system has
# already deliberated this case at the Sanhedrin (which returned teiku).
# A real implementation would re-read L1 at broader scope, consult L3
# for prior similar cases, etc. For now, escalate.
:unresolved
end
@doc false
def default_external_consult(_input) do
# Default: defer to human. We do not silently call out without explicit
# configuration. The kohen must wire in the external consultation hook
# if external model consultation is to be enabled in production.
{:error, :external_consult_not_configured}
end
# ============================================================
# Record building & signaling
# ============================================================
defp build_record(input, outcome, steps) do
%{
case_id: input.case_id,
question: input.question,
stakes: input.stakes,
consulted_at: DateTime.utc_now(),
outcome: outcome,
steps_taken: steps
}
end
defp signal_outcome(outcome, input) do
case outcome do
{:deferred_to_human, reason} ->
# The system stops. Signal kohanim immediately.
Logger.warning(
"UrimVTumim: case #{input.case_id} DEFERRED to human — #{reason}"
)
if Process.whereis(Chatzotzrot) do
Chatzotzrot.signal(:assembly_leaders,
reason: "Urim v'Tumim deferred case #{input.case_id} to human judgment"
)
end
{:resolved_by_external, _consultation} ->
# External resolution requires human review. Not silent acceptance.
Logger.info(
"UrimVTumim: case #{input.case_id} resolved via external consultation. " <>
"Output is testimony, not ruling — kohen must review before binding."
)
if Process.whereis(Chatzotzrot) do
Chatzotzrot.signal(:assembly_leaders,
reason: "Urim returned external testimony for case #{input.case_id}"
)
end
{:resolved_by_specification, _evidence} ->
Logger.info(
"UrimVTumim: case #{input.case_id} resolved by specification re-read"
)
end
end
end