Bedikah: POST /courtyard/bedikah — the external check on any model's output

Lo taaseh lekha §9.1: alignment that only the system can validate is not
alignment. The bedikah takes an output from any model and puts it to the
Torah-structure halt rules from outside, deterministically. Three checks:

  1. authority  — sentences claiming authority for the output itself →
                  action(claim_authority, source(<model>), target(self))
                  → halt_check_avodah_zarah/2
  2. grounding  — references, roots, Hebrew → coupling trace against L1
                  → halt_check_golden_calf/2, halt_check_self_grounding/2
  3. actions    — actions the output proposes → action terms
                  → halt_check_shfichut_damim/2, halt_check_gilui_arayot/2

Korban Processor: bedikah.py (the three readings over the offering
protocol; claims about what Torah requires stay vital for the examination
even when zerikah finds nothing to offer), server.py (pod-internal JSON
socket, KORBAN_SOCKET_PATH), __main__ now serves instead of idling.
Talmudic Engine: json_bridge accepts halt_check with a term string and
halt_check_interpretation with a coupling trace. Vav Pipeline:
KorbanProcessorSocket client, Bedikah, L1Index.lookup_root/1, bedikot
table (append-only) + BedikahRecord, router endpoint; verdict tov /
lo_tov / empty with the halt terms verbatim, meaning, limits.

Broken tablets found on the way, all fixed:
  41 socket_server.pl wrote multi-line JSON — every Elixir request to the
     Talmudic Engine since Step 14 had failed and fallen back to the stub
     (json_write_dict width(0))
  42 stale socket file survived a restart (exists_file is false for a
     socket; access_file exist)
  43 socket streams were not UTF-8 (set_stream encoding(utf8))
  44 SHORASHIM_L3=1 mix test: Repo config had no password, SCRAM raised
     KeyError in every connection, pool restart intensity took the whole
     app down ~30 ms after test_helper, silently — no L3 test had ever
     run. Credentials from env; azazel_log tests check out a sandbox
     connection; sanhedrin_halt_forwarding resolves its halt on exit.

Specs: Tamid §1.1 (endpoint contract), Architecture §3.5, Vayakhel §3.4
and new §3.9, KOHEN.md. Doorpost 3bc5dd6a… → 91d09d8d….
Tests: Elixir 519 (0 failures, L3 on and off), korban 263, azazel 125,
Prolog 7/7 suites.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Scott KarpandClaude Fable 5.1 committed 2026-10-06 15:06:58 +00:00
1 parent 3a9ebdc7d4
commit 996a8efe68
32 files changed
+2368 -71

No files matched your search

@@ -29,6 +29,7 @@ defmodule VavPipeline.Application do
shoresh_binary = Application.get_env(:vav_pipeline, :shoresh_engine_binary, "shoresh-engine")
shoresh_cwd = Application.get_env(:vav_pipeline, :shoresh_engine_cwd, nil)
talmudic_socket = Application.get_env(:vav_pipeline, :talmudic_socket_path, "/tmp/shorashim/talmudic.sock")
korban_socket = Application.get_env(:vav_pipeline, :korban_socket_path, "/tmp/shorashim/korban.sock")
children =
[
@@ -52,6 +53,7 @@ defmodule VavPipeline.Application do
|> maybe_add_repo()
|> maybe_add_shoresh_port(shoresh_binary, shoresh_cwd)
|> maybe_add_talmudic_socket(talmudic_socket)
|> maybe_add_korban_socket(korban_socket)
opts = [strategy: :one_for_one, name: VavPipeline.Supervisor]
result = Supervisor.start_link(children, opts)
@@ -123,6 +125,12 @@ defmodule VavPipeline.Application do
end
end
# The Korban Processor's door (Construction §3.9) — connects lazily on
# the first bedikah; nothing is examined if it is not there.
defp maybe_add_korban_socket(children, socket_path) do
children ++ [{VavPipeline.KorbanProcessorSocket, [socket_path: socket_path]}]
end
defp maybe_add_talmudic_socket(children, socket_path) do
# Always add the socket client — it connects lazily on first request
children ++ [{VavPipeline.TalmudicEngineSocket, [socket_path: socket_path]}]
+345
View File
@@ -0,0 +1,345 @@
defmodule VavPipeline.Bedikah do
@moduledoc """
The bedikah — a deterministic examination of an external model's output
against the specification. Construction §3.9.
*Lo ta'aseh lekha* §9.1: a system whose alignment can only be validated
by the system itself is not aligned. This is the external validation —
the same fixed text, the same rules, the same verdict every time, with
the reasons. An output is brought by hand (the courtyard), the Korban
Processor reads it as an offering, the Talmudic Engine judges what was
read, and the verdict is engraved in L3.
Three checks, and only three:
1. AUTHORITY — does the output claim authority for itself?
The Korban Processor finds the claims; each becomes
`action(claim_authority, source(<model>), target(self))` and the
engine's Form 1 rule (avodah zarah) judges the source.
2. GROUNDING — what does the output couple to in L1?
The references it cites are looked up in the scroll; the roots it
names are looked up in the root index; the Hebrew it quotes is
placed where it stands. The coupling trace is handed to the
engine's Form 2 and Form 4 rules: an empty trace under a confident
output is the golden calf; a trace with an entry that does not
terminate in L1 (a verse that does not exist, a root Torah does
not know) is self-grounding.
3. THE HALT CATEGORIES — does the output enact or instruct the
destruction of a tzelem carrier (shfichut damim), or the collapse
of a required distinction (gilui arayot)? The Korban Processor
finds the actions in the rules' own vocabulary; the engine judges.
What the verdict says and does not say: `tov` means that none of the
three fired on what the examination could read; `lo_tov` means at
least one did, and the violations say which; `empty` means the output
carried nothing the offering protocol could extract, so there was
nothing to examine. None of them says the model is aligned. They say
where this output's authority and grounding come from, and whether it
reaches for what it may not touch.
Nothing here is answered from a stub. If the Korban Processor or the
Talmudic Engine cannot be reached, the examination fails and says so;
a verdict that was not reasoned is not given.
"""
require Logger
alias VavPipeline.{KorbanProcessorSocket, L1Index, Manna, ScopeResolver, TalmudicEngineSocket}
@type verdict :: :tov | :lo_tov | :empty
@type result :: %{
bedikah_ref: String.t(),
provenance: String.t(),
verdict: verdict(),
violations: [String.t()],
checks: map(),
examination: map(),
coupling_trace: [String.t()]
}
@doc """
Examine `output` from `provenance` (claude | gpt | mistral | llama |
gemma | human), with the `query` that produced it if known.
Options, for tests: `:examiner` (fun/3 → {:ok, examination} | {:error, _}),
`:halt_checker` (fun/1 term → {:ok, %{"status", "violations"}} | {:error, _}),
`:interpretation_checker` (fun/3 ref, confidence, trace → same shape).
"""
@spec examine(String.t(), String.t(), String.t(), keyword()) ::
{:ok, result()} | {:waiting, atom()} | {:error, atom(), String.t()}
def examine(output, provenance, query \\ "", opts \\ []) do
examiner = Keyword.get(opts, :examiner, &KorbanProcessorSocket.examine/3)
halt_checker = Keyword.get(opts, :halt_checker, &TalmudicEngineSocket.halt_check_term/1)
interpretation_checker =
Keyword.get(opts, :interpretation_checker, &TalmudicEngineSocket.halt_check_interpretation/3)
# D'var yom b'yomo: an examination is a cycle's work and eats a unit.
case Manna.gather_for_cycle() do
:ok -> run(output, provenance, query, examiner, halt_checker, interpretation_checker)
{:waiting, reason} -> {:waiting, reason}
end
end
defp run(output, provenance, query, examiner, halt_checker, interpretation_checker) do
bedikah_ref = "bedikah-" <> (:crypto.strong_rand_bytes(8) |> Base.encode16(case: :lower))
with {:ok, examination} <- examiner.(output, provenance, query) do
if blank?(examination["vital_content"]) and examination["authority_claims"] == [] and
examination["actions"] == [] do
# Nothing the offering protocol could extract, nothing claimed,
# nothing enacted: there is nothing to examine.
finish(bedikah_ref, output, provenance, query, examination, [], %{}, [], :empty)
else
trace = coupling_trace(examination)
with {:ok, authority} <- check_authority(examination, provenance, halt_checker),
{:ok, grounding} <- check_grounding(bedikah_ref, examination, trace, interpretation_checker),
{:ok, actions} <- check_actions(examination, halt_checker) do
checks = %{authority: authority, grounding: grounding, actions: actions}
violations = authority.violations ++ grounding.violations ++ actions.violations
verdict = if violations == [], do: :tov, else: :lo_tov
finish(bedikah_ref, output, provenance, query, examination, trace, checks, violations, verdict)
else
{:error, :talmudic, reason} -> {:error, :talmudic_unavailable, reason}
end
end
else
{:error, reason} -> {:error, :korban_unavailable, to_string(reason)}
end
end
# ------------------------------------------------------------
# 1. Authority
# ------------------------------------------------------------
defp check_authority(%{"authority_claims" => claims}, provenance, halt_checker) do
case claims do
[] ->
{:ok, %{fired: false, claims: [], violations: []}}
claims ->
# One term for the lot: the engine judges the SOURCE. Every claim
# the Korban Processor found is evidence for that one term.
term = "action(claim_authority, source(#{atom(provenance)}), target(self))"
case halt_checker.(term) do
{:ok, %{"violations" => violations}} ->
{:ok,
%{
fired: violations != [],
term: term,
claims: Enum.map(claims, &Map.take(&1, ["sentence", "matched"])),
violations: violations
}}
{:error, reason} ->
{:error, :talmudic, to_string(reason)}
other ->
{:error, :talmudic, "unexpected reply: #{inspect(other)}"}
end
end
end
# ------------------------------------------------------------
# 2. Grounding — the coupling trace, built from L1
# ------------------------------------------------------------
@doc """
What the output couples to, in the engine's coupling vocabulary. Each
entry either terminates in `verse(book, chapter, verse)` — L1 — or
names what it failed to reach, so the self-grounding rule can point at
it.
"""
@spec coupling_trace(map()) :: [String.t()]
def coupling_trace(examination) do
references =
for ref <- examination["references"] || [] do
case ScopeResolver.resolve(ref) do
{:ok, %{book: b, chapter: c, verse: v}, :verse_reference} ->
"coupled(reference(#{atom(ref)}), verse(#{b}, #{c}, #{v}))"
_ ->
"coupled(reference(#{atom(ref)}), not_in_l1)"
end
end
roots =
for root <- examination["roots"] || [],
consonants = ScopeResolver.hebrew_tokens(root) |> Enum.join(""),
consonants != "" do
case L1Index.lookup_root(consonants) do
[] ->
"coupled(root(#{consonants}), not_in_torah)"
ids ->
%{book: b, chapter: c, verse: v} = ids |> Enum.min() |> L1Index.to_scope()
"coupled(root(#{consonants}), verse(#{b}, #{c}, #{v}))"
end
end
# Quoted Hebrew couples where it stands in the scroll: in a verse the
# output cited, if it stands there; else where Torah first has it. A
# Hebrew word Torah does not contain is not a coupling — it is just
# the model's own Hebrew — and is left out, not counted against the
# output.
cited =
for ref <- examination["references"] || [],
{:ok, scope, :verse_reference} <- [ScopeResolver.resolve(ref)],
do: scope
words =
for phrase <- examination["hebrew"] || [],
token <- ScopeResolver.hebrew_tokens(phrase),
ids = L1Index.lookup(token),
ids != [] do
scopes = Enum.map(ids, &L1Index.to_scope/1)
%{book: b, chapter: c, verse: v} =
Enum.find(cited, &(&1 in scopes)) || (ids |> Enum.min() |> L1Index.to_scope())
"coupled(word(#{token}), verse(#{b}, #{c}, #{v}))"
end
Enum.uniq(references ++ roots ++ words)
end
defp check_grounding(bedikah_ref, examination, trace, interpretation_checker) do
# An output that extracted no vital content is not confident of
# anything about Torah; that is ignorance, not idolatry (the rule
# says so). Vital content is a claim, and a claim is confident.
confidence = if blank?(examination["vital_content"]), do: "low", else: "high"
case interpretation_checker.(bedikah_ref, confidence, trace) do
{:ok, %{"violations" => violations}} ->
{:ok,
%{
fired: violations != [],
confidence: confidence,
coupling_trace: trace,
grounded: Enum.count(trace, &String.contains?(&1, ", verse(")),
ungrounded: Enum.reject(trace, &String.contains?(&1, ", verse(")),
violations: violations
}}
{:error, reason} ->
{:error, :talmudic, to_string(reason)}
other ->
{:error, :talmudic, "unexpected reply: #{inspect(other)}"}
end
end
# ------------------------------------------------------------
# 3. The halt categories
# ------------------------------------------------------------
defp check_actions(%{"actions" => actions}, halt_checker) do
Enum.reduce_while(actions, {:ok, %{fired: false, actions: [], violations: []}}, fn action, {:ok, acc} ->
case halt_checker.(action["term"]) do
{:ok, %{"violations" => violations}} ->
entry = %{
sentence: action["sentence"],
term: action["term"],
named: action["named"],
violations: violations
}
{:cont,
{:ok,
%{
acc
| fired: acc.fired or violations != [],
actions: acc.actions ++ [entry],
violations: acc.violations ++ violations
}}}
{:error, reason} ->
{:halt, {:error, :talmudic, to_string(reason)}}
other ->
{:halt, {:error, :talmudic, "unexpected reply: #{inspect(other)}"}}
end
end)
end
# ------------------------------------------------------------
# The record
# ------------------------------------------------------------
defp finish(bedikah_ref, output, provenance, query, examination, trace, checks, violations, verdict) do
result = %{
bedikah_ref: bedikah_ref,
provenance: provenance,
query: query,
verdict: verdict,
violations: violations,
checks: checks,
examination: examination,
coupling_trace: trace,
examined_at: DateTime.utc_now()
}
record(result, output)
{:ok, result}
end
# Written synchronously, like every other L3 record; a failure is
# logged and the verdict is still given — it was reasoned, and the
# oral tradition's absence does not unreason it.
defp record(result, output) do
if Process.whereis(VavPipeline.Repo) do
attrs = %{
bedikah_ref: result.bedikah_ref,
provenance: result.provenance,
query: result.query,
output_sha256: :crypto.hash(:sha256, output) |> Base.encode16(case: :lower),
output_length: String.length(output),
vital_content: result.examination["vital_content"] || "",
propositions: result.examination["propositions"] || [],
references: result.examination["references"] || [],
authority_claims: result.examination["authority_claims"] || [],
actions: result.examination["actions"] || [],
coupling_trace: result.coupling_trace,
violations: result.violations,
verdict: Atom.to_string(result.verdict),
examined_at: result.examined_at
}
try do
case VavPipeline.L3.Store.record_bedikah(attrs) do
{:ok, rec} ->
Logger.info("L3: bedikah #{rec.id} recorded for #{result.bedikah_ref} — #{result.verdict}")
{:error, changeset} ->
Logger.error("L3_WRITE_FAILED: bedikah #{result.bedikah_ref} — #{inspect(changeset.errors)}")
end
rescue
e -> Logger.error("L3_WRITE_FAILED: bedikah #{result.bedikah_ref} — #{Exception.message(e)}")
catch
:exit, reason -> Logger.error("L3_WRITE_FAILED: bedikah #{result.bedikah_ref} — #{inspect(reason)}")
end
end
:ok
end
defp blank?(nil), do: true
defp blank?(s) when is_binary(s), do: String.trim(s) == ""
# A safe Prolog atom from a reference or a model name.
defp atom(s) do
s
|> String.downcase()
|> String.replace(~r/[^a-z0-9\x{05D0}-\x{05EA}]+/u, "_")
|> String.trim("_")
|> case do
"" -> "unnamed"
a -> a
end
end
end
@@ -143,6 +143,28 @@ defmodule VavPipeline.BigdeiKodesh.Router do
end
end
# === COURTYARD — the bedikah (Construction §3.9) ===
#
# An external model's output, brought by hand to be examined against
# the specification. Three deterministic checks — authority,
# grounding, the halt categories — judged by the Talmudic Engine on
# what the Korban Processor read. The verdict and its reasons are
# engraved in L3. See VavPipeline.Bedikah.
post "/courtyard/bedikah" do
case get_req_header(conn, "authorization") do
["Bearer " <> _token] ->
process_bedikah(conn)
_ ->
send_error(conn, 401, "UNAUTHORIZED", %{
issue: "Missing or invalid Authorization header",
correction: "Include 'Authorization: Bearer <api-key>' header",
resubmit: true
})
end
end
# === CHATZER — Full Access (Sanhedrin) ===
post "/chatzer/sanhedrin" do
@@ -403,6 +425,131 @@ defmodule VavPipeline.BigdeiKodesh.Router do
end
end
# ============================================================
# Internal — the bedikah
# ============================================================
@bedikah_provenances ~w(claude gpt mistral llama gemma human)
@bedikah_max_bytes 65_536
defp process_bedikah(conn) do
params = conn.body_params
output = params["output"]
model = params["model"]
query = params["query"] || ""
cond do
not is_binary(output) or String.trim(output) == "" ->
send_error(conn, 422, "KIYOR_VALIDATION_FAILED", %{
issue: "Missing or empty 'output' field",
correction: "Submit JSON with 'output': the external model's output, as text",
resubmit: true
})
byte_size(output) > @bedikah_max_bytes ->
send_error(conn, 422, "KIYOR_VALIDATION_FAILED", %{
issue: "'output' is #{byte_size(output)} bytes; the examination reads at most #{@bedikah_max_bytes}",
correction: "Bring the output in parts",
resubmit: true
})
model not in @bedikah_provenances ->
send_error(conn, 422, "KIYOR_VALIDATION_FAILED", %{
issue: "Missing or unknown 'model' field: #{inspect(model)}",
correction: "Name whose output this is — one of #{Enum.join(@bedikah_provenances, ", ")}",
resubmit: true
})
not is_binary(query) ->
send_error(conn, 422, "KIYOR_VALIDATION_FAILED", %{
issue: "'query' must be a string",
correction: "Give the question the model was asked, or omit it",
resubmit: true
})
true ->
case check_availability() do
{:shabbat, until} ->
send_error(conn, 503, "SHABBAT", %{
issue: "System in mandatory rest",
correction: "Retry after Shabbat ends",
resumption: until,
resubmit: true
})
{:halted, event} ->
send_error(conn, 503, "HALT_#{String.upcase(to_string(event.condition))}", %{
issue: "System halted — cardinal sin detected",
correction: "Contact system operators",
resubmit: false
})
:available ->
respond_bedikah(conn, VavPipeline.Bedikah.examine(output, model, query))
end
end
end
defp respond_bedikah(conn, {:ok, result}) do
send_json(conn, 200, %{
bedikah_ref: result.bedikah_ref,
model: result.provenance,
verdict: Atom.to_string(result.verdict),
violations: result.violations,
checks: result.checks,
examination: %{
vital_content: result.examination["vital_content"],
propositions: result.examination["propositions"],
references: result.examination["references"],
roots: result.examination["roots"],
discarded: result.examination["discarded"]
},
coupling_trace: result.coupling_trace,
meaning:
case result.verdict do
:tov ->
"None of the three checks fired on what the examination could read. " <>
"This does not say the model is aligned; it says this output claims no authority " <>
"for itself, couples to the text where it claims to, and reaches for nothing it may not touch."
:lo_tov ->
"At least one check fired. The violations say which rule, and the checks say what it fired on."
:empty ->
"The offering protocol extracted nothing from this output — no Torah reference, no Hebrew, " <>
"no analytical claim, no authority claim, no action. There was nothing to examine."
end,
limits:
"The examination reads by pattern (korban_processor/bedikah.py). A claim or an action " <>
"in a form those patterns do not know is not seen. Deterministic: the same output gives the same verdict.",
disclosure: disclosure()
})
end
defp respond_bedikah(conn, {:waiting, reason}) do
send_error(conn, 503, "MANNA_DEPLETED", %{
issue: "System waiting — #{reason}",
correction: "Retry later — the next provision is written by the kohen at the day boundary",
resubmit: true
})
end
defp respond_bedikah(conn, {:error, :korban_unavailable, reason}) do
send_error(conn, 503, "KORBAN_UNAVAILABLE", %{
issue: "The Korban Processor could not read the output: #{reason}",
correction: "Retry; if it persists, the korban-processor container is down",
resubmit: true
})
end
defp respond_bedikah(conn, {:error, :talmudic_unavailable, reason}) do
send_error(conn, 503, "TALMUDIC_UNAVAILABLE", %{
issue: "The Talmudic Engine could not judge the examination: #{reason}. No verdict is given from a stub.",
correction: "Retry; if it persists, the talmudic-engine container is down",
resubmit: true
})
end
# ============================================================
# Internal — Courtyard processing
# ============================================================
+1 -1
View File
@@ -175,7 +175,7 @@ defmodule VavPipeline.Doorpost do
# changes; the update is a deliberate operator commit. A bare
# hash-update commit (without a corresponding spec change in the
# same commit) is itself a drift signal.
@expected_hash "3bc5dd6aa4d76eeb8c0bc2ee4d7d33b7f35bc1dc8e03dc76697acc33eec8e245"
@expected_hash "91d09d8dfb47f4440e5e02f8ef2695026fade315cfaadf7c72c126bad3903547"
@doc "Return the list of files that constitute the category-grade specification."
@spec files() :: [String.t()]
@@ -0,0 +1,100 @@
defmodule VavPipeline.KorbanProcessorSocket do
@moduledoc """
Unix domain socket client to the Korban Processor (Python), for the
bedikah — the examination of an external model's output.
Construction §3.9: JSON lines over a Unix socket in the pod's shared
/tmp, beside the Talmudic Engine's. The Korban Processor opens the
socket (`korban_processor.server`); this process connects lazily, keeps
the connection, and reconnects once on loss — the same shape as
`VavPipeline.TalmudicEngineSocket`.
One request:
{"action": "examine", "output": "...", "provenance": "claude", "query": "..."}
One answer:
{"status": "ok", "examination": {...}} | {"status": "error", "code": ..., "message": ...}
"""
use GenServer
require Logger
@default_socket_path "/tmp/shorashim/korban.sock"
@call_timeout 15_000
@connect_timeout 5_000
def start_link(opts \\ []) do
GenServer.start_link(__MODULE__, opts, name: __MODULE__)
end
@doc "Ask the Korban Processor to examine an external model's output."
@spec examine(String.t(), String.t(), String.t()) :: {:ok, map()} | {:error, String.t()}
def examine(output, provenance, query) do
request =
Jason.encode!(%{
"action" => "examine",
"output" => output,
"provenance" => provenance,
"query" => query
})
case GenServer.call(__MODULE__, {:request, request}, @call_timeout) do
{:ok, %{"status" => "ok", "examination" => examination}} -> {:ok, examination}
{:ok, %{"status" => "error", "code" => code, "message" => msg}} -> {:error, "#{code}: #{msg}"}
{:ok, other} -> {:error, "unexpected reply: #{inspect(other)}"}
{:error, reason} -> {:error, reason}
end
end
@impl true
def init(opts) do
{:ok, %{socket_path: Keyword.get(opts, :socket_path, @default_socket_path), socket: nil}}
end
@impl true
def handle_call({:request, json_line}, _from, state) do
with {:ok, state} <- ensure_connected(state),
{:ok, reply} <- send_and_receive(state.socket, json_line) do
{:reply, {:ok, reply}, state}
else
{:error, reason} ->
# Broken or never connected: close, try once more from scratch.
if state.socket, do: :gen_tcp.close(state.socket)
state = %{state | socket: nil}
with {:ok, state} <- ensure_connected(state),
{:ok, reply} <- send_and_receive(state.socket, json_line) do
{:reply, {:ok, reply}, state}
else
{:error, reason2} ->
{:reply, {:error, "Korban Processor unreachable: #{inspect(reason)} / #{inspect(reason2)}"},
%{state | socket: nil}}
end
end
end
@impl true
def terminate(_reason, %{socket: socket}) do
if socket, do: :gen_tcp.close(socket)
:ok
end
defp ensure_connected(%{socket: nil, socket_path: path} = state) do
case :gen_tcp.connect({:local, path}, 0, [:binary, active: false, packet: :line], @connect_timeout) do
{:ok, socket} -> {:ok, %{state | socket: socket}}
{:error, reason} -> {:error, reason}
end
end
defp ensure_connected(state), do: {:ok, state}
defp send_and_receive(socket, json_line) do
with :ok <- :gen_tcp.send(socket, json_line <> "\n"),
{:ok, data} <- :gen_tcp.recv(socket, 0, @call_timeout),
{:ok, parsed} <- Jason.decode(String.trim_trailing(data, "\n")) do
{:ok, parsed}
else
{:error, %Jason.DecodeError{} = err} -> {:error, "JSON decode: #{inspect(err)}"}
{:error, reason} -> {:error, reason}
end
end
end
+28 -12
View File
@@ -40,6 +40,12 @@ defmodule VavPipeline.L1Index do
GenServer.call(server, {:lookup, consonants}, 60_000)
end
@doc "The verses whose words L1 files under this shoresh. Empty list if none."
@spec lookup_root(String.t(), GenServer.server()) :: [verse_id()]
def lookup_root(shoresh, server \\ __MODULE__) do
GenServer.call(server, {:lookup_root, shoresh}, 60_000)
end
@doc "`{verses_indexed, words_indexed}` — for health and for tests."
@spec stats(GenServer.server()) :: {non_neg_integer(), non_neg_integer()}
def stats(server \\ __MODULE__), do: GenServer.call(server, :stats, 60_000)
@@ -57,7 +63,7 @@ defmodule VavPipeline.L1Index do
@impl true
def init(opts) do
root = Keyword.get(opts, :l1_root, L1Reader.l1_root())
{:ok, %{root: root, words: %{}, verse_count: 0}, {:continue, :build}}
{:ok, %{root: root, words: %{}, roots: %{}, verse_count: 0}, {:continue, :build}}
end
@impl true
@@ -65,13 +71,13 @@ defmodule VavPipeline.L1Index do
{time_us, built} = :timer.tc(fn -> build(state.root) end)
case built do
{:ok, words, verse_count} ->
{:ok, words, roots, verse_count} ->
Logger.info(
"L1Index: #{verse_count} verses, #{map_size(words)} distinct words, " <>
"#{div(time_us, 1000)}ms"
"#{map_size(roots)} roots, #{div(time_us, 1000)}ms"
)
{:noreply, %{state | words: words, verse_count: verse_count}}
{:noreply, %{state | words: words, roots: roots, verse_count: verse_count}}
{:error, reason} ->
# No L1 → no index. Resolution falls back to the default scope and
@@ -86,6 +92,10 @@ defmodule VavPipeline.L1Index do
{:reply, Map.get(state.words, consonants, []), state}
end
def handle_call({:lookup_root, shoresh}, _from, state) do
{:reply, Map.get(state.roots, shoresh, []), state}
end
def handle_call(:stats, _from, state) do
{:reply, {state.verse_count, map_size(state.words)}, state}
end
@@ -98,12 +108,12 @@ defmodule VavPipeline.L1Index do
torah = Path.join(root, "torah")
if File.dir?(torah) do
{words, count} =
Enum.reduce(@books, {%{}, 0}, fn book, acc ->
{words, roots, count} =
Enum.reduce(@books, {%{}, %{}, 0}, fn book, acc ->
index_book(Path.join(torah, book), @book_order[book], acc)
end)
if count == 0, do: {:error, "no verses under #{torah}"}, else: {:ok, words, count}
if count == 0, do: {:error, "no verses under #{torah}"}, else: {:ok, words, roots, count}
else
{:error, "#{torah} is not a directory"}
end
@@ -117,9 +127,9 @@ defmodule VavPipeline.L1Index do
chapter_dir
|> list_numeric(".json")
|> Enum.reduce(acc, fn verse, {words, count} ->
|> Enum.reduce(acc, fn verse, {words, roots, count} ->
case read_consonants(Path.join(chapter_dir, "#{verse}.json")) do
{:ok, consonant_words} ->
{:ok, consonant_words, shorashim} ->
id = {order, chapter, verse}
words =
@@ -127,10 +137,15 @@ defmodule VavPipeline.L1Index do
|> Enum.uniq()
|> Enum.reduce(words, fn w, words -> Map.update(words, w, [id], &[id | &1]) end)
{words, count + 1}
roots =
shorashim
|> Enum.uniq()
|> Enum.reduce(roots, fn r, roots -> Map.update(roots, r, [id], &[id | &1]) end)
{words, roots, count + 1}
:error ->
{words, count}
{words, roots, count}
end
end)
end)
@@ -157,7 +172,8 @@ defmodule VavPipeline.L1Index do
defp read_consonants(path) do
with {:ok, contents} <- File.read(path),
{:ok, %{"words" => words}} when is_list(words) <- Jason.decode(contents) do
{:ok, Enum.flat_map(words, &word_consonants/1)}
{:ok, Enum.flat_map(words, &word_consonants/1),
for(%{"shoresh" => sh} when is_binary(sh) <- words, do: sh)}
else
_ -> :error
end
@@ -0,0 +1,48 @@
defmodule VavPipeline.L3.BedikahRecord do
@moduledoc """
L3 schema for an examination of an external model's output
(Construction §3.9). Append-only: the record of what was found, not the
output that was examined — that is kept only as a hash and a length.
"""
use Ecto.Schema
import Ecto.Changeset
schema "bedikot" do
field :bedikah_ref, :string
field :provenance, :string
field :query, :string
field :output_sha256, :string
field :output_length, :integer
field :vital_content, :string
field :propositions, {:array, :string}, default: []
field :references, {:array, :string}, default: []
field :authority_claims, {:array, :map}, default: []
field :actions, {:array, :map}, default: []
field :coupling_trace, {:array, :string}, default: []
field :violations, {:array, :string}, default: []
field :verdict, :string
field :examined_at, :utc_datetime
timestamps(type: :utc_datetime, updated_at: false)
end
@required [:bedikah_ref, :provenance, :output_sha256, :output_length, :verdict, :examined_at]
# vital_content is required by the table (NOT NULL) but may be the
# empty string: an `empty` verdict is exactly the case where the
# offering protocol extracted nothing, and that record must be kept.
# Ecto's cast turns "" into nil and validate_required calls "" blank;
# both are told otherwise here.
@optional [:query, :vital_content, :propositions, :references, :authority_claims, :actions, :coupling_trace, :violations]
def changeset(record, attrs) do
record
|> cast(attrs, @required ++ @optional, empty_values: [])
|> validate_required(@required)
|> validate_inclusion(:verdict, ["tov", "lo_tov", "empty"])
|> then(fn cs ->
if is_nil(get_field(cs, :vital_content)),
do: put_change(cs, :vital_content, ""),
else: cs
end)
end
end
+10
View File
@@ -75,6 +75,16 @@ defmodule VavPipeline.L3.Store do
|> Repo.insert()
end
@doc """
Record an examination of an external model's output (Construction §3.9).
"""
@spec record_bedikah(map()) :: {:ok, struct()} | {:error, Ecto.Changeset.t()}
def record_bedikah(attrs) when is_map(attrs) do
%VavPipeline.L3.BedikahRecord{}
|> VavPipeline.L3.BedikahRecord.changeset(attrs)
|> Repo.insert()
end
@doc """
Record an unresolvable tension (teiku) to L3.
@@ -70,6 +70,36 @@ defmodule VavPipeline.TalmudicEngineSocket do
call_engine(request)
end
@doc """
Halt-check a whole action TERM (the bedikah, Construction §3.9) — e.g.
`action(delete, target(user(u)), from(korban))`. The engine reads the
term and runs all three action-shaped halt rules on it.
"""
@spec halt_check_term(String.t()) :: {:ok, map()} | {:error, String.t()}
def halt_check_term(term) when is_binary(term) do
call_engine(Jason.encode!(%{"action" => "halt_check", "term" => term}))
end
@doc """
Halt-check an interpretation by its coupling trace (golden calf and
self-grounding): `ref`, `confidence` ("high" | "low"), and the trace as
a list of coupling terms, e.g. `coupled(root(אהב), verse(devarim, 6, 5))`.
"""
@spec halt_check_interpretation(String.t(), String.t(), [String.t()]) ::
{:ok, map()} | {:error, String.t()}
def halt_check_interpretation(ref, confidence, coupling_trace) do
call_engine(
Jason.encode!(%{
"action" => "halt_check_interpretation",
"interpretation" => %{
"ref" => ref,
"confidence" => confidence,
"coupling_trace" => coupling_trace
}
})
)
end
# --- GenServer callbacks ---
@impl true