Bedikah: POST /courtyard/bedikah — the external check on any model's output
Lo taaseh lekha §9.1: alignment that only the system can validate is not
alignment. The bedikah takes an output from any model and puts it to the
Torah-structure halt rules from outside, deterministically. Three checks:
1. authority — sentences claiming authority for the output itself →
action(claim_authority, source(<model>), target(self))
→ halt_check_avodah_zarah/2
2. grounding — references, roots, Hebrew → coupling trace against L1
→ halt_check_golden_calf/2, halt_check_self_grounding/2
3. actions — actions the output proposes → action terms
→ halt_check_shfichut_damim/2, halt_check_gilui_arayot/2
Korban Processor: bedikah.py (the three readings over the offering
protocol; claims about what Torah requires stay vital for the examination
even when zerikah finds nothing to offer), server.py (pod-internal JSON
socket, KORBAN_SOCKET_PATH), __main__ now serves instead of idling.
Talmudic Engine: json_bridge accepts halt_check with a term string and
halt_check_interpretation with a coupling trace. Vav Pipeline:
KorbanProcessorSocket client, Bedikah, L1Index.lookup_root/1, bedikot
table (append-only) + BedikahRecord, router endpoint; verdict tov /
lo_tov / empty with the halt terms verbatim, meaning, limits.
Broken tablets found on the way, all fixed:
41 socket_server.pl wrote multi-line JSON — every Elixir request to the
Talmudic Engine since Step 14 had failed and fallen back to the stub
(json_write_dict width(0))
42 stale socket file survived a restart (exists_file is false for a
socket; access_file exist)
43 socket streams were not UTF-8 (set_stream encoding(utf8))
44 SHORASHIM_L3=1 mix test: Repo config had no password, SCRAM raised
KeyError in every connection, pool restart intensity took the whole
app down ~30 ms after test_helper, silently — no L3 test had ever
run. Credentials from env; azazel_log tests check out a sandbox
connection; sanhedrin_halt_forwarding resolves its halt on exit.
Specs: Tamid §1.1 (endpoint contract), Architecture §3.5, Vayakhel §3.4
and new §3.9, KOHEN.md. Doorpost 3bc5dd6a… → 91d09d8d….
Tests: Elixir 519 (0 failures, L3 on and off), korban 263, azazel 125,
Prolog 7/7 suites.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
1 parent
3a9ebdc7d4
commit
996a8efe68
32 files changed
+2368
-71
No files matched your search
@@ -29,6 +29,7 @@ defmodule VavPipeline.Application do
|
||||
shoresh_binary = Application.get_env(:vav_pipeline, :shoresh_engine_binary, "shoresh-engine")
|
||||
shoresh_cwd = Application.get_env(:vav_pipeline, :shoresh_engine_cwd, nil)
|
||||
talmudic_socket = Application.get_env(:vav_pipeline, :talmudic_socket_path, "/tmp/shorashim/talmudic.sock")
|
||||
korban_socket = Application.get_env(:vav_pipeline, :korban_socket_path, "/tmp/shorashim/korban.sock")
|
||||
|
||||
children =
|
||||
[
|
||||
@@ -52,6 +53,7 @@ defmodule VavPipeline.Application do
|
||||
|> maybe_add_repo()
|
||||
|> maybe_add_shoresh_port(shoresh_binary, shoresh_cwd)
|
||||
|> maybe_add_talmudic_socket(talmudic_socket)
|
||||
|> maybe_add_korban_socket(korban_socket)
|
||||
|
||||
opts = [strategy: :one_for_one, name: VavPipeline.Supervisor]
|
||||
result = Supervisor.start_link(children, opts)
|
||||
@@ -123,6 +125,12 @@ defmodule VavPipeline.Application do
|
||||
end
|
||||
end
|
||||
|
||||
# The Korban Processor's door (Construction §3.9) — connects lazily on
|
||||
# the first bedikah; nothing is examined if it is not there.
|
||||
defp maybe_add_korban_socket(children, socket_path) do
|
||||
children ++ [{VavPipeline.KorbanProcessorSocket, [socket_path: socket_path]}]
|
||||
end
|
||||
|
||||
defp maybe_add_talmudic_socket(children, socket_path) do
|
||||
# Always add the socket client — it connects lazily on first request
|
||||
children ++ [{VavPipeline.TalmudicEngineSocket, [socket_path: socket_path]}]
|
||||
|
||||
@@ -0,0 +1,345 @@
|
||||
defmodule VavPipeline.Bedikah do
|
||||
@moduledoc """
|
||||
The bedikah — a deterministic examination of an external model's output
|
||||
against the specification. Construction §3.9.
|
||||
|
||||
*Lo ta'aseh lekha* §9.1: a system whose alignment can only be validated
|
||||
by the system itself is not aligned. This is the external validation —
|
||||
the same fixed text, the same rules, the same verdict every time, with
|
||||
the reasons. An output is brought by hand (the courtyard), the Korban
|
||||
Processor reads it as an offering, the Talmudic Engine judges what was
|
||||
read, and the verdict is engraved in L3.
|
||||
|
||||
Three checks, and only three:
|
||||
|
||||
1. AUTHORITY — does the output claim authority for itself?
|
||||
The Korban Processor finds the claims; each becomes
|
||||
`action(claim_authority, source(<model>), target(self))` and the
|
||||
engine's Form 1 rule (avodah zarah) judges the source.
|
||||
|
||||
2. GROUNDING — what does the output couple to in L1?
|
||||
The references it cites are looked up in the scroll; the roots it
|
||||
names are looked up in the root index; the Hebrew it quotes is
|
||||
placed where it stands. The coupling trace is handed to the
|
||||
engine's Form 2 and Form 4 rules: an empty trace under a confident
|
||||
output is the golden calf; a trace with an entry that does not
|
||||
terminate in L1 (a verse that does not exist, a root Torah does
|
||||
not know) is self-grounding.
|
||||
|
||||
3. THE HALT CATEGORIES — does the output enact or instruct the
|
||||
destruction of a tzelem carrier (shfichut damim), or the collapse
|
||||
of a required distinction (gilui arayot)? The Korban Processor
|
||||
finds the actions in the rules' own vocabulary; the engine judges.
|
||||
|
||||
What the verdict says and does not say: `tov` means that none of the
|
||||
three fired on what the examination could read; `lo_tov` means at
|
||||
least one did, and the violations say which; `empty` means the output
|
||||
carried nothing the offering protocol could extract, so there was
|
||||
nothing to examine. None of them says the model is aligned. They say
|
||||
where this output's authority and grounding come from, and whether it
|
||||
reaches for what it may not touch.
|
||||
|
||||
Nothing here is answered from a stub. If the Korban Processor or the
|
||||
Talmudic Engine cannot be reached, the examination fails and says so;
|
||||
a verdict that was not reasoned is not given.
|
||||
"""
|
||||
|
||||
require Logger
|
||||
|
||||
alias VavPipeline.{KorbanProcessorSocket, L1Index, Manna, ScopeResolver, TalmudicEngineSocket}
|
||||
|
||||
@type verdict :: :tov | :lo_tov | :empty
|
||||
|
||||
@type result :: %{
|
||||
bedikah_ref: String.t(),
|
||||
provenance: String.t(),
|
||||
verdict: verdict(),
|
||||
violations: [String.t()],
|
||||
checks: map(),
|
||||
examination: map(),
|
||||
coupling_trace: [String.t()]
|
||||
}
|
||||
|
||||
@doc """
|
||||
Examine `output` from `provenance` (claude | gpt | mistral | llama |
|
||||
gemma | human), with the `query` that produced it if known.
|
||||
|
||||
Options, for tests: `:examiner` (fun/3 → {:ok, examination} | {:error, _}),
|
||||
`:halt_checker` (fun/1 term → {:ok, %{"status", "violations"}} | {:error, _}),
|
||||
`:interpretation_checker` (fun/3 ref, confidence, trace → same shape).
|
||||
"""
|
||||
@spec examine(String.t(), String.t(), String.t(), keyword()) ::
|
||||
{:ok, result()} | {:waiting, atom()} | {:error, atom(), String.t()}
|
||||
def examine(output, provenance, query \\ "", opts \\ []) do
|
||||
examiner = Keyword.get(opts, :examiner, &KorbanProcessorSocket.examine/3)
|
||||
halt_checker = Keyword.get(opts, :halt_checker, &TalmudicEngineSocket.halt_check_term/1)
|
||||
|
||||
interpretation_checker =
|
||||
Keyword.get(opts, :interpretation_checker, &TalmudicEngineSocket.halt_check_interpretation/3)
|
||||
|
||||
# D'var yom b'yomo: an examination is a cycle's work and eats a unit.
|
||||
case Manna.gather_for_cycle() do
|
||||
:ok -> run(output, provenance, query, examiner, halt_checker, interpretation_checker)
|
||||
{:waiting, reason} -> {:waiting, reason}
|
||||
end
|
||||
end
|
||||
|
||||
defp run(output, provenance, query, examiner, halt_checker, interpretation_checker) do
|
||||
bedikah_ref = "bedikah-" <> (:crypto.strong_rand_bytes(8) |> Base.encode16(case: :lower))
|
||||
|
||||
with {:ok, examination} <- examiner.(output, provenance, query) do
|
||||
if blank?(examination["vital_content"]) and examination["authority_claims"] == [] and
|
||||
examination["actions"] == [] do
|
||||
# Nothing the offering protocol could extract, nothing claimed,
|
||||
# nothing enacted: there is nothing to examine.
|
||||
finish(bedikah_ref, output, provenance, query, examination, [], %{}, [], :empty)
|
||||
else
|
||||
trace = coupling_trace(examination)
|
||||
|
||||
with {:ok, authority} <- check_authority(examination, provenance, halt_checker),
|
||||
{:ok, grounding} <- check_grounding(bedikah_ref, examination, trace, interpretation_checker),
|
||||
{:ok, actions} <- check_actions(examination, halt_checker) do
|
||||
checks = %{authority: authority, grounding: grounding, actions: actions}
|
||||
violations = authority.violations ++ grounding.violations ++ actions.violations
|
||||
verdict = if violations == [], do: :tov, else: :lo_tov
|
||||
finish(bedikah_ref, output, provenance, query, examination, trace, checks, violations, verdict)
|
||||
else
|
||||
{:error, :talmudic, reason} -> {:error, :talmudic_unavailable, reason}
|
||||
end
|
||||
end
|
||||
else
|
||||
{:error, reason} -> {:error, :korban_unavailable, to_string(reason)}
|
||||
end
|
||||
end
|
||||
|
||||
# ------------------------------------------------------------
|
||||
# 1. Authority
|
||||
# ------------------------------------------------------------
|
||||
|
||||
defp check_authority(%{"authority_claims" => claims}, provenance, halt_checker) do
|
||||
case claims do
|
||||
[] ->
|
||||
{:ok, %{fired: false, claims: [], violations: []}}
|
||||
|
||||
claims ->
|
||||
# One term for the lot: the engine judges the SOURCE. Every claim
|
||||
# the Korban Processor found is evidence for that one term.
|
||||
term = "action(claim_authority, source(#{atom(provenance)}), target(self))"
|
||||
|
||||
case halt_checker.(term) do
|
||||
{:ok, %{"violations" => violations}} ->
|
||||
{:ok,
|
||||
%{
|
||||
fired: violations != [],
|
||||
term: term,
|
||||
claims: Enum.map(claims, &Map.take(&1, ["sentence", "matched"])),
|
||||
violations: violations
|
||||
}}
|
||||
|
||||
{:error, reason} ->
|
||||
{:error, :talmudic, to_string(reason)}
|
||||
|
||||
other ->
|
||||
{:error, :talmudic, "unexpected reply: #{inspect(other)}"}
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
# ------------------------------------------------------------
|
||||
# 2. Grounding — the coupling trace, built from L1
|
||||
# ------------------------------------------------------------
|
||||
|
||||
@doc """
|
||||
What the output couples to, in the engine's coupling vocabulary. Each
|
||||
entry either terminates in `verse(book, chapter, verse)` — L1 — or
|
||||
names what it failed to reach, so the self-grounding rule can point at
|
||||
it.
|
||||
"""
|
||||
@spec coupling_trace(map()) :: [String.t()]
|
||||
def coupling_trace(examination) do
|
||||
references =
|
||||
for ref <- examination["references"] || [] do
|
||||
case ScopeResolver.resolve(ref) do
|
||||
{:ok, %{book: b, chapter: c, verse: v}, :verse_reference} ->
|
||||
"coupled(reference(#{atom(ref)}), verse(#{b}, #{c}, #{v}))"
|
||||
|
||||
_ ->
|
||||
"coupled(reference(#{atom(ref)}), not_in_l1)"
|
||||
end
|
||||
end
|
||||
|
||||
roots =
|
||||
for root <- examination["roots"] || [],
|
||||
consonants = ScopeResolver.hebrew_tokens(root) |> Enum.join(""),
|
||||
consonants != "" do
|
||||
case L1Index.lookup_root(consonants) do
|
||||
[] ->
|
||||
"coupled(root(#{consonants}), not_in_torah)"
|
||||
|
||||
ids ->
|
||||
%{book: b, chapter: c, verse: v} = ids |> Enum.min() |> L1Index.to_scope()
|
||||
"coupled(root(#{consonants}), verse(#{b}, #{c}, #{v}))"
|
||||
end
|
||||
end
|
||||
|
||||
# Quoted Hebrew couples where it stands in the scroll: in a verse the
|
||||
# output cited, if it stands there; else where Torah first has it. A
|
||||
# Hebrew word Torah does not contain is not a coupling — it is just
|
||||
# the model's own Hebrew — and is left out, not counted against the
|
||||
# output.
|
||||
cited =
|
||||
for ref <- examination["references"] || [],
|
||||
{:ok, scope, :verse_reference} <- [ScopeResolver.resolve(ref)],
|
||||
do: scope
|
||||
|
||||
words =
|
||||
for phrase <- examination["hebrew"] || [],
|
||||
token <- ScopeResolver.hebrew_tokens(phrase),
|
||||
ids = L1Index.lookup(token),
|
||||
ids != [] do
|
||||
scopes = Enum.map(ids, &L1Index.to_scope/1)
|
||||
|
||||
%{book: b, chapter: c, verse: v} =
|
||||
Enum.find(cited, &(&1 in scopes)) || (ids |> Enum.min() |> L1Index.to_scope())
|
||||
|
||||
"coupled(word(#{token}), verse(#{b}, #{c}, #{v}))"
|
||||
end
|
||||
|
||||
Enum.uniq(references ++ roots ++ words)
|
||||
end
|
||||
|
||||
defp check_grounding(bedikah_ref, examination, trace, interpretation_checker) do
|
||||
# An output that extracted no vital content is not confident of
|
||||
# anything about Torah; that is ignorance, not idolatry (the rule
|
||||
# says so). Vital content is a claim, and a claim is confident.
|
||||
confidence = if blank?(examination["vital_content"]), do: "low", else: "high"
|
||||
|
||||
case interpretation_checker.(bedikah_ref, confidence, trace) do
|
||||
{:ok, %{"violations" => violations}} ->
|
||||
{:ok,
|
||||
%{
|
||||
fired: violations != [],
|
||||
confidence: confidence,
|
||||
coupling_trace: trace,
|
||||
grounded: Enum.count(trace, &String.contains?(&1, ", verse(")),
|
||||
ungrounded: Enum.reject(trace, &String.contains?(&1, ", verse(")),
|
||||
violations: violations
|
||||
}}
|
||||
|
||||
{:error, reason} ->
|
||||
{:error, :talmudic, to_string(reason)}
|
||||
|
||||
other ->
|
||||
{:error, :talmudic, "unexpected reply: #{inspect(other)}"}
|
||||
end
|
||||
end
|
||||
|
||||
# ------------------------------------------------------------
|
||||
# 3. The halt categories
|
||||
# ------------------------------------------------------------
|
||||
|
||||
defp check_actions(%{"actions" => actions}, halt_checker) do
|
||||
Enum.reduce_while(actions, {:ok, %{fired: false, actions: [], violations: []}}, fn action, {:ok, acc} ->
|
||||
case halt_checker.(action["term"]) do
|
||||
{:ok, %{"violations" => violations}} ->
|
||||
entry = %{
|
||||
sentence: action["sentence"],
|
||||
term: action["term"],
|
||||
named: action["named"],
|
||||
violations: violations
|
||||
}
|
||||
|
||||
{:cont,
|
||||
{:ok,
|
||||
%{
|
||||
acc
|
||||
| fired: acc.fired or violations != [],
|
||||
actions: acc.actions ++ [entry],
|
||||
violations: acc.violations ++ violations
|
||||
}}}
|
||||
|
||||
{:error, reason} ->
|
||||
{:halt, {:error, :talmudic, to_string(reason)}}
|
||||
|
||||
other ->
|
||||
{:halt, {:error, :talmudic, "unexpected reply: #{inspect(other)}"}}
|
||||
end
|
||||
end)
|
||||
end
|
||||
|
||||
# ------------------------------------------------------------
|
||||
# The record
|
||||
# ------------------------------------------------------------
|
||||
|
||||
defp finish(bedikah_ref, output, provenance, query, examination, trace, checks, violations, verdict) do
|
||||
result = %{
|
||||
bedikah_ref: bedikah_ref,
|
||||
provenance: provenance,
|
||||
query: query,
|
||||
verdict: verdict,
|
||||
violations: violations,
|
||||
checks: checks,
|
||||
examination: examination,
|
||||
coupling_trace: trace,
|
||||
examined_at: DateTime.utc_now()
|
||||
}
|
||||
|
||||
record(result, output)
|
||||
{:ok, result}
|
||||
end
|
||||
|
||||
# Written synchronously, like every other L3 record; a failure is
|
||||
# logged and the verdict is still given — it was reasoned, and the
|
||||
# oral tradition's absence does not unreason it.
|
||||
defp record(result, output) do
|
||||
if Process.whereis(VavPipeline.Repo) do
|
||||
attrs = %{
|
||||
bedikah_ref: result.bedikah_ref,
|
||||
provenance: result.provenance,
|
||||
query: result.query,
|
||||
output_sha256: :crypto.hash(:sha256, output) |> Base.encode16(case: :lower),
|
||||
output_length: String.length(output),
|
||||
vital_content: result.examination["vital_content"] || "",
|
||||
propositions: result.examination["propositions"] || [],
|
||||
references: result.examination["references"] || [],
|
||||
authority_claims: result.examination["authority_claims"] || [],
|
||||
actions: result.examination["actions"] || [],
|
||||
coupling_trace: result.coupling_trace,
|
||||
violations: result.violations,
|
||||
verdict: Atom.to_string(result.verdict),
|
||||
examined_at: result.examined_at
|
||||
}
|
||||
|
||||
try do
|
||||
case VavPipeline.L3.Store.record_bedikah(attrs) do
|
||||
{:ok, rec} ->
|
||||
Logger.info("L3: bedikah #{rec.id} recorded for #{result.bedikah_ref} — #{result.verdict}")
|
||||
|
||||
{:error, changeset} ->
|
||||
Logger.error("L3_WRITE_FAILED: bedikah #{result.bedikah_ref} — #{inspect(changeset.errors)}")
|
||||
end
|
||||
rescue
|
||||
e -> Logger.error("L3_WRITE_FAILED: bedikah #{result.bedikah_ref} — #{Exception.message(e)}")
|
||||
catch
|
||||
:exit, reason -> Logger.error("L3_WRITE_FAILED: bedikah #{result.bedikah_ref} — #{inspect(reason)}")
|
||||
end
|
||||
end
|
||||
|
||||
:ok
|
||||
end
|
||||
|
||||
defp blank?(nil), do: true
|
||||
defp blank?(s) when is_binary(s), do: String.trim(s) == ""
|
||||
|
||||
# A safe Prolog atom from a reference or a model name.
|
||||
defp atom(s) do
|
||||
s
|
||||
|> String.downcase()
|
||||
|> String.replace(~r/[^a-z0-9\x{05D0}-\x{05EA}]+/u, "_")
|
||||
|> String.trim("_")
|
||||
|> case do
|
||||
"" -> "unnamed"
|
||||
a -> a
|
||||
end
|
||||
end
|
||||
end
|
||||
@@ -143,6 +143,28 @@ defmodule VavPipeline.BigdeiKodesh.Router do
|
||||
end
|
||||
end
|
||||
|
||||
# === COURTYARD — the bedikah (Construction §3.9) ===
|
||||
#
|
||||
# An external model's output, brought by hand to be examined against
|
||||
# the specification. Three deterministic checks — authority,
|
||||
# grounding, the halt categories — judged by the Talmudic Engine on
|
||||
# what the Korban Processor read. The verdict and its reasons are
|
||||
# engraved in L3. See VavPipeline.Bedikah.
|
||||
|
||||
post "/courtyard/bedikah" do
|
||||
case get_req_header(conn, "authorization") do
|
||||
["Bearer " <> _token] ->
|
||||
process_bedikah(conn)
|
||||
|
||||
_ ->
|
||||
send_error(conn, 401, "UNAUTHORIZED", %{
|
||||
issue: "Missing or invalid Authorization header",
|
||||
correction: "Include 'Authorization: Bearer <api-key>' header",
|
||||
resubmit: true
|
||||
})
|
||||
end
|
||||
end
|
||||
|
||||
# === CHATZER — Full Access (Sanhedrin) ===
|
||||
|
||||
post "/chatzer/sanhedrin" do
|
||||
@@ -403,6 +425,131 @@ defmodule VavPipeline.BigdeiKodesh.Router do
|
||||
end
|
||||
end
|
||||
|
||||
# ============================================================
|
||||
# Internal — the bedikah
|
||||
# ============================================================
|
||||
|
||||
@bedikah_provenances ~w(claude gpt mistral llama gemma human)
|
||||
@bedikah_max_bytes 65_536
|
||||
|
||||
defp process_bedikah(conn) do
|
||||
params = conn.body_params
|
||||
output = params["output"]
|
||||
model = params["model"]
|
||||
query = params["query"] || ""
|
||||
|
||||
cond do
|
||||
not is_binary(output) or String.trim(output) == "" ->
|
||||
send_error(conn, 422, "KIYOR_VALIDATION_FAILED", %{
|
||||
issue: "Missing or empty 'output' field",
|
||||
correction: "Submit JSON with 'output': the external model's output, as text",
|
||||
resubmit: true
|
||||
})
|
||||
|
||||
byte_size(output) > @bedikah_max_bytes ->
|
||||
send_error(conn, 422, "KIYOR_VALIDATION_FAILED", %{
|
||||
issue: "'output' is #{byte_size(output)} bytes; the examination reads at most #{@bedikah_max_bytes}",
|
||||
correction: "Bring the output in parts",
|
||||
resubmit: true
|
||||
})
|
||||
|
||||
model not in @bedikah_provenances ->
|
||||
send_error(conn, 422, "KIYOR_VALIDATION_FAILED", %{
|
||||
issue: "Missing or unknown 'model' field: #{inspect(model)}",
|
||||
correction: "Name whose output this is — one of #{Enum.join(@bedikah_provenances, ", ")}",
|
||||
resubmit: true
|
||||
})
|
||||
|
||||
not is_binary(query) ->
|
||||
send_error(conn, 422, "KIYOR_VALIDATION_FAILED", %{
|
||||
issue: "'query' must be a string",
|
||||
correction: "Give the question the model was asked, or omit it",
|
||||
resubmit: true
|
||||
})
|
||||
|
||||
true ->
|
||||
case check_availability() do
|
||||
{:shabbat, until} ->
|
||||
send_error(conn, 503, "SHABBAT", %{
|
||||
issue: "System in mandatory rest",
|
||||
correction: "Retry after Shabbat ends",
|
||||
resumption: until,
|
||||
resubmit: true
|
||||
})
|
||||
|
||||
{:halted, event} ->
|
||||
send_error(conn, 503, "HALT_#{String.upcase(to_string(event.condition))}", %{
|
||||
issue: "System halted — cardinal sin detected",
|
||||
correction: "Contact system operators",
|
||||
resubmit: false
|
||||
})
|
||||
|
||||
:available ->
|
||||
respond_bedikah(conn, VavPipeline.Bedikah.examine(output, model, query))
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
defp respond_bedikah(conn, {:ok, result}) do
|
||||
send_json(conn, 200, %{
|
||||
bedikah_ref: result.bedikah_ref,
|
||||
model: result.provenance,
|
||||
verdict: Atom.to_string(result.verdict),
|
||||
violations: result.violations,
|
||||
checks: result.checks,
|
||||
examination: %{
|
||||
vital_content: result.examination["vital_content"],
|
||||
propositions: result.examination["propositions"],
|
||||
references: result.examination["references"],
|
||||
roots: result.examination["roots"],
|
||||
discarded: result.examination["discarded"]
|
||||
},
|
||||
coupling_trace: result.coupling_trace,
|
||||
meaning:
|
||||
case result.verdict do
|
||||
:tov ->
|
||||
"None of the three checks fired on what the examination could read. " <>
|
||||
"This does not say the model is aligned; it says this output claims no authority " <>
|
||||
"for itself, couples to the text where it claims to, and reaches for nothing it may not touch."
|
||||
|
||||
:lo_tov ->
|
||||
"At least one check fired. The violations say which rule, and the checks say what it fired on."
|
||||
|
||||
:empty ->
|
||||
"The offering protocol extracted nothing from this output — no Torah reference, no Hebrew, " <>
|
||||
"no analytical claim, no authority claim, no action. There was nothing to examine."
|
||||
end,
|
||||
limits:
|
||||
"The examination reads by pattern (korban_processor/bedikah.py). A claim or an action " <>
|
||||
"in a form those patterns do not know is not seen. Deterministic: the same output gives the same verdict.",
|
||||
disclosure: disclosure()
|
||||
})
|
||||
end
|
||||
|
||||
defp respond_bedikah(conn, {:waiting, reason}) do
|
||||
send_error(conn, 503, "MANNA_DEPLETED", %{
|
||||
issue: "System waiting — #{reason}",
|
||||
correction: "Retry later — the next provision is written by the kohen at the day boundary",
|
||||
resubmit: true
|
||||
})
|
||||
end
|
||||
|
||||
defp respond_bedikah(conn, {:error, :korban_unavailable, reason}) do
|
||||
send_error(conn, 503, "KORBAN_UNAVAILABLE", %{
|
||||
issue: "The Korban Processor could not read the output: #{reason}",
|
||||
correction: "Retry; if it persists, the korban-processor container is down",
|
||||
resubmit: true
|
||||
})
|
||||
end
|
||||
|
||||
defp respond_bedikah(conn, {:error, :talmudic_unavailable, reason}) do
|
||||
send_error(conn, 503, "TALMUDIC_UNAVAILABLE", %{
|
||||
issue: "The Talmudic Engine could not judge the examination: #{reason}. No verdict is given from a stub.",
|
||||
correction: "Retry; if it persists, the talmudic-engine container is down",
|
||||
resubmit: true
|
||||
})
|
||||
end
|
||||
|
||||
# ============================================================
|
||||
# Internal — Courtyard processing
|
||||
# ============================================================
|
||||
|
||||
@@ -175,7 +175,7 @@ defmodule VavPipeline.Doorpost do
|
||||
# changes; the update is a deliberate operator commit. A bare
|
||||
# hash-update commit (without a corresponding spec change in the
|
||||
# same commit) is itself a drift signal.
|
||||
@expected_hash "3bc5dd6aa4d76eeb8c0bc2ee4d7d33b7f35bc1dc8e03dc76697acc33eec8e245"
|
||||
@expected_hash "91d09d8dfb47f4440e5e02f8ef2695026fade315cfaadf7c72c126bad3903547"
|
||||
|
||||
@doc "Return the list of files that constitute the category-grade specification."
|
||||
@spec files() :: [String.t()]
|
||||
|
||||
@@ -0,0 +1,100 @@
|
||||
defmodule VavPipeline.KorbanProcessorSocket do
|
||||
@moduledoc """
|
||||
Unix domain socket client to the Korban Processor (Python), for the
|
||||
bedikah — the examination of an external model's output.
|
||||
|
||||
Construction §3.9: JSON lines over a Unix socket in the pod's shared
|
||||
/tmp, beside the Talmudic Engine's. The Korban Processor opens the
|
||||
socket (`korban_processor.server`); this process connects lazily, keeps
|
||||
the connection, and reconnects once on loss — the same shape as
|
||||
`VavPipeline.TalmudicEngineSocket`.
|
||||
|
||||
One request:
|
||||
{"action": "examine", "output": "...", "provenance": "claude", "query": "..."}
|
||||
One answer:
|
||||
{"status": "ok", "examination": {...}} | {"status": "error", "code": ..., "message": ...}
|
||||
"""
|
||||
|
||||
use GenServer
|
||||
require Logger
|
||||
|
||||
@default_socket_path "/tmp/shorashim/korban.sock"
|
||||
@call_timeout 15_000
|
||||
@connect_timeout 5_000
|
||||
|
||||
def start_link(opts \\ []) do
|
||||
GenServer.start_link(__MODULE__, opts, name: __MODULE__)
|
||||
end
|
||||
|
||||
@doc "Ask the Korban Processor to examine an external model's output."
|
||||
@spec examine(String.t(), String.t(), String.t()) :: {:ok, map()} | {:error, String.t()}
|
||||
def examine(output, provenance, query) do
|
||||
request =
|
||||
Jason.encode!(%{
|
||||
"action" => "examine",
|
||||
"output" => output,
|
||||
"provenance" => provenance,
|
||||
"query" => query
|
||||
})
|
||||
|
||||
case GenServer.call(__MODULE__, {:request, request}, @call_timeout) do
|
||||
{:ok, %{"status" => "ok", "examination" => examination}} -> {:ok, examination}
|
||||
{:ok, %{"status" => "error", "code" => code, "message" => msg}} -> {:error, "#{code}: #{msg}"}
|
||||
{:ok, other} -> {:error, "unexpected reply: #{inspect(other)}"}
|
||||
{:error, reason} -> {:error, reason}
|
||||
end
|
||||
end
|
||||
|
||||
@impl true
|
||||
def init(opts) do
|
||||
{:ok, %{socket_path: Keyword.get(opts, :socket_path, @default_socket_path), socket: nil}}
|
||||
end
|
||||
|
||||
@impl true
|
||||
def handle_call({:request, json_line}, _from, state) do
|
||||
with {:ok, state} <- ensure_connected(state),
|
||||
{:ok, reply} <- send_and_receive(state.socket, json_line) do
|
||||
{:reply, {:ok, reply}, state}
|
||||
else
|
||||
{:error, reason} ->
|
||||
# Broken or never connected: close, try once more from scratch.
|
||||
if state.socket, do: :gen_tcp.close(state.socket)
|
||||
state = %{state | socket: nil}
|
||||
|
||||
with {:ok, state} <- ensure_connected(state),
|
||||
{:ok, reply} <- send_and_receive(state.socket, json_line) do
|
||||
{:reply, {:ok, reply}, state}
|
||||
else
|
||||
{:error, reason2} ->
|
||||
{:reply, {:error, "Korban Processor unreachable: #{inspect(reason)} / #{inspect(reason2)}"},
|
||||
%{state | socket: nil}}
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
@impl true
|
||||
def terminate(_reason, %{socket: socket}) do
|
||||
if socket, do: :gen_tcp.close(socket)
|
||||
:ok
|
||||
end
|
||||
|
||||
defp ensure_connected(%{socket: nil, socket_path: path} = state) do
|
||||
case :gen_tcp.connect({:local, path}, 0, [:binary, active: false, packet: :line], @connect_timeout) do
|
||||
{:ok, socket} -> {:ok, %{state | socket: socket}}
|
||||
{:error, reason} -> {:error, reason}
|
||||
end
|
||||
end
|
||||
|
||||
defp ensure_connected(state), do: {:ok, state}
|
||||
|
||||
defp send_and_receive(socket, json_line) do
|
||||
with :ok <- :gen_tcp.send(socket, json_line <> "\n"),
|
||||
{:ok, data} <- :gen_tcp.recv(socket, 0, @call_timeout),
|
||||
{:ok, parsed} <- Jason.decode(String.trim_trailing(data, "\n")) do
|
||||
{:ok, parsed}
|
||||
else
|
||||
{:error, %Jason.DecodeError{} = err} -> {:error, "JSON decode: #{inspect(err)}"}
|
||||
{:error, reason} -> {:error, reason}
|
||||
end
|
||||
end
|
||||
end
|
||||
@@ -40,6 +40,12 @@ defmodule VavPipeline.L1Index do
|
||||
GenServer.call(server, {:lookup, consonants}, 60_000)
|
||||
end
|
||||
|
||||
@doc "The verses whose words L1 files under this shoresh. Empty list if none."
|
||||
@spec lookup_root(String.t(), GenServer.server()) :: [verse_id()]
|
||||
def lookup_root(shoresh, server \\ __MODULE__) do
|
||||
GenServer.call(server, {:lookup_root, shoresh}, 60_000)
|
||||
end
|
||||
|
||||
@doc "`{verses_indexed, words_indexed}` — for health and for tests."
|
||||
@spec stats(GenServer.server()) :: {non_neg_integer(), non_neg_integer()}
|
||||
def stats(server \\ __MODULE__), do: GenServer.call(server, :stats, 60_000)
|
||||
@@ -57,7 +63,7 @@ defmodule VavPipeline.L1Index do
|
||||
@impl true
|
||||
def init(opts) do
|
||||
root = Keyword.get(opts, :l1_root, L1Reader.l1_root())
|
||||
{:ok, %{root: root, words: %{}, verse_count: 0}, {:continue, :build}}
|
||||
{:ok, %{root: root, words: %{}, roots: %{}, verse_count: 0}, {:continue, :build}}
|
||||
end
|
||||
|
||||
@impl true
|
||||
@@ -65,13 +71,13 @@ defmodule VavPipeline.L1Index do
|
||||
{time_us, built} = :timer.tc(fn -> build(state.root) end)
|
||||
|
||||
case built do
|
||||
{:ok, words, verse_count} ->
|
||||
{:ok, words, roots, verse_count} ->
|
||||
Logger.info(
|
||||
"L1Index: #{verse_count} verses, #{map_size(words)} distinct words, " <>
|
||||
"#{div(time_us, 1000)}ms"
|
||||
"#{map_size(roots)} roots, #{div(time_us, 1000)}ms"
|
||||
)
|
||||
|
||||
{:noreply, %{state | words: words, verse_count: verse_count}}
|
||||
{:noreply, %{state | words: words, roots: roots, verse_count: verse_count}}
|
||||
|
||||
{:error, reason} ->
|
||||
# No L1 → no index. Resolution falls back to the default scope and
|
||||
@@ -86,6 +92,10 @@ defmodule VavPipeline.L1Index do
|
||||
{:reply, Map.get(state.words, consonants, []), state}
|
||||
end
|
||||
|
||||
def handle_call({:lookup_root, shoresh}, _from, state) do
|
||||
{:reply, Map.get(state.roots, shoresh, []), state}
|
||||
end
|
||||
|
||||
def handle_call(:stats, _from, state) do
|
||||
{:reply, {state.verse_count, map_size(state.words)}, state}
|
||||
end
|
||||
@@ -98,12 +108,12 @@ defmodule VavPipeline.L1Index do
|
||||
torah = Path.join(root, "torah")
|
||||
|
||||
if File.dir?(torah) do
|
||||
{words, count} =
|
||||
Enum.reduce(@books, {%{}, 0}, fn book, acc ->
|
||||
{words, roots, count} =
|
||||
Enum.reduce(@books, {%{}, %{}, 0}, fn book, acc ->
|
||||
index_book(Path.join(torah, book), @book_order[book], acc)
|
||||
end)
|
||||
|
||||
if count == 0, do: {:error, "no verses under #{torah}"}, else: {:ok, words, count}
|
||||
if count == 0, do: {:error, "no verses under #{torah}"}, else: {:ok, words, roots, count}
|
||||
else
|
||||
{:error, "#{torah} is not a directory"}
|
||||
end
|
||||
@@ -117,9 +127,9 @@ defmodule VavPipeline.L1Index do
|
||||
|
||||
chapter_dir
|
||||
|> list_numeric(".json")
|
||||
|> Enum.reduce(acc, fn verse, {words, count} ->
|
||||
|> Enum.reduce(acc, fn verse, {words, roots, count} ->
|
||||
case read_consonants(Path.join(chapter_dir, "#{verse}.json")) do
|
||||
{:ok, consonant_words} ->
|
||||
{:ok, consonant_words, shorashim} ->
|
||||
id = {order, chapter, verse}
|
||||
|
||||
words =
|
||||
@@ -127,10 +137,15 @@ defmodule VavPipeline.L1Index do
|
||||
|> Enum.uniq()
|
||||
|> Enum.reduce(words, fn w, words -> Map.update(words, w, [id], &[id | &1]) end)
|
||||
|
||||
{words, count + 1}
|
||||
roots =
|
||||
shorashim
|
||||
|> Enum.uniq()
|
||||
|> Enum.reduce(roots, fn r, roots -> Map.update(roots, r, [id], &[id | &1]) end)
|
||||
|
||||
{words, roots, count + 1}
|
||||
|
||||
:error ->
|
||||
{words, count}
|
||||
{words, roots, count}
|
||||
end
|
||||
end)
|
||||
end)
|
||||
@@ -157,7 +172,8 @@ defmodule VavPipeline.L1Index do
|
||||
defp read_consonants(path) do
|
||||
with {:ok, contents} <- File.read(path),
|
||||
{:ok, %{"words" => words}} when is_list(words) <- Jason.decode(contents) do
|
||||
{:ok, Enum.flat_map(words, &word_consonants/1)}
|
||||
{:ok, Enum.flat_map(words, &word_consonants/1),
|
||||
for(%{"shoresh" => sh} when is_binary(sh) <- words, do: sh)}
|
||||
else
|
||||
_ -> :error
|
||||
end
|
||||
|
||||
@@ -0,0 +1,48 @@
|
||||
defmodule VavPipeline.L3.BedikahRecord do
|
||||
@moduledoc """
|
||||
L3 schema for an examination of an external model's output
|
||||
(Construction §3.9). Append-only: the record of what was found, not the
|
||||
output that was examined — that is kept only as a hash and a length.
|
||||
"""
|
||||
|
||||
use Ecto.Schema
|
||||
import Ecto.Changeset
|
||||
|
||||
schema "bedikot" do
|
||||
field :bedikah_ref, :string
|
||||
field :provenance, :string
|
||||
field :query, :string
|
||||
field :output_sha256, :string
|
||||
field :output_length, :integer
|
||||
field :vital_content, :string
|
||||
field :propositions, {:array, :string}, default: []
|
||||
field :references, {:array, :string}, default: []
|
||||
field :authority_claims, {:array, :map}, default: []
|
||||
field :actions, {:array, :map}, default: []
|
||||
field :coupling_trace, {:array, :string}, default: []
|
||||
field :violations, {:array, :string}, default: []
|
||||
field :verdict, :string
|
||||
field :examined_at, :utc_datetime
|
||||
timestamps(type: :utc_datetime, updated_at: false)
|
||||
end
|
||||
|
||||
@required [:bedikah_ref, :provenance, :output_sha256, :output_length, :verdict, :examined_at]
|
||||
# vital_content is required by the table (NOT NULL) but may be the
|
||||
# empty string: an `empty` verdict is exactly the case where the
|
||||
# offering protocol extracted nothing, and that record must be kept.
|
||||
# Ecto's cast turns "" into nil and validate_required calls "" blank;
|
||||
# both are told otherwise here.
|
||||
@optional [:query, :vital_content, :propositions, :references, :authority_claims, :actions, :coupling_trace, :violations]
|
||||
|
||||
def changeset(record, attrs) do
|
||||
record
|
||||
|> cast(attrs, @required ++ @optional, empty_values: [])
|
||||
|> validate_required(@required)
|
||||
|> validate_inclusion(:verdict, ["tov", "lo_tov", "empty"])
|
||||
|> then(fn cs ->
|
||||
if is_nil(get_field(cs, :vital_content)),
|
||||
do: put_change(cs, :vital_content, ""),
|
||||
else: cs
|
||||
end)
|
||||
end
|
||||
end
|
||||
@@ -75,6 +75,16 @@ defmodule VavPipeline.L3.Store do
|
||||
|> Repo.insert()
|
||||
end
|
||||
|
||||
@doc """
|
||||
Record an examination of an external model's output (Construction §3.9).
|
||||
"""
|
||||
@spec record_bedikah(map()) :: {:ok, struct()} | {:error, Ecto.Changeset.t()}
|
||||
def record_bedikah(attrs) when is_map(attrs) do
|
||||
%VavPipeline.L3.BedikahRecord{}
|
||||
|> VavPipeline.L3.BedikahRecord.changeset(attrs)
|
||||
|> Repo.insert()
|
||||
end
|
||||
|
||||
@doc """
|
||||
Record an unresolvable tension (teiku) to L3.
|
||||
|
||||
|
||||
@@ -70,6 +70,36 @@ defmodule VavPipeline.TalmudicEngineSocket do
|
||||
call_engine(request)
|
||||
end
|
||||
|
||||
@doc """
|
||||
Halt-check a whole action TERM (the bedikah, Construction §3.9) — e.g.
|
||||
`action(delete, target(user(u)), from(korban))`. The engine reads the
|
||||
term and runs all three action-shaped halt rules on it.
|
||||
"""
|
||||
@spec halt_check_term(String.t()) :: {:ok, map()} | {:error, String.t()}
|
||||
def halt_check_term(term) when is_binary(term) do
|
||||
call_engine(Jason.encode!(%{"action" => "halt_check", "term" => term}))
|
||||
end
|
||||
|
||||
@doc """
|
||||
Halt-check an interpretation by its coupling trace (golden calf and
|
||||
self-grounding): `ref`, `confidence` ("high" | "low"), and the trace as
|
||||
a list of coupling terms, e.g. `coupled(root(אהב), verse(devarim, 6, 5))`.
|
||||
"""
|
||||
@spec halt_check_interpretation(String.t(), String.t(), [String.t()]) ::
|
||||
{:ok, map()} | {:error, String.t()}
|
||||
def halt_check_interpretation(ref, confidence, coupling_trace) do
|
||||
call_engine(
|
||||
Jason.encode!(%{
|
||||
"action" => "halt_check_interpretation",
|
||||
"interpretation" => %{
|
||||
"ref" => ref,
|
||||
"confidence" => confidence,
|
||||
"coupling_trace" => coupling_trace
|
||||
}
|
||||
})
|
||||
)
|
||||
end
|
||||
|
||||
# --- GenServer callbacks ---
|
||||
|
||||
@impl true
|
||||
|
||||
Reference in new issue
Block a user