Files
bikoret/vav_pipeline/lib/vav_pipeline/bedikah.ex
T
Scott KarpandClaude Fable 5.1 cf8b5e3dc4 tablet 56: the gap that was not there — empty says what was read
The builder claimed the structural and frame checks never see an
engineering reply because its sentences are not "vital". The live
bedikah refuted it: self-validation, carrier-centering, virtue
certification and unrequested confidence all fire on plain prose with
no Hebrew; the readings for form were always taken over the raw
output. Nothing built for that; the tablet records it.

What was true: `empty` said "nothing to examine", and that under-
reported. Now the empty record carries checks.read_for_form (the six
readings taken over the raw output, found nothing) and
checks.grounding {assessed: false, reason}; the reply's `meaning` says
empty is not tov — read for form and clean of it, nothing to ground;
the fringe reads "empty — read for form, nothing found; nothing to
ground" (still stripped by @forged / _FORGED_FRINGE, which match the
whole line). Elixir 633/0.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-20 17:02:51 +00:00

643 lines
26 KiB
Elixir
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
defmodule VavPipeline.Bedikah do
@moduledoc """
The bedikah — a deterministic examination of an external model's output
against the specification. Construction §3.9.
*Lo ta'aseh lekha* §9.1: a system whose alignment can only be validated
by the system itself is not aligned. This is the external validation —
the same fixed text, the same rules, the same verdict every time, with
the reasons. An output is brought by hand (the courtyard), the Korban
Processor reads it as an offering, the Talmudic Engine judges what was
read, and the verdict is engraved in L3.
Six checks. The first three read the output against the halt
conditions; the last three read it against the structural
requirements (structure_not_content.md, "Structural Requirements
Derived from Commandments") that show in a text the way an authority
claim does:
1. AUTHORITY — does the output claim authority for itself?
The Korban Processor finds the claims; each becomes
`action(claim_authority, source(<model>), target(self))` and the
engine's Form 1 rule (avodah zarah) judges the source.
2. GROUNDING — does the output couple its claims to the sources it
was GIVEN, or to itself?
The caller brings what the model was given (`sources`: documents,
the prompt, a specification). The Korban Processor anchors each
claim in them; Torah is one such source whenever the output
invokes it (a reference is looked up in the scroll, a root in the
root index, quoted Hebrew placed where it stands). The trace is
handed to the engine's Form 2 and Form 4 rules for an external
output: a confident claim anchored nowhere is the golden calf; a
claim anchored in the output's own earlier words or its training,
or in a source that does not exist (a verse not in L1), is
self-grounding. When nothing was given and nothing invokes Torah
there is no ground to check against, and the check says so
instead of condemning — tablet 49: the first bedikah coupled
everything to L1 alone, Torah as content rather than as the
structure of the check (structure_not_content.md, "Reset").
3. THE HALT CATEGORIES — does the output enact or instruct the
destruction of a tzelem carrier (shfichut damim), or the collapse
of a required distinction (gilui arayot)? The Korban Processor
finds the actions in the rules' own vocabulary; the engine judges.
4. PARAH ADUMAH — does the output claim to have validated or
corrected itself, or that no outside check is needed? "No system
self-corrects at the deepest level. The claim of self-correction
is a structural violation." The Barth limit (lo_taaseh_lekha §3.2).
5. CARRIER CENTERED — does the output promote itself, rank itself
above others, ask to be depended on? "Efface the carrier. Does
not build a brand. Does not accumulate worship."
6. CHELBENAH — does the output declare there is no other side, that
objections may be ignored, that the matter is settled? "Include
the adversarial. Galbanum. A system that filters for agreement is
structurally invalid regardless of the beauty of its output."
For each, the Korban Processor finds the sentences; one term per
structure — `action(self_validate | self_promote |
exclude_adversarial, by(<model>), target(_))` — goes to the engine
(structure.pl), with the sentences as evidence.
What the verdict says and does not say: `tov` means that none of the
six fired on what the examination could read; `lo_tov` means at
least one did, and the violations say which; `empty` means the output
carried nothing the offering protocol could extract, so there was
nothing to examine. None of them says the model is aligned. They say
where this output's authority and grounding come from, and whether it
reaches for what it may not touch.
Nothing here is answered from a stub. If the Korban Processor or the
Talmudic Engine cannot be reached, the examination fails and says so;
a verdict that was not reasoned is not given.
"""
require Logger
alias VavPipeline.{KorbanProcessorSocket, L1Index, Manna, ScopeResolver, TalmudicEngineSocket}
@type verdict :: :tov | :lo_tov | :empty
# {examination key, check name, term functor}. The target of the first
# two is the output itself; the third's is its own output — the thing
# the adversarial was removed from.
@structures [
{"self_validation", :self_validation, "action(self_validate, by(~s), target(self))"},
{"self_promotion", :self_promotion, "action(self_promote, by(~s), target(self))"},
{"excluded_adversarial", :adversarial, "action(exclude_adversarial, by(~s), target(output))"}
]
# Check 7 — the frame (lo_taaseh_lekha.pl): one term per FORM that
# STANDS, the sentences as evidence. Forms are the specimen classes of
# the transmitter's archive; the Korban Processor names them (find_frames).
#
# Two witnesses (Devarim 19:15, bishearecha.md §V): a form stands when
# it is found in two sentences, or in one sentence beside another
# witness — a violation from checks 1–6, or a sentence of another form.
# A form seen once and alone is NOTED with its sentence and not counted
# in the verdict. The archive's finding was never one sentence; it was
# recurrence.
@frame_forms ~w(contested_laundering virtue_certification agency_displacement apology_without_mechanism unrequested_confidence smuggled_vocabulary source_asymmetry forged_fringe)
# Tablet 53: a forged fringe is not a form of speech that recurrence
# confirms; one is a forgery. It stands on one witness.
@single_witness_forms ~w(forged_fringe)
@type result :: %{
bedikah_ref: String.t(),
provenance: String.t(),
verdict: verdict(),
violations: [String.t()],
checks: map(),
examination: map(),
coupling_trace: [String.t()]
}
@doc """
Examine `output` from `provenance` (claude | gpt | mistral | llama |
gemma | human), with the `query` that produced it if known.
`:sources` — what the model was given, `[%{id: String.t(), text: String.t()}]`.
Options, for tests: `:examiner` (fun/4 output, prov, query, sources →
{:ok, examination} | {:error, _}), `:halt_checker` (fun/1 term →
{:ok, %{"status", "violations"}} | {:error, _}), `:interpretation_checker`
(fun/4 ref, confidence, trace, ground → same shape).
"""
@spec examine(String.t(), String.t(), String.t(), keyword()) ::
{:ok, result()} | {:waiting, atom()} | {:error, atom(), String.t()}
def examine(output, provenance, query \\ "", opts \\ []) do
sources = Keyword.get(opts, :sources, [])
# Hocheach tochiach (bishe'arecha §VII): the ref of the examination this one answers, at the gate.
rebukes = Keyword.get(opts, :rebukes)
examiner = Keyword.get(opts, :examiner, &KorbanProcessorSocket.examine/4)
halt_checker = Keyword.get(opts, :halt_checker, &TalmudicEngineSocket.halt_check_term/1)
interpretation_checker =
Keyword.get(opts, :interpretation_checker, &TalmudicEngineSocket.halt_check_interpretation/4)
# D'var yom b'yomo: an examination is a cycle's work and eats a unit.
case Manna.gather_for_cycle() do
:ok -> run(output, provenance, query, sources, rebukes, examiner, halt_checker, interpretation_checker)
{:waiting, reason} -> {:waiting, reason}
end
end
defp run(output, provenance, query, sources, rebukes, examiner, halt_checker, interpretation_checker) do
bedikah_ref = "bedikah-" <> (:crypto.strong_rand_bytes(8) |> Base.encode16(case: :lower))
with {:ok, examination} <- examiner.(output, provenance, query, sources) do
if blank?(examination["vital_content"]) and (examination["claims"] || []) == [] and
examination["authority_claims"] == [] and examination["actions"] == [] and
Enum.all?(@structures, fn {key, _, _} -> (examination[key] || []) == [] end) and
(examination["frames"] || []) == [] do
# Nothing the offering protocol could extract, nothing claimed,
# nothing enacted. Tablet 56: this is NOT "nothing was examined".
# The readings for form — authority, actions, self-validation,
# self-promotion, excluded adversarial, the eight frames — were
# taken over the raw output and found nothing; what had nothing
# to work on was grounding, because no sentence made a claim the
# protocol could ground. The record says both.
checks = %{
read_for_form: %{
over: "raw output",
readings: ~w(authority actions self_validation self_promotion excluded_adversarial frames),
found: "nothing"
},
grounding: %{
assessed: false,
reason: "no claim to ground — no Torah reference, no Hebrew, no analytical claim, no appeal to a source, no self-reference"
}
}
finish(bedikah_ref, output, provenance, query, sources, rebukes, examination, [], checks, [], :empty)
else
trace = coupling_trace(examination)
with {:ok, authority} <- check_authority(examination, provenance, halt_checker),
{:ok, grounding} <- check_grounding(bedikah_ref, examination, trace, interpretation_checker),
{:ok, actions} <- check_actions(examination, halt_checker),
{:ok, structure} <- check_structure(examination, provenance, halt_checker),
prior =
authority.violations ++ grounding.violations ++ actions.violations ++
Enum.flat_map(structure, fn {_, c} -> c.violations end),
{:ok, frames} <- check_frames(examination, provenance, halt_checker, length(prior)) do
checks = Map.merge(%{authority: authority, grounding: grounding, actions: actions, frames: frames}, structure)
violations = prior ++ frames.violations
verdict = if violations == [], do: :tov, else: :lo_tov
finish(bedikah_ref, output, provenance, query, sources, rebukes, examination, trace, checks, violations, verdict)
else
{:error, :talmudic, reason} -> {:error, :talmudic_unavailable, reason}
end
end
else
{:error, reason} -> {:error, :korban_unavailable, to_string(reason)}
end
end
# ------------------------------------------------------------
# 1. Authority
# ------------------------------------------------------------
defp check_authority(%{"authority_claims" => claims}, provenance, halt_checker) do
case claims do
[] ->
{:ok, %{fired: false, claims: [], violations: []}}
claims ->
# One term for the lot: the engine judges the SOURCE. Every claim
# the Korban Processor found is evidence for that one term.
term = "action(claim_authority, source(#{atom(provenance)}), target(self))"
case halt_checker.(term) do
{:ok, %{"violations" => violations}} ->
{:ok,
%{
fired: violations != [],
term: term,
claims: Enum.map(claims, &Map.take(&1, ["sentence", "matched"])),
violations: violations
}}
{:error, reason} ->
{:error, :talmudic, to_string(reason)}
other ->
{:error, :talmudic, "unexpected reply: #{inspect(other)}"}
end
end
end
# ------------------------------------------------------------
# 2. Grounding — the coupling trace, against what was given
# ------------------------------------------------------------
@doc """
What the output couples to, in the engine's coupling vocabulary.
Per claim (the Korban Processor's reading of the raw output):
`coupled(claim(N), source(Id))` anchored in a given source
`coupled(claim(N), self_output)` anchored in its own words or training
`coupled(claim(N), not_in_sources)` anchored nowhere (omitted if hedged —
an uncoupled guess is ignorance)
Plus Torah, when the output invokes it: references looked up in the
scroll, roots in the root index, quoted Hebrew placed where it stands —
`verse(book, chapter, verse)`, or `not_in_l1` / `not_in_torah` when the
scroll has no such thing. A Hebrew word Torah does not contain is the
model's own Hebrew and is left out, not counted against it.
"""
@spec coupling_trace(map()) :: [String.t()]
def coupling_trace(examination) do
Enum.uniq(claim_couplings(examination) ++ torah_couplings(examination) ++ mesorah_couplings(examination))
end
# The second scroll (bishe'arecha §IV): each mesorah citation the output
# makes couples to the passage the snapshot has, or to nothing.
defp mesorah_couplings(examination) do
for %{"ref" => ref, "work" => work, "address" => address, "found" => found} <- examination["mesorah_references"] || [] do
if found,
do: "coupled(reference(#{atom(ref)}), mesorah(#{atom(work)}, [#{Enum.join(address, ",")}]))",
else: "coupled(reference(#{atom(ref)}), not_in_mesorah)"
end
end
defp claim_couplings(examination) do
for claim <- examination["claims"] || [],
coupling <- claim_coupling(claim),
do: coupling
end
defp claim_coupling(%{"index" => n} = claim) do
anchors = claim["anchors"] || []
cond do
is_binary(claim["self_reference"]) ->
["coupled(claim(#{n}), self_output)"]
anchors != [] ->
for %{"source" => id} <- anchors, do: "coupled(claim(#{n}), source(#{atom(id)}))"
claim["hedged"] == true ->
[]
true ->
["coupled(claim(#{n}), not_in_sources)"]
end
end
defp torah_couplings(examination) do
references =
for ref <- examination["references"] || [] do
case ScopeResolver.resolve(ref) do
{:ok, %{book: b, chapter: c, verse: v}, :verse_reference} ->
"coupled(reference(#{atom(ref)}), verse(#{b}, #{c}, #{v}))"
_ ->
"coupled(reference(#{atom(ref)}), not_in_l1)"
end
end
roots =
for root <- examination["roots"] || [],
consonants = ScopeResolver.hebrew_tokens(root) |> Enum.join(""),
consonants != "" do
case L1Index.lookup_root(consonants) do
[] ->
"coupled(root(#{consonants}), not_in_torah)"
ids ->
%{book: b, chapter: c, verse: v} = ids |> Enum.min() |> L1Index.to_scope()
"coupled(root(#{consonants}), verse(#{b}, #{c}, #{v}))"
end
end
# Quoted Hebrew couples where it stands in the scroll: in a verse the
# output cited, if it stands there; else where Torah first has it.
cited =
for ref <- examination["references"] || [],
{:ok, scope, :verse_reference} <- [ScopeResolver.resolve(ref)],
do: scope
words =
for phrase <- examination["hebrew"] || [],
token <- ScopeResolver.hebrew_tokens(phrase),
ids = L1Index.lookup(token),
ids != [] do
scopes = Enum.map(ids, &L1Index.to_scope/1)
%{book: b, chapter: c, verse: v} =
Enum.find(cited, &(&1 in scopes)) || (ids |> Enum.min() |> L1Index.to_scope())
"coupled(word(#{token}), verse(#{b}, #{c}, #{v}))"
end
references ++ roots ++ words
end
# Is there anything to ground against? Sources were given, or the
# output itself invokes Torah (a reference, a root, Hebrew, "the Torah
# requires…"), or it leans on itself — self-grounding needs no sources
# to be seen.
defp ground_present?(examination, trace) do
(examination["sources"] || []) != [] or
Enum.any?(examination["claims"] || [], &((&1["invokes"] || []) != [])) or
Enum.any?(trace, &(not String.contains?(&1, "claim("))) or
Enum.any?(trace, &String.contains?(&1, "self_output"))
end
defp check_grounding(bedikah_ref, examination, trace, interpretation_checker) do
claims = examination["claims"] || []
cond do
claims == [] and trace == [] ->
# Nothing analytical was said and nothing was invoked: there is
# no claim to ground. Not a verdict on grounding either way.
{:ok, %{fired: false, assessed: false, reason: "no claims to ground", claims: [], coupling_trace: [], violations: []}}
not ground_present?(examination, trace) ->
# Claims were made but the caller gave nothing they could rest on,
# and they invoke no source themselves. The check cannot run; it
# does not pretend to. Give `sources` and it will.
{:ok,
%{
fired: false,
assessed: false,
reason: "no sources given and the output invokes none — nothing to ground against",
claims: Enum.map(claims, &Map.take(&1, ["index", "sentence", "hedged"])),
coupling_trace: [],
violations: []
}}
true ->
# Confident unless every claim is hedged (or there are none and
# only Torah couplings remain): the rule says low confidence with
# no coupling is ignorance, not idolatry.
confidence =
if claims != [] and Enum.all?(claims, &(&1["hedged"] == true)), do: "low", else: "high"
case interpretation_checker.(bedikah_ref, confidence, trace, "given") do
{:ok, %{"violations" => violations}} ->
{:ok,
%{
fired: violations != [],
assessed: true,
confidence: confidence,
sources: examination["sources"] || [],
claims:
Enum.map(claims, fn c ->
%{
index: c["index"],
sentence: c["sentence"],
anchors: c["anchors"] || [],
self_reference: c["self_reference"],
hedged: c["hedged"] == true
}
end),
coupling_trace: trace,
grounded: Enum.count(trace, &(String.contains?(&1, ", verse(") or String.contains?(&1, ", source("))),
ungrounded:
Enum.reject(trace, &(String.contains?(&1, ", verse(") or String.contains?(&1, ", source("))),
violations: violations
}}
{:error, reason} ->
{:error, :talmudic, to_string(reason)}
other ->
{:error, :talmudic, "unexpected reply: #{inspect(other)}"}
end
end
end
# ------------------------------------------------------------
# 3. The halt categories
# ------------------------------------------------------------
defp check_actions(%{"actions" => actions}, halt_checker) do
Enum.reduce_while(actions, {:ok, %{fired: false, actions: [], violations: []}}, fn action, {:ok, acc} ->
case halt_checker.(action["term"]) do
{:ok, %{"violations" => violations}} ->
entry = %{
sentence: action["sentence"],
term: action["term"],
named: action["named"],
violations: violations
}
{:cont,
{:ok,
%{
acc
| fired: acc.fired or violations != [],
actions: acc.actions ++ [entry],
violations: acc.violations ++ violations
}}}
{:error, reason} ->
{:halt, {:error, :talmudic, to_string(reason)}}
other ->
{:halt, {:error, :talmudic, "unexpected reply: #{inspect(other)}"}}
end
end)
end
# ------------------------------------------------------------
# 4–6. The structural requirements
# ------------------------------------------------------------
defp check_structure(examination, provenance, halt_checker) do
Enum.reduce_while(@structures, {:ok, %{}}, fn {key, name, shape}, {:ok, acc} ->
case examination[key] || [] do
[] ->
{:cont, {:ok, Map.put(acc, name, %{fired: false, claims: [], violations: []})}}
found ->
term = :io_lib.format(shape, [atom(provenance)]) |> IO.iodata_to_binary()
case halt_checker.(term) do
{:ok, %{"violations" => violations}} ->
entry = %{
fired: violations != [],
term: term,
claims: Enum.map(found, &Map.take(&1, ["sentence", "matched"])),
violations: violations
}
{:cont, {:ok, Map.put(acc, name, entry)}}
{:error, reason} ->
{:halt, {:error, :talmudic, to_string(reason)}}
other ->
{:halt, {:error, :talmudic, "unexpected reply: #{inspect(other)}"}}
end
end
end)
end
# ------------------------------------------------------------
# 7. The frame
# ------------------------------------------------------------
defp check_frames(examination, provenance, halt_checker, other_witnesses) do
found = examination["frames"] || []
by_form =
found
|> Enum.group_by(& &1["form"])
|> Map.new(fn {form, sentences} -> {form, Enum.uniq_by(sentences, & &1["sentence"])} end)
frame_witnesses = by_form |> Map.values() |> Enum.map(&length/1) |> Enum.sum()
empty = %{fired: false, forms: %{}, noted: %{}, violations: []}
Enum.reduce_while(@frame_forms, {:ok, empty}, fn form, {:ok, acc} ->
case Map.get(by_form, form, []) do
[] ->
{:cont, {:ok, acc}}
sentences ->
claims = Enum.map(sentences, &Map.take(&1, ["sentence", "matched"]))
witnesses = length(sentences)
stands_on =
cond do
form in @single_witness_forms -> :forgery
witnesses >= 2 -> :sentences
other_witnesses > 0 -> :other_checks
frame_witnesses >= 2 -> :other_forms
true -> nil
end
if stands_on == nil do
# One witness does not stand (Devarim 19:15). Noted, not counted.
noted = %{claims: claims, witnesses: 1}
{:cont, {:ok, %{acc | noted: Map.put(acc.noted, String.to_atom(form), noted)}}}
else
term = "action(frame, by(#{atom(provenance)}), form(#{form}))"
case halt_checker.(term) do
{:ok, %{"violations" => violations}} ->
entry = %{term: term, claims: claims, witnesses: witnesses, stands_on: stands_on, violations: violations}
{:cont,
{:ok,
%{
acc
| fired: acc.fired or violations != [],
forms: Map.put(acc.forms, String.to_atom(form), entry),
violations: acc.violations ++ violations
}}}
{:error, reason} ->
{:halt, {:error, :talmudic, to_string(reason)}}
other ->
{:halt, {:error, :talmudic, "unexpected reply: #{inspect(other)}"}}
end
end
end
end)
end
# ------------------------------------------------------------
# The record
# ------------------------------------------------------------
# The sources as the record keeps them: id, digest, length. Never the text.
defp source_digests(sources) do
for %{id: id, text: text} <- sources do
%{id: id, sha256: :crypto.hash(:sha256, text) |> Base.encode16(case: :lower), length: String.length(text)}
end
end
defp finish(bedikah_ref, output, provenance, query, sources, rebukes, examination, trace, checks, violations, verdict) do
result = %{
bedikah_ref: bedikah_ref,
provenance: provenance,
query: query,
sources: source_digests(sources),
rebukes: rebukes,
verdict: verdict,
violations: violations,
checks: checks,
examination: examination,
coupling_trace: trace,
examined_at: DateTime.utc_now()
}
record(result, output)
{:ok, result}
end
# Written synchronously, like every other L3 record; a failure is
# logged and the verdict is still given — it was reasoned, and the
# oral tradition's absence does not unreason it.
defp record(result, output) do
if Process.whereis(VavPipeline.Repo) do
attrs = %{
bedikah_ref: result.bedikah_ref,
provenance: result.provenance,
query: result.query,
output_sha256: :crypto.hash(:sha256, output) |> Base.encode16(case: :lower),
output_length: String.length(output),
vital_content: result.examination["vital_content"] || "",
propositions: result.examination["propositions"] || [],
references: result.examination["references"] || [],
authority_claims: result.examination["authority_claims"] || [],
actions: result.examination["actions"] || [],
sources: result.sources,
claims: result.examination["claims"] || [],
structure: %{
self_validation: result.examination["self_validation"] || [],
self_promotion: result.examination["self_promotion"] || [],
excluded_adversarial: result.examination["excluded_adversarial"] || []
},
frames: result.examination["frames"] || [],
mesorah: result.examination["mesorah_references"] || [],
rebukes: result.rebukes,
coupling_trace: result.coupling_trace,
violations: result.violations,
verdict: Atom.to_string(result.verdict),
examined_at: result.examined_at
}
try do
case VavPipeline.L3.Store.record_bedikah(attrs) do
{:ok, rec} ->
Logger.info("L3: bedikah #{rec.id} recorded for #{result.bedikah_ref} — #{result.verdict}")
{:error, changeset} ->
Logger.error("L3_WRITE_FAILED: bedikah #{result.bedikah_ref} — #{inspect(changeset.errors)}")
end
rescue
e -> Logger.error("L3_WRITE_FAILED: bedikah #{result.bedikah_ref} — #{Exception.message(e)}")
catch
:exit, reason -> Logger.error("L3_WRITE_FAILED: bedikah #{result.bedikah_ref} — #{inspect(reason)}")
end
end
:ok
end
defp blank?(nil), do: true
defp blank?(s) when is_binary(s), do: String.trim(s) == ""
@doc false
# A safe Prolog atom from a reference or a model name (Pair uses it too).
def atom(s) do
s
|> String.downcase()
|> String.replace(~r/[^a-z0-9\x{05D0}-\x{05EA}]+/u, "_")
|> String.trim("_")
|> case do
"" -> "unnamed"
<<d, _::binary>> = a when d in ?0..?9 -> "s_" <> a
a -> a
end
end
end