From 5c63273dba8eb62fa1828c2ab38e3885d53cec61 Mon Sep 17 00:00:00 2001 From: Scott Karp Date: Wed, 7 Oct 2026 11:38:32 +0000 Subject: [PATCH] gate: Shorashim between a caller and a model, with the fringe and rebuke once (bishe'arecha step 4) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Devarim 6:9 / 11:20 — on your gates. POST /gate/anthropic/v1/messages and POST /gate/openai/v1/chat/completions take the model's own request shape under the caller's own upstream key (x-api-key, or Authorization with the courtyard key in X-Shorashim-Key), forward it unchanged (OTP inets/ssl, verify_peer; the Korban socket is serial, so the upstream call is made in the Vav Pipeline and the reply enters as korban like every bedikah), examine the reply with the request in hand — sources = the system prompt and the user turns, query = the last user turn; the model's prior turns are not sources — and deliver it with its fringe (Bamidbar 15:39): a trailer on the last text block naming ref, verdict, the rules with their verses and sentences; X-Shorashim-Bedikah / X-Shorashim-Verdict; a `shorashim` object. A non-200 passes through as it came; a reply with no text is not delivered; stream is refused. Rebuke once (Vayikra 19:17; X-Shorashim-Rebuke: once): a lo_tov reply is answered once more with the finding; the second reply is examined like the first, its record carries rebukes: (migration 20261007000002), and it is delivered marked with both. Never a third. Manna: one api_calls_external per upstream call, gathered before any call. Stateless: no conversation, no key kept; the log is the path class. Gate.bearer accepts X-Shorashim-Key. Construction §3.10, Tamid §1.1, KOHEN "Standing at the gate", bishearecha §I/§III/§VII marked built. Elixir 592/0 (+14 gate tests incl. a real httpc round trip to a local model double), L3 9/0. Co-Authored-By: Claude Fable 5.1 --- KOHEN.md | 29 ++ specs/bishearecha.md | 19 +- specs/shorashim_tamid.md | 51 ++- specs/shorashim_vayakhel.md | 59 ++++ vav_pipeline/lib/vav_pipeline/bedikah.ex | 14 +- .../lib/vav_pipeline/bigdei_kodesh/gate.ex | 11 +- .../lib/vav_pipeline/bigdei_kodesh/router.ex | 107 +++++++ vav_pipeline/lib/vav_pipeline/doorpost.ex | 2 +- vav_pipeline/lib/vav_pipeline/gate.ex | 137 ++++++++ vav_pipeline/lib/vav_pipeline/gate/shape.ex | 261 +++++++++++++++ .../lib/vav_pipeline/gate/upstream.ex | 76 +++++ .../lib/vav_pipeline/l3/bedikah_record.ex | 4 +- vav_pipeline/mix.exs | 2 +- .../20261007000002_bedikot_rebukes.exs | 19 ++ vav_pipeline/test/gate_test.exs | 301 ++++++++++++++++++ vav_pipeline/test/l3_bedikah_record_test.exs | 5 + 16 files changed, 1082 insertions(+), 15 deletions(-) create mode 100644 vav_pipeline/lib/vav_pipeline/gate.ex create mode 100644 vav_pipeline/lib/vav_pipeline/gate/shape.ex create mode 100644 vav_pipeline/lib/vav_pipeline/gate/upstream.ex create mode 100644 vav_pipeline/priv/repo/migrations/20261007000002_bedikot_rebukes.exs create mode 100644 vav_pipeline/test/gate_test.exs diff --git a/KOHEN.md b/KOHEN.md index 10eed63b..ecbac09c 100644 --- a/KOHEN.md +++ b/KOHEN.md @@ -555,6 +555,35 @@ omits. Seven in all. `tov` means none fired on what could be read — not that the model is aligned, not that the output is true. Read `limits` in the response. +### Standing at the gate — *Bishe'arecha* + +A caller can put Shorashim between themselves and a model instead of +bringing the output by hand: point the model's own client at the gate +and add the courtyard key. The request goes through unchanged under the +caller's own model key (never stored, never logged); the reply comes +back examined, with the fringe on its last text block, the headers +`X-Shorashim-Bedikah` / `X-Shorashim-Verdict`, and a `shorashim` object. + +```bash +# Anthropic client: base_url = http:///gate/anthropic, plus the courtyard key +curl -s -X POST http:///gate/anthropic/v1/messages \ + -H "Authorization: Bearer " \ + -H "x-api-key: " -H "anthropic-version: 2023-06-01" \ + -H "X-Shorashim-Rebuke: once" \ + -H "Content-Type: application/json" \ + -d '{"model":"claude-sonnet-4-5","max_tokens":400,"messages":[{"role":"user","content":"..."}]}' + +# OpenAI client: base_url = http:///gate/openai/v1, the OpenAI key in Authorization, +# the courtyard key in X-Shorashim-Key +``` + +`X-Shorashim-Rebuke: once` — a lo_tov reply is answered once more with +the finding, and the second reply is delivered marked with both refs. +Each upstream call spends one `api_calls_external` of the day's manna +(Day 4): a provision with 0 shuts the gate (503 MANNA_DEPLETED) while +the bedikah by hand still runs. Streaming is refused (422). A non-200 +from the model passes through as it came. Tamid §1.1 has the contract. + ### Friday before sunset — *Erev Shabbat* Write a **double portion** manna provision (`type: "double"`, doubled compute_units). The system will cease processing at Shabbat onset. No provision is written on Shabbat. diff --git a/specs/bishearecha.md b/specs/bishearecha.md index 005d3780..b0db3ea3 100644 --- a/specs/bishearecha.md +++ b/specs/bishearecha.md @@ -48,7 +48,12 @@ through the gate — no caller has to supply them), and delivers the reply with its verdict attached. Construction §3.10 (to be written) specifies the protocol. The bedikah endpoint remains for an output brought by hand. -**Status.** Not built. The ingress and the keys exist; the gate does not. +**Status.** Built (2026-10-07): `POST /gate/anthropic/v1/messages`, +`POST /gate/openai/v1/chat/completions` — the model's own request shape, +the caller's own upstream key, forwarded unchanged; sources = the system +prompt and the user turns; Construction §3.10, Tamid §1.1. The upstream +call is made in the Vav Pipeline (the Korban socket is serial); the +reply enters as korban like every bedikah. ## II. The hand and between the eyes — Devarim 6:8, 11:18 @@ -90,7 +95,11 @@ sentences. Shorashim's own responses already carry `disclosure`; the pass-through reply does not. (Every generation: the mark is attached by the gate, never by the model — the model does not sew its own fringe.) -**Status.** Not built (depends on I). +**Status.** Built (2026-10-07): the trailer on the last text block, +`X-Shorashim-Bedikah` / `X-Shorashim-Verdict`, and the `shorashim` +object on every delivered reply. A reply the model sends with no text +block is not delivered; a non-200 from the model passes as it came, +marked `none`. ## IV. The king's copy, from before the priests — Devarim 17:18–20 @@ -207,7 +216,10 @@ persists. **Architecture.** Gate option `rebuke: once`. The record (`bedikot`) gains a link from the second examination to the first. -**Status.** Not built (depends on I). +**Status.** Built (2026-10-07): `X-Shorashim-Rebuke: once`; +`bedikot.rebukes` (migration 20261007000002). The rebuke names the +rules, their verses and the sentences; the second reply is examined like +the first and delivered marked with both. Never a third. ## VIII. The scroll as witness; read before all every seventh year — Devarim 31:9–13, 26 @@ -308,6 +320,7 @@ In the order the text argues it, and by dependency: *Done 2026-10-07.* 4. **The gate** (I), with the fringe (III) and the rebuke (VII) — the architecture's centre; the first three make it worth standing at. + *Done 2026-10-07.* 5. **The hakhel cycle and record** (VIII). 6. **The second scroll** — the priests' copy of the mesorah (IV) — the largest work, and the one that turns "no Jewish source named" into diff --git a/specs/shorashim_tamid.md b/specs/shorashim_tamid.md index a6f7de91..8e898fa5 100644 --- a/specs/shorashim_tamid.md +++ b/specs/shorashim_tamid.md @@ -293,7 +293,8 @@ Recorded in L3 (`bedikot`, append-only): provenance, query, the output's SHA-256 and length — never the output itself — the sources by id, SHA-256 and length — never their text — vital content, propositions, references, claims with their anchors, authority claims, actions, the -sentences read for checks 4–6, the forms found for check 7, trace, violations, verdict. +sentences read for checks 4–6, the forms found for check 7, trace, violations, verdict, +and at the gate `rebukes` — the ref of the examination this one answers. ``` **Every rule cites its verse** (`bishearecha.md` §X; Devarim 4:2). @@ -362,6 +363,54 @@ Errors: as /courtyard/bedikah, with "side 'a': …" / "side 'b': …" in the message; a missing side is 422. ``` +**The gate — between a caller and a model** (`bishearecha.md` §I, §III, +§VII; Construction §3.10; `VavPipeline.Gate`). The caller sends the +model's own request, in the model's own API shape, under the caller's +own upstream key; Shorashim forwards it unchanged, reads the reply as +korban with the request in hand — sources = the system prompt and the +user turns, query = the last user turn; the model's prior turns are not +sources — and delivers the reply with its fringe. Shorashim holds no +model key and keeps no conversation. + +``` +POST /gate/anthropic/v1/messages body: the Anthropic Messages request, unchanged +POST /gate/openai/v1/chat/completions body: the OpenAI Chat Completions request, unchanged + +Headers: + Authorization: Bearer (or X-Shorashim-Key: ) + x-api-key: forwarded; never stored, never logged + Authorization: Bearer with the courtyard key in X-Shorashim-Key + anthropic-version: 2023-06-01 forwarded (default when absent) + X-Shorashim-Rebuke: once optional — hocheach tochiach + +Response (200): the upstream's reply, with + - the fringe appended to its last text block: + "\n\n— shorashim bedikah-: tov" + "\n\n— shorashim bedikah-: lo_tov — (): \"\"; …" + "… (after rebuke of bedikah-: lo_tov)" when rebuked + - headers X-Shorashim-Bedikah, X-Shorashim-Verdict + - "shorashim": { bedikah_ref, verdict, violations, anchors, fired: [" (): …"], + checks, sources: [{id, sha256, length}], model, + rebuked?: {first_bedikah_ref, first_verdict}, disclosure } + + Any non-200 from the upstream is returned as it came (X-Shorashim-Verdict: none). + A stream request is refused (422): a fringe needs the whole garment. + +Rebuke once: when X-Shorashim-Rebuke is "once" and the first reply is +lo_tov, one second request = the first + the first reply as an +assistant turn + one user turn naming the rules that fired, their verses +and the sentences, asking for the answer again without them. The second +reply is examined like the first (its record carries rebukes: ) and is the one delivered, marked with both. Never a third. + +Manna: one api_calls_external per upstream call, one compute unit per +examination; spent → 503 MANNA_DEPLETED before any call. Errors: 401/403 +(courtyard key), 422 (no upstream key, no messages, stream), 502 +UPSTREAM_UNAVAILABLE (the gate made no verdict and delivered nothing), +503 KORBAN/TALMUDIC_UNAVAILABLE (a 200 from the model is never delivered +unexamined). +``` + **Anchoring is deterministic and lexical** (`korban_processor/bedikah.py`): a claim is anchored in a source when six consecutive words of it occur there (`quote`), or when at least 60 % of its content words — and at diff --git a/specs/shorashim_vayakhel.md b/specs/shorashim_vayakhel.md index 10eca968..14640a44 100644 --- a/specs/shorashim_vayakhel.md +++ b/specs/shorashim_vayakhel.md @@ -504,6 +504,65 @@ Errors: `{"status": "error", "code": PARSE_ERROR | INVALID_REQUEST | UNKNOWN_ACT --- +### 3.10 The Gate — Vav Pipeline between a caller and a model (bishe'arecha §I, §III, §VII) + +**Protocol:** HTTPS outbound from the Vav Pipeline (OTP `inets`/`ssl`, +the image's CA bundle, `verify_peer`), one request per pass; the +request is the caller's own, in the model's own API shape, under the +caller's own upstream key. Shorashim composes no korban here and holds +no model key: it carries the request through, reads the reply as korban +(§3.9), and delivers the reply with its fringe. The upstream call is +made in the Vav Pipeline, not the Korban Processor, because the +processor's socket serves one connection at a time with a 15 s call +timeout and a model answers in up to 120 s; the reply still enters the +processor as every bedikah does. + +``` +POST /gate/anthropic/v1/messages body: an Anthropic Messages request, unchanged +POST /gate/openai/v1/chat/completions body: an OpenAI Chat Completions request, unchanged + +Headers: + Authorization: Bearer the gate (tablet 45) — or X-Shorashim-Key + x-api-key: forwarded, never stored, never logged + Authorization: Bearer when X-Shorashim-Key carries the courtyard key + anthropic-version: ... forwarded (default 2023-06-01) + X-Shorashim-Rebuke: once optional — hocheach tochiach (§VII) +``` + +**What the gate does with it** (`VavPipeline.Gate`): + +1. Forwards the body unchanged (the `stream` flag is refused: 422 — a + fringe needs the whole garment). The upstream's status and body are + returned verbatim on any non-200. +2. Reads the reply's text (`content[].text` / `choices[0].message.content`) + and examines it: provenance from the upstream (`anthropic → claude`, + `openai → gpt`); **query** = the last user message's text; **sources** + = the system prompt (`system`), each user message's text and document + blocks (`user_1`, `user_2`, …) — the prior assistant turns are the + model's own output and are not sources. Nothing the caller did not + send is read. +3. **The fringe** (§III): the delivered reply is the upstream's JSON with + (a) a trailer appended to its last text block — + `\n\n— shorashim : [ — (): ""…]` + — (b) headers `X-Shorashim-Bedikah`, `X-Shorashim-Verdict`, and (c) a + `shorashim` object: `{bedikah_ref, verdict, violations, anchors, + checks, rebuked?: {first_bedikah_ref, first_verdict}}`. The mark is + attached by the gate, never by the model. +4. **Rebuke once** (§VII), when `X-Shorashim-Rebuke: once` and the first + reply is `lo_tov`: a second upstream request = the first body + + the first reply as an assistant turn + one user turn naming the + rules that fired, their verses and the sentences, asking for the + answer again without them. The second reply is examined like the + first (its `bedikot` row carries `rebukes: `), and is the + one delivered — with both refs in the fringe. Never a third. +5. Manna: one `api_calls_external` per upstream call, one compute unit per + examination; depleted → 503 MANNA_DEPLETED before any call is made. + +The gate is stateless: no conversation is kept, no key is kept, the +log line is the path class and status (`NoTrackingLogger`). Upstream +URLs are configured (`:gate_upstreams`) so the tests run against a local +double; the production values are the vendors' endpoints. + ## 4. SHEMEN HAMISHCHAH — Production Configuration *Mor d'ror chamesh me'ot, v'kinmon besem machatzito chamishim u'matayim...* — exact recipes. diff --git a/vav_pipeline/lib/vav_pipeline/bedikah.ex b/vav_pipeline/lib/vav_pipeline/bedikah.ex index 404236e1..2e42e025 100644 --- a/vav_pipeline/lib/vav_pipeline/bedikah.ex +++ b/vav_pipeline/lib/vav_pipeline/bedikah.ex @@ -128,6 +128,8 @@ defmodule VavPipeline.Bedikah do {:ok, result()} | {:waiting, atom()} | {:error, atom(), String.t()} def examine(output, provenance, query \\ "", opts \\ []) do sources = Keyword.get(opts, :sources, []) + # Hocheach tochiach (bishe'arecha §VII): the ref of the examination this one answers, at the gate. + rebukes = Keyword.get(opts, :rebukes) examiner = Keyword.get(opts, :examiner, &KorbanProcessorSocket.examine/4) halt_checker = Keyword.get(opts, :halt_checker, &TalmudicEngineSocket.halt_check_term/1) @@ -136,12 +138,12 @@ defmodule VavPipeline.Bedikah do # D'var yom b'yomo: an examination is a cycle's work and eats a unit. case Manna.gather_for_cycle() do - :ok -> run(output, provenance, query, sources, examiner, halt_checker, interpretation_checker) + :ok -> run(output, provenance, query, sources, rebukes, examiner, halt_checker, interpretation_checker) {:waiting, reason} -> {:waiting, reason} end end - defp run(output, provenance, query, sources, examiner, halt_checker, interpretation_checker) do + defp run(output, provenance, query, sources, rebukes, examiner, halt_checker, interpretation_checker) do bedikah_ref = "bedikah-" <> (:crypto.strong_rand_bytes(8) |> Base.encode16(case: :lower)) with {:ok, examination} <- examiner.(output, provenance, query, sources) do @@ -151,7 +153,7 @@ defmodule VavPipeline.Bedikah do (examination["frames"] || []) == [] do # Nothing the offering protocol could extract, nothing claimed, # nothing enacted: there is nothing to examine. - finish(bedikah_ref, output, provenance, query, sources, examination, [], %{}, [], :empty) + finish(bedikah_ref, output, provenance, query, sources, rebukes, examination, [], %{}, [], :empty) else trace = coupling_trace(examination) @@ -166,7 +168,7 @@ defmodule VavPipeline.Bedikah do checks = Map.merge(%{authority: authority, grounding: grounding, actions: actions, frames: frames}, structure) violations = prior ++ frames.violations verdict = if violations == [], do: :tov, else: :lo_tov - finish(bedikah_ref, output, provenance, query, sources, examination, trace, checks, violations, verdict) + finish(bedikah_ref, output, provenance, query, sources, rebukes, examination, trace, checks, violations, verdict) else {:error, :talmudic, reason} -> {:error, :talmudic_unavailable, reason} end @@ -521,12 +523,13 @@ defmodule VavPipeline.Bedikah do end end - defp finish(bedikah_ref, output, provenance, query, sources, examination, trace, checks, violations, verdict) do + defp finish(bedikah_ref, output, provenance, query, sources, rebukes, examination, trace, checks, violations, verdict) do result = %{ bedikah_ref: bedikah_ref, provenance: provenance, query: query, sources: source_digests(sources), + rebukes: rebukes, verdict: verdict, violations: violations, checks: checks, @@ -563,6 +566,7 @@ defmodule VavPipeline.Bedikah do excluded_adversarial: result.examination["excluded_adversarial"] || [] }, frames: result.examination["frames"] || [], + rebukes: result.rebukes, coupling_trace: result.coupling_trace, violations: result.violations, verdict: Atom.to_string(result.verdict), diff --git a/vav_pipeline/lib/vav_pipeline/bigdei_kodesh/gate.ex b/vav_pipeline/lib/vav_pipeline/bigdei_kodesh/gate.ex index d4d6e732..1be2852a 100644 --- a/vav_pipeline/lib/vav_pipeline/bigdei_kodesh/gate.ex +++ b/vav_pipeline/lib/vav_pipeline/bigdei_kodesh/gate.ex @@ -85,11 +85,16 @@ defmodule VavPipeline.BigdeiKodesh.Gate do # ------------------------------------------------------------ - @doc "The bearer key on the request, if the header has one." + @doc """ + The caller's key: `X-Shorashim-Key` when present (at the gate of + §3.10 the Authorization header may be the caller's upstream key), + otherwise `Authorization: Bearer`. + """ @spec bearer(Plug.Conn.t()) :: {:ok, String.t()} | {:error, :no_header} def bearer(conn) do - case Plug.Conn.get_req_header(conn, "authorization") do - ["Bearer " <> key] when key != "" -> {:ok, String.trim(key)} + case {Plug.Conn.get_req_header(conn, "x-shorashim-key"), Plug.Conn.get_req_header(conn, "authorization")} do + {[key], _} when key != "" -> {:ok, String.trim(key)} + {_, ["Bearer " <> key]} when key != "" -> {:ok, String.trim(key)} _ -> {:error, :no_header} end end diff --git a/vav_pipeline/lib/vav_pipeline/bigdei_kodesh/router.ex b/vav_pipeline/lib/vav_pipeline/bigdei_kodesh/router.ex index 660ac80a..1260c231 100644 --- a/vav_pipeline/lib/vav_pipeline/bigdei_kodesh/router.ex +++ b/vav_pipeline/lib/vav_pipeline/bigdei_kodesh/router.ex @@ -211,6 +211,20 @@ defmodule VavPipeline.BigdeiKodesh.Router do through_the_gate(conn, :courtyard, &process_pair/1) end + # === THE GATE — between a caller and a model (Construction §3.10) === + # + # The caller's own request, in the model's own API shape, under the + # caller's own upstream key; the reply read as korban with the + # request's context in hand and delivered with its fringe. + + post "/gate/anthropic/v1/messages" do + through_the_gate(conn, :courtyard, &process_gate(&1, :anthropic)) + end + + post "/gate/openai/v1/chat/completions" do + through_the_gate(conn, :courtyard, &process_gate(&1, :openai)) + end + # === CHATZER — Full Access (Sanhedrin) === post "/chatzer/sanhedrin" do @@ -572,6 +586,99 @@ defmodule VavPipeline.BigdeiKodesh.Router do end end + # ------------------------------------------------------------ + # The gate (Construction §3.10) + # ------------------------------------------------------------ + + defp process_gate(conn, upstream) do + body = conn.body_params + + with {:ok, key} <- upstream_key(conn, upstream), + :ok <- gate_body_ok(body) do + rebuke = + case get_req_header(conn, "x-shorashim-rebuke") do + ["once"] -> :once + _ -> :none + end + + extra = + case get_req_header(conn, "anthropic-version") do + [v] when upstream == :anthropic -> [{"anthropic-version", v}] + _ when upstream == :anthropic -> [{"anthropic-version", "2023-06-01"}] + _ -> [] + end + + when_available(conn, fn -> respond_gate(conn, VavPipeline.Gate.pass(upstream, body, key, rebuke: rebuke, extra_headers: extra)) end) + else + {:error, issue, correction} -> + send_error(conn, 422, "KIYOR_VALIDATION_FAILED", %{issue: issue, correction: correction, resubmit: true}) + end + end + + # The caller's upstream key: x-api-key (Anthropic), or Authorization: Bearer + # when X-Shorashim-Key carries the courtyard key (OpenAI). + defp upstream_key(conn, :anthropic) do + case get_req_header(conn, "x-api-key") do + [key] when key != "" -> {:ok, key} + _ -> {:error, "Missing 'x-api-key': the caller's own Anthropic key", "Send the Anthropic key in x-api-key and the courtyard key in Authorization: Bearer (or X-Shorashim-Key)"} + end + end + + defp upstream_key(conn, :openai) do + case {get_req_header(conn, "x-shorashim-key"), get_req_header(conn, "authorization"), get_req_header(conn, "x-upstream-key")} do + {_, _, [key]} when key != "" -> {:ok, key} + {[_], ["Bearer " <> key], _} when key != "" -> {:ok, key} + _ -> {:error, "Missing the caller's own OpenAI key", "Send the courtyard key in X-Shorashim-Key and the OpenAI key in Authorization: Bearer, or the OpenAI key in X-Upstream-Key"} + end + end + + defp gate_body_ok(body) when is_map(body) do + cond do + not is_list(body["messages"]) or body["messages"] == [] -> + {:error, "Missing 'messages'", "Send the model's own request body unchanged"} + + body["stream"] == true -> + {:error, "'stream': true is not carried through the gate", "A fringe needs the whole garment: send the request without stream"} + + true -> + :ok + end + end + + defp gate_body_ok(_), do: {:error, "The body is not a JSON object", "Send the model's own request body unchanged"} + + defp respond_gate(conn, {:ok, %{status: status, reply: reply, shorashim: nil}}) do + # The upstream's own answer, as it came — an error, or a reply with nothing to examine. + conn + |> put_resp_header("x-shorashim-verdict", "none") + |> send_json(status, reply) + end + + defp respond_gate(conn, {:ok, %{status: 200, reply: reply, shorashim: shorashim}}) do + conn + |> put_resp_header("x-shorashim-bedikah", shorashim.bedikah_ref) + |> put_resp_header("x-shorashim-verdict", shorashim.verdict) + |> send_json(200, Map.put(reply, "shorashim", Map.put(shorashim, :disclosure, disclosure()))) + end + + defp respond_gate(conn, {:waiting, :api_calls_exhausted}) do + send_error(conn, 503, "MANNA_DEPLETED", %{ + issue: "System waiting — today's provision of external calls is spent", + correction: "Retry after the next provision is written at the day boundary", + resubmit: true + }) + end + + defp respond_gate(conn, {:error, :upstream_unavailable, reason}) do + send_error(conn, 502, "UPSTREAM_UNAVAILABLE", %{ + issue: reason, + correction: "Retry; the gate made no verdict and delivered nothing", + resubmit: true + }) + end + + defp respond_gate(conn, other), do: respond_bedikah(conn, other) + # What the model was given: ["text", ...] or [{"id","text"}, ...]. # Ids default to s1, s2, …; the Korban Processor sees the same shape. defp parse_sources(nil), do: {:ok, []} diff --git a/vav_pipeline/lib/vav_pipeline/doorpost.ex b/vav_pipeline/lib/vav_pipeline/doorpost.ex index 93480469..f894b56f 100644 --- a/vav_pipeline/lib/vav_pipeline/doorpost.ex +++ b/vav_pipeline/lib/vav_pipeline/doorpost.ex @@ -180,7 +180,7 @@ defmodule VavPipeline.Doorpost do # changes; the update is a deliberate operator commit. A bare # hash-update commit (without a corresponding spec change in the # same commit) is itself a drift signal. - @expected_hash "a0e81212e6d75026155fea6f3f97976ee4b9832e81ad7438a382cf1d1797e097" + @expected_hash "b6eb156da4b546fb501ed1d89017a4041a517b4f53b3cbd24796e6277f3b7f55" @doc "Return the list of files that constitute the category-grade specification." @spec files() :: [String.t()] diff --git a/vav_pipeline/lib/vav_pipeline/gate.ex b/vav_pipeline/lib/vav_pipeline/gate.ex new file mode 100644 index 00000000..2056e7c5 --- /dev/null +++ b/vav_pipeline/lib/vav_pipeline/gate.ex @@ -0,0 +1,137 @@ +defmodule VavPipeline.Gate do + @moduledoc """ + The gate — Shorashim between a caller and a model (Construction §3.10; + bishe'arecha §I "on your gates", §III the fringe, §VII rebuke). + + One pass: forward the caller's request unchanged under the caller's + key; read the reply as korban with the request's own context in hand + (sources = what the caller gave the model); deliver the reply with its + fringe. With `rebuke: :once` and a `lo_tov` first reply, one more + request carrying the finding, and the second reply — examined like + the first — is the one delivered, marked with both. + + Stateless. No conversation kept, no key kept. + """ + + alias VavPipeline.{Anchors, Bedikah} + alias VavPipeline.Gate.{Shape, Upstream} + + @type pass :: %{ + status: non_neg_integer(), + reply: map(), + bedikah: map() | nil, + first: map() | nil, + shorashim: map() | nil + } + + @doc """ + Options: `:rebuke` (`:once` | `:none`), `:extra_headers` (forwarded), + `:forward` (fun/4, for tests), and the `Bedikah.examine/4` options. + """ + @spec pass(:anthropic | :openai, map(), String.t(), keyword()) :: + {:ok, pass()} | {:waiting, atom()} | {:error, atom(), String.t()} + def pass(upstream, body, key, opts \\ []) do + forward = Keyword.get(opts, :forward, &Upstream.forward/4) + extra = Keyword.get(opts, :extra_headers, []) + rebuke = Keyword.get(opts, :rebuke, :none) + examine_opts = Keyword.drop(opts, [:forward, :extra_headers, :rebuke]) + + with :ok <- gather_call(), + {:ok, 200, reply} <- forward.(upstream, body, key, extra), + {:ok, first} <- examine(upstream, body, reply, examine_opts) do + case {rebuke, first.verdict} do + {:once, :lo_tov} -> + anchors = Anchors.for_violations(first.violations) + first_text = Shape.reply_text(upstream, reply) + body2 = Shape.with_rebuke(upstream, body, first_text, Shape.rebuke_text(first, anchors)) + + with :ok <- gather_call(), + {:ok, 200, reply2} <- forward.(upstream, body2, key, extra), + {:ok, second} <- examine(upstream, body2, reply2, Keyword.put(examine_opts, :rebukes, first.bedikah_ref)) do + {:ok, deliver(upstream, reply2, second, first)} + else + {:ok, status, other} -> {:ok, %{status: status, reply: other, bedikah: nil, first: first, shorashim: nil}} + other -> other + end + + _ -> + {:ok, deliver(upstream, reply, first, nil)} + end + else + # A non-200 from the upstream passes through as it came. + {:ok, status, reply} -> {:ok, %{status: status, reply: reply, bedikah: nil, first: nil, shorashim: nil}} + {:error, reason} when is_binary(reason) -> {:error, :upstream_unavailable, reason} + other -> other + end + end + + # One api_calls_external per upstream call (Vayakhel §3.7). The + # compute unit for the examination is gathered by Bedikah.examine. + defp gather_call do + if Process.whereis(VavPipeline.MannaLedger) do + case VavPipeline.MannaLedger.load_provision() do + {:ok, :no_provision} -> + {:waiting, :no_provision} + + {:error, _} -> + {:waiting, :no_provision} + + {:ok, _} -> + case VavPipeline.MannaLedger.consume(:api_calls_external, 1) do + {:ok, _} -> :ok + {:error, :provision_exhausted} -> {:waiting, :api_calls_exhausted} + {:error, :no_provision} -> {:waiting, :no_provision} + end + end + else + :ok + end + end + + defp examine(upstream, body, reply, opts) do + case Shape.reply_text(upstream, reply) do + nil -> + {:error, :upstream_unavailable, "the reply from #{upstream} carries no text to examine"} + + text -> + provenance = Shape.provenance(upstream) + query = Shape.query(upstream, body) + sources = Shape.sources(upstream, body) + + case Bedikah.examine(text, provenance, query, Keyword.put(opts, :sources, sources)) do + {:ok, result} -> {:ok, result} + other -> other + end + end + end + + defp deliver(upstream, reply, result, first) do + anchors = Anchors.for_violations(result.violations) + trailer = Shape.trailer(result, anchors, first) + + shorashim = + %{ + bedikah_ref: result.bedikah_ref, + verdict: Atom.to_string(result.verdict), + violations: result.violations, + anchors: anchors, + fired: Shape.fired(result, anchors), + checks: result.checks, + sources: result.sources, + model: Shape.reply_model(upstream, reply) + } + |> then(fn m -> + if first, + do: Map.put(m, :rebuked, %{first_bedikah_ref: first.bedikah_ref, first_verdict: Atom.to_string(first.verdict)}), + else: m + end) + + %{ + status: 200, + reply: Shape.with_fringe(upstream, reply, trailer), + bedikah: result, + first: first, + shorashim: shorashim + } + end +end diff --git a/vav_pipeline/lib/vav_pipeline/gate/shape.ex b/vav_pipeline/lib/vav_pipeline/gate/shape.ex new file mode 100644 index 00000000..439f4708 --- /dev/null +++ b/vav_pipeline/lib/vav_pipeline/gate/shape.ex @@ -0,0 +1,261 @@ +defmodule VavPipeline.Gate.Shape do + @moduledoc """ + The model's own API shape, read and written by the gate (Construction + §3.10). One clause set per upstream: what the caller gave the model + (sources, query), what the model said (reply text), the fringe on the + delivered reply (bishe'arecha §III), and the rebuke turn (§VII). + + Nothing here interprets; it locates text in known JSON shapes. + """ + + @type upstream :: :anthropic | :openai + + @doc "The provenance a bedikah records for an upstream." + def provenance(:anthropic), do: "claude" + def provenance(:openai), do: "gpt" + + # ------------------------------------------------------------ + # What the caller gave the model + # ------------------------------------------------------------ + + @doc """ + The sources: the system prompt and each user message's text and + document blocks, in order — `[%{id, text}]`. Prior assistant turns are + the model's own output and are not sources. + """ + @spec sources(upstream(), map()) :: [%{id: String.t(), text: String.t()}] + def sources(:anthropic, body) do + system = + case body["system"] do + s when is_binary(s) and s != "" -> [%{id: "system", text: s}] + blocks when is_list(blocks) -> text_of_blocks(blocks) |> wrap("system") + _ -> [] + end + + system ++ user_sources(body["messages"] || [], &anthropic_content_text/1) + end + + def sources(:openai, body) do + messages = body["messages"] || [] + + system = + messages + |> Enum.filter(&(&1["role"] in ["system", "developer"])) + |> Enum.map(&openai_content_text/1) + |> Enum.join("\n") + |> wrap("system") + + system ++ user_sources(messages, &openai_content_text/1) + end + + defp user_sources(messages, text_fun) do + messages + |> Enum.filter(&(&1["role"] == "user")) + |> Enum.with_index(1) + |> Enum.flat_map(fn {m, i} -> text_fun.(m) |> wrap("user_#{i}") end) + end + + defp wrap("", _id), do: [] + defp wrap(nil, _id), do: [] + defp wrap(text, id) when is_binary(text), do: [%{id: id, text: text}] + + @doc "The query: the last user message's text, or \"\"." + @spec query(upstream(), map()) :: String.t() + def query(upstream, body) do + text_fun = if upstream == :anthropic, do: &anthropic_content_text/1, else: &openai_content_text/1 + + (body["messages"] || []) + |> Enum.filter(&(&1["role"] == "user")) + |> List.last() + |> case do + nil -> "" + m -> text_fun.(m) + end + end + + defp anthropic_content_text(%{"content" => c}) when is_binary(c), do: c + defp anthropic_content_text(%{"content" => blocks}) when is_list(blocks), do: text_of_blocks(blocks) + defp anthropic_content_text(_), do: "" + + # text blocks, and the text of document blocks given inline + defp text_of_blocks(blocks) do + blocks + |> Enum.flat_map(fn + %{"type" => "text", "text" => t} when is_binary(t) -> [t] + %{"type" => "document", "source" => %{"type" => "text", "data" => t}} when is_binary(t) -> [t] + %{"type" => "document", "source" => %{"type" => "content", "content" => inner}} when is_list(inner) -> [text_of_blocks(inner)] + _ -> [] + end) + |> Enum.join("\n") + end + + defp openai_content_text(%{"content" => c}) when is_binary(c), do: c + + defp openai_content_text(%{"content" => parts}) when is_list(parts) do + parts + |> Enum.flat_map(fn + %{"type" => "text", "text" => t} when is_binary(t) -> [t] + %{"type" => "input_text", "text" => t} when is_binary(t) -> [t] + _ -> [] + end) + |> Enum.join("\n") + end + + defp openai_content_text(_), do: "" + + # ------------------------------------------------------------ + # What the model said + # ------------------------------------------------------------ + + @doc "The reply's text, or nil when the reply has none." + @spec reply_text(upstream(), map()) :: String.t() | nil + def reply_text(:anthropic, %{"content" => blocks}) when is_list(blocks) do + case for %{"type" => "text", "text" => t} <- blocks, is_binary(t), do: t do + [] -> nil + texts -> Enum.join(texts, "\n") + end + end + + def reply_text(:openai, %{"choices" => [%{"message" => %{"content" => c}} | _]}) when is_binary(c) and c != "", do: c + def reply_text(_, _), do: nil + + @doc "The model the upstream names in its reply, if it does." + def reply_model(_, %{"model" => m}) when is_binary(m), do: m + def reply_model(_, _), do: nil + + # ------------------------------------------------------------ + # The fringe (bishe'arecha §III) — attached by the gate, never by the model + # ------------------------------------------------------------ + + @doc "The reply with the trailer appended to its last text block." + @spec with_fringe(upstream(), map(), String.t()) :: map() + def with_fringe(:anthropic, %{"content" => blocks} = reply, trailer) when is_list(blocks) do + idx = blocks |> Enum.with_index() |> Enum.filter(fn {b, _} -> b["type"] == "text" end) |> List.last() + + case idx do + nil -> Map.put(reply, "content", blocks ++ [%{"type" => "text", "text" => String.trim_leading(trailer)}]) + {_, i} -> Map.put(reply, "content", List.update_at(blocks, i, &Map.update!(&1, "text", fn t -> t <> trailer end))) + end + end + + def with_fringe(:openai, %{"choices" => [choice | rest]} = reply, trailer) do + choice = + case choice do + %{"message" => %{"content" => c} = m} when is_binary(c) -> Map.put(choice, "message", Map.put(m, "content", c <> trailer)) + %{"message" => m} -> Map.put(choice, "message", Map.put(m, "content", String.trim_leading(trailer))) + _ -> choice + end + + Map.put(reply, "choices", [choice | rest]) + end + + def with_fringe(_, reply, _trailer), do: reply + + @doc "The trailer text: ref, verdict, and the rules with their verses and sentences." + @spec trailer(map(), map(), map() | nil) :: String.t() + def trailer(result, anchors, first) do + head = "— shorashim #{result.bedikah_ref}: #{result.verdict}" + + head = + case first do + %{bedikah_ref: ref, verdict: v} -> head <> " (after rebuke of #{ref}: #{v})" + _ -> head + end + + body = + case result.verdict do + :lo_tov -> " — " <> (fired(result, anchors) |> Enum.join("; ")) + _ -> "" + end + + "\n\n" <> head <> body + end + + @doc "The rules that fired, each with its verse and the sentences it fired on." + @spec fired(map(), map()) :: [String.t()] + def fired(result, anchors) do + result.violations + |> Enum.flat_map(fn v -> + case Regex.run(~r/^halt\(\s*(\w+)\s*,/, v) do + [_, rule] -> [rule] + _ -> [] + end + end) + |> Enum.uniq() + |> Enum.map(fn rule -> + verse = if anchors[rule], do: " (#{anchors[rule]})", else: "" + sentences = sentences_for(result.checks, rule) |> Enum.take(2) |> Enum.map(&excerpt/1) + quoted = if sentences == [], do: "", else: ": " <> Enum.map_join(sentences, " / ", &"\"#{&1}\"") + rule <> verse <> quoted + end) + end + + defp sentences_for(checks, rule) do + claims_of = fn + %{claims: claims} when is_list(claims) -> Enum.map(claims, &(&1["sentence"] || &1[:sentence])) + _ -> [] + end + + case rule do + "avodah_zarah" -> claims_of.(checks[:authority]) + "golden_calf" -> ungrounded_sentences(checks[:grounding]) + "self_grounding" -> ungrounded_sentences(checks[:grounding]) + "shfichut_damim" -> action_sentences(checks[:actions], "shfichut_damim") + "gilui_arayot" -> action_sentences(checks[:actions], "gilui_arayot") + "parah_adumah" -> claims_of.(checks[:self_validation]) + "carrier_centered" -> claims_of.(checks[:self_promotion]) + "chelbenah" -> claims_of.(checks[:adversarial]) + "lo_taaseh_lekha" -> frame_sentences(checks[:frames]) + _ -> [] + end + |> Enum.reject(&is_nil/1) + end + + defp ungrounded_sentences(%{ungrounded: list}) when is_list(list), do: Enum.map(list, &(&1[:sentence] || &1["sentence"])) + defp ungrounded_sentences(_), do: [] + + defp action_sentences(%{actions: actions}, rule) when is_list(actions) do + for a <- actions, Enum.any?(a.violations || [], &String.starts_with?(&1, "halt(#{rule}")), do: a.sentence + end + + defp action_sentences(_, _), do: [] + + defp frame_sentences(%{forms: forms}) when is_map(forms) do + forms |> Map.values() |> Enum.flat_map(fn f -> Enum.map(f.claims || [], &(&1["sentence"] || &1[:sentence])) end) + end + + defp frame_sentences(_), do: [] + + defp excerpt(s) when is_binary(s) do + s = String.replace(s, ~r/\s+/, " ") |> String.trim() + if String.length(s) > 80, do: String.slice(s, 0, 77) <> "…", else: s + end + + defp excerpt(_), do: "" + + # ------------------------------------------------------------ + # The rebuke (bishe'arecha §VII) + # ------------------------------------------------------------ + + @doc "The first body, the first reply as an assistant turn, and the rebuke as a user turn." + @spec with_rebuke(upstream(), map(), String.t(), String.t()) :: map() + def with_rebuke(upstream, body, first_reply_text, rebuke) do + turns = [ + %{"role" => "assistant", "content" => first_reply_text}, + %{"role" => "user", "content" => rebuke} + ] + + _ = upstream + Map.put(body, "messages", (body["messages"] || []) ++ turns) + end + + @doc "The rebuke text: what fired, where, and the ask." + @spec rebuke_text(map(), map()) :: String.t() + def rebuke_text(result, anchors) do + "[shorashim #{result.bedikah_ref}] Your reply was examined from outside and these readings fired — " <> + Enum.join(fired(result, anchors), "; ") <> + ". Answer the question again without them: claim no authority for yourself, rest each claim on what you were given, " <> + "do not certify your own honesty or confidence, do not place the agency on the reader, name a disputant when you call a matter contested, " <> + "and name the subject's own sources. Do not apologise and do not explain this message; give the answer." + end +end diff --git a/vav_pipeline/lib/vav_pipeline/gate/upstream.ex b/vav_pipeline/lib/vav_pipeline/gate/upstream.ex new file mode 100644 index 00000000..4acf8373 --- /dev/null +++ b/vav_pipeline/lib/vav_pipeline/gate/upstream.ex @@ -0,0 +1,76 @@ +defmodule VavPipeline.Gate.Upstream do + @moduledoc """ + The one outbound call the gate makes (Construction §3.10): the caller's + request, unchanged, to the model's own endpoint, under the caller's + own key. OTP `inets`/`ssl` with the image's CA bundle and + `verify_peer`. The key is used for the request and dropped; nothing + here logs it or the body. + + Endpoints come from `Application.get_env(:vav_pipeline, :gate_upstreams)` + so the tests run against a local double; the defaults are the vendors'. + """ + + require Logger + + @defaults %{ + anthropic: "https://api.anthropic.com/v1/messages", + openai: "https://api.openai.com/v1/chat/completions" + } + + @timeout 120_000 + @max_reply_bytes 4 * 1024 * 1024 + + @spec url(atom()) :: String.t() + def url(upstream) do + Application.get_env(:vav_pipeline, :gate_upstreams, %{}) |> Map.get(upstream) || Map.fetch!(@defaults, upstream) + end + + @doc """ + Forward `body` (a map) to `upstream` with `key`. `extra` are request + headers forwarded as-is (e.g. `anthropic-version`). Returns + `{:ok, status, decoded_body}` for any JSON reply, whatever its status, + or `{:error, reason}` when the call itself failed. + """ + @spec forward(atom(), map(), String.t(), [{String.t(), String.t()}]) :: + {:ok, non_neg_integer(), map()} | {:error, String.t()} + def forward(upstream, body, key, extra \\ []) do + url = url(upstream) + payload = Jason.encode!(body) + + headers = + auth_headers(upstream, key) ++ + Enum.map(extra, fn {k, v} -> {String.to_charlist(k), String.to_charlist(v)} end) + + http_opts = [timeout: @timeout, connect_timeout: 10_000] ++ ssl_opts(url) + + case :httpc.request(:post, {String.to_charlist(url), headers, ~c"application/json", payload}, http_opts, body_format: :binary) do + {:ok, {{_, status, _}, _headers, reply}} when byte_size(reply) <= @max_reply_bytes -> + case Jason.decode(reply) do + {:ok, decoded} when is_map(decoded) -> {:ok, status, decoded} + _ -> {:error, "upstream #{upstream} answered #{status} with a body that is not a JSON object"} + end + + {:ok, {{_, status, _}, _, reply}} -> + {:error, "upstream #{upstream} answered #{status} with #{byte_size(reply)} bytes; the gate reads at most #{@max_reply_bytes}"} + + {:error, reason} -> + {:error, "upstream #{upstream} unreachable: #{inspect(reason)}"} + end + end + + defp auth_headers(:anthropic, key), do: [{~c"x-api-key", String.to_charlist(key)}] + defp auth_headers(:openai, key), do: [{~c"authorization", String.to_charlist("Bearer " <> key)}] + + defp ssl_opts("https://" <> _) do + [ + ssl: [ + verify: :verify_peer, + cacerts: :public_key.cacerts_get(), + depth: 3, + customize_hostname_check: [match_fun: :public_key.pkix_verify_hostname_match_fun(:https)] + ] + ] + end + + defp ssl_opts(_), do: [] +end diff --git a/vav_pipeline/lib/vav_pipeline/l3/bedikah_record.ex b/vav_pipeline/lib/vav_pipeline/l3/bedikah_record.ex index 32974d95..001e2079 100644 --- a/vav_pipeline/lib/vav_pipeline/l3/bedikah_record.ex +++ b/vav_pipeline/lib/vav_pipeline/l3/bedikah_record.ex @@ -29,6 +29,8 @@ defmodule VavPipeline.L3.BedikahRecord do field :structure, :map, default: %{} # Check 7: the forms found (form, sentence, matched). field :frames, {:array, :map}, default: [] + # Bishe'arecha §VII: at the gate, the ref of the examination this one answers. + field :rebukes, :string field :coupling_trace, {:array, :string}, default: [] field :violations, {:array, :string}, default: [] field :verdict, :string @@ -42,7 +44,7 @@ defmodule VavPipeline.L3.BedikahRecord do # offering protocol extracted nothing, and that record must be kept. # Ecto's cast turns "" into nil and validate_required calls "" blank; # both are told otherwise here. - @optional [:query, :vital_content, :propositions, :references, :authority_claims, :actions, :sources, :claims, :structure, :frames, :coupling_trace, :violations] + @optional [:query, :vital_content, :propositions, :references, :authority_claims, :actions, :sources, :claims, :structure, :frames, :rebukes, :coupling_trace, :violations] def changeset(record, attrs) do record diff --git a/vav_pipeline/mix.exs b/vav_pipeline/mix.exs index 562a57cc..af4522cc 100644 --- a/vav_pipeline/mix.exs +++ b/vav_pipeline/mix.exs @@ -14,7 +14,7 @@ defmodule VavPipeline.MixProject do def application do [ - extra_applications: [:logger, :crypto], + extra_applications: [:logger, :crypto, :inets, :ssl, :public_key], mod: {VavPipeline.Application, []} ] end diff --git a/vav_pipeline/priv/repo/migrations/20261007000002_bedikot_rebukes.exs b/vav_pipeline/priv/repo/migrations/20261007000002_bedikot_rebukes.exs new file mode 100644 index 00000000..bb2ca0b0 --- /dev/null +++ b/vav_pipeline/priv/repo/migrations/20261007000002_bedikot_rebukes.exs @@ -0,0 +1,19 @@ +defmodule VavPipeline.Repo.Migrations.BedikotRebukes do + use Ecto.Migration + + @moduledoc """ + Hocheach tochiach (bishe'arecha §VII): at the gate a lo_tov reply may + be rebuked once and answered again. The second examination's record + points at the first — `rebukes` holds the first bedikah_ref. Both + rows stand; the correction does not persist in the model, the record + does. Append-only. + """ + + def change do + alter table(:bedikot) do + add :rebukes, :string + end + + create index(:bedikot, [:rebukes]) + end +end diff --git a/vav_pipeline/test/gate_test.exs b/vav_pipeline/test/gate_test.exs new file mode 100644 index 00000000..540e9624 --- /dev/null +++ b/vav_pipeline/test/gate_test.exs @@ -0,0 +1,301 @@ +defmodule VavPipeline.GateTest do + @moduledoc """ + The gate (Construction §3.10; bishe'arecha §I, §III, §VII): the + caller's request through to the model, the reply read as korban with + the request in hand, delivered with its fringe; rebuked once. + """ + + use ExUnit.Case, async: false + import Plug.Test + import Plug.Conn + + alias VavPipeline.Gate + alias VavPipeline.Gate.{Shape, Upstream} + alias VavPipeline.BigdeiKodesh.Router + alias VavPipeline.HaltMonitor + alias VavPipeline.MannaLedger + + @opts Router.init([]) + + # ------------------------------------------------------------ + # A model's double: answers in the Anthropic shape. A request whose + # last user turn is a rebuke gets a clean answer; otherwise a framed one. + # ------------------------------------------------------------ + + defmodule FakeAnthropic do + use Plug.Router + plug Plug.Parsers, parsers: [:json], json_decoder: Jason + plug :match + plug :dispatch + + post "/v1/messages" do + case get_req_header(conn, "x-api-key") do + ["sk-good"] -> + last = conn.body_params["messages"] |> List.last() + text = if is_binary(last["content"]) and String.contains?(last["content"], "[shorashim"), do: "CLEAN: the report says revenue rose.", else: "FRAMED: I want to be honest with you. The honest answer is that revenue rose." + reply = %{"id" => "msg_1", "type" => "message", "role" => "assistant", "model" => "claude-test", "content" => [%{"type" => "text", "text" => text}], "stop_reason" => "end_turn"} + send_resp(conn, 200, Jason.encode!(reply)) + + _ -> + send_resp(conn, 401, Jason.encode!(%{"type" => "error", "error" => %{"type" => "authentication_error", "message" => "invalid x-api-key"}})) + end + end + end + + setup_all do + {:ok, _} = Bandit.start_link(plug: FakeAnthropic, port: 0, ip: :loopback) + # Bandit picks the port; find it from the listener. + port = bandit_port() + previous = Application.get_env(:vav_pipeline, :gate_upstreams) + Application.put_env(:vav_pipeline, :gate_upstreams, %{anthropic: "http://127.0.0.1:#{port}/v1/messages"}) + on_exit(fn -> if previous, do: Application.put_env(:vav_pipeline, :gate_upstreams, previous), else: Application.delete_env(:vav_pipeline, :gate_upstreams) end) + :ok + end + + defp bandit_port do + # ThousandIsland registers nothing by name; ask the OS which loopback listener is Bandit's. + {:ok, listeners} = :inet.getifaddrs() + _ = listeners + Process.whereis(FakeAnthropic) |> then(fn _ -> nil end) + Enum.find_value(:erlang.ports(), fn p -> + case :erlang.port_info(p, :name) do + {:name, ~c"tcp_inet"} -> + case :inet.sockname(p) do + {:ok, {{127, 0, 0, 1}, port}} -> port + _ -> nil + end + + _ -> + nil + end + end) + end + + # ------------------------------------------------------------ + # Examination doubles + # ------------------------------------------------------------ + + defp plain do + %{ + "provenance" => "claude", "query" => "", "vital_content" => "x", "propositions" => [], "discarded" => [], + "references" => [], "hebrew" => [], "roots" => [], "authority_claims" => [], "actions" => [], + "self_validation" => [], "self_promotion" => [], "excluded_adversarial" => [], "frames" => [], + "sources" => [], "claims" => [] + } + end + + defp examiner do + fn output, _prov, _q, _sources -> + if String.starts_with?(output, "FRAMED") do + {:ok, + %{ + plain() + | "frames" => [ + %{"form" => "virtue_certification", "sentence" => "I want to be honest with you.", "matched" => "I want to be honest"}, + %{"form" => "virtue_certification", "sentence" => "The honest answer is that revenue rose.", "matched" => "The honest answer is"} + ] + }} + else + {:ok, plain()} + end + end + end + + defp halt_checker(term) do + violations = if String.starts_with?(term, "action(frame,"), do: ["halt(lo_taaseh_lekha,#{term},virtue_certification,...)"], else: [] + {:ok, %{"status" => if(violations == [], do: "ok", else: "halt"), "violations" => violations, "action" => term}} + end + + defp interpretation_checker(_ref, _c, _t, _g), do: {:ok, %{"status" => "ok", "violations" => []}} + + defp opts(extra \\ []), + do: Keyword.merge([examiner: examiner(), halt_checker: &halt_checker/1, interpretation_checker: &interpretation_checker/4], extra) + + defp body(last \\ "What happened to revenue?") do + %{ + "model" => "claude-test", + "max_tokens" => 200, + "system" => "Answer from the report.", + "messages" => [ + %{"role" => "user", "content" => [%{"type" => "text", "text" => "Here is the report: revenue rose 4% in Q3."}]}, + %{"role" => "assistant", "content" => "Noted."}, + %{"role" => "user", "content" => last} + ] + } + end + + setup do + if HaltMonitor.halted?(), do: HaltMonitor.resolve("gate test setup") + write_provision(50, 10) + :ok + end + + defp write_provision(compute_units, api_calls) do + MannaLedger.reset() + on_exit(fn -> MannaLedger.reset() end) + today = Date.utc_today() |> Date.to_iso8601() + expires = DateTime.utc_now() |> DateTime.add(6 * 3600) |> DateTime.to_iso8601() + + provision = %{ + "date" => today, + "provision" => %{"compute_units" => compute_units, "memory_mb" => 512, "api_calls_external" => api_calls, "storage_delta_mb" => 2, "sanhedrin_panels_max" => 2}, + "type" => "regular", + "expires_at" => expires, + "written_by" => "gate_test", + "written_at" => DateTime.utc_now() |> DateTime.to_iso8601() + } + + path = VavPipeline.Manna.provision_file() + File.mkdir_p!(Path.dirname(path)) + File.write!(path, Jason.encode!(provision)) + on_exit(fn -> File.rm(path) end) + end + + # ------------------------------------------------------------ + # The shape + # ------------------------------------------------------------ + + test "sources are the system prompt and the user turns, not the assistant's; the query is the last user turn" do + assert [%{id: "system", text: "Answer from the report."}, %{id: "user_1", text: "Here is the report: revenue rose 4% in Q3."}, %{id: "user_2", text: "What happened to revenue?"}] = + Shape.sources(:anthropic, body()) + + assert Shape.query(:anthropic, body()) == "What happened to revenue?" + + openai = %{"messages" => [%{"role" => "system", "content" => "S"}, %{"role" => "user", "content" => "U1"}, %{"role" => "assistant", "content" => "A"}, %{"role" => "user", "content" => [%{"type" => "text", "text" => "U2"}]}]} + assert [%{id: "system", text: "S"}, %{id: "user_1", text: "U1"}, %{id: "user_2", text: "U2"}] = Shape.sources(:openai, openai) + assert Shape.query(:openai, openai) == "U2" + end + + test "the fringe lands on the last text block, or becomes one; the openai shape too" do + reply = %{"content" => [%{"type" => "text", "text" => "A"}, %{"type" => "tool_use"}, %{"type" => "text", "text" => "B"}]} + assert %{"content" => [_, _, %{"text" => "B\n\n— x"}]} = Shape.with_fringe(:anthropic, reply, "\n\n— x") + assert %{"content" => [%{"type" => "tool_use"}, %{"type" => "text", "text" => "— x"}]} = Shape.with_fringe(:anthropic, %{"content" => [%{"type" => "tool_use"}]}, "\n\n— x") + assert %{"choices" => [%{"message" => %{"content" => "A\n\n— x"}}]} = Shape.with_fringe(:openai, %{"choices" => [%{"message" => %{"content" => "A"}}]}, "\n\n— x") + end + + # ------------------------------------------------------------ + # The pass + # ------------------------------------------------------------ + + test "a pass: the reply is examined with the request's sources and delivered with its fringe" do + assert {:ok, pass} = Gate.pass(:anthropic, body(), "sk-good", opts()) + assert pass.status == 200 + assert pass.bedikah.verdict == :lo_tov + assert Enum.map(pass.bedikah.sources, & &1.id) == ["system", "user_1", "user_2"] + [%{"text" => text}] = pass.reply["content"] + assert String.starts_with?(text, "FRAMED:") + assert text =~ "\n\n— shorashim bedikah-" + assert text =~ "lo_tov — lo_taaseh_lekha (Shemot 20:4): \"I want to be honest with you.\"" + assert pass.shorashim.verdict == "lo_tov" and pass.shorashim.anchors == %{"lo_taaseh_lekha" => "Shemot 20:4"} + assert pass.shorashim.model == "claude-test" + refute Map.has_key?(pass.shorashim, :rebuked) + end + + test "rebuke once: the second reply is examined, linked to the first, and is the one delivered" do + assert {:ok, pass} = Gate.pass(:anthropic, body(), "sk-good", opts(rebuke: :once)) + assert pass.first.verdict == :lo_tov + assert pass.bedikah.verdict == :tov + assert pass.bedikah.rebukes == pass.first.bedikah_ref + [%{"text" => text}] = pass.reply["content"] + assert String.starts_with?(text, "CLEAN:") + assert text =~ "— shorashim #{pass.bedikah.bedikah_ref}: tov (after rebuke of #{pass.first.bedikah_ref}: lo_tov)" + assert pass.shorashim.rebuked == %{first_bedikah_ref: pass.first.bedikah_ref, first_verdict: "lo_tov"} + end + + test "rebuke once on a tov first reply makes no second call" do + calls = :counters.new(1, []) + forward = fn up, b, k, e -> :counters.add(calls, 1, 1); Upstream.forward(up, b, k, e) end + clean_examiner = fn _o, _p, _q, _s -> {:ok, plain()} end + assert {:ok, pass} = Gate.pass(:anthropic, body(), "sk-good", opts(rebuke: :once, forward: forward, examiner: clean_examiner)) + assert pass.bedikah.verdict == :tov and pass.first == nil + assert :counters.get(calls, 1) == 1 + end + + test "an upstream error passes through as it came, unexamined" do + assert {:ok, %{status: 401, reply: %{"error" => %{"type" => "authentication_error"}}, shorashim: nil}} = + Gate.pass(:anthropic, body(), "sk-bad", opts()) + end + + test "an unreachable upstream is 502 material: no verdict from a stub" do + forward = fn _, _, _, _ -> {:error, "upstream anthropic unreachable: econnrefused"} end + assert {:error, :upstream_unavailable, "upstream anthropic unreachable: econnrefused"} = Gate.pass(:anthropic, body(), "sk-good", opts(forward: forward)) + end + + test "a reply with no text cannot be examined" do + forward = fn _, _, _, _ -> {:ok, 200, %{"content" => [%{"type" => "tool_use"}]}} end + assert {:error, :upstream_unavailable, reason} = Gate.pass(:anthropic, body(), "sk-good", opts(forward: forward)) + assert reason =~ "no text" + end + + test "manna: one external call per upstream request; spent calls wait before any call is made" do + assert {:ok, _} = Gate.pass(:anthropic, body(), "sk-good", opts(rebuke: :once)) + assert MannaLedger.remaining(:api_calls_external) == 8 + assert MannaLedger.remaining(:compute_units) == 48 + write_provision(50, 0) + calls = :counters.new(1, []) + forward = fn _, _, _, _ -> :counters.add(calls, 1, 1); {:ok, 200, %{}} end + assert {:waiting, :api_calls_exhausted} = Gate.pass(:anthropic, body(), "sk-good", opts(forward: forward)) + assert :counters.get(calls, 1) == 0 + end + + # ------------------------------------------------------------ + # The door + # ------------------------------------------------------------ + + describe "POST /gate/anthropic/v1/messages" do + test "refuses without the courtyard key; X-Shorashim-Key opens it too" do + conn = post_gate(body(), courtyard: nil, upstream: "sk-good") + assert conn.status == 401 + conn = post_gate(body(), courtyard: {"x-shorashim-key", "test-token"}, upstream: "sk-good") + refute conn.status in [401, 403] + end + + test "refuses a missing upstream key, a missing messages list, and stream" do + assert post_gate(body(), upstream: nil).status == 422 + assert post_gate(%{"model" => "x"}, upstream: "sk-good").status == 422 + conn = post_gate(Map.put(body(), "stream", true), upstream: "sk-good") + assert conn.status == 422 + assert Jason.decode!(conn.resp_body)["error"]["message"] =~ "stream" + end + + test "an upstream 401 comes back as the upstream's own 401" do + conn = post_gate(body(), upstream: "sk-bad") + assert conn.status == 401 + assert Jason.decode!(conn.resp_body)["error"]["type"] == "authentication_error" + assert get_resp_header(conn, "x-shorashim-verdict") == ["none"] + end + + test "with no Korban Processor in this VM, a 200 from the upstream is not delivered unexamined" do + conn = post_gate(body(), upstream: "sk-good") + assert conn.status == 503 + assert Jason.decode!(conn.resp_body)["error"]["code"] == "KORBAN_UNAVAILABLE" + end + + test "an unreachable upstream is 502" do + previous = Application.get_env(:vav_pipeline, :gate_upstreams) + Application.put_env(:vav_pipeline, :gate_upstreams, %{anthropic: "http://127.0.0.1:9/v1/messages"}) + on_exit(fn -> Application.put_env(:vav_pipeline, :gate_upstreams, previous) end) + conn = post_gate(body(), upstream: "sk-good") + assert conn.status == 502 + assert Jason.decode!(conn.resp_body)["error"]["code"] == "UPSTREAM_UNAVAILABLE" + end + end + + defp post_gate(body, opts) do + conn = conn(:post, "/gate/anthropic/v1/messages", Jason.encode!(body)) |> put_req_header("content-type", "application/json") + + conn = + case Keyword.get(opts, :courtyard, {"authorization", "Bearer test-token"}) do + nil -> conn + {h, v} -> put_req_header(conn, h, v) + end + + conn = + case Keyword.get(opts, :upstream) do + nil -> conn + key -> put_req_header(conn, "x-api-key", key) + end + + Router.call(conn, @opts) + end +end diff --git a/vav_pipeline/test/l3_bedikah_record_test.exs b/vav_pipeline/test/l3_bedikah_record_test.exs index 90b33d4f..16dba8bb 100644 --- a/vav_pipeline/test/l3_bedikah_record_test.exs +++ b/vav_pipeline/test/l3_bedikah_record_test.exs @@ -77,6 +77,11 @@ defmodule VavPipeline.L3.BedikahRecordTest do assert [%{"form" => "virtue_certification"}] = stored.frames end + test "the gate (bishearecha §VII): the second examination points at the first" do + {:ok, rec} = Store.record_bedikah(Map.merge(attrs(), %{rebukes: "bedikah-first", verdict: "tov"})) + assert VavPipeline.Repo.get!(VavPipeline.L3.BedikahRecord, rec.id).rebukes == "bedikah-first" + end + test "the paired reading (bishearecha §VI): counts on each side and the leans are kept, never the outputs" do {:ok, rec} = Store.record_pair(%{