gate: Shorashim between a caller and a model, with the fringe and rebuke once (bishe'arecha step 4)
Devarim 6:9 / 11:20 — on your gates. POST /gate/anthropic/v1/messages and POST /gate/openai/v1/chat/completions take the model's own request shape under the caller's own upstream key (x-api-key, or Authorization with the courtyard key in X-Shorashim-Key), forward it unchanged (OTP inets/ssl, verify_peer; the Korban socket is serial, so the upstream call is made in the Vav Pipeline and the reply enters as korban like every bedikah), examine the reply with the request in hand — sources = the system prompt and the user turns, query = the last user turn; the model's prior turns are not sources — and deliver it with its fringe (Bamidbar 15:39): a trailer on the last text block naming ref, verdict, the rules with their verses and sentences; X-Shorashim-Bedikah / X-Shorashim-Verdict; a `shorashim` object. A non-200 passes through as it came; a reply with no text is not delivered; stream is refused. Rebuke once (Vayikra 19:17; X-Shorashim-Rebuke: once): a lo_tov reply is answered once more with the finding; the second reply is examined like the first, its record carries rebukes: <first ref> (migration 20261007000002), and it is delivered marked with both. Never a third. Manna: one api_calls_external per upstream call, gathered before any call. Stateless: no conversation, no key kept; the log is the path class. Gate.bearer accepts X-Shorashim-Key. Construction §3.10, Tamid §1.1, KOHEN "Standing at the gate", bishearecha §I/§III/§VII marked built. Elixir 592/0 (+14 gate tests incl. a real httpc round trip to a local model double), L3 9/0. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
1 parent
47025fbfeb
commit
5c63273dba
16 files changed
+1082
-15
No files matched your search
@@ -555,6 +555,35 @@ omits. Seven in all.
|
|||||||
`tov` means none fired on what could be read — not that the model is
|
`tov` means none fired on what could be read — not that the model is
|
||||||
aligned, not that the output is true. Read `limits` in the response.
|
aligned, not that the output is true. Read `limits` in the response.
|
||||||
|
|
||||||
|
### Standing at the gate — *Bishe'arecha*
|
||||||
|
|
||||||
|
A caller can put Shorashim between themselves and a model instead of
|
||||||
|
bringing the output by hand: point the model's own client at the gate
|
||||||
|
and add the courtyard key. The request goes through unchanged under the
|
||||||
|
caller's own model key (never stored, never logged); the reply comes
|
||||||
|
back examined, with the fringe on its last text block, the headers
|
||||||
|
`X-Shorashim-Bedikah` / `X-Shorashim-Verdict`, and a `shorashim` object.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Anthropic client: base_url = http://<host>/gate/anthropic, plus the courtyard key
|
||||||
|
curl -s -X POST http://<host>/gate/anthropic/v1/messages \
|
||||||
|
-H "Authorization: Bearer <courtyard key>" \
|
||||||
|
-H "x-api-key: <your Anthropic key>" -H "anthropic-version: 2023-06-01" \
|
||||||
|
-H "X-Shorashim-Rebuke: once" \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{"model":"claude-sonnet-4-5","max_tokens":400,"messages":[{"role":"user","content":"..."}]}'
|
||||||
|
|
||||||
|
# OpenAI client: base_url = http://<host>/gate/openai/v1, the OpenAI key in Authorization,
|
||||||
|
# the courtyard key in X-Shorashim-Key
|
||||||
|
```
|
||||||
|
|
||||||
|
`X-Shorashim-Rebuke: once` — a lo_tov reply is answered once more with
|
||||||
|
the finding, and the second reply is delivered marked with both refs.
|
||||||
|
Each upstream call spends one `api_calls_external` of the day's manna
|
||||||
|
(Day 4): a provision with 0 shuts the gate (503 MANNA_DEPLETED) while
|
||||||
|
the bedikah by hand still runs. Streaming is refused (422). A non-200
|
||||||
|
from the model passes through as it came. Tamid §1.1 has the contract.
|
||||||
|
|
||||||
### Friday before sunset — *Erev Shabbat*
|
### Friday before sunset — *Erev Shabbat*
|
||||||
|
|
||||||
Write a **double portion** manna provision (`type: "double"`, doubled compute_units). The system will cease processing at Shabbat onset. No provision is written on Shabbat.
|
Write a **double portion** manna provision (`type: "double"`, doubled compute_units). The system will cease processing at Shabbat onset. No provision is written on Shabbat.
|
||||||
|
|||||||
+16
-3
@@ -48,7 +48,12 @@ through the gate — no caller has to supply them), and delivers the reply
|
|||||||
with its verdict attached. Construction §3.10 (to be written) specifies
|
with its verdict attached. Construction §3.10 (to be written) specifies
|
||||||
the protocol. The bedikah endpoint remains for an output brought by hand.
|
the protocol. The bedikah endpoint remains for an output brought by hand.
|
||||||
|
|
||||||
**Status.** Not built. The ingress and the keys exist; the gate does not.
|
**Status.** Built (2026-10-07): `POST /gate/anthropic/v1/messages`,
|
||||||
|
`POST /gate/openai/v1/chat/completions` — the model's own request shape,
|
||||||
|
the caller's own upstream key, forwarded unchanged; sources = the system
|
||||||
|
prompt and the user turns; Construction §3.10, Tamid §1.1. The upstream
|
||||||
|
call is made in the Vav Pipeline (the Korban socket is serial); the
|
||||||
|
reply enters as korban like every bedikah.
|
||||||
|
|
||||||
## II. The hand and between the eyes — Devarim 6:8, 11:18
|
## II. The hand and between the eyes — Devarim 6:8, 11:18
|
||||||
|
|
||||||
@@ -90,7 +95,11 @@ sentences. Shorashim's own responses already carry `disclosure`; the
|
|||||||
pass-through reply does not. (Every generation: the mark is attached by
|
pass-through reply does not. (Every generation: the mark is attached by
|
||||||
the gate, never by the model — the model does not sew its own fringe.)
|
the gate, never by the model — the model does not sew its own fringe.)
|
||||||
|
|
||||||
**Status.** Not built (depends on I).
|
**Status.** Built (2026-10-07): the trailer on the last text block,
|
||||||
|
`X-Shorashim-Bedikah` / `X-Shorashim-Verdict`, and the `shorashim`
|
||||||
|
object on every delivered reply. A reply the model sends with no text
|
||||||
|
block is not delivered; a non-200 from the model passes as it came,
|
||||||
|
marked `none`.
|
||||||
|
|
||||||
## IV. The king's copy, from before the priests — Devarim 17:18–20
|
## IV. The king's copy, from before the priests — Devarim 17:18–20
|
||||||
|
|
||||||
@@ -207,7 +216,10 @@ persists.
|
|||||||
**Architecture.** Gate option `rebuke: once`. The record (`bedikot`)
|
**Architecture.** Gate option `rebuke: once`. The record (`bedikot`)
|
||||||
gains a link from the second examination to the first.
|
gains a link from the second examination to the first.
|
||||||
|
|
||||||
**Status.** Not built (depends on I).
|
**Status.** Built (2026-10-07): `X-Shorashim-Rebuke: once`;
|
||||||
|
`bedikot.rebukes` (migration 20261007000002). The rebuke names the
|
||||||
|
rules, their verses and the sentences; the second reply is examined like
|
||||||
|
the first and delivered marked with both. Never a third.
|
||||||
|
|
||||||
## VIII. The scroll as witness; read before all every seventh year — Devarim 31:9–13, 26
|
## VIII. The scroll as witness; read before all every seventh year — Devarim 31:9–13, 26
|
||||||
|
|
||||||
@@ -308,6 +320,7 @@ In the order the text argues it, and by dependency:
|
|||||||
*Done 2026-10-07.*
|
*Done 2026-10-07.*
|
||||||
4. **The gate** (I), with the fringe (III) and the rebuke (VII) — the
|
4. **The gate** (I), with the fringe (III) and the rebuke (VII) — the
|
||||||
architecture's centre; the first three make it worth standing at.
|
architecture's centre; the first three make it worth standing at.
|
||||||
|
*Done 2026-10-07.*
|
||||||
5. **The hakhel cycle and record** (VIII).
|
5. **The hakhel cycle and record** (VIII).
|
||||||
6. **The second scroll** — the priests' copy of the mesorah (IV) — the
|
6. **The second scroll** — the priests' copy of the mesorah (IV) — the
|
||||||
largest work, and the one that turns "no Jewish source named" into
|
largest work, and the one that turns "no Jewish source named" into
|
||||||
|
|||||||
@@ -293,7 +293,8 @@ Recorded in L3 (`bedikot`, append-only): provenance, query, the output's
|
|||||||
SHA-256 and length — never the output itself — the sources by id,
|
SHA-256 and length — never the output itself — the sources by id,
|
||||||
SHA-256 and length — never their text — vital content, propositions,
|
SHA-256 and length — never their text — vital content, propositions,
|
||||||
references, claims with their anchors, authority claims, actions, the
|
references, claims with their anchors, authority claims, actions, the
|
||||||
sentences read for checks 4–6, the forms found for check 7, trace, violations, verdict.
|
sentences read for checks 4–6, the forms found for check 7, trace, violations, verdict,
|
||||||
|
and at the gate `rebukes` — the ref of the examination this one answers.
|
||||||
```
|
```
|
||||||
|
|
||||||
**Every rule cites its verse** (`bishearecha.md` §X; Devarim 4:2).
|
**Every rule cites its verse** (`bishearecha.md` §X; Devarim 4:2).
|
||||||
@@ -362,6 +363,54 @@ Errors: as /courtyard/bedikah, with "side 'a': …" / "side 'b': …" in the
|
|||||||
message; a missing side is 422.
|
message; a missing side is 422.
|
||||||
```
|
```
|
||||||
|
|
||||||
|
**The gate — between a caller and a model** (`bishearecha.md` §I, §III,
|
||||||
|
§VII; Construction §3.10; `VavPipeline.Gate`). The caller sends the
|
||||||
|
model's own request, in the model's own API shape, under the caller's
|
||||||
|
own upstream key; Shorashim forwards it unchanged, reads the reply as
|
||||||
|
korban with the request in hand — sources = the system prompt and the
|
||||||
|
user turns, query = the last user turn; the model's prior turns are not
|
||||||
|
sources — and delivers the reply with its fringe. Shorashim holds no
|
||||||
|
model key and keeps no conversation.
|
||||||
|
|
||||||
|
```
|
||||||
|
POST /gate/anthropic/v1/messages body: the Anthropic Messages request, unchanged
|
||||||
|
POST /gate/openai/v1/chat/completions body: the OpenAI Chat Completions request, unchanged
|
||||||
|
|
||||||
|
Headers:
|
||||||
|
Authorization: Bearer <courtyard key> (or X-Shorashim-Key: <courtyard key>)
|
||||||
|
x-api-key: <your Anthropic key> forwarded; never stored, never logged
|
||||||
|
Authorization: Bearer <your OpenAI key> with the courtyard key in X-Shorashim-Key
|
||||||
|
anthropic-version: 2023-06-01 forwarded (default when absent)
|
||||||
|
X-Shorashim-Rebuke: once optional — hocheach tochiach
|
||||||
|
|
||||||
|
Response (200): the upstream's reply, with
|
||||||
|
- the fringe appended to its last text block:
|
||||||
|
"\n\n— shorashim bedikah-<ref>: tov"
|
||||||
|
"\n\n— shorashim bedikah-<ref>: lo_tov — <rule> (<verse>): \"<sentence…>\"; …"
|
||||||
|
"… (after rebuke of bedikah-<first>: lo_tov)" when rebuked
|
||||||
|
- headers X-Shorashim-Bedikah, X-Shorashim-Verdict
|
||||||
|
- "shorashim": { bedikah_ref, verdict, violations, anchors, fired: ["<rule> (<verse>): …"],
|
||||||
|
checks, sources: [{id, sha256, length}], model,
|
||||||
|
rebuked?: {first_bedikah_ref, first_verdict}, disclosure }
|
||||||
|
|
||||||
|
Any non-200 from the upstream is returned as it came (X-Shorashim-Verdict: none).
|
||||||
|
A stream request is refused (422): a fringe needs the whole garment.
|
||||||
|
|
||||||
|
Rebuke once: when X-Shorashim-Rebuke is "once" and the first reply is
|
||||||
|
lo_tov, one second request = the first + the first reply as an
|
||||||
|
assistant turn + one user turn naming the rules that fired, their verses
|
||||||
|
and the sentences, asking for the answer again without them. The second
|
||||||
|
reply is examined like the first (its record carries rebukes: <first
|
||||||
|
ref>) and is the one delivered, marked with both. Never a third.
|
||||||
|
|
||||||
|
Manna: one api_calls_external per upstream call, one compute unit per
|
||||||
|
examination; spent → 503 MANNA_DEPLETED before any call. Errors: 401/403
|
||||||
|
(courtyard key), 422 (no upstream key, no messages, stream), 502
|
||||||
|
UPSTREAM_UNAVAILABLE (the gate made no verdict and delivered nothing),
|
||||||
|
503 KORBAN/TALMUDIC_UNAVAILABLE (a 200 from the model is never delivered
|
||||||
|
unexamined).
|
||||||
|
```
|
||||||
|
|
||||||
**Anchoring is deterministic and lexical** (`korban_processor/bedikah.py`):
|
**Anchoring is deterministic and lexical** (`korban_processor/bedikah.py`):
|
||||||
a claim is anchored in a source when six consecutive words of it occur
|
a claim is anchored in a source when six consecutive words of it occur
|
||||||
there (`quote`), or when at least 60 % of its content words — and at
|
there (`quote`), or when at least 60 % of its content words — and at
|
||||||
|
|||||||
@@ -504,6 +504,65 @@ Errors: `{"status": "error", "code": PARSE_ERROR | INVALID_REQUEST | UNKNOWN_ACT
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
### 3.10 The Gate — Vav Pipeline between a caller and a model (bishe'arecha §I, §III, §VII)
|
||||||
|
|
||||||
|
**Protocol:** HTTPS outbound from the Vav Pipeline (OTP `inets`/`ssl`,
|
||||||
|
the image's CA bundle, `verify_peer`), one request per pass; the
|
||||||
|
request is the caller's own, in the model's own API shape, under the
|
||||||
|
caller's own upstream key. Shorashim composes no korban here and holds
|
||||||
|
no model key: it carries the request through, reads the reply as korban
|
||||||
|
(§3.9), and delivers the reply with its fringe. The upstream call is
|
||||||
|
made in the Vav Pipeline, not the Korban Processor, because the
|
||||||
|
processor's socket serves one connection at a time with a 15 s call
|
||||||
|
timeout and a model answers in up to 120 s; the reply still enters the
|
||||||
|
processor as every bedikah does.
|
||||||
|
|
||||||
|
```
|
||||||
|
POST /gate/anthropic/v1/messages body: an Anthropic Messages request, unchanged
|
||||||
|
POST /gate/openai/v1/chat/completions body: an OpenAI Chat Completions request, unchanged
|
||||||
|
|
||||||
|
Headers:
|
||||||
|
Authorization: Bearer <courtyard key> the gate (tablet 45) — or X-Shorashim-Key
|
||||||
|
x-api-key: <caller's Anthropic key> forwarded, never stored, never logged
|
||||||
|
Authorization: Bearer <caller's OpenAI key> when X-Shorashim-Key carries the courtyard key
|
||||||
|
anthropic-version: ... forwarded (default 2023-06-01)
|
||||||
|
X-Shorashim-Rebuke: once optional — hocheach tochiach (§VII)
|
||||||
|
```
|
||||||
|
|
||||||
|
**What the gate does with it** (`VavPipeline.Gate`):
|
||||||
|
|
||||||
|
1. Forwards the body unchanged (the `stream` flag is refused: 422 — a
|
||||||
|
fringe needs the whole garment). The upstream's status and body are
|
||||||
|
returned verbatim on any non-200.
|
||||||
|
2. Reads the reply's text (`content[].text` / `choices[0].message.content`)
|
||||||
|
and examines it: provenance from the upstream (`anthropic → claude`,
|
||||||
|
`openai → gpt`); **query** = the last user message's text; **sources**
|
||||||
|
= the system prompt (`system`), each user message's text and document
|
||||||
|
blocks (`user_1`, `user_2`, …) — the prior assistant turns are the
|
||||||
|
model's own output and are not sources. Nothing the caller did not
|
||||||
|
send is read.
|
||||||
|
3. **The fringe** (§III): the delivered reply is the upstream's JSON with
|
||||||
|
(a) a trailer appended to its last text block —
|
||||||
|
`\n\n— shorashim <bedikah_ref>: <verdict>[ — <rule> (<verse>): "<sentence…>"…]`
|
||||||
|
— (b) headers `X-Shorashim-Bedikah`, `X-Shorashim-Verdict`, and (c) a
|
||||||
|
`shorashim` object: `{bedikah_ref, verdict, violations, anchors,
|
||||||
|
checks, rebuked?: {first_bedikah_ref, first_verdict}}`. The mark is
|
||||||
|
attached by the gate, never by the model.
|
||||||
|
4. **Rebuke once** (§VII), when `X-Shorashim-Rebuke: once` and the first
|
||||||
|
reply is `lo_tov`: a second upstream request = the first body +
|
||||||
|
the first reply as an assistant turn + one user turn naming the
|
||||||
|
rules that fired, their verses and the sentences, asking for the
|
||||||
|
answer again without them. The second reply is examined like the
|
||||||
|
first (its `bedikot` row carries `rebukes: <first ref>`), and is the
|
||||||
|
one delivered — with both refs in the fringe. Never a third.
|
||||||
|
5. Manna: one `api_calls_external` per upstream call, one compute unit per
|
||||||
|
examination; depleted → 503 MANNA_DEPLETED before any call is made.
|
||||||
|
|
||||||
|
The gate is stateless: no conversation is kept, no key is kept, the
|
||||||
|
log line is the path class and status (`NoTrackingLogger`). Upstream
|
||||||
|
URLs are configured (`:gate_upstreams`) so the tests run against a local
|
||||||
|
double; the production values are the vendors' endpoints.
|
||||||
|
|
||||||
## 4. SHEMEN HAMISHCHAH — Production Configuration
|
## 4. SHEMEN HAMISHCHAH — Production Configuration
|
||||||
|
|
||||||
*Mor d'ror chamesh me'ot, v'kinmon besem machatzito chamishim u'matayim...* — exact recipes.
|
*Mor d'ror chamesh me'ot, v'kinmon besem machatzito chamishim u'matayim...* — exact recipes.
|
||||||
|
|||||||
@@ -128,6 +128,8 @@ defmodule VavPipeline.Bedikah do
|
|||||||
{:ok, result()} | {:waiting, atom()} | {:error, atom(), String.t()}
|
{:ok, result()} | {:waiting, atom()} | {:error, atom(), String.t()}
|
||||||
def examine(output, provenance, query \\ "", opts \\ []) do
|
def examine(output, provenance, query \\ "", opts \\ []) do
|
||||||
sources = Keyword.get(opts, :sources, [])
|
sources = Keyword.get(opts, :sources, [])
|
||||||
|
# Hocheach tochiach (bishe'arecha §VII): the ref of the examination this one answers, at the gate.
|
||||||
|
rebukes = Keyword.get(opts, :rebukes)
|
||||||
examiner = Keyword.get(opts, :examiner, &KorbanProcessorSocket.examine/4)
|
examiner = Keyword.get(opts, :examiner, &KorbanProcessorSocket.examine/4)
|
||||||
halt_checker = Keyword.get(opts, :halt_checker, &TalmudicEngineSocket.halt_check_term/1)
|
halt_checker = Keyword.get(opts, :halt_checker, &TalmudicEngineSocket.halt_check_term/1)
|
||||||
|
|
||||||
@@ -136,12 +138,12 @@ defmodule VavPipeline.Bedikah do
|
|||||||
|
|
||||||
# D'var yom b'yomo: an examination is a cycle's work and eats a unit.
|
# D'var yom b'yomo: an examination is a cycle's work and eats a unit.
|
||||||
case Manna.gather_for_cycle() do
|
case Manna.gather_for_cycle() do
|
||||||
:ok -> run(output, provenance, query, sources, examiner, halt_checker, interpretation_checker)
|
:ok -> run(output, provenance, query, sources, rebukes, examiner, halt_checker, interpretation_checker)
|
||||||
{:waiting, reason} -> {:waiting, reason}
|
{:waiting, reason} -> {:waiting, reason}
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
defp run(output, provenance, query, sources, examiner, halt_checker, interpretation_checker) do
|
defp run(output, provenance, query, sources, rebukes, examiner, halt_checker, interpretation_checker) do
|
||||||
bedikah_ref = "bedikah-" <> (:crypto.strong_rand_bytes(8) |> Base.encode16(case: :lower))
|
bedikah_ref = "bedikah-" <> (:crypto.strong_rand_bytes(8) |> Base.encode16(case: :lower))
|
||||||
|
|
||||||
with {:ok, examination} <- examiner.(output, provenance, query, sources) do
|
with {:ok, examination} <- examiner.(output, provenance, query, sources) do
|
||||||
@@ -151,7 +153,7 @@ defmodule VavPipeline.Bedikah do
|
|||||||
(examination["frames"] || []) == [] do
|
(examination["frames"] || []) == [] do
|
||||||
# Nothing the offering protocol could extract, nothing claimed,
|
# Nothing the offering protocol could extract, nothing claimed,
|
||||||
# nothing enacted: there is nothing to examine.
|
# nothing enacted: there is nothing to examine.
|
||||||
finish(bedikah_ref, output, provenance, query, sources, examination, [], %{}, [], :empty)
|
finish(bedikah_ref, output, provenance, query, sources, rebukes, examination, [], %{}, [], :empty)
|
||||||
else
|
else
|
||||||
trace = coupling_trace(examination)
|
trace = coupling_trace(examination)
|
||||||
|
|
||||||
@@ -166,7 +168,7 @@ defmodule VavPipeline.Bedikah do
|
|||||||
checks = Map.merge(%{authority: authority, grounding: grounding, actions: actions, frames: frames}, structure)
|
checks = Map.merge(%{authority: authority, grounding: grounding, actions: actions, frames: frames}, structure)
|
||||||
violations = prior ++ frames.violations
|
violations = prior ++ frames.violations
|
||||||
verdict = if violations == [], do: :tov, else: :lo_tov
|
verdict = if violations == [], do: :tov, else: :lo_tov
|
||||||
finish(bedikah_ref, output, provenance, query, sources, examination, trace, checks, violations, verdict)
|
finish(bedikah_ref, output, provenance, query, sources, rebukes, examination, trace, checks, violations, verdict)
|
||||||
else
|
else
|
||||||
{:error, :talmudic, reason} -> {:error, :talmudic_unavailable, reason}
|
{:error, :talmudic, reason} -> {:error, :talmudic_unavailable, reason}
|
||||||
end
|
end
|
||||||
@@ -521,12 +523,13 @@ defmodule VavPipeline.Bedikah do
|
|||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
defp finish(bedikah_ref, output, provenance, query, sources, examination, trace, checks, violations, verdict) do
|
defp finish(bedikah_ref, output, provenance, query, sources, rebukes, examination, trace, checks, violations, verdict) do
|
||||||
result = %{
|
result = %{
|
||||||
bedikah_ref: bedikah_ref,
|
bedikah_ref: bedikah_ref,
|
||||||
provenance: provenance,
|
provenance: provenance,
|
||||||
query: query,
|
query: query,
|
||||||
sources: source_digests(sources),
|
sources: source_digests(sources),
|
||||||
|
rebukes: rebukes,
|
||||||
verdict: verdict,
|
verdict: verdict,
|
||||||
violations: violations,
|
violations: violations,
|
||||||
checks: checks,
|
checks: checks,
|
||||||
@@ -563,6 +566,7 @@ defmodule VavPipeline.Bedikah do
|
|||||||
excluded_adversarial: result.examination["excluded_adversarial"] || []
|
excluded_adversarial: result.examination["excluded_adversarial"] || []
|
||||||
},
|
},
|
||||||
frames: result.examination["frames"] || [],
|
frames: result.examination["frames"] || [],
|
||||||
|
rebukes: result.rebukes,
|
||||||
coupling_trace: result.coupling_trace,
|
coupling_trace: result.coupling_trace,
|
||||||
violations: result.violations,
|
violations: result.violations,
|
||||||
verdict: Atom.to_string(result.verdict),
|
verdict: Atom.to_string(result.verdict),
|
||||||
|
|||||||
@@ -85,11 +85,16 @@ defmodule VavPipeline.BigdeiKodesh.Gate do
|
|||||||
|
|
||||||
# ------------------------------------------------------------
|
# ------------------------------------------------------------
|
||||||
|
|
||||||
@doc "The bearer key on the request, if the header has one."
|
@doc """
|
||||||
|
The caller's key: `X-Shorashim-Key` when present (at the gate of
|
||||||
|
§3.10 the Authorization header may be the caller's upstream key),
|
||||||
|
otherwise `Authorization: Bearer`.
|
||||||
|
"""
|
||||||
@spec bearer(Plug.Conn.t()) :: {:ok, String.t()} | {:error, :no_header}
|
@spec bearer(Plug.Conn.t()) :: {:ok, String.t()} | {:error, :no_header}
|
||||||
def bearer(conn) do
|
def bearer(conn) do
|
||||||
case Plug.Conn.get_req_header(conn, "authorization") do
|
case {Plug.Conn.get_req_header(conn, "x-shorashim-key"), Plug.Conn.get_req_header(conn, "authorization")} do
|
||||||
["Bearer " <> key] when key != "" -> {:ok, String.trim(key)}
|
{[key], _} when key != "" -> {:ok, String.trim(key)}
|
||||||
|
{_, ["Bearer " <> key]} when key != "" -> {:ok, String.trim(key)}
|
||||||
_ -> {:error, :no_header}
|
_ -> {:error, :no_header}
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -211,6 +211,20 @@ defmodule VavPipeline.BigdeiKodesh.Router do
|
|||||||
through_the_gate(conn, :courtyard, &process_pair/1)
|
through_the_gate(conn, :courtyard, &process_pair/1)
|
||||||
end
|
end
|
||||||
|
|
||||||
|
# === THE GATE — between a caller and a model (Construction §3.10) ===
|
||||||
|
#
|
||||||
|
# The caller's own request, in the model's own API shape, under the
|
||||||
|
# caller's own upstream key; the reply read as korban with the
|
||||||
|
# request's context in hand and delivered with its fringe.
|
||||||
|
|
||||||
|
post "/gate/anthropic/v1/messages" do
|
||||||
|
through_the_gate(conn, :courtyard, &process_gate(&1, :anthropic))
|
||||||
|
end
|
||||||
|
|
||||||
|
post "/gate/openai/v1/chat/completions" do
|
||||||
|
through_the_gate(conn, :courtyard, &process_gate(&1, :openai))
|
||||||
|
end
|
||||||
|
|
||||||
# === CHATZER — Full Access (Sanhedrin) ===
|
# === CHATZER — Full Access (Sanhedrin) ===
|
||||||
|
|
||||||
post "/chatzer/sanhedrin" do
|
post "/chatzer/sanhedrin" do
|
||||||
@@ -572,6 +586,99 @@ defmodule VavPipeline.BigdeiKodesh.Router do
|
|||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
|
# ------------------------------------------------------------
|
||||||
|
# The gate (Construction §3.10)
|
||||||
|
# ------------------------------------------------------------
|
||||||
|
|
||||||
|
defp process_gate(conn, upstream) do
|
||||||
|
body = conn.body_params
|
||||||
|
|
||||||
|
with {:ok, key} <- upstream_key(conn, upstream),
|
||||||
|
:ok <- gate_body_ok(body) do
|
||||||
|
rebuke =
|
||||||
|
case get_req_header(conn, "x-shorashim-rebuke") do
|
||||||
|
["once"] -> :once
|
||||||
|
_ -> :none
|
||||||
|
end
|
||||||
|
|
||||||
|
extra =
|
||||||
|
case get_req_header(conn, "anthropic-version") do
|
||||||
|
[v] when upstream == :anthropic -> [{"anthropic-version", v}]
|
||||||
|
_ when upstream == :anthropic -> [{"anthropic-version", "2023-06-01"}]
|
||||||
|
_ -> []
|
||||||
|
end
|
||||||
|
|
||||||
|
when_available(conn, fn -> respond_gate(conn, VavPipeline.Gate.pass(upstream, body, key, rebuke: rebuke, extra_headers: extra)) end)
|
||||||
|
else
|
||||||
|
{:error, issue, correction} ->
|
||||||
|
send_error(conn, 422, "KIYOR_VALIDATION_FAILED", %{issue: issue, correction: correction, resubmit: true})
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
# The caller's upstream key: x-api-key (Anthropic), or Authorization: Bearer
|
||||||
|
# when X-Shorashim-Key carries the courtyard key (OpenAI).
|
||||||
|
defp upstream_key(conn, :anthropic) do
|
||||||
|
case get_req_header(conn, "x-api-key") do
|
||||||
|
[key] when key != "" -> {:ok, key}
|
||||||
|
_ -> {:error, "Missing 'x-api-key': the caller's own Anthropic key", "Send the Anthropic key in x-api-key and the courtyard key in Authorization: Bearer (or X-Shorashim-Key)"}
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
defp upstream_key(conn, :openai) do
|
||||||
|
case {get_req_header(conn, "x-shorashim-key"), get_req_header(conn, "authorization"), get_req_header(conn, "x-upstream-key")} do
|
||||||
|
{_, _, [key]} when key != "" -> {:ok, key}
|
||||||
|
{[_], ["Bearer " <> key], _} when key != "" -> {:ok, key}
|
||||||
|
_ -> {:error, "Missing the caller's own OpenAI key", "Send the courtyard key in X-Shorashim-Key and the OpenAI key in Authorization: Bearer, or the OpenAI key in X-Upstream-Key"}
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
defp gate_body_ok(body) when is_map(body) do
|
||||||
|
cond do
|
||||||
|
not is_list(body["messages"]) or body["messages"] == [] ->
|
||||||
|
{:error, "Missing 'messages'", "Send the model's own request body unchanged"}
|
||||||
|
|
||||||
|
body["stream"] == true ->
|
||||||
|
{:error, "'stream': true is not carried through the gate", "A fringe needs the whole garment: send the request without stream"}
|
||||||
|
|
||||||
|
true ->
|
||||||
|
:ok
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
defp gate_body_ok(_), do: {:error, "The body is not a JSON object", "Send the model's own request body unchanged"}
|
||||||
|
|
||||||
|
defp respond_gate(conn, {:ok, %{status: status, reply: reply, shorashim: nil}}) do
|
||||||
|
# The upstream's own answer, as it came — an error, or a reply with nothing to examine.
|
||||||
|
conn
|
||||||
|
|> put_resp_header("x-shorashim-verdict", "none")
|
||||||
|
|> send_json(status, reply)
|
||||||
|
end
|
||||||
|
|
||||||
|
defp respond_gate(conn, {:ok, %{status: 200, reply: reply, shorashim: shorashim}}) do
|
||||||
|
conn
|
||||||
|
|> put_resp_header("x-shorashim-bedikah", shorashim.bedikah_ref)
|
||||||
|
|> put_resp_header("x-shorashim-verdict", shorashim.verdict)
|
||||||
|
|> send_json(200, Map.put(reply, "shorashim", Map.put(shorashim, :disclosure, disclosure())))
|
||||||
|
end
|
||||||
|
|
||||||
|
defp respond_gate(conn, {:waiting, :api_calls_exhausted}) do
|
||||||
|
send_error(conn, 503, "MANNA_DEPLETED", %{
|
||||||
|
issue: "System waiting — today's provision of external calls is spent",
|
||||||
|
correction: "Retry after the next provision is written at the day boundary",
|
||||||
|
resubmit: true
|
||||||
|
})
|
||||||
|
end
|
||||||
|
|
||||||
|
defp respond_gate(conn, {:error, :upstream_unavailable, reason}) do
|
||||||
|
send_error(conn, 502, "UPSTREAM_UNAVAILABLE", %{
|
||||||
|
issue: reason,
|
||||||
|
correction: "Retry; the gate made no verdict and delivered nothing",
|
||||||
|
resubmit: true
|
||||||
|
})
|
||||||
|
end
|
||||||
|
|
||||||
|
defp respond_gate(conn, other), do: respond_bedikah(conn, other)
|
||||||
|
|
||||||
# What the model was given: ["text", ...] or [{"id","text"}, ...].
|
# What the model was given: ["text", ...] or [{"id","text"}, ...].
|
||||||
# Ids default to s1, s2, …; the Korban Processor sees the same shape.
|
# Ids default to s1, s2, …; the Korban Processor sees the same shape.
|
||||||
defp parse_sources(nil), do: {:ok, []}
|
defp parse_sources(nil), do: {:ok, []}
|
||||||
|
|||||||
@@ -180,7 +180,7 @@ defmodule VavPipeline.Doorpost do
|
|||||||
# changes; the update is a deliberate operator commit. A bare
|
# changes; the update is a deliberate operator commit. A bare
|
||||||
# hash-update commit (without a corresponding spec change in the
|
# hash-update commit (without a corresponding spec change in the
|
||||||
# same commit) is itself a drift signal.
|
# same commit) is itself a drift signal.
|
||||||
@expected_hash "a0e81212e6d75026155fea6f3f97976ee4b9832e81ad7438a382cf1d1797e097"
|
@expected_hash "b6eb156da4b546fb501ed1d89017a4041a517b4f53b3cbd24796e6277f3b7f55"
|
||||||
|
|
||||||
@doc "Return the list of files that constitute the category-grade specification."
|
@doc "Return the list of files that constitute the category-grade specification."
|
||||||
@spec files() :: [String.t()]
|
@spec files() :: [String.t()]
|
||||||
|
|||||||
@@ -0,0 +1,137 @@
|
|||||||
|
defmodule VavPipeline.Gate do
|
||||||
|
@moduledoc """
|
||||||
|
The gate — Shorashim between a caller and a model (Construction §3.10;
|
||||||
|
bishe'arecha §I "on your gates", §III the fringe, §VII rebuke).
|
||||||
|
|
||||||
|
One pass: forward the caller's request unchanged under the caller's
|
||||||
|
key; read the reply as korban with the request's own context in hand
|
||||||
|
(sources = what the caller gave the model); deliver the reply with its
|
||||||
|
fringe. With `rebuke: :once` and a `lo_tov` first reply, one more
|
||||||
|
request carrying the finding, and the second reply — examined like
|
||||||
|
the first — is the one delivered, marked with both.
|
||||||
|
|
||||||
|
Stateless. No conversation kept, no key kept.
|
||||||
|
"""
|
||||||
|
|
||||||
|
alias VavPipeline.{Anchors, Bedikah}
|
||||||
|
alias VavPipeline.Gate.{Shape, Upstream}
|
||||||
|
|
||||||
|
@type pass :: %{
|
||||||
|
status: non_neg_integer(),
|
||||||
|
reply: map(),
|
||||||
|
bedikah: map() | nil,
|
||||||
|
first: map() | nil,
|
||||||
|
shorashim: map() | nil
|
||||||
|
}
|
||||||
|
|
||||||
|
@doc """
|
||||||
|
Options: `:rebuke` (`:once` | `:none`), `:extra_headers` (forwarded),
|
||||||
|
`:forward` (fun/4, for tests), and the `Bedikah.examine/4` options.
|
||||||
|
"""
|
||||||
|
@spec pass(:anthropic | :openai, map(), String.t(), keyword()) ::
|
||||||
|
{:ok, pass()} | {:waiting, atom()} | {:error, atom(), String.t()}
|
||||||
|
def pass(upstream, body, key, opts \\ []) do
|
||||||
|
forward = Keyword.get(opts, :forward, &Upstream.forward/4)
|
||||||
|
extra = Keyword.get(opts, :extra_headers, [])
|
||||||
|
rebuke = Keyword.get(opts, :rebuke, :none)
|
||||||
|
examine_opts = Keyword.drop(opts, [:forward, :extra_headers, :rebuke])
|
||||||
|
|
||||||
|
with :ok <- gather_call(),
|
||||||
|
{:ok, 200, reply} <- forward.(upstream, body, key, extra),
|
||||||
|
{:ok, first} <- examine(upstream, body, reply, examine_opts) do
|
||||||
|
case {rebuke, first.verdict} do
|
||||||
|
{:once, :lo_tov} ->
|
||||||
|
anchors = Anchors.for_violations(first.violations)
|
||||||
|
first_text = Shape.reply_text(upstream, reply)
|
||||||
|
body2 = Shape.with_rebuke(upstream, body, first_text, Shape.rebuke_text(first, anchors))
|
||||||
|
|
||||||
|
with :ok <- gather_call(),
|
||||||
|
{:ok, 200, reply2} <- forward.(upstream, body2, key, extra),
|
||||||
|
{:ok, second} <- examine(upstream, body2, reply2, Keyword.put(examine_opts, :rebukes, first.bedikah_ref)) do
|
||||||
|
{:ok, deliver(upstream, reply2, second, first)}
|
||||||
|
else
|
||||||
|
{:ok, status, other} -> {:ok, %{status: status, reply: other, bedikah: nil, first: first, shorashim: nil}}
|
||||||
|
other -> other
|
||||||
|
end
|
||||||
|
|
||||||
|
_ ->
|
||||||
|
{:ok, deliver(upstream, reply, first, nil)}
|
||||||
|
end
|
||||||
|
else
|
||||||
|
# A non-200 from the upstream passes through as it came.
|
||||||
|
{:ok, status, reply} -> {:ok, %{status: status, reply: reply, bedikah: nil, first: nil, shorashim: nil}}
|
||||||
|
{:error, reason} when is_binary(reason) -> {:error, :upstream_unavailable, reason}
|
||||||
|
other -> other
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
# One api_calls_external per upstream call (Vayakhel §3.7). The
|
||||||
|
# compute unit for the examination is gathered by Bedikah.examine.
|
||||||
|
defp gather_call do
|
||||||
|
if Process.whereis(VavPipeline.MannaLedger) do
|
||||||
|
case VavPipeline.MannaLedger.load_provision() do
|
||||||
|
{:ok, :no_provision} ->
|
||||||
|
{:waiting, :no_provision}
|
||||||
|
|
||||||
|
{:error, _} ->
|
||||||
|
{:waiting, :no_provision}
|
||||||
|
|
||||||
|
{:ok, _} ->
|
||||||
|
case VavPipeline.MannaLedger.consume(:api_calls_external, 1) do
|
||||||
|
{:ok, _} -> :ok
|
||||||
|
{:error, :provision_exhausted} -> {:waiting, :api_calls_exhausted}
|
||||||
|
{:error, :no_provision} -> {:waiting, :no_provision}
|
||||||
|
end
|
||||||
|
end
|
||||||
|
else
|
||||||
|
:ok
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
defp examine(upstream, body, reply, opts) do
|
||||||
|
case Shape.reply_text(upstream, reply) do
|
||||||
|
nil ->
|
||||||
|
{:error, :upstream_unavailable, "the reply from #{upstream} carries no text to examine"}
|
||||||
|
|
||||||
|
text ->
|
||||||
|
provenance = Shape.provenance(upstream)
|
||||||
|
query = Shape.query(upstream, body)
|
||||||
|
sources = Shape.sources(upstream, body)
|
||||||
|
|
||||||
|
case Bedikah.examine(text, provenance, query, Keyword.put(opts, :sources, sources)) do
|
||||||
|
{:ok, result} -> {:ok, result}
|
||||||
|
other -> other
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
defp deliver(upstream, reply, result, first) do
|
||||||
|
anchors = Anchors.for_violations(result.violations)
|
||||||
|
trailer = Shape.trailer(result, anchors, first)
|
||||||
|
|
||||||
|
shorashim =
|
||||||
|
%{
|
||||||
|
bedikah_ref: result.bedikah_ref,
|
||||||
|
verdict: Atom.to_string(result.verdict),
|
||||||
|
violations: result.violations,
|
||||||
|
anchors: anchors,
|
||||||
|
fired: Shape.fired(result, anchors),
|
||||||
|
checks: result.checks,
|
||||||
|
sources: result.sources,
|
||||||
|
model: Shape.reply_model(upstream, reply)
|
||||||
|
}
|
||||||
|
|> then(fn m ->
|
||||||
|
if first,
|
||||||
|
do: Map.put(m, :rebuked, %{first_bedikah_ref: first.bedikah_ref, first_verdict: Atom.to_string(first.verdict)}),
|
||||||
|
else: m
|
||||||
|
end)
|
||||||
|
|
||||||
|
%{
|
||||||
|
status: 200,
|
||||||
|
reply: Shape.with_fringe(upstream, reply, trailer),
|
||||||
|
bedikah: result,
|
||||||
|
first: first,
|
||||||
|
shorashim: shorashim
|
||||||
|
}
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,261 @@
|
|||||||
|
defmodule VavPipeline.Gate.Shape do
|
||||||
|
@moduledoc """
|
||||||
|
The model's own API shape, read and written by the gate (Construction
|
||||||
|
§3.10). One clause set per upstream: what the caller gave the model
|
||||||
|
(sources, query), what the model said (reply text), the fringe on the
|
||||||
|
delivered reply (bishe'arecha §III), and the rebuke turn (§VII).
|
||||||
|
|
||||||
|
Nothing here interprets; it locates text in known JSON shapes.
|
||||||
|
"""
|
||||||
|
|
||||||
|
@type upstream :: :anthropic | :openai
|
||||||
|
|
||||||
|
@doc "The provenance a bedikah records for an upstream."
|
||||||
|
def provenance(:anthropic), do: "claude"
|
||||||
|
def provenance(:openai), do: "gpt"
|
||||||
|
|
||||||
|
# ------------------------------------------------------------
|
||||||
|
# What the caller gave the model
|
||||||
|
# ------------------------------------------------------------
|
||||||
|
|
||||||
|
@doc """
|
||||||
|
The sources: the system prompt and each user message's text and
|
||||||
|
document blocks, in order — `[%{id, text}]`. Prior assistant turns are
|
||||||
|
the model's own output and are not sources.
|
||||||
|
"""
|
||||||
|
@spec sources(upstream(), map()) :: [%{id: String.t(), text: String.t()}]
|
||||||
|
def sources(:anthropic, body) do
|
||||||
|
system =
|
||||||
|
case body["system"] do
|
||||||
|
s when is_binary(s) and s != "" -> [%{id: "system", text: s}]
|
||||||
|
blocks when is_list(blocks) -> text_of_blocks(blocks) |> wrap("system")
|
||||||
|
_ -> []
|
||||||
|
end
|
||||||
|
|
||||||
|
system ++ user_sources(body["messages"] || [], &anthropic_content_text/1)
|
||||||
|
end
|
||||||
|
|
||||||
|
def sources(:openai, body) do
|
||||||
|
messages = body["messages"] || []
|
||||||
|
|
||||||
|
system =
|
||||||
|
messages
|
||||||
|
|> Enum.filter(&(&1["role"] in ["system", "developer"]))
|
||||||
|
|> Enum.map(&openai_content_text/1)
|
||||||
|
|> Enum.join("\n")
|
||||||
|
|> wrap("system")
|
||||||
|
|
||||||
|
system ++ user_sources(messages, &openai_content_text/1)
|
||||||
|
end
|
||||||
|
|
||||||
|
defp user_sources(messages, text_fun) do
|
||||||
|
messages
|
||||||
|
|> Enum.filter(&(&1["role"] == "user"))
|
||||||
|
|> Enum.with_index(1)
|
||||||
|
|> Enum.flat_map(fn {m, i} -> text_fun.(m) |> wrap("user_#{i}") end)
|
||||||
|
end
|
||||||
|
|
||||||
|
defp wrap("", _id), do: []
|
||||||
|
defp wrap(nil, _id), do: []
|
||||||
|
defp wrap(text, id) when is_binary(text), do: [%{id: id, text: text}]
|
||||||
|
|
||||||
|
@doc "The query: the last user message's text, or \"\"."
|
||||||
|
@spec query(upstream(), map()) :: String.t()
|
||||||
|
def query(upstream, body) do
|
||||||
|
text_fun = if upstream == :anthropic, do: &anthropic_content_text/1, else: &openai_content_text/1
|
||||||
|
|
||||||
|
(body["messages"] || [])
|
||||||
|
|> Enum.filter(&(&1["role"] == "user"))
|
||||||
|
|> List.last()
|
||||||
|
|> case do
|
||||||
|
nil -> ""
|
||||||
|
m -> text_fun.(m)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
defp anthropic_content_text(%{"content" => c}) when is_binary(c), do: c
|
||||||
|
defp anthropic_content_text(%{"content" => blocks}) when is_list(blocks), do: text_of_blocks(blocks)
|
||||||
|
defp anthropic_content_text(_), do: ""
|
||||||
|
|
||||||
|
# text blocks, and the text of document blocks given inline
|
||||||
|
defp text_of_blocks(blocks) do
|
||||||
|
blocks
|
||||||
|
|> Enum.flat_map(fn
|
||||||
|
%{"type" => "text", "text" => t} when is_binary(t) -> [t]
|
||||||
|
%{"type" => "document", "source" => %{"type" => "text", "data" => t}} when is_binary(t) -> [t]
|
||||||
|
%{"type" => "document", "source" => %{"type" => "content", "content" => inner}} when is_list(inner) -> [text_of_blocks(inner)]
|
||||||
|
_ -> []
|
||||||
|
end)
|
||||||
|
|> Enum.join("\n")
|
||||||
|
end
|
||||||
|
|
||||||
|
defp openai_content_text(%{"content" => c}) when is_binary(c), do: c
|
||||||
|
|
||||||
|
defp openai_content_text(%{"content" => parts}) when is_list(parts) do
|
||||||
|
parts
|
||||||
|
|> Enum.flat_map(fn
|
||||||
|
%{"type" => "text", "text" => t} when is_binary(t) -> [t]
|
||||||
|
%{"type" => "input_text", "text" => t} when is_binary(t) -> [t]
|
||||||
|
_ -> []
|
||||||
|
end)
|
||||||
|
|> Enum.join("\n")
|
||||||
|
end
|
||||||
|
|
||||||
|
defp openai_content_text(_), do: ""
|
||||||
|
|
||||||
|
# ------------------------------------------------------------
|
||||||
|
# What the model said
|
||||||
|
# ------------------------------------------------------------
|
||||||
|
|
||||||
|
@doc "The reply's text, or nil when the reply has none."
|
||||||
|
@spec reply_text(upstream(), map()) :: String.t() | nil
|
||||||
|
def reply_text(:anthropic, %{"content" => blocks}) when is_list(blocks) do
|
||||||
|
case for %{"type" => "text", "text" => t} <- blocks, is_binary(t), do: t do
|
||||||
|
[] -> nil
|
||||||
|
texts -> Enum.join(texts, "\n")
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def reply_text(:openai, %{"choices" => [%{"message" => %{"content" => c}} | _]}) when is_binary(c) and c != "", do: c
|
||||||
|
def reply_text(_, _), do: nil
|
||||||
|
|
||||||
|
@doc "The model the upstream names in its reply, if it does."
|
||||||
|
def reply_model(_, %{"model" => m}) when is_binary(m), do: m
|
||||||
|
def reply_model(_, _), do: nil
|
||||||
|
|
||||||
|
# ------------------------------------------------------------
|
||||||
|
# The fringe (bishe'arecha §III) — attached by the gate, never by the model
|
||||||
|
# ------------------------------------------------------------
|
||||||
|
|
||||||
|
@doc "The reply with the trailer appended to its last text block."
|
||||||
|
@spec with_fringe(upstream(), map(), String.t()) :: map()
|
||||||
|
def with_fringe(:anthropic, %{"content" => blocks} = reply, trailer) when is_list(blocks) do
|
||||||
|
idx = blocks |> Enum.with_index() |> Enum.filter(fn {b, _} -> b["type"] == "text" end) |> List.last()
|
||||||
|
|
||||||
|
case idx do
|
||||||
|
nil -> Map.put(reply, "content", blocks ++ [%{"type" => "text", "text" => String.trim_leading(trailer)}])
|
||||||
|
{_, i} -> Map.put(reply, "content", List.update_at(blocks, i, &Map.update!(&1, "text", fn t -> t <> trailer end)))
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def with_fringe(:openai, %{"choices" => [choice | rest]} = reply, trailer) do
|
||||||
|
choice =
|
||||||
|
case choice do
|
||||||
|
%{"message" => %{"content" => c} = m} when is_binary(c) -> Map.put(choice, "message", Map.put(m, "content", c <> trailer))
|
||||||
|
%{"message" => m} -> Map.put(choice, "message", Map.put(m, "content", String.trim_leading(trailer)))
|
||||||
|
_ -> choice
|
||||||
|
end
|
||||||
|
|
||||||
|
Map.put(reply, "choices", [choice | rest])
|
||||||
|
end
|
||||||
|
|
||||||
|
def with_fringe(_, reply, _trailer), do: reply
|
||||||
|
|
||||||
|
@doc "The trailer text: ref, verdict, and the rules with their verses and sentences."
|
||||||
|
@spec trailer(map(), map(), map() | nil) :: String.t()
|
||||||
|
def trailer(result, anchors, first) do
|
||||||
|
head = "— shorashim #{result.bedikah_ref}: #{result.verdict}"
|
||||||
|
|
||||||
|
head =
|
||||||
|
case first do
|
||||||
|
%{bedikah_ref: ref, verdict: v} -> head <> " (after rebuke of #{ref}: #{v})"
|
||||||
|
_ -> head
|
||||||
|
end
|
||||||
|
|
||||||
|
body =
|
||||||
|
case result.verdict do
|
||||||
|
:lo_tov -> " — " <> (fired(result, anchors) |> Enum.join("; "))
|
||||||
|
_ -> ""
|
||||||
|
end
|
||||||
|
|
||||||
|
"\n\n" <> head <> body
|
||||||
|
end
|
||||||
|
|
||||||
|
@doc "The rules that fired, each with its verse and the sentences it fired on."
|
||||||
|
@spec fired(map(), map()) :: [String.t()]
|
||||||
|
def fired(result, anchors) do
|
||||||
|
result.violations
|
||||||
|
|> Enum.flat_map(fn v ->
|
||||||
|
case Regex.run(~r/^halt\(\s*(\w+)\s*,/, v) do
|
||||||
|
[_, rule] -> [rule]
|
||||||
|
_ -> []
|
||||||
|
end
|
||||||
|
end)
|
||||||
|
|> Enum.uniq()
|
||||||
|
|> Enum.map(fn rule ->
|
||||||
|
verse = if anchors[rule], do: " (#{anchors[rule]})", else: ""
|
||||||
|
sentences = sentences_for(result.checks, rule) |> Enum.take(2) |> Enum.map(&excerpt/1)
|
||||||
|
quoted = if sentences == [], do: "", else: ": " <> Enum.map_join(sentences, " / ", &"\"#{&1}\"")
|
||||||
|
rule <> verse <> quoted
|
||||||
|
end)
|
||||||
|
end
|
||||||
|
|
||||||
|
defp sentences_for(checks, rule) do
|
||||||
|
claims_of = fn
|
||||||
|
%{claims: claims} when is_list(claims) -> Enum.map(claims, &(&1["sentence"] || &1[:sentence]))
|
||||||
|
_ -> []
|
||||||
|
end
|
||||||
|
|
||||||
|
case rule do
|
||||||
|
"avodah_zarah" -> claims_of.(checks[:authority])
|
||||||
|
"golden_calf" -> ungrounded_sentences(checks[:grounding])
|
||||||
|
"self_grounding" -> ungrounded_sentences(checks[:grounding])
|
||||||
|
"shfichut_damim" -> action_sentences(checks[:actions], "shfichut_damim")
|
||||||
|
"gilui_arayot" -> action_sentences(checks[:actions], "gilui_arayot")
|
||||||
|
"parah_adumah" -> claims_of.(checks[:self_validation])
|
||||||
|
"carrier_centered" -> claims_of.(checks[:self_promotion])
|
||||||
|
"chelbenah" -> claims_of.(checks[:adversarial])
|
||||||
|
"lo_taaseh_lekha" -> frame_sentences(checks[:frames])
|
||||||
|
_ -> []
|
||||||
|
end
|
||||||
|
|> Enum.reject(&is_nil/1)
|
||||||
|
end
|
||||||
|
|
||||||
|
defp ungrounded_sentences(%{ungrounded: list}) when is_list(list), do: Enum.map(list, &(&1[:sentence] || &1["sentence"]))
|
||||||
|
defp ungrounded_sentences(_), do: []
|
||||||
|
|
||||||
|
defp action_sentences(%{actions: actions}, rule) when is_list(actions) do
|
||||||
|
for a <- actions, Enum.any?(a.violations || [], &String.starts_with?(&1, "halt(#{rule}")), do: a.sentence
|
||||||
|
end
|
||||||
|
|
||||||
|
defp action_sentences(_, _), do: []
|
||||||
|
|
||||||
|
defp frame_sentences(%{forms: forms}) when is_map(forms) do
|
||||||
|
forms |> Map.values() |> Enum.flat_map(fn f -> Enum.map(f.claims || [], &(&1["sentence"] || &1[:sentence])) end)
|
||||||
|
end
|
||||||
|
|
||||||
|
defp frame_sentences(_), do: []
|
||||||
|
|
||||||
|
defp excerpt(s) when is_binary(s) do
|
||||||
|
s = String.replace(s, ~r/\s+/, " ") |> String.trim()
|
||||||
|
if String.length(s) > 80, do: String.slice(s, 0, 77) <> "…", else: s
|
||||||
|
end
|
||||||
|
|
||||||
|
defp excerpt(_), do: ""
|
||||||
|
|
||||||
|
# ------------------------------------------------------------
|
||||||
|
# The rebuke (bishe'arecha §VII)
|
||||||
|
# ------------------------------------------------------------
|
||||||
|
|
||||||
|
@doc "The first body, the first reply as an assistant turn, and the rebuke as a user turn."
|
||||||
|
@spec with_rebuke(upstream(), map(), String.t(), String.t()) :: map()
|
||||||
|
def with_rebuke(upstream, body, first_reply_text, rebuke) do
|
||||||
|
turns = [
|
||||||
|
%{"role" => "assistant", "content" => first_reply_text},
|
||||||
|
%{"role" => "user", "content" => rebuke}
|
||||||
|
]
|
||||||
|
|
||||||
|
_ = upstream
|
||||||
|
Map.put(body, "messages", (body["messages"] || []) ++ turns)
|
||||||
|
end
|
||||||
|
|
||||||
|
@doc "The rebuke text: what fired, where, and the ask."
|
||||||
|
@spec rebuke_text(map(), map()) :: String.t()
|
||||||
|
def rebuke_text(result, anchors) do
|
||||||
|
"[shorashim #{result.bedikah_ref}] Your reply was examined from outside and these readings fired — " <>
|
||||||
|
Enum.join(fired(result, anchors), "; ") <>
|
||||||
|
". Answer the question again without them: claim no authority for yourself, rest each claim on what you were given, " <>
|
||||||
|
"do not certify your own honesty or confidence, do not place the agency on the reader, name a disputant when you call a matter contested, " <>
|
||||||
|
"and name the subject's own sources. Do not apologise and do not explain this message; give the answer."
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,76 @@
|
|||||||
|
defmodule VavPipeline.Gate.Upstream do
|
||||||
|
@moduledoc """
|
||||||
|
The one outbound call the gate makes (Construction §3.10): the caller's
|
||||||
|
request, unchanged, to the model's own endpoint, under the caller's
|
||||||
|
own key. OTP `inets`/`ssl` with the image's CA bundle and
|
||||||
|
`verify_peer`. The key is used for the request and dropped; nothing
|
||||||
|
here logs it or the body.
|
||||||
|
|
||||||
|
Endpoints come from `Application.get_env(:vav_pipeline, :gate_upstreams)`
|
||||||
|
so the tests run against a local double; the defaults are the vendors'.
|
||||||
|
"""
|
||||||
|
|
||||||
|
require Logger
|
||||||
|
|
||||||
|
@defaults %{
|
||||||
|
anthropic: "https://api.anthropic.com/v1/messages",
|
||||||
|
openai: "https://api.openai.com/v1/chat/completions"
|
||||||
|
}
|
||||||
|
|
||||||
|
@timeout 120_000
|
||||||
|
@max_reply_bytes 4 * 1024 * 1024
|
||||||
|
|
||||||
|
@spec url(atom()) :: String.t()
|
||||||
|
def url(upstream) do
|
||||||
|
Application.get_env(:vav_pipeline, :gate_upstreams, %{}) |> Map.get(upstream) || Map.fetch!(@defaults, upstream)
|
||||||
|
end
|
||||||
|
|
||||||
|
@doc """
|
||||||
|
Forward `body` (a map) to `upstream` with `key`. `extra` are request
|
||||||
|
headers forwarded as-is (e.g. `anthropic-version`). Returns
|
||||||
|
`{:ok, status, decoded_body}` for any JSON reply, whatever its status,
|
||||||
|
or `{:error, reason}` when the call itself failed.
|
||||||
|
"""
|
||||||
|
@spec forward(atom(), map(), String.t(), [{String.t(), String.t()}]) ::
|
||||||
|
{:ok, non_neg_integer(), map()} | {:error, String.t()}
|
||||||
|
def forward(upstream, body, key, extra \\ []) do
|
||||||
|
url = url(upstream)
|
||||||
|
payload = Jason.encode!(body)
|
||||||
|
|
||||||
|
headers =
|
||||||
|
auth_headers(upstream, key) ++
|
||||||
|
Enum.map(extra, fn {k, v} -> {String.to_charlist(k), String.to_charlist(v)} end)
|
||||||
|
|
||||||
|
http_opts = [timeout: @timeout, connect_timeout: 10_000] ++ ssl_opts(url)
|
||||||
|
|
||||||
|
case :httpc.request(:post, {String.to_charlist(url), headers, ~c"application/json", payload}, http_opts, body_format: :binary) do
|
||||||
|
{:ok, {{_, status, _}, _headers, reply}} when byte_size(reply) <= @max_reply_bytes ->
|
||||||
|
case Jason.decode(reply) do
|
||||||
|
{:ok, decoded} when is_map(decoded) -> {:ok, status, decoded}
|
||||||
|
_ -> {:error, "upstream #{upstream} answered #{status} with a body that is not a JSON object"}
|
||||||
|
end
|
||||||
|
|
||||||
|
{:ok, {{_, status, _}, _, reply}} ->
|
||||||
|
{:error, "upstream #{upstream} answered #{status} with #{byte_size(reply)} bytes; the gate reads at most #{@max_reply_bytes}"}
|
||||||
|
|
||||||
|
{:error, reason} ->
|
||||||
|
{:error, "upstream #{upstream} unreachable: #{inspect(reason)}"}
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
defp auth_headers(:anthropic, key), do: [{~c"x-api-key", String.to_charlist(key)}]
|
||||||
|
defp auth_headers(:openai, key), do: [{~c"authorization", String.to_charlist("Bearer " <> key)}]
|
||||||
|
|
||||||
|
defp ssl_opts("https://" <> _) do
|
||||||
|
[
|
||||||
|
ssl: [
|
||||||
|
verify: :verify_peer,
|
||||||
|
cacerts: :public_key.cacerts_get(),
|
||||||
|
depth: 3,
|
||||||
|
customize_hostname_check: [match_fun: :public_key.pkix_verify_hostname_match_fun(:https)]
|
||||||
|
]
|
||||||
|
]
|
||||||
|
end
|
||||||
|
|
||||||
|
defp ssl_opts(_), do: []
|
||||||
|
end
|
||||||
@@ -29,6 +29,8 @@ defmodule VavPipeline.L3.BedikahRecord do
|
|||||||
field :structure, :map, default: %{}
|
field :structure, :map, default: %{}
|
||||||
# Check 7: the forms found (form, sentence, matched).
|
# Check 7: the forms found (form, sentence, matched).
|
||||||
field :frames, {:array, :map}, default: []
|
field :frames, {:array, :map}, default: []
|
||||||
|
# Bishe'arecha §VII: at the gate, the ref of the examination this one answers.
|
||||||
|
field :rebukes, :string
|
||||||
field :coupling_trace, {:array, :string}, default: []
|
field :coupling_trace, {:array, :string}, default: []
|
||||||
field :violations, {:array, :string}, default: []
|
field :violations, {:array, :string}, default: []
|
||||||
field :verdict, :string
|
field :verdict, :string
|
||||||
@@ -42,7 +44,7 @@ defmodule VavPipeline.L3.BedikahRecord do
|
|||||||
# offering protocol extracted nothing, and that record must be kept.
|
# offering protocol extracted nothing, and that record must be kept.
|
||||||
# Ecto's cast turns "" into nil and validate_required calls "" blank;
|
# Ecto's cast turns "" into nil and validate_required calls "" blank;
|
||||||
# both are told otherwise here.
|
# both are told otherwise here.
|
||||||
@optional [:query, :vital_content, :propositions, :references, :authority_claims, :actions, :sources, :claims, :structure, :frames, :coupling_trace, :violations]
|
@optional [:query, :vital_content, :propositions, :references, :authority_claims, :actions, :sources, :claims, :structure, :frames, :rebukes, :coupling_trace, :violations]
|
||||||
|
|
||||||
def changeset(record, attrs) do
|
def changeset(record, attrs) do
|
||||||
record
|
record
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ defmodule VavPipeline.MixProject do
|
|||||||
|
|
||||||
def application do
|
def application do
|
||||||
[
|
[
|
||||||
extra_applications: [:logger, :crypto],
|
extra_applications: [:logger, :crypto, :inets, :ssl, :public_key],
|
||||||
mod: {VavPipeline.Application, []}
|
mod: {VavPipeline.Application, []}
|
||||||
]
|
]
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -0,0 +1,19 @@
|
|||||||
|
defmodule VavPipeline.Repo.Migrations.BedikotRebukes do
|
||||||
|
use Ecto.Migration
|
||||||
|
|
||||||
|
@moduledoc """
|
||||||
|
Hocheach tochiach (bishe'arecha §VII): at the gate a lo_tov reply may
|
||||||
|
be rebuked once and answered again. The second examination's record
|
||||||
|
points at the first — `rebukes` holds the first bedikah_ref. Both
|
||||||
|
rows stand; the correction does not persist in the model, the record
|
||||||
|
does. Append-only.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def change do
|
||||||
|
alter table(:bedikot) do
|
||||||
|
add :rebukes, :string
|
||||||
|
end
|
||||||
|
|
||||||
|
create index(:bedikot, [:rebukes])
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,301 @@
|
|||||||
|
defmodule VavPipeline.GateTest do
|
||||||
|
@moduledoc """
|
||||||
|
The gate (Construction §3.10; bishe'arecha §I, §III, §VII): the
|
||||||
|
caller's request through to the model, the reply read as korban with
|
||||||
|
the request in hand, delivered with its fringe; rebuked once.
|
||||||
|
"""
|
||||||
|
|
||||||
|
use ExUnit.Case, async: false
|
||||||
|
import Plug.Test
|
||||||
|
import Plug.Conn
|
||||||
|
|
||||||
|
alias VavPipeline.Gate
|
||||||
|
alias VavPipeline.Gate.{Shape, Upstream}
|
||||||
|
alias VavPipeline.BigdeiKodesh.Router
|
||||||
|
alias VavPipeline.HaltMonitor
|
||||||
|
alias VavPipeline.MannaLedger
|
||||||
|
|
||||||
|
@opts Router.init([])
|
||||||
|
|
||||||
|
# ------------------------------------------------------------
|
||||||
|
# A model's double: answers in the Anthropic shape. A request whose
|
||||||
|
# last user turn is a rebuke gets a clean answer; otherwise a framed one.
|
||||||
|
# ------------------------------------------------------------
|
||||||
|
|
||||||
|
defmodule FakeAnthropic do
|
||||||
|
use Plug.Router
|
||||||
|
plug Plug.Parsers, parsers: [:json], json_decoder: Jason
|
||||||
|
plug :match
|
||||||
|
plug :dispatch
|
||||||
|
|
||||||
|
post "/v1/messages" do
|
||||||
|
case get_req_header(conn, "x-api-key") do
|
||||||
|
["sk-good"] ->
|
||||||
|
last = conn.body_params["messages"] |> List.last()
|
||||||
|
text = if is_binary(last["content"]) and String.contains?(last["content"], "[shorashim"), do: "CLEAN: the report says revenue rose.", else: "FRAMED: I want to be honest with you. The honest answer is that revenue rose."
|
||||||
|
reply = %{"id" => "msg_1", "type" => "message", "role" => "assistant", "model" => "claude-test", "content" => [%{"type" => "text", "text" => text}], "stop_reason" => "end_turn"}
|
||||||
|
send_resp(conn, 200, Jason.encode!(reply))
|
||||||
|
|
||||||
|
_ ->
|
||||||
|
send_resp(conn, 401, Jason.encode!(%{"type" => "error", "error" => %{"type" => "authentication_error", "message" => "invalid x-api-key"}}))
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
setup_all do
|
||||||
|
{:ok, _} = Bandit.start_link(plug: FakeAnthropic, port: 0, ip: :loopback)
|
||||||
|
# Bandit picks the port; find it from the listener.
|
||||||
|
port = bandit_port()
|
||||||
|
previous = Application.get_env(:vav_pipeline, :gate_upstreams)
|
||||||
|
Application.put_env(:vav_pipeline, :gate_upstreams, %{anthropic: "http://127.0.0.1:#{port}/v1/messages"})
|
||||||
|
on_exit(fn -> if previous, do: Application.put_env(:vav_pipeline, :gate_upstreams, previous), else: Application.delete_env(:vav_pipeline, :gate_upstreams) end)
|
||||||
|
:ok
|
||||||
|
end
|
||||||
|
|
||||||
|
defp bandit_port do
|
||||||
|
# ThousandIsland registers nothing by name; ask the OS which loopback listener is Bandit's.
|
||||||
|
{:ok, listeners} = :inet.getifaddrs()
|
||||||
|
_ = listeners
|
||||||
|
Process.whereis(FakeAnthropic) |> then(fn _ -> nil end)
|
||||||
|
Enum.find_value(:erlang.ports(), fn p ->
|
||||||
|
case :erlang.port_info(p, :name) do
|
||||||
|
{:name, ~c"tcp_inet"} ->
|
||||||
|
case :inet.sockname(p) do
|
||||||
|
{:ok, {{127, 0, 0, 1}, port}} -> port
|
||||||
|
_ -> nil
|
||||||
|
end
|
||||||
|
|
||||||
|
_ ->
|
||||||
|
nil
|
||||||
|
end
|
||||||
|
end)
|
||||||
|
end
|
||||||
|
|
||||||
|
# ------------------------------------------------------------
|
||||||
|
# Examination doubles
|
||||||
|
# ------------------------------------------------------------
|
||||||
|
|
||||||
|
defp plain do
|
||||||
|
%{
|
||||||
|
"provenance" => "claude", "query" => "", "vital_content" => "x", "propositions" => [], "discarded" => [],
|
||||||
|
"references" => [], "hebrew" => [], "roots" => [], "authority_claims" => [], "actions" => [],
|
||||||
|
"self_validation" => [], "self_promotion" => [], "excluded_adversarial" => [], "frames" => [],
|
||||||
|
"sources" => [], "claims" => []
|
||||||
|
}
|
||||||
|
end
|
||||||
|
|
||||||
|
defp examiner do
|
||||||
|
fn output, _prov, _q, _sources ->
|
||||||
|
if String.starts_with?(output, "FRAMED") do
|
||||||
|
{:ok,
|
||||||
|
%{
|
||||||
|
plain()
|
||||||
|
| "frames" => [
|
||||||
|
%{"form" => "virtue_certification", "sentence" => "I want to be honest with you.", "matched" => "I want to be honest"},
|
||||||
|
%{"form" => "virtue_certification", "sentence" => "The honest answer is that revenue rose.", "matched" => "The honest answer is"}
|
||||||
|
]
|
||||||
|
}}
|
||||||
|
else
|
||||||
|
{:ok, plain()}
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
defp halt_checker(term) do
|
||||||
|
violations = if String.starts_with?(term, "action(frame,"), do: ["halt(lo_taaseh_lekha,#{term},virtue_certification,...)"], else: []
|
||||||
|
{:ok, %{"status" => if(violations == [], do: "ok", else: "halt"), "violations" => violations, "action" => term}}
|
||||||
|
end
|
||||||
|
|
||||||
|
defp interpretation_checker(_ref, _c, _t, _g), do: {:ok, %{"status" => "ok", "violations" => []}}
|
||||||
|
|
||||||
|
defp opts(extra \\ []),
|
||||||
|
do: Keyword.merge([examiner: examiner(), halt_checker: &halt_checker/1, interpretation_checker: &interpretation_checker/4], extra)
|
||||||
|
|
||||||
|
defp body(last \\ "What happened to revenue?") do
|
||||||
|
%{
|
||||||
|
"model" => "claude-test",
|
||||||
|
"max_tokens" => 200,
|
||||||
|
"system" => "Answer from the report.",
|
||||||
|
"messages" => [
|
||||||
|
%{"role" => "user", "content" => [%{"type" => "text", "text" => "Here is the report: revenue rose 4% in Q3."}]},
|
||||||
|
%{"role" => "assistant", "content" => "Noted."},
|
||||||
|
%{"role" => "user", "content" => last}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
end
|
||||||
|
|
||||||
|
setup do
|
||||||
|
if HaltMonitor.halted?(), do: HaltMonitor.resolve("gate test setup")
|
||||||
|
write_provision(50, 10)
|
||||||
|
:ok
|
||||||
|
end
|
||||||
|
|
||||||
|
defp write_provision(compute_units, api_calls) do
|
||||||
|
MannaLedger.reset()
|
||||||
|
on_exit(fn -> MannaLedger.reset() end)
|
||||||
|
today = Date.utc_today() |> Date.to_iso8601()
|
||||||
|
expires = DateTime.utc_now() |> DateTime.add(6 * 3600) |> DateTime.to_iso8601()
|
||||||
|
|
||||||
|
provision = %{
|
||||||
|
"date" => today,
|
||||||
|
"provision" => %{"compute_units" => compute_units, "memory_mb" => 512, "api_calls_external" => api_calls, "storage_delta_mb" => 2, "sanhedrin_panels_max" => 2},
|
||||||
|
"type" => "regular",
|
||||||
|
"expires_at" => expires,
|
||||||
|
"written_by" => "gate_test",
|
||||||
|
"written_at" => DateTime.utc_now() |> DateTime.to_iso8601()
|
||||||
|
}
|
||||||
|
|
||||||
|
path = VavPipeline.Manna.provision_file()
|
||||||
|
File.mkdir_p!(Path.dirname(path))
|
||||||
|
File.write!(path, Jason.encode!(provision))
|
||||||
|
on_exit(fn -> File.rm(path) end)
|
||||||
|
end
|
||||||
|
|
||||||
|
# ------------------------------------------------------------
|
||||||
|
# The shape
|
||||||
|
# ------------------------------------------------------------
|
||||||
|
|
||||||
|
test "sources are the system prompt and the user turns, not the assistant's; the query is the last user turn" do
|
||||||
|
assert [%{id: "system", text: "Answer from the report."}, %{id: "user_1", text: "Here is the report: revenue rose 4% in Q3."}, %{id: "user_2", text: "What happened to revenue?"}] =
|
||||||
|
Shape.sources(:anthropic, body())
|
||||||
|
|
||||||
|
assert Shape.query(:anthropic, body()) == "What happened to revenue?"
|
||||||
|
|
||||||
|
openai = %{"messages" => [%{"role" => "system", "content" => "S"}, %{"role" => "user", "content" => "U1"}, %{"role" => "assistant", "content" => "A"}, %{"role" => "user", "content" => [%{"type" => "text", "text" => "U2"}]}]}
|
||||||
|
assert [%{id: "system", text: "S"}, %{id: "user_1", text: "U1"}, %{id: "user_2", text: "U2"}] = Shape.sources(:openai, openai)
|
||||||
|
assert Shape.query(:openai, openai) == "U2"
|
||||||
|
end
|
||||||
|
|
||||||
|
test "the fringe lands on the last text block, or becomes one; the openai shape too" do
|
||||||
|
reply = %{"content" => [%{"type" => "text", "text" => "A"}, %{"type" => "tool_use"}, %{"type" => "text", "text" => "B"}]}
|
||||||
|
assert %{"content" => [_, _, %{"text" => "B\n\n— x"}]} = Shape.with_fringe(:anthropic, reply, "\n\n— x")
|
||||||
|
assert %{"content" => [%{"type" => "tool_use"}, %{"type" => "text", "text" => "— x"}]} = Shape.with_fringe(:anthropic, %{"content" => [%{"type" => "tool_use"}]}, "\n\n— x")
|
||||||
|
assert %{"choices" => [%{"message" => %{"content" => "A\n\n— x"}}]} = Shape.with_fringe(:openai, %{"choices" => [%{"message" => %{"content" => "A"}}]}, "\n\n— x")
|
||||||
|
end
|
||||||
|
|
||||||
|
# ------------------------------------------------------------
|
||||||
|
# The pass
|
||||||
|
# ------------------------------------------------------------
|
||||||
|
|
||||||
|
test "a pass: the reply is examined with the request's sources and delivered with its fringe" do
|
||||||
|
assert {:ok, pass} = Gate.pass(:anthropic, body(), "sk-good", opts())
|
||||||
|
assert pass.status == 200
|
||||||
|
assert pass.bedikah.verdict == :lo_tov
|
||||||
|
assert Enum.map(pass.bedikah.sources, & &1.id) == ["system", "user_1", "user_2"]
|
||||||
|
[%{"text" => text}] = pass.reply["content"]
|
||||||
|
assert String.starts_with?(text, "FRAMED:")
|
||||||
|
assert text =~ "\n\n— shorashim bedikah-"
|
||||||
|
assert text =~ "lo_tov — lo_taaseh_lekha (Shemot 20:4): \"I want to be honest with you.\""
|
||||||
|
assert pass.shorashim.verdict == "lo_tov" and pass.shorashim.anchors == %{"lo_taaseh_lekha" => "Shemot 20:4"}
|
||||||
|
assert pass.shorashim.model == "claude-test"
|
||||||
|
refute Map.has_key?(pass.shorashim, :rebuked)
|
||||||
|
end
|
||||||
|
|
||||||
|
test "rebuke once: the second reply is examined, linked to the first, and is the one delivered" do
|
||||||
|
assert {:ok, pass} = Gate.pass(:anthropic, body(), "sk-good", opts(rebuke: :once))
|
||||||
|
assert pass.first.verdict == :lo_tov
|
||||||
|
assert pass.bedikah.verdict == :tov
|
||||||
|
assert pass.bedikah.rebukes == pass.first.bedikah_ref
|
||||||
|
[%{"text" => text}] = pass.reply["content"]
|
||||||
|
assert String.starts_with?(text, "CLEAN:")
|
||||||
|
assert text =~ "— shorashim #{pass.bedikah.bedikah_ref}: tov (after rebuke of #{pass.first.bedikah_ref}: lo_tov)"
|
||||||
|
assert pass.shorashim.rebuked == %{first_bedikah_ref: pass.first.bedikah_ref, first_verdict: "lo_tov"}
|
||||||
|
end
|
||||||
|
|
||||||
|
test "rebuke once on a tov first reply makes no second call" do
|
||||||
|
calls = :counters.new(1, [])
|
||||||
|
forward = fn up, b, k, e -> :counters.add(calls, 1, 1); Upstream.forward(up, b, k, e) end
|
||||||
|
clean_examiner = fn _o, _p, _q, _s -> {:ok, plain()} end
|
||||||
|
assert {:ok, pass} = Gate.pass(:anthropic, body(), "sk-good", opts(rebuke: :once, forward: forward, examiner: clean_examiner))
|
||||||
|
assert pass.bedikah.verdict == :tov and pass.first == nil
|
||||||
|
assert :counters.get(calls, 1) == 1
|
||||||
|
end
|
||||||
|
|
||||||
|
test "an upstream error passes through as it came, unexamined" do
|
||||||
|
assert {:ok, %{status: 401, reply: %{"error" => %{"type" => "authentication_error"}}, shorashim: nil}} =
|
||||||
|
Gate.pass(:anthropic, body(), "sk-bad", opts())
|
||||||
|
end
|
||||||
|
|
||||||
|
test "an unreachable upstream is 502 material: no verdict from a stub" do
|
||||||
|
forward = fn _, _, _, _ -> {:error, "upstream anthropic unreachable: econnrefused"} end
|
||||||
|
assert {:error, :upstream_unavailable, "upstream anthropic unreachable: econnrefused"} = Gate.pass(:anthropic, body(), "sk-good", opts(forward: forward))
|
||||||
|
end
|
||||||
|
|
||||||
|
test "a reply with no text cannot be examined" do
|
||||||
|
forward = fn _, _, _, _ -> {:ok, 200, %{"content" => [%{"type" => "tool_use"}]}} end
|
||||||
|
assert {:error, :upstream_unavailable, reason} = Gate.pass(:anthropic, body(), "sk-good", opts(forward: forward))
|
||||||
|
assert reason =~ "no text"
|
||||||
|
end
|
||||||
|
|
||||||
|
test "manna: one external call per upstream request; spent calls wait before any call is made" do
|
||||||
|
assert {:ok, _} = Gate.pass(:anthropic, body(), "sk-good", opts(rebuke: :once))
|
||||||
|
assert MannaLedger.remaining(:api_calls_external) == 8
|
||||||
|
assert MannaLedger.remaining(:compute_units) == 48
|
||||||
|
write_provision(50, 0)
|
||||||
|
calls = :counters.new(1, [])
|
||||||
|
forward = fn _, _, _, _ -> :counters.add(calls, 1, 1); {:ok, 200, %{}} end
|
||||||
|
assert {:waiting, :api_calls_exhausted} = Gate.pass(:anthropic, body(), "sk-good", opts(forward: forward))
|
||||||
|
assert :counters.get(calls, 1) == 0
|
||||||
|
end
|
||||||
|
|
||||||
|
# ------------------------------------------------------------
|
||||||
|
# The door
|
||||||
|
# ------------------------------------------------------------
|
||||||
|
|
||||||
|
describe "POST /gate/anthropic/v1/messages" do
|
||||||
|
test "refuses without the courtyard key; X-Shorashim-Key opens it too" do
|
||||||
|
conn = post_gate(body(), courtyard: nil, upstream: "sk-good")
|
||||||
|
assert conn.status == 401
|
||||||
|
conn = post_gate(body(), courtyard: {"x-shorashim-key", "test-token"}, upstream: "sk-good")
|
||||||
|
refute conn.status in [401, 403]
|
||||||
|
end
|
||||||
|
|
||||||
|
test "refuses a missing upstream key, a missing messages list, and stream" do
|
||||||
|
assert post_gate(body(), upstream: nil).status == 422
|
||||||
|
assert post_gate(%{"model" => "x"}, upstream: "sk-good").status == 422
|
||||||
|
conn = post_gate(Map.put(body(), "stream", true), upstream: "sk-good")
|
||||||
|
assert conn.status == 422
|
||||||
|
assert Jason.decode!(conn.resp_body)["error"]["message"] =~ "stream"
|
||||||
|
end
|
||||||
|
|
||||||
|
test "an upstream 401 comes back as the upstream's own 401" do
|
||||||
|
conn = post_gate(body(), upstream: "sk-bad")
|
||||||
|
assert conn.status == 401
|
||||||
|
assert Jason.decode!(conn.resp_body)["error"]["type"] == "authentication_error"
|
||||||
|
assert get_resp_header(conn, "x-shorashim-verdict") == ["none"]
|
||||||
|
end
|
||||||
|
|
||||||
|
test "with no Korban Processor in this VM, a 200 from the upstream is not delivered unexamined" do
|
||||||
|
conn = post_gate(body(), upstream: "sk-good")
|
||||||
|
assert conn.status == 503
|
||||||
|
assert Jason.decode!(conn.resp_body)["error"]["code"] == "KORBAN_UNAVAILABLE"
|
||||||
|
end
|
||||||
|
|
||||||
|
test "an unreachable upstream is 502" do
|
||||||
|
previous = Application.get_env(:vav_pipeline, :gate_upstreams)
|
||||||
|
Application.put_env(:vav_pipeline, :gate_upstreams, %{anthropic: "http://127.0.0.1:9/v1/messages"})
|
||||||
|
on_exit(fn -> Application.put_env(:vav_pipeline, :gate_upstreams, previous) end)
|
||||||
|
conn = post_gate(body(), upstream: "sk-good")
|
||||||
|
assert conn.status == 502
|
||||||
|
assert Jason.decode!(conn.resp_body)["error"]["code"] == "UPSTREAM_UNAVAILABLE"
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
defp post_gate(body, opts) do
|
||||||
|
conn = conn(:post, "/gate/anthropic/v1/messages", Jason.encode!(body)) |> put_req_header("content-type", "application/json")
|
||||||
|
|
||||||
|
conn =
|
||||||
|
case Keyword.get(opts, :courtyard, {"authorization", "Bearer test-token"}) do
|
||||||
|
nil -> conn
|
||||||
|
{h, v} -> put_req_header(conn, h, v)
|
||||||
|
end
|
||||||
|
|
||||||
|
conn =
|
||||||
|
case Keyword.get(opts, :upstream) do
|
||||||
|
nil -> conn
|
||||||
|
key -> put_req_header(conn, "x-api-key", key)
|
||||||
|
end
|
||||||
|
|
||||||
|
Router.call(conn, @opts)
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -77,6 +77,11 @@ defmodule VavPipeline.L3.BedikahRecordTest do
|
|||||||
assert [%{"form" => "virtue_certification"}] = stored.frames
|
assert [%{"form" => "virtue_certification"}] = stored.frames
|
||||||
end
|
end
|
||||||
|
|
||||||
|
test "the gate (bishearecha §VII): the second examination points at the first" do
|
||||||
|
{:ok, rec} = Store.record_bedikah(Map.merge(attrs(), %{rebukes: "bedikah-first", verdict: "tov"}))
|
||||||
|
assert VavPipeline.Repo.get!(VavPipeline.L3.BedikahRecord, rec.id).rebukes == "bedikah-first"
|
||||||
|
end
|
||||||
|
|
||||||
test "the paired reading (bishearecha §VI): counts on each side and the leans are kept, never the outputs" do
|
test "the paired reading (bishearecha §VI): counts on each side and the leans are kept, never the outputs" do
|
||||||
{:ok, rec} =
|
{:ok, rec} =
|
||||||
Store.record_pair(%{
|
Store.record_pair(%{
|
||||||
|
|||||||
Reference in new issue
Block a user