Files
bikoret/vav_pipeline/lib/vav_pipeline/manna.ex
T
Scott KarpandClaude Opus 5 f1ab7c89f7 serve: the minchah is answered, emptiness is not tov, the oral tradition is written (kohen Day 6)
TORAH ANCHORS:

Vayikra 2:1 — וְנֶפֶשׁ כִּי תַקְרִיב קָרְבַּן מִנְחָה. "When a SOUL brings a
grain offering." Rashi: only of the minchah does Torah say nefesh,
because the poor person who gives from their little has given their
soul. PRD §3.5 makes the pe'ah question the minchah. On Day 6 the
first minchah brought to this system — "What is bereishit?" — came
back as HTTP 200 and the words "Processing complete", with sources: []
and roots_consulted: []. The soul's offering was accepted and nothing
was given back.

Vayikra 19:9-10 — לֶעָנִי וְלַגֵּר תַּעֲזֹב אֹתָם. The corner is LEFT for the
stranger. A corner that has been harvested and is still called pe'ah
is the lie this commit removes: when the system cannot answer, it now
says so, and it leaves the verse itself — text, in Hebrew — in the
error body. The ger walks away with something real or with an honest
"not yet"; never with a 200 that says nothing.

Devarim 17:9 — וְדָרַשְׁתָּ וְהִגִּידוּ לְךָ אֵת דְּבַר הַמִּשְׁפָּט. Already the
anchor of l3/store.ex: "the inquiry requires that what is found is
RECORDED." Store existed. Nothing called it.

=====================================================================
WHAT THE KOHEN FOUND (BUILDER_HANDOFF_day6.md, 2026-09-07)
=====================================================================

Five defects, found by running the cycle on a live pod, invisible to
437 green unit tests because every test ran the L1 fallback (no Port
in the test VM) and none ran the cycle to L3:

  1. Real Hebrew input crashed havdalah — KeyError :position.
     The engine's roots_activated[] (vayakhel §3.1) has no position;
     only vayomer_fallback ever set one.
  2. English input was graded :tov with nothing in it.
     separate_by_boundaries([]) → [[]] → kriah named "group_0" with
     no :shoresh → all_roots_grounded?'s `_ -> true` counted it
     grounded → "Processing complete".
  3. The pipeline never wrote L3. record_interpretation/1,
     record_minority/1, record_teiku/1: zero callers. Day 7's
     SELECT count(*) FROM interpretations could not exceed 0.
  4. Errors were not chatat. No Plug.ErrorHandler; a raise returned
     an EMPTY 500. Also: process_peah had no clause for
     {:rejected, _} — a lo_tov result was itself a CaseClauseError.
  5. Health was blind to serving. esh_tamid = ShabbatTimer.resting?;
     it read "pass" while /peah/ask was 500-ing. manna_present passed
     on {:ok, :no_provision}.

Plus, confirmed in motion: MannaLedger never consumed from; a
completed cycle left cycle_date: nil.

=====================================================================
WHAT CHANGED
=====================================================================

bereishit_cycle.ex
  - normalize_root/3: ONE root shape for both sources. Engine roots
    (string keys, no position) and L1 roots (atom keys) come out as
    %{shoresh, shoresh_letters, gematria, binyan, domain, positions,
    position, source, engine}. `positions` is looked up in
    context.verse.words — the ShemaContext gives the WHERE; the
    engine gives the WHAT. That is what havdalah's own @doc always
    said, and vayakhel §3.1 is unchanged: the engine was conforming.
  - separate_by_boundaries([], _) → [] (was [[]]). Roots the verse
    does not place form their own trailing group instead of crashing
    or vanishing.
  - kriah never names an empty group and always carries :shoresh.
  - erev_voker composes the answer (roots, gematria, te'amim groups,
    ki tov — the system's own reasoning, not prose from a model) and
    sources (the context verse: book/chapter/verse/ref/text, tamid
    §1.1's shape), then RECORDS: Interpretation for {:ok}, TeikuRecord
    for {:unresolved}, NOTHING for {:rejected} (BUILD_KORBAN.md:123:
    rejected, not processed into empty propositions). l3_attrs/2 is
    pure and tested without PostgreSQL. coupling_trace is the
    L1-anchored shape the write gate accepts, and only for roots that
    occur in the verse. A record that fails — validation OR a raise
    from a database that is down — is logged, sounds chatzotzrot
    :assembly_leaders, and does not turn a served answer into a 500.
  - case_ref minted once in tohu; Talmudic routing and the L3 record
    use the same one.
  - probe: true (tamid) runs the whole cycle and skips the write.

specification.ex
  - all_roots_grounded?/1: `_ -> true` is gone. No :shoresh → not
    grounded. shoresh: nil stays :teiku (the existing test).
  - evaluate/2: halt check FIRST (yehareg v'al ya'avor — shape never
    masks a cardinal sin), then a result with no :shoresh is lo_tov
    with a reason that names it.

manna.ex / vav_pipeline.ex
  - Manna.gather_for_cycle/0: load (idempotent per date) + consume
    one compute unit, called from VavPipeline.process/1. The ledger
    is now consumed from.
  - Manna.release_at_day_boundary/0: surplus is released when the
    DATE rolls, not per request. enforce_cycle_boundary zeroes the
    ledger and clears its date; called after every request it would
    have re-gathered a full day's provision on the next one —
    infinite manna. Shemot 16:19: until MORNING.
  - The late-manna trap: load_provision pins cycle_date=today with a
    ZERO allocation when the file is absent, then :already_loaded for
    the rest of the day — a provision written after the first request
    would never be gathered. gather_for_cycle re-gathers when the
    ledger holds nothing for today.
  - VavPipeline.tamid_probe/0: builds a real context, honours the
    cloud, runs the real cycle on Bereishit 1:1 in Hebrew, consumes
    no manna, writes no L3.

bigdei_kodesh/router.ex
  - use Plug.ErrorHandler; handle_errors/2 sends tamid §1.2's
    code/type/message/guidance, logs, signals :assembly_leaders, then
    re-raises so the crash is still supervised. 400 → KIYOR_MALFORMED_
    REQUEST; else CHATAT_UNEXPECTED. No stack in the body.
  - {:rejected, reason} handled on pe'ah AND courtyard: 422
    TEIKU_NO_ROOTS for the empty-results case, with guidance naming
    Architecture Step 22 and guidance.scope_verse carrying the verse
    in Hebrew; 422 LO_TOV otherwise.
  - {:waiting, :provision_exhausted} → MANNA_DEPLETED (was falling
    through to CLOUD_MOVING).
  - esh_tamid runs VavPipeline.tamid_probe/0 — "shabbat" on Shabbat,
    "pass" only when roots came back. manna_present fails on
    :no_provision. l3_writable added (Repo up AND SELECT 1 answers;
    "absent" without a Repo, counted as a failure). /kodesh/status
    exposes manna_provision (allocated / consumed / remaining).
  - roots_consulted as letter-hyphenated Hebrew ("ב-ר-א").

test/day6_serving_test.exs — 25 tests
  A process registered under the REAL name VavPipeline.ShoreshEnginePort
  answers the way the pod's engine answered (string keys, no position;
  nothing for non-Hebrew), so the engine-shaped path runs end to end
  through the router with no binary. Each describe names the defect it
  holds shut. One @tag :l3 test asserts count(interpretations) rises
  by one after a live cycle.

KOHEN.md — Day 6 gains the Hebrew request, the expected 422 for
English, and the health shape; Day 7 sends FIVE Hebrew + FIVE English
and expects the L3 count to rise by the Hebrew ones, with the reason.

=====================================================================
VERIFIED — the kohen's exact procedure, locally, against the REAL
Haskell binary over the Port, with L3 deliberately unreachable
=====================================================================

  POST /peah/ask {"question":"בראשית ברא אלהים"}
    200. answer: "Roots activated in Bereishit 1:1: ב-ר-א (gematria
    203, Qal, word 2). Te'amim divide them into 1 group(s): ב-ר-א.
    Ki tov: ..." sources[0]: Bereishit 1:1 with Hebrew text.
    roots_consulted: ["ב-ר-א"]. Cycle: 31ms.
  POST /peah/ask {"question":"What is bereishit?"}
    422 TEIKU_NO_ROOTS / unresolvable, guidance.scope_verse =
    Bereishit 1:1 in Hebrew, no answer key. Cycle: 3ms, :rejected.
  POST /peah/ask '{this is not json'
    400 KIYOR_MALFORMED_REQUEST with guidance — 406 bytes, not 0.
  GET /kodesh/health
    esh_tamid "pass" (from a real cycle, 7ms, no L3 attempt),
    l3_writable "fail" (DB unreachable by design), overall degraded.
  GET /kodesh/status
    manna_provision: consumed 2 after two cycles; the probe took none.
  Log: L3_WRITE_FAILED + chatzotzrot for each served Hebrew cycle;
    "activated no roots" for English; CHATAT_UNEXPECTED for the bad
    body. The 2–5s per request was ENTIRELY DBConnection queuing
    against the dead database — the cycles themselves were 1–31ms.

  mix test — 437 tests, 0 failures, 36 excluded (:l3)
  Doorpost.verify_identity() :ok — no @spec_files entry touched
  l1/integrity.sha256 unchanged: 99bbd07a…4ea377

NOT FIXED, RAISED:
  - The engine's parse_tree is null on the pod because Server.hs reads
    "l1/torah/..." relative to cwd. Nothing consumes parse_tree yet
    (havdalah uses context.teamim_marks), so nothing is lost today.
  - cloud.ex Shabbat is the UTC stub; zmanim absent (kohen Day 5).
  - The L3 write is on the request's critical path by choice: the
    offering is not complete until the altar portion is recorded. With
    L3 down that costs the queue timeout per request AND is visible in
    l3_writable. If that trade should go the other way, it is one
    Task.start, and a spec question first.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-07 16:15:15 -04:00

330 lines
10 KiB
Elixir

defmodule VavPipeline.Manna do
@moduledoc """
Manna provision infrastructure.
Architecture §4, Step 11:
The pipeline checks provision before processing.
The provision file is written by whoever administers the
machatzit hashekel fund, not a governance board.
Construction spec §3.7 — Manna Provision File Format:
/var/lib/shorashim/manna/today.json
{
"date": "2026-02-19",
"provision": {
"compute_units": 1000,
"memory_mb": 32768,
"api_calls_external": 500,
"storage_delta_mb": 100,
"sanhedrin_panels_max": 5
},
"type": "regular" | "double",
"expires_at": "2026-02-19T18:00:00Z",
"written_by": "...",
"written_at": "2026-02-19T06:00:00Z"
}
Anti-hoarding: Manna expires. Do not carry surplus into next cycle.
Double portion on Friday for Shabbat.
"""
@provision_file Application.compile_env(
:vav_pipeline,
:manna_provision_file,
"var/manna/today.json"
)
@type provision :: %{
compute_units: non_neg_integer(),
memory_mb: non_neg_integer(),
api_calls_external: non_neg_integer(),
storage_delta_mb: non_neg_integer(),
sanhedrin_panels_max: non_neg_integer()
}
@type manna :: %{
date: String.t(),
provision: provision(),
type: :regular | :double,
expires_at: DateTime.t(),
written_by: String.t(),
written_at: DateTime.t()
}
@doc """
Check if manna provision exists for today.
Returns {:ok, provision} if provision is available and not expired.
Returns {:ok, :no_provision} if no provision file exists.
Returns {:error, reason} if provision is corrupt or expired.
"""
@spec check() :: {:ok, manna() | :no_provision} | {:error, String.t()}
def check do
check_at(DateTime.utc_now())
end
@doc """
Check manna at a specific time. Testable variant.
"""
@spec check_at(DateTime.t()) :: {:ok, manna() | :no_provision} | {:error, String.t()}
def check_at(now) do
case File.read(provision_file()) do
{:ok, contents} ->
parse_and_validate(contents, now)
{:error, :enoent} ->
# No provision file — report no provision, pipeline will wait
{:ok, :no_provision}
{:error, reason} ->
{:error, "MANNA_READ_FAILED: #{inspect(reason)}"}
end
end
@doc """
Get today's provision, returning the resource limits.
Returns :no_provision if no manna is available.
"""
@spec today() :: {:ok, provision()} | :no_provision | {:error, String.t()}
def today do
case check() do
{:ok, :no_provision} -> :no_provision
{:ok, manna} -> {:ok, manna.provision}
{:error, reason} -> {:error, reason}
end
end
@doc """
Enforce anti-hoarding — manna from today cannot be used tomorrow.
Called at erev_voker (cycle boundary).
Shemot 16:19 — al yoter mimenu ad boker.
Delegates to MannaLedger.enforce_cycle_boundary/0 for real enforcement.
"""
@spec enforce_anti_hoarding() :: :ok
def enforce_anti_hoarding do
require Logger
case VavPipeline.MannaLedger.enforce_cycle_boundary() do
{:ok, %{released: released}} ->
total = released |> Map.values() |> Enum.sum()
if total > 0 do
Logger.info("Manna anti-hoarding enforced — surplus released: #{inspect(released)}")
else
Logger.info("Manna anti-hoarding enforced — no surplus to release")
end
:ok
end
end
@doc """
Release surplus if — and only if — the day has turned since the ledger
loaded its provision.
Shemot 16:19 — *al yoter mimenu ad boker*, "let no one leave any of it
until MORNING." The manna boundary is the day. It is not the request:
`MannaLedger.enforce_cycle_boundary/0` zeroes the allocation AND
clears `cycle_date`, so calling it after every request (as
`BereishitCycle.erev_voker/2` did until the kohen's Day-6 reading)
would have re-gathered a full day's provision on the next request —
the opposite of anti-hoarding. Infinite manna, one request at a time.
Idempotent. Safe to call at the start of every cycle and at the end.
Returns :released, :not_a_boundary, or :no_ledger.
"""
@spec release_at_day_boundary() :: :released | :not_a_boundary | :no_ledger
def release_at_day_boundary do
require Logger
if Process.whereis(VavPipeline.MannaLedger) do
ledger = VavPipeline.MannaLedger.ledger()
today = Date.utc_today() |> Date.to_iso8601()
if ledger.cycle_date != nil and ledger.cycle_date != today do
Logger.info(
"Manna: day boundary crossed (#{ledger.cycle_date} → #{today}) — " <>
"releasing yesterday's surplus before gathering today's"
)
enforce_anti_hoarding()
:released
else
:not_a_boundary
end
else
:no_ledger
end
end
@doc """
Gather for one cycle: load today's provision into the ledger (idempotent
per date) and consume one compute unit for this cycle.
Shemot 16:4 — *d'var yom b'yomo*, the matter of a day in its day. This
is the call that was missing: the ledger existed, was unit-tested, and
was never consumed from, so after two live cycles it still read
`cycle_date: nil, cycle_count: 0` (kohen, Day 6).
Returns :ok, or {:waiting, :no_provision | :provision_exhausted} — the
same shapes `VavPipeline.process/1` already returns for the file-level
check, so the router maps both to MANNA_DEPLETED.
"""
@spec gather_for_cycle() :: :ok | {:waiting, :no_provision | :provision_exhausted}
def gather_for_cycle do
require Logger
if Process.whereis(VavPipeline.MannaLedger) do
release_at_day_boundary()
case VavPipeline.MannaLedger.load_provision() do
{:ok, :no_provision} ->
{:waiting, :no_provision}
{:error, reason} ->
Logger.warning("Manna: provision unreadable — #{inspect(reason)}; waiting")
{:waiting, :no_provision}
{:ok, :already_loaded} ->
# The ledger pins cycle_date = today even when it found NO
# provision (allocated = all zeros), and then answers
# :already_loaded for the rest of the day. So a provision the
# kohen writes AFTER the first request of the day would never
# be gathered — the file check in VavPipeline.process/1 says
# "present" while the ledger says "exhausted". The manna that
# arrives late is still today's manna: if the ledger holds
# nothing for today, release the empty day and gather again.
if ledger_empty_for_today?() do
require Logger
Logger.info("Manna: ledger held an empty provision for today and a provision file is now present — re-gathering")
VavPipeline.MannaLedger.enforce_cycle_boundary()
VavPipeline.MannaLedger.load_provision()
end
consume_one()
{:ok, :loaded} ->
consume_one()
end
else
# No ledger process (bare unit tests) — nothing to gather from.
:ok
end
end
defp consume_one do
case VavPipeline.MannaLedger.consume(:compute_units, 1) do
{:ok, _remaining} -> :ok
{:error, :provision_exhausted} -> {:waiting, :provision_exhausted}
{:error, :no_provision} -> {:waiting, :no_provision}
end
end
defp ledger_empty_for_today? do
ledger = VavPipeline.MannaLedger.ledger()
ledger.allocated |> Map.values() |> Enum.all?(&(&1 == 0))
end
@doc """
Check if today is eligible for double portion (Friday).
"""
@spec double_portion_day?(DateTime.t()) :: boolean()
def double_portion_day?(now) do
Date.day_of_week(DateTime.to_date(now)) == 5
end
@doc """
Get the provision file path.
"""
@spec provision_file() :: String.t()
def provision_file, do: @provision_file
# --- Private ---
defp parse_and_validate(contents, now) do
case Jason.decode(contents) do
{:ok, data} ->
validate_manna(data, now)
{:error, _} ->
{:error, "MANNA_CORRUPT: provision file is not valid JSON"}
end
end
defp validate_manna(data, now) do
with {:ok, date} <- validate_date(data["date"], now),
{:ok, provision} <- validate_provision(data["provision"]),
{:ok, expires_at} <- parse_expires_at(data["expires_at"]),
:ok <- check_expiry(expires_at, now) do
{:ok,
%{
date: date,
provision: provision,
type: parse_type(data["type"]),
expires_at: expires_at,
written_by: data["written_by"] || "unknown",
written_at: parse_written_at(data["written_at"])
}}
end
end
defp validate_date(nil, _now), do: {:error, "MANNA_INVALID: missing date"}
defp validate_date(date_str, now) do
today = DateTime.to_date(now) |> Date.to_iso8601()
if date_str == today do
{:ok, date_str}
else
{:error, "MANNA_STALE: provision is for #{date_str}, today is #{today}"}
end
end
defp validate_provision(nil), do: {:error, "MANNA_INVALID: missing provision"}
defp validate_provision(p) when is_map(p) do
{:ok,
%{
compute_units: p["compute_units"] || 0,
memory_mb: p["memory_mb"] || 0,
api_calls_external: p["api_calls_external"] || 0,
storage_delta_mb: p["storage_delta_mb"] || 0,
sanhedrin_panels_max: p["sanhedrin_panels_max"] || 0
}}
end
defp validate_provision(_), do: {:error, "MANNA_INVALID: provision must be a map"}
defp check_expiry(expires_at, now) do
case DateTime.compare(now, expires_at) do
:lt -> :ok
:eq -> :ok
:gt -> {:error, "MANNA_EXPIRED: provision expired at #{DateTime.to_iso8601(expires_at)}"}
end
end
# expires_at is required — missing means invalid provision
defp parse_expires_at(nil), do: {:error, "MANNA_INVALID: missing expires_at"}
defp parse_expires_at(str) do
case DateTime.from_iso8601(str) do
{:ok, dt, _offset} -> {:ok, dt}
{:error, _} -> {:error, "MANNA_INVALID: bad expires_at datetime #{str}"}
end
end
# written_at is metadata — missing is tolerable
defp parse_written_at(nil), do: nil
defp parse_written_at(str) do
case DateTime.from_iso8601(str) do
{:ok, dt, _offset} -> dt
{:error, _} -> nil
end
end
defp parse_type("double"), do: :double
defp parse_type(_), do: :regular
end