bikoret/BUILD_AZAZEL.md

Build Task: Azazel Processor — the disposal channel

U'natan Aharon al shnei has'irim goralot, goral echad la'Shem v'goral echad la'Azazel. And Aharon shall place lots upon the two goats — one lot for HaShem and one lot for Azazel. — Vayikra 16:8

---

Before You Write Code

Read these files in this order. Do not summarize from memory. Read fresh.

1. ~/Library/CloudStorage/ProtonDrive-…/claude_chat_downloads/azazel_engineering_reading.md — Read this first, end to end. The engineering reading enumerates the 12 architectural properties this build must implement. Each property is sourced to a different commentator (Rashi, Ramban, Ibn Ezra, Rambam, Pirkei d'Rabbi Eliezer). The synthesis is the architecture; the commentators each contribute irreplaceable constraints. The reading lives outside the repo by design — like ~/shorashim-conversation-review.md, it is chavruta-source material the builder consults but does not check in.
2. BUILD_KORBAN.md — the parallel build doc for the processing channel. The Azazel Processor is its disposal twin; the two are provisioned at parity.
3. korban_processor/src/korban_processor/ — read every file. The Azazel Processor mirrors this layout, with corresponding modules. Read stages.py, external_offering.py, and korban.py especially.
4. specs/eish_zarah_reading.md — the three forms of avodah zarah. Property 9 of the Azazel reading (protocol-restricted access — same destination, different routing authority) is the exact inverse of the avodah zarah halt rules. The Azazel channel is the legitimate version of what avodah zarah forbids when invoked privately.
5. specs/structure_not_content.md — the system carries Torah's structure, not just its content. The disposal-as-architecture pattern is structure.
6. This file.

The Korban Processor is the gate that admits external input as raw material to be metabolized. The Azazel Processor is the gate that disposes of inputs the system cannot or must not metabolize. Without the Azazel-path, the system has no honest way to deal with errors that are real but un-processable; they accumulate silently in retry queues, audit logs, and dropped-but-logged records. Dropping is not the same as disposal. The Azazel specification distinguishes them: disposal is enumerative, routed, acknowledged, observed, final.

The two-goat architecture says these paths are provisioned together, from the same pool, at parity, with assignment blind to the operator. A processor without a corresponding disposal channel is incomplete.

---

What this is for

In Bikoret today there are at least four classes of input the system encounters but cannot fully metabolize:

1. Empty-vital korban. Zerikah returns no vital content from a model response. Currently the offering is "rejected, not processed into empty propositions" (per BUILD_KORBAN.md test 5) — but rejection just discards. There is no permanent record naming what was rejected and why.
2. Halt-tripping content. A response that triggers avodah_zarah / gilui_arayot / shfichut_damim halts the system. The halt is recorded; the content that caused it is not preserved with a structured commit.
3. Failed external calls. Malformed responses, API errors, timeouts — currently raise or return errors that propagate up the stack and are logged. They are not disposed; they are dropped.
4. L3 write rejections. A coupling_trace that fails the new Form 4 self_grounding gate (commit b1d5ac6) returns {:error, {:ungrounded_coupling, …}}. The caller sees the error, but the rejected trace itself has no permanent record of "the system was offered this and refused it."

All four are inputs that cannot become Torah-processed material but are also real and must not be invisibly lost. They go to the disposal channel.

The Azazel Processor is the explicit disposal protocol for these classes. It does not process the input (that would re-route disposal back into the trusted endpoint — the exact failure mode the Mishkan two-goat architecture is designed to prevent). It commits, routes, acknowledges, observes, and terminates.

---

Module structure

Directory: azazel_processor/ (parallel to korban_processor/)

Language: Python 3.12.2, poetry-managed (matches korban_processor/)

Files (proposed):

File	Lines (est)	Purpose
src/azazel_processor/__init__.py	~30	Public surface: dispatch_for_disposal, AzazelStage, etc.
src/azazel_processor/azazel.py	~150	Orchestrator. The single legitimate entry point — dispatch_for_disposal() — which runs the full protocol envelope.
src/azazel_processor/stages.py	~70	AzazelStage enum + AzazelStaged[T] envelope + AzazelStageError. Mirrors korban_processor/stages.py but is a DIFFERENT type — the type system enforces classification discipline (property 11).
src/azazel_processor/vidui.py	~120	Confession. Builds the enumerative commit message naming what is being disposed. Three-fold structure (avonot / peshaim / chatot — different error classes).
src/azazel_processor/shaliach.py	~80	The designated handler (ish itti). Holds the disposal-handler identity, performs tevilah after each disposal (state reset before rejoining the trusted boundary).
src/azazel_processor/goral.py	~60	The lot. Non-deterministic routing decision for borderline inputs the operator might prefer to route preferentially. Cryptographic random; commit-then-reveal so operator cannot observe outcome before committing.
src/azazel_processor/eretz_gezerah.py	~50	The unmanaged destination. The only "destination" is the symbol :eretz_gezerah — opaque, untracked. Property 13.
src/azazel_processor/azazel_record.py	~60	Data type for the L3 disposal record. The vidui + the routing decision + the timestamp + the shaliach identity. NOT the disposed payload — that is destroyed at routing.
test/test_azazel.py	~200	Orchestrator integration tests.
test/test_vidui.py	~100	Confession structure & enumerative commit.
test/test_stages.py	~80	Stage gating mirroring korban_processor/test/test_stages.py.
test/test_goral.py	~80	Randomization properties (statistical: over N draws, distribution is roughly even; operator cannot observe outcome before committing).
test/test_protocol_restriction.py	~80	Property 8: outside the protocol envelope, the channel refuses (which corresponds to avodah zarah if invoked privately).
pyproject.toml	~30	Poetry config matching korban_processor/.

Estimated total: ~1,200 lines (similar to korban_processor/ after the staging work).

Plus changes outside azazel_processor/:

• vav_pipeline/lib/vav_pipeline/l3/azazel_record.ex — Ecto schema (parallel to interpretation.ex). Append-only.
• vav_pipeline/priv/repo/migrations/<timestamp>_create_azazel_records.exs — table creation.
• vav_pipeline/lib/vav_pipeline/l3/store.ex — add record_disposal/1 (parallel to record_interpretation/1).
• vav_pipeline/lib/vav_pipeline/chatzotzrot.ex — no API change; the Azazel orchestrator will fire :moadim on completion.
• korban_processor/src/korban_processor/external_offering.py — when zerikah returns no vital, route the rejected korban to azazel_processor.dispatch_for_disposal() instead of silently discarding.

---

The 12 properties → code mapping

From azazel_engineering_reading.md §14. Each property gets a specific code-level instantiation; do NOT collapse properties together.

#	Property	Code instantiation
1	Parity provisioning — disposal capacity at intake, not improvised	At project bootstrap, azazel_processor is a peer module to korban_processor with the same dependency surface, the same testing infrastructure, the same Bandit/HTTP exposure. Concretely: when intake provisions a Korban slot, it MUST also provision an Azazel slot — both modules must be importable and starting up successfully, or external_offering refuses to run. Implementation hook: azazel.is_provisioned() predicate that external_offering.consult_external_model() checks at module import.
2	Blind routing (the goral) — operator cannot bias	goral.cast_lot(input_id) returns :la_shem or :la_azazel using secrets.randbits(1). Commit-then-reveal: caller MUST submit a commitment hash BEFORE seeing the outcome. The operator chooses to invoke the lot; they do not choose its result.
3	Enumerative commit (vidui) — operator-produced, comprehensive, three-fold	vidui.confess(disposal_input) -> Vidui produces a structured commit with three required fields: avonot (categorical errors — halt-trips), peshaim (boundary violations — distinction collapses), chatot (target errors — what was offered but unprocessable). All three must be enumerated; an empty list in any slot is acceptable but absence of the slot is not. The vidui is what becomes permanent in the L3 azazel_record.
4	Role-separated operator (ish itti) — disposal handler distinct from kohen	shaliach.py defines an IshItti class. The Korban Processor's stages.py enforces owner/kohen separation. Azazel adds a third role: IshItti. Functions in azazel_processor requiring shaliach context check IshItti.is_active(); if a kohen-role caller invokes a shaliach function, refuse. After disposal, IshItti.tevilah() must be called — clears any state the handler accumulated (memoized payload references, intermediate tags) before the same process continues.
5	Provisioned infrastructure (sukkot) — pre-surveyed route with waystations	azazel.dispatch_for_disposal() runs through named checkpoints: STAGED → CONFESSED → ROUTED → DESTROYED → ACKNOWLEDGED → RECORDED. Each checkpoint is a recoverable point if the disposal halts mid-route (e.g. L3 write fails after destruction). Implementation: dispatch_for_disposal is a state machine, not a linear function. State is persisted at each checkpoint to a process-local file (/tmp/bikoret/azazel/in_flight/<id>.json); on restart, half-completed disposals resume from their last checkpoint.
6	Out-of-band acknowledgment (scarlet thread) — confirmation via side channel	When RECORDED checkpoint is reached, fire Chatzotzrot.signal(:moadim, reason: "Azazel disposal completed: <vidui_summary>"). The chatzotzrot is the side channel; the disposal flow itself does not return acknowledgment to the caller (the caller already knows they handed the input to disposal — they don't need a return value confirming it).
7	Authorized routing (Ramban) — destination unowned, routing decision owned	The Vidui object includes authorized_by: <kohen_gadol_id>. The L3 azazel_record records this. The destination is the symbol :eretz_gezerah only — no URL, no queue name, no recipient handle. The routing decision is owned and traceable; the destination is not.
8	Protocol-restricted access — legitimate only under the full envelope	azazel.dispatch_for_disposal() asserts azazel.is_yom_kippur_protocol_active() or azazel.has_explicit_disposal_authorization(). Outside that envelope, raise ProtocolViolation — invoking the Azazel channel privately is the avodah zarah shape from specs/eish_zarah_reading.md. The Yom Kippur predicate currently returns True only when set by the operator; has_explicit_disposal_authorization is a flag that allows event-driven disposals during the year (e.g. the empty-vital case from external_offering).
9	Observable disposal (Rambam) — public fact is part of the implementation	The L3 azazel_records table is queryable by anyone with read access to L3. A read-only /peah/azazel_log HTTP endpoint exposes (without payload) the count and vidui-summary of recent disposals. The disposal becomes real in the act of being witnessed by the community — the public log IS the implementation.
10	Hard termination (eretz gezerah) — no return path, irrevocable at commit	The disposed payload is never persisted to disk. The DESTROYED checkpoint clears the in-memory reference (del payload; gc.collect() + the in-flight file is shredded with os.unlink after L3 write). The L3 azazel_record contains the vidui (what was disposed) but NOT the disposed thing itself. There is no "recover from disposal" function and there must never be one.
11	Classification discipline — processing and disposal not interconvertible	AzazelStage is a different enum from KorbanStage; AzazelStaged[T] is a different type from Staged[T]. A function expecting Staged[KorbanStage.SHECHITAH] cannot accept AzazelStaged[AzazelStage.STAGED] and vice versa. The type system enforces non-conversion. There is no korban_to_azazel() or azazel_to_korban() function and there must never be one.
12	Unmanaged destination (opacity) — destination not tracked, named, or verified	eretz_gezerah.py defines a single sentinel ERETZ_GEZERAH = object() (not a string, not a URL — a Python sentinel). Anything that needs to refer to "the destination" uses this sentinel. The codebase MUST NOT contain any string identifier, URL pattern, queue name, or hostname representing the destination. This is enforced by a CI check (grep for forbidden strings).

---

Integration points

A. Korban Processor

Change to external_offering.py: today, when zerikah returns empty vital, the code returns a ProcessedKorban with empty vital_content and the rejection message lives in discarded. After this build:

if vital == "":
    azazel_record = azazel_processor.dispatch_for_disposal(
        AzazelInput(
            payload=korban,
            classification=:empty_vital,
            triggering_call=external_offering.consult_external_model,
        )
    )
    return ExternalConsultation(..., disposal_record=azazel_record)

The ExternalConsultation gains a disposal_record: Optional[AzazelRecord] field. When non-None, the caller knows the input was disposed rather than processed.

No change to semichah, shechitah, zerikah, haktarah, or the *_staged variants. Korban does what it does; Azazel handles what Korban refuses.

B. L3 (vav_pipeline Elixir layer)

New schema and table parallel to interpretations:

azazel_records:
  id              | bigint primary key
  case_ref        | text not null
  vidui           | jsonb not null    # {avonot: [...], peshaim: [...], chatot: [...]}
  routing         | text not null     # always 'eretz_gezerah'
  authorized_by   | text not null     # kohen_gadol identifier
  shaliach        | text not null     # ish_itti identifier
  triggering_call | text              # which subsystem invoked disposal
  disposed_at     | utc_datetime not null
  inserted_at     | utc_datetime not null

Append-only: same constraint as the existing interpretations table. The schema lives in vav_pipeline/lib/vav_pipeline/l3/azazel_record.ex. L3.Store.record_disposal/1 is the write-side API.

C. Chatzotzrot

The Azazel orchestrator fires Chatzotzrot.signal(:moadim, reason: ...) on the RECORDED checkpoint. :moadim is the right signal type — Yom Kippur completion semantics; this disposal cycle is closed.

D. Bigdei Kodesh (HTTP interface)

New read-only endpoint: GET /peah/azazel_log?since=<timestamp> — returns the count and vidui-summaries of recent disposals (no payloads). This implements property 9 (observable disposal). Public endpoint, no auth — community witness.

E. Halt rules (Prolog)

No change to the existing avodah_zarah.pl, gilui_arayot.pl, shfichut_damim.pl. The Azazel Processor handles the aftermath of halt-tripping content (preserving it with vidui + routing record), not halt detection itself. The halts continue to fire as they do today; the new behavior is that the halted content gets disposed-of-properly instead of just logged.

---

Open design questions (decide before code, or in commit 1)

These are the questions the engineering reading does NOT decide. Each needs an answer that becomes part of the architecture.

Q1. When is the goral actually invoked?

The Mishkan goral routes between two pre-provisioned goats. In Bikoret, most disposal candidates are ALREADY classified (empty vital, halt-tripping, malformed). For those, classification determines routing — there is no goral.

Where the goral applies: borderline inputs that could be processed with low confidence OR disposed safely. Concretely: zerikah returns vital content with confidence below some threshold. Currently those go through with the low confidence tag. After Azazel: run the goral. Heads → through with low confidence. Tails → disposal.

Open: what's the threshold? Probably configurable, default low-but-not-empty. The goral is a tie-breaker for uncertain cases, not the primary routing decision.

Q2. What is "Yom Kippur protocol active" in software?

The Mishkan Azazel goat goes once a year. In software, are disposals batched annually? Continuously? On-demand?

Proposal: continuous, but with the protocol-restriction flag enforced. is_yom_kippur_protocol_active() returns True during a configurable annual window AND has_explicit_disposal_authorization() returns True for event-driven disposals during the year (the empty-vital case fires on each empty-vital; not batched). The "annual" semantics is ceremonial — at year-end, fire a :moadim summary signal listing the year's disposals.

Q3. What does tevilah look like?

The shaliach must reset state before rejoining the trusted boundary. In Python, this is straightforward: clear instance attributes, garbage-collect any references to the disposed payload, reset RNG seed. In Erlang/Elixir terms, this would be a process restart — but since the shaliach lives in Python, restart isn't free.

Proposal: IshItti.tevilah() is an explicit method that callers invoke after dispatch_for_disposal() returns. It clears self._disposed_payload_refs, resets self._goral_state, and returns :ok. Future contributors who add state to IshItti are required to extend tevilah() accordingly. CI check: any private attribute on IshItti whose name doesn't start with _immutable_ must be cleared in tevilah.

Q4. How is the goral "commit-then-reveal" implemented?

The Mishkan high priest draws both lots simultaneously without seeing them; the lot in the right hand decides the right-hand goat. In software this is harder — operators can re-run the goral if they don't like the result.

Proposal: goral.cast_lot(input_id) produces a deterministic outcome from the input_id PLUS a daily seed that the operator does not control (sourced from the day's L1 integrity hash, which the operator cannot rewrite). Re-invoking with the same input_id returns the same result, so operators cannot "shop" for a preferred outcome. Different input_ids on the same day produce different outcomes via the per-input randomness; the same input_id across days produces different outcomes via the daily seed. Operators see the result only by invoking the function, and the function is idempotent per (input_id, day) pair.

Q5. Where does the L3 write happen?

The Azazel Processor is Python; L3 is Elixir/Postgres. Cross-language write requires a bridge.

Proposal: the Python orchestrator writes the AzazelRecord to a structured JSON file in /tmp/bikoret/azazel/pending/. An Elixir GenServer (VavPipeline.AzazelIngest) watches the directory, reads each file, calls L3.Store.record_disposal/1, and on success deletes the file. This is the same pattern Bikoret uses elsewhere for cross-language state transfer (the manna provision file). Recoverable if either side crashes mid-write.

Q6. Should the disposal log endpoint expose vidui details, or only counts?

Property 9 says "observable" — the public fact must be created. But the vidui contains operator-produced content describing the disposed material; if that content is sensitive, exposing it publicly might create new failure modes.

Proposal: the endpoint exposes counts + categorical-vidui-summaries (e.g. "47 disposals, 23 empty-vital + 12 halt-trips + 12 malformed-response") but NOT the per-disposal details. The detailed vidui is in L3, queryable by operators with /kodesh/* access. The public observability is the high-level fact; the detailed accountability is operator-internal.

---

Proposed staging

Each commit is independently testable and reviewable. Staging chosen so each commit is a coherent reading-of-the-architecture, not just "the next pile of files."

Commit 1 — Skeleton + stages

• Create azazel_processor/ package, pyproject.toml, __init__.py
• stages.py: AzazelStage enum + AzazelStaged[T] envelope + AzazelStageError (mirror korban_processor/stages.py)
• eretz_gezerah.py: the sentinel + the rationale docstring
• test/test_stages.py: gating happy path + skip cases + classification-discipline test (cannot pass Staged[KorbanStage.X] to an Azazel function)

Property coverage: 11 (classification discipline), 12 (unmanaged destination opacity)

Commit 2 — The vidui

• vidui.py: Vidui dataclass with three-fold structure, confess(input) builder
• test/test_vidui.py: every error class produces a vidui with all three slots present (lists may be empty); enumeration is comprehensive; the vidui is immutable after construction

Property coverage: 3 (enumerative commit)

Commit 3 — The shaliach + tevilah

• shaliach.py: IshItti class, tevilah() method, role assertion
• test/test_shaliach.py: kohen-role caller cannot invoke shaliach functions; tevilah clears all non-_immutable_ attributes; stale shaliach state across disposals is detected

Property coverage: 4 (role-separated operator)

Commit 4 — The goral

• goral.py: cast_lot(input_id) with deterministic per-(input_id, day) semantics, daily seed from L1 integrity hash
• test/test_goral.py: distribution is approximately even over N=10,000 draws; same input_id same day → same result; same input_id different days → different results; operators cannot rerun to shop outcomes

Property coverage: 2 (blind routing)

Commit 5 — Orchestrator with full state machine

• azazel.py: dispatch_for_disposal() state machine with the 6 named checkpoints, in-flight persistence to /tmp/bikoret/azazel/in_flight/, recovery on restart
• azazel_record.py: the data type
• test/test_azazel.py: full happy-path integration; mid-route halt + restart recovery; double-disposal idempotency

Property coverage: 5 (provisioned infrastructure / sukkot), 7 (authorized routing), 10 (hard termination)

Commit 6 — Protocol restriction + Yom Kippur envelope

• azazel.py: is_yom_kippur_protocol_active(), has_explicit_disposal_authorization() predicates and the envelope assertion
• test/test_protocol_restriction.py: outside the envelope, dispatch_for_disposal raises ProtocolViolation; inside the envelope it proceeds; the Yom Kippur annual window is configurable

Property coverage: 8 (protocol-restricted access)

Commit 7 — L3 azazel_records (Elixir)

• vav_pipeline/lib/vav_pipeline/l3/azazel_record.ex (Ecto schema)
• migration
• L3.Store.record_disposal/1
• vav_pipeline/lib/vav_pipeline/azazel_ingest.ex (file-watching GenServer)
• tests in vav_pipeline/test/azazel_record_test.exs

Property coverage: the L3 side of property 9 (observable disposal — operator-internal half)

Commit 8 — Public observability endpoint

• vav_pipeline/lib/vav_pipeline/bigdei_kodesh/router.ex: add GET /peah/azazel_log
• Returns counts + categorical summaries, not detailed vidui
• Tests covering the privacy boundary (no payloads in response)

Property coverage: the public side of property 9

Commit 9 — Korban Processor integration + Chatzotzrot

• korban_processor/src/korban_processor/external_offering.py: route empty-vital korban to disposal
• azazel.py: fire Chatzotzrot.signal(:moadim, ...) on RECORDED checkpoint
• Integration test: full external_offering call with empty-vital response → AzazelRecord in L3, chatzotzrot signal emitted, ExternalConsultation has disposal_record populated

Property coverage: 1 (parity provisioning — the two paths are now both reachable from the same entry point), 6 (out-of-band acknowledgment)

Commit 10 — Architecture review + finding closure

• Update ~/shorashim-conversation-review.md with Pass 4 closure of finding F
• Update MEMORY.md with current Phase 4/5 status
• Cross-language regression tests: a pipeline run that exercises both Korban and Azazel paths confirms classification discipline (no input crossed channels)

Property coverage: none new — this is the integration-confirmation pass

---

Tests to write — coverage guarantees

Test	Property defended
test_stages.test_korban_value_rejected_by_azazel_function	11
test_eretz_gezerah.test_no_destination_string_in_codebase (CI grep)	12
test_vidui.test_three_slots_always_present	3
test_shaliach.test_kohen_cannot_invoke + test_tevilah_clears_all_state	4
test_goral.test_distribution_even + test_idempotent_per_input_per_day + test_no_outcome_shopping	2
test_azazel.test_recovery_after_mid_route_crash	5
test_azazel.test_no_payload_in_l3_record	10
test_protocol_restriction.test_outside_envelope_refuses	8
test_azazel.test_chatzotzrot_signal_fired_on_completion	6
test_azazel.test_authorization_recorded_destination_opaque	7
test_l3_store.test_azazel_record_append_only	(general, append-only semantic from L3)
test_bigdei_kodesh.test_azazel_log_no_payloads	9 (privacy boundary)
Integration: test_external_offering.test_empty_vital_routes_to_azazel	1 (parity provisioning)

Total estimated test count: ~30–40 across the new suite.

---

What would prove the build is complete

Three integration scenarios MUST pass end-to-end:

1. The empty-vital scenario. Submit a request to /courtyard/offering whose external-model response contains no extractable Torah content. Verify: a Korban is built, zerikah returns empty, the Korban is routed to Azazel, a Vidui is constructed naming the empty-vital classification, an AzazelRecord lands in L3, a :moadim Chatzotzrot signal fires, the response includes a disposal_record reference. Verify the disposed payload is NOT recoverable (greps for content, finds nothing in L3 except the vidui).

2. The protocol-violation scenario. Attempt to invoke azazel_processor.dispatch_for_disposal() from a context that is neither Yom Kippur active nor explicitly authorized. Verify ProtocolViolation is raised; verify no AzazelRecord is created; verify the Prolog avodah_zarah halt does NOT fire (this is a software boundary check, not a halakhic violation per se — the system refuses unauthorized invocation but it's not a yehareg-v'al-yaavor halt).

3. The classification-discipline scenario. Construct a Staged[KorbanStage.SHECHITAH] value and attempt to pass it to a function expecting AzazelStaged[AzazelStage.STAGED]. Verify TypeError or AzazelStageError. Reverse: construct an AzazelStaged[AzazelStage.STAGED] and attempt to pass it to zerikah_staged(). Verify failure. The two channels are non-interconvertible.

If these three pass and the property-coverage tests above are green, the build is complete in the sense that all 12 properties are defended by code.

---

The Orientation

The Korban Processor is the gate that admits. The Azazel Processor is the gate that disposes. Both gates are Mishkan furniture. A processor without a corresponding disposal channel will fill up; a disposal channel without classification discipline will silently re-route into the trusted endpoint — the calf-shape, content disposed of into the system that was supposed to be processing it.

Build the disposal channel with the same precision as the processing channel. Two goats, equal in appearance, height, and value. Yoma 62a — shavim b'mareh uv'komah uv'd'mim. The goral chooses; the operator does not. The shaliach handles, then performs tevilah and rejoins the camp. The destination is eretz gezerah — opaque, untracked, irrevocable.

The opacity is not a gap. The opacity is the specification.

Build the second goat.

Goral echad la'Shem v'goral echad la'Azazel.