f493b7df6502f55a4ef7ff783cae9b88bc5b8577
2
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Scott KarpandClaude Opus 4.7
|
2932fb7447 |
vav_pipeline: commit 7 — L3 azazel_records + cross-language ingest GenServer
Pass 4 / Finding F, commit 7 of 10 from BUILD_AZAZEL.md. The
operator-internal half of property 9 (observable disposal). Crosses
the Python/Elixir boundary: the Python orchestrator drops completed
disposal records as JSON files; this commit's GenServer reads them
and persists to L3.
The public half (a /peah/azazel_log read endpoint) lands in commit 8;
the routing-from-Korban + Chatzotzrot wiring lands in commit 9. After
those, the Azazel Processor is end-to-end functional.
CREATED:
vav_pipeline/priv/repo/migrations/20260428000001_create_azazel_records.exs
azazel_records table — append-only, parallel to interpretations.
Columns: case_ref, vidui (jsonb), routing, authorized_by,
shaliach, triggering_call, goral_outcome_id, disposed_at.
Indexes on case_ref, disposed_at, shaliach.
Property-12-defending CHECK constraint:
routing_must_be_eretz_gezerah CHECK (routing = 'eretz_gezerah').
A future migration that wants to add a destination handle will
fail this check; the operator must explicitly justify removing
the constraint, which is the right friction.
Append-only enforcement matches interpretations: REVOKE
UPDATE/DELETE on the shorashim_app role.
vav_pipeline/lib/vav_pipeline/l3/azazel_record.ex
Ecto schema mirroring the Python AzazelRecord.to_storage_dict()
shape. Changeset validates:
- all required fields present
- routing == "eretz_gezerah" (mirrors the DB CHECK at app layer)
- vidui is a map with avonot/peshaim/chatot/confessor/
confessed_at slots present
- vidui's three categories not all empty (mirrors the Python
Vidui.__post_init__ EmptyConfessionError)
vav_pipeline/lib/vav_pipeline/azazel_ingest.ex ~190 lines
GenServer that polls a configurable pending-l3 directory
(default /tmp/shorashim/azazel/pending_l3). For each *.json file:
1. Read + JSON-parse
2. Translate Python serialization → Ecto attrs
3. Call recorder (default L3.Store.record_disposal/1)
4. On success, shred the source file (property-10 follow-through)
5. On failure, leave file in place for retry; log warning
Recorder is injectable for tests — mock recorder lets us verify
the pipeline without PostgreSQL.
Poll interval default 5s, configurable. Polling rather than
file_system events to keep the dependency surface minimal; the
Azazel cadence is exceptional content, not high-throughput.
Not added to the supervision tree in this commit — the producer
side (Python orchestrator dropping files) is wired in commit 9.
Adding to the supervisor before there's a producer means a
poller doing nothing useful. Operator can start manually for
early testing: VavPipeline.AzazelIngest.start_link(...).
CHANGED:
vav_pipeline/lib/vav_pipeline/l3/store.ex
+ record_disposal/1
Inserts an AzazelRecord. Returns {:ok, record} or
{:error, changeset}.
+ count_disposals/1
Total or since-a-timestamp count. Used by the public
observability endpoint (commit 8).
+ disposal_summary/1
Returns categorical info per disposal — case_ref, shaliach,
triggering_call, disposed_at, categories_filled (which of
avonot/peshaim/chatot had any items, NOT the items
themselves). PROPERTY-9 PRIVACY BOUNDARY at the public
side: items are operator-internal only, accessed via
future /kodesh/* endpoints.
TESTS:
vav_pipeline/test/azazel_ingest_test.exs 15 tests, no DB
Happy path:
- poll_now ingests valid file and shreds it
- multiple files processed in sorted order
- vidui preserved as map (jsonb shape)
Retry semantics:
- recorder error → file left in place + state.errors++
- malformed JSON → file left in place
- missing disposed_at → file left in place
State accounting:
- polls_completed, records_ingested, errors all tracked
- state survives multiple polls
- non-json files ignored
Changeset (no DB):
- valid attrs
- rejects routing != eretz_gezerah
- rejects vidui missing slots
- rejects vidui all-empty (untagged disposal)
- rejects missing required fields
- accepts optional triggering_call + goral_outcome_id
vav_pipeline/test/l3_store_test.exs +9 :l3-tagged
record_disposal:
- inserts valid disposal
- rejects routing != eretz_gezerah
- rejects all-empty vidui
- rejects missing required fields
Counts:
- count_disposals == 0 initially
- count_disposals after several disposals
- count_disposals(since) filters by timestamp
Summary:
- disposal_summary returns categorical info but NOT items
(privacy boundary verification)
- categories_filled accurately reflects vidui shape
Tests: vav_pipeline 353 passed, 0 failures, 33 excluded (was
329/24; +15 visible azazel_ingest + 9 :l3-tagged disposal store).
korban_processor + azazel_processor unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Scott KarpandClaude Opus 4.6
|
d34e2799ac |
L3 Oral Tradition: PostgreSQL persistence layer for interpretations, minority opinions, teiku
Devarim 17:9 — v'darashta v'higidu lekha et d'var hamishpat. The inquiry requires that what is found is RECORDED. - Ecto Repo with graceful degradation (maybe_add_repo, like other engines) - Migration: interpretations, minority_opinions, teiku_registry tables - Append-only enforcement via REVOKE UPDATE/DELETE - Includes eish_zarah fields: coupling_trace, grounding_type, challenge_sequence - Ecto schemas: Interpretation, MinorityOpinion, TeikuRecord - Changeset validations: status inclusion, panel_size > 0, grounding_type enum - TeikuRecord.resolve_changeset restricts updates to resolved_at/resolution only - L3.Store bridge module: record, query, count operations - L3 tests tagged @moduletag :l3, excluded by default (no local PostgreSQL) - Enable with: SHORASHIM_L3=1 mix test --include l3 - All 76 existing Elixir tests pass, 195 Prolog tests pass, 33 Haskell tests pass Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |