Files
Scott KarpandClaude Opus 4.7 e6fee82dd7 azazel_processor: commit 4 — goral (property 2, blind routing)
Pass 4 / Finding F, commit 4 of 10 from BUILD_AZAZEL.md.

Vayikra 16:8 — וְנָתַן אַהֲרֹן עַל־שְׁנֵי הַשְּׂעִירִם גֹּרָלוֹת,
גּוֹרָל אֶחָד לַה׳ וְגוֹרָל אֶחָד לַעֲזָאזֵל.
And Aharon shall place lots on the two goats — one lot for HaShem,
one lot for Azazel.

Yoma 62a: a kalpi (shuffled vessel), drawn with both hands
simultaneously. The high priest cannot see the lots in advance.
He cannot shape the outcome.

THREAT MODEL:

An operator who, given the choice, would route preferentially —
assigning the "better" goat to processing and the "worse" to
disposal. The specification makes that preference structurally
impossible.

DESIGN (BUILD_AZAZEL.md §6, Q4):

cast_lot(input_id) is DETERMINISTIC per (input_id, today). The
seed is SHA-256 over (L1 integrity hash, today's UTC date,
input_id). The first bit of the digest is the outcome.

Properties guaranteed:
  - Same (input_id, today) → SAME outcome (no shopping by re-roll)
  - Different input_ids same day → outcomes can differ (~50/50)
  - Same input_id across days → outcomes can differ (~50/50)
  - Operators cannot pre-compute without invoking (SHA-256 over
    L1 hash + date is not mentally computable)
  - Seed inherits L1 tamper-resistance: rewriting the daily seed
    requires rewriting L1, which has its own halt rules

The L1 hash mirrored from vav_pipeline/lib/vav_pipeline/l1_reader.ex
(@l1_expected_hash). Both must move together if L1 is rebuilt.

CREATED:

  src/azazel_processor/goral.py          ~155 lines
    GoralOutcome enum                    LA_SHEM | LA_AZAZEL
                                         (both legitimate; the goral
                                         enforces blindness, not a
                                         "good vs. bad" distinction)
    cast_lot(input_id, *, today=None)    deterministic per (id, day)
                                         outcome
    cast_lot_outcome_id(input_id,        full SHA-256 hex digest;
                          *, today=None) used by L3 records (commit 7)
                                         so audit can recompute
    GoralError                           on malformed input

  test/test_goral.py                     17 tests covering:
    - outcome shape (one of two; enum members distinct)
    - same input same day → same outcome (anti-shopping; verified
      across 50 re-invocations + 1000-iteration audit-shopping test)
    - outcome_id is reproducible 64-char hex
    - 100 different input_ids → 100 distinct outcome_ids
    - same input across 30 days → 30 distinct outcome_ids
    - distribution across 10,000 input_ids on one day → ~50/50
      (±5% tolerance, well within SHA-256 binomial expectations)
    - distribution across 365 days for one input → ~50/50 (±10%)
    - input validation (empty, whitespace, None, non-string raise)
    - default today = UTC date
    - audit can recompute outcome_id from (input_id, today) — the
      audit-trail half of property 2

NOTE: when the orchestrator (commit 5) decides WHEN to invoke the
goral. For inputs that are cleanly processable or cleanly
un-processable, the goral is NOT invoked — they go to their channel
deterministically. The goral is the tie-breaker for genuinely
borderline inputs (Q1 in BUILD_AZAZEL.md §6).

Tests: azazel_processor 88 passed (was 71, +17). korban_processor
unchanged at 203 passed, 1 skipped. No regressions.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 07:54:26 -04:00

213 lines
7.3 KiB
Python

"""
Tests for goral — property 2 (blind routing) of
azazel_engineering_reading.md §14.
The goral protects against operator preference. Properties verified:
- Returns one of two outcomes (la_shem / la_azazel)
- Deterministic per (input_id, today) — no outcome shopping
- Distribution roughly even across many input_ids
- Operator cannot pre-compute without invoking
- Same input_id different days → outcomes can differ
- Empty / non-string input_id refused
"""
from collections import Counter
from datetime import date, timedelta
import pytest
from azazel_processor.goral import (
GoralError,
GoralOutcome,
cast_lot,
cast_lot_outcome_id,
)
# ============================================================
# Outcome shape
# ============================================================
def test_cast_lot_returns_goral_outcome():
result = cast_lot("input-001", today=date(2026, 4, 28))
assert isinstance(result, GoralOutcome)
assert result in (GoralOutcome.LA_SHEM, GoralOutcome.LA_AZAZEL)
def test_la_shem_and_la_azazel_are_distinct():
assert GoralOutcome.LA_SHEM != GoralOutcome.LA_AZAZEL
assert {GoralOutcome.LA_SHEM, GoralOutcome.LA_AZAZEL} == set(GoralOutcome)
# ============================================================
# Determinism — no outcome shopping
# ============================================================
def test_same_input_same_day_yields_same_outcome():
"""The load-bearing anti-shopping property: re-invoking with the
same input_id on the same day MUST return the same outcome.
Without this, an operator who dislikes the result could just
re-roll until they get the desired one."""
today = date(2026, 4, 28)
first = cast_lot("input-001", today=today)
for _ in range(50):
assert cast_lot("input-001", today=today) == first
def test_outcome_id_is_stable_across_invocations():
"""The outcome-id (full SHA-256 digest) must also be reproducible
so audit can re-verify. Property 2's audit-trail half."""
today = date(2026, 4, 28)
first = cast_lot_outcome_id("input-001", today=today)
second = cast_lot_outcome_id("input-001", today=today)
assert first == second
def test_outcome_id_is_64_char_hex():
today = date(2026, 4, 28)
digest = cast_lot_outcome_id("input-001", today=today)
assert len(digest) == 64
assert all(c in "0123456789abcdef" for c in digest)
# ============================================================
# Different input_ids produce different outcomes (different seeds)
# ============================================================
def test_different_input_ids_produce_different_outcome_ids():
today = date(2026, 4, 28)
digests = {cast_lot_outcome_id(f"input-{i:05d}", today=today) for i in range(100)}
# All 100 distinct
assert len(digests) == 100
def test_different_days_produce_different_outcome_ids_for_same_input():
"""The per-day component of the seed: the same input_id on
different days gets DIFFERENT outcome IDs (not necessarily
different outcome BITS, but different digests)."""
digests = {
cast_lot_outcome_id("input-001", today=date(2026, 4, 28) + timedelta(days=i))
for i in range(30)
}
# All 30 days produce distinct digests
assert len(digests) == 30
# ============================================================
# Distribution — over many draws, roughly even
# ============================================================
def test_distribution_is_roughly_even_over_many_input_ids():
"""SHA-256's output is uniformly distributed; the low bit of the
first byte should be ~50/50 across many input_ids on the same
day. Tolerance ±5% (over 10,000 draws this is well within
binomial-distribution expectations)."""
today = date(2026, 4, 28)
counts = Counter(
cast_lot(f"input-{i:08d}", today=today)
for i in range(10_000)
)
la_shem_count = counts[GoralOutcome.LA_SHEM]
la_azazel_count = counts[GoralOutcome.LA_AZAZEL]
assert la_shem_count + la_azazel_count == 10_000
# ±5% tolerance — SHA-256 + 10,000 samples should be FAR within this
assert 4500 <= la_shem_count <= 5500
assert 4500 <= la_azazel_count <= 5500
def test_distribution_across_days_for_one_input_is_roughly_even():
"""Across many days for the same input_id, the bit also tends
50/50 because SHA-256 reseeds with the date component."""
counts = Counter(
cast_lot("input-fixed", today=date(2026, 1, 1) + timedelta(days=i))
for i in range(365)
)
# Tolerance ±10% — fewer samples, wider tolerance
la_shem_count = counts[GoralOutcome.LA_SHEM]
la_azazel_count = counts[GoralOutcome.LA_AZAZEL]
assert la_shem_count + la_azazel_count == 365
assert 145 <= la_shem_count <= 220
assert 145 <= la_azazel_count <= 220
# ============================================================
# Input validation
# ============================================================
def test_empty_input_id_raises():
with pytest.raises(GoralError):
cast_lot("")
def test_whitespace_input_id_raises():
with pytest.raises(GoralError):
cast_lot(" ")
def test_none_input_id_raises():
with pytest.raises(GoralError):
cast_lot(None) # type: ignore[arg-type]
def test_non_string_input_id_raises():
with pytest.raises(GoralError):
cast_lot(42) # type: ignore[arg-type]
def test_outcome_id_validates_input_id_too():
with pytest.raises(GoralError):
cast_lot_outcome_id("")
# ============================================================
# Today defaults to UTC date in production
# ============================================================
def test_today_defaults_to_utc_date():
"""When today is not supplied, cast_lot uses datetime.now(UTC).date().
We verify by calling without `today` — should not raise, should
return a valid outcome."""
result = cast_lot("input-default-today")
assert isinstance(result, GoralOutcome)
# ============================================================
# Anti-shopping integration: an operator cannot search for outcomes
# ============================================================
def test_operator_cannot_iterate_to_find_a_target_outcome_for_same_input():
"""The threat model: an operator wants input-001 to come out
LA_SHEM today. They re-invoke cast_lot many times hoping for a
different result. They cannot — the outcome is fixed by the
seed."""
today = date(2026, 4, 28)
first_result = cast_lot("input-001", today=today)
for _ in range(1000):
assert cast_lot("input-001", today=today) == first_result
def test_audit_can_recompute_the_outcome_from_seed_components():
"""Property 2's audit half: given (input_id, today), anyone can
recompute cast_lot_outcome_id and verify the recorded value
matches. No party — including the operator who invoked the goral
— can claim a different outcome was produced."""
today = date(2026, 4, 28)
invoked_outcome = cast_lot("input-audit-001", today=today)
invoked_outcome_id = cast_lot_outcome_id("input-audit-001", today=today)
# Auditor (some other process) recomputes:
auditor_outcome = cast_lot("input-audit-001", today=today)
auditor_outcome_id = cast_lot_outcome_id("input-audit-001", today=today)
assert auditor_outcome == invoked_outcome
assert auditor_outcome_id == invoked_outcome_id