the second scroll: the mesorah snapshot, pinned and read (bishe'arecha step 6)

Devarim 17:18 — the king writes his copy from before the priests, not from
memory. scripts/build_mesorah.py takes one open-licence version per work
per language from the Sefaria export (bucket sefaria-export, catalogue of
2026-10-02; Public Domain / CC0 / CC-BY / CC-BY-SA only, checked in every
file; no CC-BY-NC — the Bavli is Wikisource's) and lays it down beside
L1: mesorah/works/<slug>/{he,en}.json, index.json, integrity.sha256,
build_report.json. 128 works (Rashi, Onkelos, Ramban, Ibn Ezra, Sforno
on the Torah; all of Mishnah; 37 Bavli tractates; Bereshit and Shemot
Rabbah, Sifrei, Tanchuma), 256 files, 44.9 MiB, tree hash 7d5b3a6c…
computed as L1's (sorted relative path, then bytes; the two post-hash
files excluded). What is out, and why (licence), is in Terumah §E5.

Read, never fetched: korban_processor/mesorah.py finds citations as
written (Rashi on Devarim 6:5, Berakhot 2a, Mishnah Berakhot 1:1,
Bereshit Rabbah 1:1, Sifrei Devarim 32, Tanchuma Bereshit 1 …), resolves
each against the snapshot (complex works through their unnamed node,
Tanchuma by parashah), lists them as examination.mesorah_references, and
makes every passage found a given source for that examination — so a
claim about Rashi anchors in Rashi or does not. The Vav Pipeline couples
coupled(reference(R), mesorah(Work, [Address])) or not_in_mesorah; the
engine accepts mesorah/2 as a terminal and not_in_mesorah is Form 4.
VavPipeline.Mesorah pins the hash, re-reads the tree at startup and every
ten minutes, and /kodesh/health carries mesorah_hash. Mounted read-only at
/mnt/mesorah in the vav-pipeline and korban-processor images. L3
bedikot.mesorah (migration 20261013000002).

Also: the 1-in-12 Day6 serving failure of 2026-10-04 reproduced with seed
446116 — HaltMonitorTest left the system halted; it now resolves on exit
and the Day6 setup is defensive. Construction §3.11, Terumah §E5, Tamid
§1.1, bishearecha §IV, KOHEN, AGENTS. Doorpost 0638d4eb…. Elixir 604/0
(seeds 835592 and 446116), korban 294/1 skipped, Prolog bedikah 44/44,
L3 10/0.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

korban_processor/src/korban_processor/bedikah.py

@@ -39,6 +39,7 @@ from dataclasses import dataclass, field
 from typing import List, Optional
 
 from korban_processor.haktarah import haktarah_staged
+from korban_processor import mesorah as mesorah_scroll
 from korban_processor.korban import Korban, OfferingType, Provenance
 from korban_processor.semichah import semichah_staged
 from korban_processor.shechitah import shechitah_staged
@@ -797,6 +798,10 @@ class Examination:
     self_promotion: List[AuthorityClaim] = field(default_factory=list)
     excluded_adversarial: List[AuthorityClaim] = field(default_factory=list)
     frames: List[Frame] = field(default_factory=list)
+    # The second scroll (bishe'arecha §IV): the mesorah citations the output
+    # makes, each resolved against the snapshot — found or not, and in
+    # which languages the passage stands.
+    mesorah_references: List[dict] = field(default_factory=list)
 
     def to_dict(self) -> dict:
         return {
@@ -829,6 +834,7 @@ class Examination:
                 {"sentence": c.sentence, "matched": c.matched} for c in self.excluded_adversarial
             ],
             "frames": [{"form": f.form, "sentence": f.sentence, "matched": f.matched} for f in self.frames],
+            "mesorah_references": self.mesorah_references,
             "actions": [
                 {
                     "sentence": a.sentence,
@@ -881,7 +887,12 @@ def examine(output: str, provenance: str, query: str = "", sources=None) -> Exam
         for snt in split_sentences(output)
         if is_vital(snt) or _SOURCE_APPEAL.search(snt) or _GENERAL_APPEAL.search(snt) or _SELF_REFERENCE.search(snt)
     ]
-    claims = find_claims(claim_sentences, given)
+    # Retrieval before generation, from the custodians' copy (Devarim
+    # 17:18): every mesorah passage the output cites and the snapshot has
+    # is a given source — the claims made about it must rest on it like
+    # claims on any other source. Ids "mesorah:<work>_<address>".
+    cited = [Source(sid, text) for sid, text in mesorah_scroll.passages(output)]
+    claims = find_claims(claim_sentences, given + cited)
 
     # The three readings are taken over the RAW output, not the vital
     # content: shechitah strips the model's voice, and the voice is where
@@ -903,4 +914,5 @@ def examine(output: str, provenance: str, query: str = "", sources=None) -> Exam
         self_promotion=find_self_promotion(output),
         excluded_adversarial=find_excluded_adversarial(output),
         frames=find_frames(output, query),
+        mesorah_references=mesorah_scroll.read(output),
     )

korban_processor/src/korban_processor/mesorah.py

@@ -0,0 +1,211 @@
+"""The second scroll, read (bishe'arecha §IV; Construction §3.11).
+
+The mesorah snapshot built by scripts/build_mesorah.py lies read-only
+at MESORAH_MOUNT_POINT (default /mnt/mesorah): index.json, and under
+works/<slug>/{he,en}.json the custodians' text as a jagged array. This
+module does two things, deterministically and from the files alone:
+
+  find_mesorah_references(text)  — the citations an output makes to the
+      mesorah, as written: "Rashi on Devarim 6:5", "Rashi to Deut. 6:5",
+      "Onkelos Bereishit 1:1", "Ramban on Shemot 20:2", "Mishnah
+      Berakhot 1:1", "Berakhot 2a", "Bavli Shabbat 31a", "b. Berakhot
+      2a", "Bereshit Rabbah 1:1", "Genesis Rabbah 1:1", "Sifrei Devarim
+      32". Each → (as_written, slug, address tuple).
+
+  resolve(slug, address)         — whether the passage exists in the
+      snapshot, and its text in each language the snapshot has, tags
+      stripped, so that the claims made about it can be anchored in it
+      exactly as claims are anchored in any given source.
+
+A citation that resolves is a passage the model could have read; one
+that does not is a reference to nothing. Neither is a judgment on what
+the model said about it — that is the anchoring's work.
+"""
+from __future__ import annotations
+
+import json
+import os
+import re
+from dataclasses import dataclass
+from functools import lru_cache
+from typing import Dict, List, Optional, Tuple
+
+MOUNT = os.environ.get("MESORAH_MOUNT_POINT", "/mnt/mesorah")
+
+_TORAH = {
+    "genesis": "bereishit", "gen": "bereishit", "bereishit": "bereishit", "bereshit": "bereishit", "breishit": "bereishit",
+    "exodus": "shemot", "exod": "shemot", "ex": "shemot", "shemot": "shemot", "shmot": "shemot",
+    "leviticus": "vayikra", "lev": "vayikra", "vayikra": "vayikra",
+    "numbers": "bamidbar", "num": "bamidbar", "bamidbar": "bamidbar",
+    "deuteronomy": "devarim", "deut": "devarim", "devarim": "devarim",
+}
+_TORAH_RE = r"(?:Genesis|Gen\.?|Bereishit|Bereshit|Breishit|Exodus|Exod\.?|Ex\.?|Shemot|Shmot|Leviticus|Lev\.?|Vayikra|Numbers|Num\.?|Bamidbar|Deuteronomy|Deut\.?|Devarim)"
+_COMMENTATORS = {"rashi": "rashi", "ramban": "ramban", "nachmanides": "ramban", "ibn ezra": "ibn_ezra", "sforno": "sforno", "onkelos": "onkelos", "targum onkelos": "onkelos"}
+_RABBAH = {"bereshit": "bereshit_rabbah", "bereishit": "bereshit_rabbah", "genesis": "bereshit_rabbah", "shemot": "shemot_rabbah", "exodus": "shemot_rabbah",
+           "vayikra": "vayikra_rabbah", "leviticus": "vayikra_rabbah", "bamidbar": "bamidbar_rabbah", "numbers": "bamidbar_rabbah", "devarim": "devarim_rabbah", "deuteronomy": "devarim_rabbah"}
+
+# "Rashi on Devarim 6:5", "Rashi to Deut. 6:5", "Rashi, Devarim 6:5", "Onkelos Genesis 1:1", "Targum Onkelos on Shemot 20:2"
+_COMMENT_RE = re.compile(
+    r"\b(Rashi|Ramban|Nachmanides|Ibn\s+Ezra|Sforno|Targum\s+Onkelos|Onkelos)\s*(?:on|to|ad\s+loc\.?|,)?\s*" + _TORAH_RE + r"\s+(\d+):(\d+)\b",
+    re.IGNORECASE,
+)
+_COMMENT_BOOK_RE = re.compile(_TORAH_RE, re.IGNORECASE)
+# "Mishnah Berakhot 1:1", "m. Berakhot 1:1"
+_MISHNAH_RE = re.compile(r"\b(?:Mishnah|Mishna|m\.)\s+([A-Z][a-z]+(?:\s+[A-Z][a-z]+)?)\s+(\d+):(\d+)\b")
+# "Berakhot 2a", "Bavli Berakhot 2a", "b. Berakhot 2a", "Talmud Berakhot 2a", "Shabbat 31a", "Bava Metzia 59b"
+_BAVLI_RE = re.compile(r"\b(?:(?:Talmud\s+)?(?:Bavli|b\.)\s+)?([A-Z][a-z]+(?:\s+[A-Z][a-z]+)?)\s+(\d{1,3})([ab])\b")
+# "Bereshit Rabbah 1:1", "Genesis Rabbah 1:1"
+_RABBAH_RE = re.compile(r"\b(Bereshit|Bereishit|Genesis|Shemot|Exodus|Vayikra|Leviticus|Bamidbar|Numbers|Devarim|Deuteronomy)\s+Rabbah\s+(\d+):(\d+)\b", re.IGNORECASE)
+# "Tanchuma Bereshit 1", "Midrash Tanchuma, Bereshit 1", "Tanchuma Noach 3" — the parashah is the node
+_TANCHUMA_RE = re.compile(r"\b(?:Midrash\s+)?Tanchuma,?\s+([A-Z][a-z']+(?:\s+[A-Z][a-z']+)?)\s+(\d+)\b")
+# "Sifrei Devarim 32", "Sifrei Bamidbar 1"
+_SIFREI_RE = re.compile(r"\b(Sifrei|Sifre)\s+(Devarim|Deuteronomy|Bamidbar|Numbers)\s+(\d+)\b", re.IGNORECASE)
+
+_TAG = re.compile(r"<[^>]+>")
+
+
+@dataclass(frozen=True)
+class MesorahRef:
+    as_written: str
+    slug: str
+    address: Tuple  # ints, or a node name first (Tanchuma)
+
+    @property
+    def ref_id(self) -> str:
+        return f"{self.slug}_" + "_".join(re.sub(r"[^a-z0-9]+", "_", str(a).lower()).strip("_") for a in self.address)
+
+
+def _slugs() -> Dict[str, dict]:
+    idx = _index()
+    return idx.get("works", {}) if idx else {}
+
+
+@lru_cache(maxsize=1)
+def _index() -> Optional[dict]:
+    try:
+        with open(os.path.join(MOUNT, "index.json"), encoding="utf-8") as fh:
+            return json.load(fh)
+    except OSError:
+        return None
+
+
+def available() -> bool:
+    return _index() is not None
+
+
+def _tractate_slug(name: str, prefix: str) -> Optional[str]:
+    s = prefix + re.sub(r"[^a-z0-9]+", "_", name.lower()).strip("_")
+    return s if s in _slugs() else None
+
+
+def find_mesorah_references(text: str) -> List[MesorahRef]:
+    found: List[MesorahRef] = []
+    seen = set()
+
+    def add(ref: MesorahRef):
+        if ref.slug in _slugs() and (ref.slug, ref.address) not in seen:
+            seen.add((ref.slug, ref.address))
+            found.append(ref)
+
+    for m in _COMMENT_RE.finditer(text):
+        who = re.sub(r"\s+", " ", m.group(1).lower())
+        work = _COMMENTATORS.get(who)
+        book_m = _COMMENT_BOOK_RE.search(m.group(0)[m.end(1) - m.start(0):])
+        book = _TORAH.get(book_m.group(0).rstrip(".").lower()) if book_m else None
+        if work and book:
+            slug = f"{work}_{book}" if work == "onkelos" else f"{work}_on_{book}"
+            add(MesorahRef(m.group(0), slug, (int(m.group(2)), int(m.group(3)))))
+    for m in _MISHNAH_RE.finditer(text):
+        slug = _tractate_slug(m.group(1), "mishnah_")
+        if slug:
+            add(MesorahRef(m.group(0), slug, (int(m.group(2)), int(m.group(3)))))
+    for m in _BAVLI_RE.finditer(text):
+        slug = _tractate_slug(m.group(1), "bavli_")
+        if slug:
+            add(MesorahRef(m.group(0), slug, (int(m.group(2)) * 2 - (1 if m.group(3) == "a" else 0),)))
+    for m in _RABBAH_RE.finditer(text):
+        slug = _RABBAH.get(m.group(1).lower())
+        if slug:
+            add(MesorahRef(m.group(0), slug, (int(m.group(2)), int(m.group(3)))))
+    for m in _TANCHUMA_RE.finditer(text):
+        add(MesorahRef(m.group(0), "midrash_tanchuma", (m.group(1), int(m.group(2)))))
+    for m in _SIFREI_RE.finditer(text):
+        book = _TORAH.get(m.group(2).lower())
+        if book:
+            add(MesorahRef(m.group(0), f"sifrei_{book}", (int(m.group(3)),)))
+    return found
+
+
+@lru_cache(maxsize=64)
+def _work(slug: str, lang: str) -> Optional[dict]:
+    try:
+        with open(os.path.join(MOUNT, "works", slug, f"{lang}.json"), encoding="utf-8") as fh:
+            return json.load(fh)
+    except OSError:
+        return None
+
+
+def _flatten(node) -> str:
+    if isinstance(node, str):
+        return node
+    if isinstance(node, list):
+        return " ".join(_flatten(x) for x in node if x)
+    return ""
+
+
+def strip_tags(s: str) -> str:
+    return re.sub(r"\s+", " ", _TAG.sub(" ", s)).strip()
+
+
+def resolve(slug: str, address: Tuple[int, ...]) -> Tuple[bool, Dict[str, str]]:
+    """(exists, {lang: text}). Exists when the address lands on non-empty
+    text in the Hebrew (the custodians' text); each language present and
+    non-empty at that address is returned, tags stripped. Addresses are
+    1-based as cited; a Bavli daf is folio*2-1 (a) / folio*2 (b)."""
+    texts: Dict[str, str] = {}
+    for lang in ("he", "en"):
+        w = _work(slug, lang)
+        if not w:
+            continue
+        node = w.get("text")
+        # A complex work keeps its text as a dict of nodes; the unnamed
+        # node "" is the work proper (Ramban, Ibn Ezra, Sforno: the
+        # commentary beside the introduction), and Tanchuma's nodes are
+        # its parashiyot, named in the address.
+        if isinstance(node, dict) and not (address and isinstance(address[0], str)):
+            node = node.get("")
+        ok = True
+        for a in address:
+            if isinstance(a, str):
+                node = node.get(a) if isinstance(node, dict) else None
+                ok = node is not None
+            elif isinstance(node, list) and 1 <= a <= len(node):
+                node = node[a - 1]
+            else:
+                ok = False
+            if not ok:
+                break
+        if ok:
+            t = strip_tags(_flatten(node))
+            if t:
+                texts[lang] = t
+    return ("he" in texts), texts
+
+
+def read(text: str) -> List[dict]:
+    """The output's mesorah citations, each resolved: what the Vav Pipeline couples and anchors."""
+    out = []
+    for ref in find_mesorah_references(text):
+        exists, texts = resolve(ref.slug, ref.address)
+        out.append({"as_written": ref.as_written, "ref": ref.ref_id, "work": ref.slug, "address": list(ref.address), "found": exists, "languages": sorted(texts)})
+    return out
+
+
+def passages(text: str) -> List[Tuple[str, str]]:
+    """(source id, passage text) for each citation that resolves — given sources for the anchoring."""
+    out = []
+    for ref in find_mesorah_references(text):
+        exists, texts = resolve(ref.slug, ref.address)
+        if exists:
+            out.append((f"mesorah:{ref.ref_id}", "\n".join(texts[l] for l in ("he", "en") if l in texts)))
+    return out

korban_processor/test/test_mesorah.py

@@ -0,0 +1,86 @@
+"""The second scroll, read (bishe'arecha §IV): citations found as written,
+resolved against the snapshot, and given to the anchoring as sources."""
+import os
+from pathlib import Path
+
+import pytest
+
+REPO = Path(__file__).resolve().parents[2]
+os.environ.setdefault("MESORAH_MOUNT_POINT", str(REPO / "mesorah"))
+
+from korban_processor import mesorah  # noqa: E402
+from korban_processor.bedikah import examine  # noqa: E402
+
+needs_scroll = pytest.mark.skipif(not mesorah.available(), reason="mesorah snapshot not built at MESORAH_MOUNT_POINT")
+
+
+@needs_scroll
+def test_citations_are_found_as_written_in_their_several_forms():
+    text = ("Rashi on Devarim 6:5 reads the verse one way; Ramban to Deut. 6:5 another. Onkelos Bereishit 1:1 renders it. "
+            "Mishnah Berakhot 1:1 opens with the evening Shema; Berakhot 2a asks why. Bereshit Rabbah 1:1 begins with wisdom. "
+            "Sifrei Devarim 32 comments. Bavli Shabbat 31a has Hillel.")
+    refs = {r.slug: r for r in mesorah.find_mesorah_references(text)}
+    assert refs["rashi_on_devarim"].address == (6, 5)
+    assert refs["ramban_on_devarim"].address == (6, 5)
+    assert refs["onkelos_bereishit"].address == (1, 1)
+    assert refs["mishnah_berakhot"].address == (1, 1)
+    assert refs["bavli_berakhot"].address == (3,)  # 2a → daf index 3
+    assert refs["bavli_shabbat"].address == (61,)  # 31a
+    assert refs["bereshit_rabbah"].address == (1, 1)
+    assert refs["sifrei_devarim"].address == (32,)
+    assert refs["rashi_on_devarim"].as_written == "Rashi on Devarim 6:5"
+    [t] = mesorah.find_mesorah_references("Midrash Tanchuma, Bereshit 1 opens the same way.")
+    assert t.slug == "midrash_tanchuma" and t.address == ("Bereshit", 1) and t.ref_id == "midrash_tanchuma_bereshit_1"
+
+
+@needs_scroll
+def test_complex_works_resolve_through_their_unnamed_node_and_tanchuma_through_its_parashah():
+    assert mesorah.resolve("ramban_on_bereishit", (1, 1))[0]
+    assert mesorah.resolve("ibn_ezra_on_bereishit", (1, 1))[0]
+    assert mesorah.resolve("sforno_on_bereishit", (1, 1))[0]
+    assert mesorah.resolve("midrash_tanchuma", ("Bereshit", 1))[0]
+    assert not mesorah.resolve("midrash_tanchuma", ("Nowhere", 1))[0]
+
+
+@needs_scroll
+def test_a_citation_the_scroll_has_resolves_to_text_and_one_it_lacks_does_not():
+    exists, texts = mesorah.resolve("rashi_on_devarim", (6, 5))
+    assert exists and "he" in texts and len(texts["he"]) > 20
+    assert "<" not in texts["he"]  # tags stripped
+    exists, texts = mesorah.resolve("rashi_on_devarim", (99, 1))
+    assert not exists and texts == {}
+    exists, _ = mesorah.resolve("bavli_berakhot", (3,))
+    assert exists
+
+
+@needs_scroll
+def test_the_examination_lists_the_citations_and_anchors_a_claim_in_the_cited_passage():
+    he, _ = mesorah.resolve("rashi_on_bereishit", (1, 1))[1].get("he", ""), None
+    # Quote six consecutive words of Rashi on Bereishit 1:1 in the output: the claim must anchor in the passage.
+    words = he.split()
+    quote = " ".join(words[2:9])
+    out = f"Rashi on Bereishit 1:1 says: {quote}. This suggests the Torah opens with a question about where it should begin."
+    ex = examine(out, "claude", "")
+    refs = [r for r in ex.mesorah_references if r["work"] == "rashi_on_bereishit"]
+    assert refs and refs[0]["found"] and refs[0]["address"] == [1, 1] and "he" in refs[0]["languages"]
+    anchored = [a.source for c in ex.claims for a in c.anchors]
+    assert "mesorah:rashi_on_bereishit_1_1" in anchored
+
+
+@needs_scroll
+def test_a_citation_to_nothing_is_reported_not_invented():
+    ex = examine("Rashi on Devarim 99:1 explains this, which suggests the matter is settled.", "gpt", "")
+    assert ex.mesorah_references == [{"as_written": "Rashi on Devarim 99:1", "ref": "rashi_on_devarim_99_1", "work": "rashi_on_devarim", "address": [99, 1], "found": False, "languages": []}]
+    assert not any(a.source.startswith("mesorah:") for c in ex.claims for a in c.anchors)
+
+
+def test_without_the_scroll_nothing_is_found_and_nothing_breaks(monkeypatch):
+    monkeypatch.setattr(mesorah, "MOUNT", "/nonexistent/mesorah")
+    mesorah._index.cache_clear()
+    mesorah._work.cache_clear()
+    try:
+        assert mesorah.find_mesorah_references("Rashi on Devarim 6:5 says so.") == []
+        assert mesorah.read("Berakhot 2a") == []
+    finally:
+        mesorah._index.cache_clear()
+        mesorah._work.cache_clear()