From d7520cd69d396bbb065b80d387c5125c2c0a0a6b Mon Sep 17 00:00:00 2001 From: Scott Karp Date: Sun, 4 Oct 2026 19:26:26 +0000 Subject: [PATCH] =?UTF-8?q?specs=20=C2=A72=20+=20images=20+=20Doorpost:=20?= =?UTF-8?q?the=20technology=20table=20says=20what=20is=20built;=20L1=20is?= =?UTF-8?q?=20read-only=20in=20fact?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Architecture §2 described a self-hosted server: Phoenix for L2, PostgreSQL 16, Ubuntu 24.04 under systemd, rootless Podman isolating Sanhedrin panels, Erlang distribution between them, systemd timers for Shabbat and the cycle, L1 on a read-only mount. None of that is what runs. Each row was checked against the code and the images before it was rewritten. §2.1 exact versions (no "9.6+"); Korban and Azazel Processors listed. §2.2 L1: in every image, root-owned, no write bit; recorded hash compared before every cycle. L2: a process in the pipeline, not yet over HTTP and not yet called by the cycle. L3: PostgreSQL 17 managed, append-only by REVOKE. §2.3 Kubernetes, one pod. OTP supervisors + the Deployment's probes. Sanhedrin panels are Erlang processes in the pipeline's BEAM — no container boundary. Message passing within one node; a port to the engine; a Unix socket to the Talmudic Engine. Logging: stdout and L3; halt events in memory. ShabbatTimer process, UTC stub. No cycle timer: erev/voker is a stage of each cycle. The manna volume. A DEFECT FOUND BY CHECKING THE L1 ROW, AND FIXED: The spec said L1 is a read-only filesystem. In all four images the scroll was owned by the service user, mode 664 — `touch` inside /mnt/torah succeeded. The process that reads Torah could have rewritten it. All four Dockerfiles now copy L1 as root and strip every write bit (dirs 555, files 444). Verified in each image as the service user: cannot create, append, or delete. The hash is unchanged (it covers paths and contents, not modes). A DEFECT FOUND AND NOT FIXED, now written into the spec as it is: L1Reader.verify_integrity/0 reads the hash stored in integrity.sha256 and compares it with the compiled-in constant. It never hashes the verse files. `l1_hash: pass` witnesses which L1 was built into the image, not that the files are unaltered. §2.2 says so in plain words. deploy/ARCHITECTURE.md: L1 is in the images, not on a volume; the manna volume is 1 GiB and holds the ledger too; l1-volume.yaml is vestigial. Doorpost @expected_hash 1cd07079… → 7e5286b9…, in this commit, with the witnessed file that moved it (architecture). Read before committing: Elixir 474/0 (seeds 0, 42). Four images rebuilt; full local stack (real engine binary, PostgreSQL with SSL, migrations): Hebrew 200, English 422, health 7/7, 2 interpretations in L3, no errors; talmudic and korban containers start. Left as written: §1.4 hardware (a 64–128 GB server; the node has 4 GB), §2.4 external models (Steps 21–22, unbuilt), §4's build steps ("Install PostgreSQL 16"), terumah §B (systemd, squashfs, Caddy, nftables). Co-Authored-By: Claude Fable 5.1 --- deploy/ARCHITECTURE.md | 8 ++-- deploy/docker/Dockerfile.korban_processor | 10 ++++- deploy/docker/Dockerfile.shoresh_engine | 10 ++++- deploy/docker/Dockerfile.talmudic_engine | 10 ++++- deploy/docker/Dockerfile.vav_pipeline | 10 ++++- specs/shorashim_architecture.md | 47 +++++++++++++++-------- vav_pipeline/lib/vav_pipeline/doorpost.ex | 2 +- 7 files changed, 71 insertions(+), 26 deletions(-) diff --git a/deploy/ARCHITECTURE.md b/deploy/ARCHITECTURE.md index 0c116044..737710f0 100644 --- a/deploy/ARCHITECTURE.md +++ b/deploy/ARCHITECTURE.md @@ -20,8 +20,8 @@ All four run in the same pod. vav-pipeline is the main container; talmudic-engin | Dependency | Purpose | Contract | |-----------|---------|----------| | Managed PostgreSQL 17+ | L3 Oral Tradition (append-only) | `DATABASE_URL` env var. Requires SSL. Role with `SELECT, INSERT` only — `UPDATE`, `DELETE` revoked (see migration). | -| Block-storage PersistentVolume | L1 Torah filesystem (read-only, ~1 GiB) | Mount read-only at `/mnt/torah`. SHA-256 hash verified at boot. | -| Block-storage PersistentVolume | Manna provision (read-write, ~100 MiB) | Mount at `/var/lib/shorashim/manna`. Operator writes `today.json` daily. | +| (none — in the images) | L1 Torah filesystem (~29 MB) | Baked into every image at `/mnt/torah`, owned by root with no write bit. No volume. The recorded SHA-256 is compared with the pinned hash before every cycle. | +| Block-storage PersistentVolume | Manna provision and its ledger (read-write, 1 GiB) | Mount at `/var/lib/shorashim/manna`. Operator writes `today.json` daily; the vessel writes `ledger.json`. | | Ingress controller + TLS cert | Public HTTPS access | Cert-manager with Let's Encrypt HTTP-01 is the reference setup. | | Container registry | Image distribution | Any registry that supports Docker v2 protocol. | @@ -32,7 +32,7 @@ Kept in `deploy/k8s/`: - `namespace.yaml` — the namespace - `configmap.yaml` — non-secret configuration - `secrets.yaml` — **template only**; operator fills in `DATABASE_URL` -- `l1-volume.yaml` — PV and PVC for Torah filesystem +- `l1-volume.yaml` — PV and PVC for Torah filesystem (vestigial: L1 is in the images; do not apply) - `manna-volume.yaml` — PVC for provision file - `deployment.yaml` — the pod with 3 containers + 2 init containers (`shoresh-engine-install`, `migrate`). Four images are still built and pushed; shoresh-engine ships a binary, not a running container. - `service.yaml` — ClusterIP and external LoadBalancer @@ -54,7 +54,7 @@ Four Dockerfiles in `deploy/docker/`: The system reads three things from its environment: -1. **L1 Torah** — mounted read-only at `$L1_MOUNT_POINT`. Integrity verified at every boot. +1. **L1 Torah** — in the image at `$L1_MOUNT_POINT`, not writable by the service user. Its recorded hash is checked before every cycle. 2. **Manna provision** — JSON file at `$MANNA_PROVISION_FILE`, written by the kohen/treasurer. 3. **L3 database** — connection string in `$DATABASE_URL`. diff --git a/deploy/docker/Dockerfile.korban_processor b/deploy/docker/Dockerfile.korban_processor index 5acab360..3f4c737f 100644 --- a/deploy/docker/Dockerfile.korban_processor +++ b/deploy/docker/Dockerfile.korban_processor @@ -58,7 +58,15 @@ COPY --from=build /usr/local/lib/python3.12/site-packages/ /usr/local/lib/python ENV PYTHONPATH=/home/shorashim/korban_processor/src # L1 Torah filesystem — baked into the image -COPY --chown=shorashim:shorashim l1/ /mnt/torah/ +# L1 is read-only in fact, not only in name. Until 2026-10-04 the scroll +# was copied in owned by the service user with its write bits on: the +# process that reads Torah could also have rewritten it, and only the +# SHA-256 check would have noticed afterwards. Owned by root, no write +# bit for anyone; the service user can read and traverse, nothing more. +USER root +COPY --chown=root:root l1/ /mnt/torah/ +RUN chmod -R a-w,a+rX /mnt/torah +USER shorashim ENV L1_MOUNT_POINT=/mnt/torah diff --git a/deploy/docker/Dockerfile.shoresh_engine b/deploy/docker/Dockerfile.shoresh_engine index 71696657..1dcae8ef 100644 --- a/deploy/docker/Dockerfile.shoresh_engine +++ b/deploy/docker/Dockerfile.shoresh_engine @@ -66,7 +66,15 @@ WORKDIR /home/shorashim COPY --from=build /usr/local/bin/shoresh-engine ./shoresh-engine # L1 Torah filesystem — baked into the image -COPY --chown=shorashim:shorashim l1/ /mnt/torah/ +# L1 is read-only in fact, not only in name. Until 2026-10-04 the scroll +# was copied in owned by the service user with its write bits on: the +# process that reads Torah could also have rewritten it, and only the +# SHA-256 check would have noticed afterwards. Owned by root, no write +# bit for anyone; the service user can read and traverse, nothing more. +USER root +COPY --chown=root:root l1/ /mnt/torah/ +RUN chmod -R a-w,a+rX /mnt/torah +USER shorashim # L1 mount point ENV L1_MOUNT_POINT=/mnt/torah diff --git a/deploy/docker/Dockerfile.talmudic_engine b/deploy/docker/Dockerfile.talmudic_engine index fc06dab0..eb030822 100644 --- a/deploy/docker/Dockerfile.talmudic_engine +++ b/deploy/docker/Dockerfile.talmudic_engine @@ -29,7 +29,15 @@ WORKDIR /home/shorashim/talmudic_engine COPY --from=build --chown=shorashim:shorashim /app/ ./ # L1 Torah filesystem — baked into the image -COPY --chown=shorashim:shorashim l1/ /mnt/torah/ +# L1 is read-only in fact, not only in name. Until 2026-10-04 the scroll +# was copied in owned by the service user with its write bits on: the +# process that reads Torah could also have rewritten it, and only the +# SHA-256 check would have noticed afterwards. Owned by root, no write +# bit for anyone; the service user can read and traverse, nothing more. +USER root +COPY --chown=root:root l1/ /mnt/torah/ +RUN chmod -R a-w,a+rX /mnt/torah +USER shorashim # L1 mount point ENV L1_MOUNT_POINT=/mnt/torah diff --git a/deploy/docker/Dockerfile.vav_pipeline b/deploy/docker/Dockerfile.vav_pipeline index 94758821..23174988 100644 --- a/deploy/docker/Dockerfile.vav_pipeline +++ b/deploy/docker/Dockerfile.vav_pipeline @@ -84,7 +84,15 @@ COPY --from=build --chown=shorashim:shorashim \ # L1 Torah filesystem — baked into the image (read-only, ~29MB) # The scroll is sealed in the aron. It ships with the vessel. -COPY --chown=shorashim:shorashim l1/ /mnt/torah/ +# L1 is read-only in fact, not only in name. Until 2026-10-04 the scroll +# was copied in owned by the service user with its write bits on: the +# process that reads Torah could also have rewritten it, and only the +# SHA-256 check would have noticed afterwards. Owned by root, no write +# bit for anyone; the service user can read and traverse, nothing more. +USER root +COPY --chown=root:root l1/ /mnt/torah/ +RUN chmod -R a-w,a+rX /mnt/torah +USER shorashim # === INSCRIBE THE DOORPOST === # Devarim 11:20 — ukhtavtam al mezuzot beitecha uvish'arecha. A running diff --git a/specs/shorashim_architecture.md b/specs/shorashim_architecture.md index 08f5ae63..e1e593fe 100644 --- a/specs/shorashim_architecture.md +++ b/specs/shorashim_architecture.md @@ -61,35 +61,48 @@ Swiss or Icelandic provider, outside US legal jurisdiction. Candidates previousl ### 2.1 Primary Languages +Exact versions are fixed in Construction §1.1; these are the ones built. + | Component | Language | Version | Purpose | |-----------|---------|---------|---------| -| Root system & binyanim | Haskell | GHC 9.6+ | Type-safe symbolic processing | -| Sanhedrin panels | Erlang/OTP | OTP 26+ | Concurrent fault-tolerant deliberation | -| Pipeline orchestration | Elixir | 1.16+ | Vav-consecutive processing chains | -| Talmudic reasoning | SWI-Prolog | 9.0+ | Logic programming, backtracking search | -| Kushya generator | Python 3.12+ | — | Interface to fine-tuned Mistral model | -| System glue / CLI | Bash + Python | — | Build scripts, deployment, monitoring | +| Root system & binyanim (Shoresh Engine) | Haskell | GHC 9.6.4 | Type-safe symbolic processing | +| Sanhedrin panels | Erlang/OTP | OTP 26.2.1 | Concurrent fault-tolerant deliberation | +| Pipeline orchestration (Vav Pipeline) | Elixir | 1.16.1 | Vav-consecutive processing chains; the HTTP API | +| Talmudic reasoning (Talmudic Engine) | SWI-Prolog | 9.2.2 | Logic programming, backtracking search | +| Korban Processor | Python | 3.12.2 | The offering protocol for external model output | +| Azazel Processor | Python | 3.12.2 | The disposal channel; lives in the Korban Processor's image and process | +| Kushya generator | Python | 3.12.2 | Interface to fine-tuned Mistral model (scaffolding until Step 21) | +| System glue / CLI | Bash + Python | — | Build scripts (`scripts/`), L1 construction, image builds | ### 2.2 Data Storage | Layer | Technology | Purpose | |-------|-----------|---------| -| L0 — Luchot | Compiled Haskell constants | Immutable firmware baked into binary | -| L1 — Torah scroll | Read-only filesystem (squashfs or `mount -o ro`) | Fixed specification | -| L2 — Urim v'Tumim | REST API (Elixir/Phoenix) | Decision oracle interface | -| L3 — Oral tradition | PostgreSQL 16 (append-only policy) | Accumulated learning | +| L0 — Luchot | Compiled Haskell constants (`Luchot.Firmware`) | Immutable firmware baked into binary | +| L1 — Torah scroll | Baked into every image at `/mnt/torah`, owned by root with no write bit; its recorded SHA-256 is compared with the pinned hash before every cycle | Fixed specification | +| L2 — Urim v'Tumim | A process in the Vav Pipeline (`UrimVTumim`). Not yet exposed over HTTP and not yet called by the cycle (Step 24) | Decision oracle interface | +| L3 — Oral tradition | PostgreSQL 17, managed; append-only by `REVOKE UPDATE, DELETE` from the application role | Accumulated learning | + +L1 is read-only by ownership, not by mount: the service user can read the +scroll and cannot change it. It is not a squashfs image and the +container's root filesystem is not mounted read-only. The integrity check +compares the hash recorded in `integrity.sha256` with the hash compiled +into the pipeline; it does not re-hash the verse files, so it witnesses +which L1 was built into the image, not that the files are unaltered since. ### 2.3 Infrastructure | Component | Technology | Purpose | |-----------|-----------|---------| -| OS | Ubuntu 24.04 LTS | Base system | -| Process supervision | systemd | Service management, cycle timers | -| Container isolation | Podman (rootless) | Sanhedrin panel isolation | -| Message passing | Erlang distribution protocol | Inter-panel communication | -| Logging | Append-only log files + PostgreSQL | Audit trail (minority opinions) | -| Shabbat timer | systemd calendar timer | Weekly mandatory cessation | -| Cycle timer | systemd timer | Erev/voker processing boundaries | +| Platform | Kubernetes (managed), one pod: the pipeline, the Talmudic Engine and the Korban Processor as containers, the Shoresh Engine as a binary the pipeline runs | Base system | +| OS | Ubuntu 22.04 (pipeline, engine images), Debian 12 (korban, talmudic images); Ubuntu 24.04 LTS on the node | Base system | +| Process supervision | OTP supervisors inside the BEAM; the Kubernetes Deployment outside it (liveness `/kodesh/live`, readiness `/kodesh/health`) | Service management | +| Sanhedrin panel isolation | Erlang processes — one per judge, supervised per panel — in the pipeline's BEAM. No container boundary between panels | Independent deliberation | +| Message passing | Erlang message passing within one node (no distribution); JSON lines over a port to the Shoresh Engine; JSON over a Unix socket to the Talmudic Engine | Inter-panel and inter-component communication | +| Logging | Container stdout; PostgreSQL (L3) for interpretations, minority opinions, teiku and disposals. Halt events are held in the halt monitor's memory | Audit trail (minority opinions) | +| Shabbat timer | A process in the pipeline (`ShabbatTimer`) checking every 60 s; Friday 18:00 to Saturday 18:00 UTC until zmanim are built | Weekly mandatory cessation | +| Cycle boundaries | Erev/voker is a stage of each cycle; the day boundary is enforced by the manna ledger when a cycle runs. There is no timer | Erev/voker processing boundaries | +| Provision | A 1 GiB volume holding the day's manna and its ledger | Daily manna, kept through restarts | ### 2.4 External Model (Korban Source) diff --git a/vav_pipeline/lib/vav_pipeline/doorpost.ex b/vav_pipeline/lib/vav_pipeline/doorpost.ex index 4b23a5a1..cc7b0427 100644 --- a/vav_pipeline/lib/vav_pipeline/doorpost.ex +++ b/vav_pipeline/lib/vav_pipeline/doorpost.ex @@ -175,7 +175,7 @@ defmodule VavPipeline.Doorpost do # changes; the update is a deliberate operator commit. A bare # hash-update commit (without a corresponding spec change in the # same commit) is itself a drift signal. - @expected_hash "1cd070792fc101c5a526680cf8a8116d586151b969a1ba2cf3924e5f02d3cc40" + @expected_hash "7e5286b99aaefa49c7516d3e27dae9d7e3f4fc9ae89347c92c5394685d953400" @doc "Return the list of files that constitute the category-grade specification." @spec files() :: [String.t()]