From e17436c4ca2bd90f36b3c90e5367a9bdcb8eb80f Mon Sep 17 00:00:00 2001 From: Scott Karp Date: Tue, 6 Oct 2026 17:59:23 +0000 Subject: [PATCH] One gate to the world: Shorashim and Gitea behind the shared ingress MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three NLBs became one (185.19.28.170, ingress-nginx). Shorashim: `shorashim-external` (NLB 185.19.29.180, NodePort 30923) is replaced by an Ingress — a host-less catch-all (works by IP, no DNS) plus a `shorashim.ch` rule ready for the A record; TLS deliberately not requested until the name resolves. Sanhedrin-length proxy timeouts. The watch behind the ingress (completes tablet 46): the TCP peer is now the ingress controller, so Mishmar.peer/1 takes the caller from X-Real-IP — only when the peer is inside TRUSTED_PROXIES (the pod network, configmap); from anyone else the header is the caller's own claim and is ignored. The ingress Service was already externalTrafficPolicy: Local with its health-check port open, so the caller's address arrives without the security-group change the direct route needed. Verified live: the 11th knock is 429, a forged X-Real-IP is still 429, and in the same minute the old route (pod sees the node) was a different subject. Gitea: no name of its resolves, so it rides the controller's TCP passthrough on the same NLB — :3000 → gitea:80, :22 → gitea:22 (ingress-tcp-services.yaml), on the two NodePorts the security group already allowed. gitea-mcp now reaches Gitea over the cluster network instead of out through an NLB and back. Tamid §1.1 note, KOHEN.md, AGENTS.md. Doorpost e4e3d56d… → a2ec3842…. Elixir 539/0 (L3 on and off). Co-Authored-By: Claude Fable 5.1 --- AGENTS.md | 3 +- KOHEN.md | 6 +- deploy/k8s/configmap.yaml | 6 ++ deploy/k8s/gitea-mcp-deployment.yaml | 11 ++- deploy/k8s/gitea-mcp-secret.yaml | 4 +- deploy/k8s/ingress-tcp-services.yaml | 38 +++++++++ deploy/k8s/service.yaml | 82 ++++++++++++------- specs/shorashim_tamid.md | 6 +- vav_pipeline/config/config.exs | 5 +- vav_pipeline/config/runtime.exs | 6 ++ .../lib/vav_pipeline/bigdei_kodesh/mishmar.ex | 57 ++++++++++++- vav_pipeline/lib/vav_pipeline/doorpost.ex | 2 +- vav_pipeline/test/tablet46_mishmar_test.exs | 65 +++++++++++++++ 13 files changed, 247 insertions(+), 44 deletions(-) create mode 100644 deploy/k8s/ingress-tcp-services.yaml diff --git a/AGENTS.md b/AGENTS.md index 1a9bc692..4b28808d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -103,7 +103,8 @@ approve decisions, or supervise work. The specification does all of this. | Service | Location | Access | |---------|----------|--------| -| Gitea | `http://185.19.29.94` | User: heaviness510 | +| Gitea | `http://185.19.28.170:3000` (SSH :22) — behind the shared ingress NLB | User: heaviness510 | +| Shorashim | `http://185.19.28.170` — behind the same ingress (catch-all; `shorashim.ch` once DNS points there) | keys: KOHEN.md "Issuing a key" | | PostgreSQL | Exoscale DBaaS ch-gva-2 | `temple-council-db` | | Kubernetes | Exoscale SKS ch-gva-2 | `~/.kube/exoscale-temple-council.yaml` | diff --git a/KOHEN.md b/KOHEN.md index 7dfe3d6c..bd25cd5f 100644 --- a/KOHEN.md +++ b/KOHEN.md @@ -73,7 +73,7 @@ You inherit infrastructure that has been provisioned and verified: | Item | Where | |------|-------| -| Repo | `git clone http://185.19.29.94/heaviness510/shorashim.git` | +| Repo | `git clone http://185.19.28.170:3000/heaviness510/shorashim.git` | | Kubeconfig | `~/.kube/exoscale-temple-council.yaml` | | `kubectl` | already installed | | `exo` CLI | already installed | @@ -463,7 +463,7 @@ kubectl logs -n shorashim deploy/shorashim -c vav-pipeline -f If the seven days passed without halt events, without Shabbat violations, without manna exhaustion, without any pod restarts: -1. **Point DNS** — `shorashim.ch` A record to the LoadBalancer external IP +1. **Point DNS** — `shorashim.ch` A record to the ingress IP (`kubectl get svc -n ingress-nginx ingress-nginx-controller`; `185.19.28.170`). Then add the TLS block noted in `deploy/k8s/service.yaml` — until then `` is that IP over plain HTTP 2. **Open the pe'ah endpoint to the world** — the stranger approaches first 3. **Observe the first real interaction** — a question from outside 4. If *ki tov*: open courtyard and chatzer endpoints. The system is consecrated. @@ -583,7 +583,7 @@ Three things are kept outside the mishkan repository, each for a Torah-grounded | Repo / directory | Where | Why | |---|---|---| -| `shorashim-readings` | `http://185.19.29.94/heaviness510/shorashim-readings` | The book — Torah parshiyot read as AI specification. Companion to the mishkan. Lives beside the aron, not inside (Devarim 31:26). | +| `shorashim-readings` | `http://185.19.28.170:3000/heaviness510/shorashim-readings` | The book — Torah parshiyot read as AI specification. Companion to the mishkan. Lives beside the aron, not inside (Devarim 31:26). | | `~/shorashim-source-data/morphhb` | local only | The quarry — raw WLC source. The carved tablets go in the aron; the quarry does not. | | `~/shorashim-operator-notes` | local only / private | The kohen's runbook — environment-specific details, hostnames, account names. Private operator notes. Not part of the system's specification. | diff --git a/deploy/k8s/configmap.yaml b/deploy/k8s/configmap.yaml index a3b7884e..e779e522 100644 --- a/deploy/k8s/configmap.yaml +++ b/deploy/k8s/configmap.yaml @@ -25,6 +25,12 @@ data: # keys never are. API_KEYS_FILE: "/etc/shorashim/secrets/api_keys" + # The pod network. The ingress controller lives in it; only from here + # is X-Real-IP believed as the caller's address (the pe'ah's per-IP + # watch, Tamid §1.1). Nothing outside the cluster can reach the pod + # directly any more (service.yaml), so nothing outside can forge it. + TRUSTED_PROXIES: "192.168.0.0/16" + # Shoresh Engine binary — staged onto the shared shoresh-bin volume # by the shoresh-engine-install init container (see deployment.yaml). # vav_pipeline spawns it as an Erlang Port, i.e. as a child process of diff --git a/deploy/k8s/gitea-mcp-deployment.yaml b/deploy/k8s/gitea-mcp-deployment.yaml index e214632a..92b7cc85 100644 --- a/deploy/k8s/gitea-mcp-deployment.yaml +++ b/deploy/k8s/gitea-mcp-deployment.yaml @@ -3,8 +3,11 @@ # Deploys the official Gitea MCP server (gitea.com/gitea/gitea-mcp) # in HTTP mode, exposing /mcp endpoint for Streamable HTTP transport. # -# The MCP server connects to the Gitea backend at 185.19.29.94 -# using a Personal Access Token stored in gitea-mcp-secrets. +# The MCP server connects to the Gitea backend over the cluster network +# (gitea.temple-council.svc.cluster.local) using a Personal Access Token +# stored in gitea-mcp-secrets. Until 2026-10-06 it went out to Gitea's +# own NLB (185.19.29.94) and back in; that NLB is gone — Gitea is behind +# the shared ingress now (ingress-tcp-services.yaml). --- apiVersion: apps/v1 kind: Deployment @@ -31,7 +34,7 @@ spec: - "-t" - "http" - "--host" - - "http://185.19.29.94" + - "http://gitea.temple-council.svc.cluster.local" - "--port" - "8080" env: @@ -41,7 +44,7 @@ spec: name: gitea-mcp-secrets key: GITEA_TOKEN - name: GITEA_HOST - value: "http://185.19.29.94" + value: "http://gitea.temple-council.svc.cluster.local" ports: - containerPort: 8080 name: http diff --git a/deploy/k8s/gitea-mcp-secret.yaml b/deploy/k8s/gitea-mcp-secret.yaml index dd4b4f05..0f5d3876 100644 --- a/deploy/k8s/gitea-mcp-secret.yaml +++ b/deploy/k8s/gitea-mcp-secret.yaml @@ -3,11 +3,11 @@ # # Two secrets needed: # 1. GITEA_TOKEN: Personal Access Token from Gitea -# Create at: http://185.19.29.94/user/settings/applications +# Create at: http://185.19.28.170:3000/user/settings/applications # Permissions: repo (read/write) # # 2. OAuth Client ID/Secret: Registered OAuth app in Gitea -# Create at: http://185.19.29.94/user/settings/applications +# Create at: http://185.19.28.170:3000/user/settings/applications # Redirect URI: https://claude.ai/api/mcp/auth_callback # (These go into claude.ai connector settings, not this secret) --- diff --git a/deploy/k8s/ingress-tcp-services.yaml b/deploy/k8s/ingress-tcp-services.yaml new file mode 100644 index 00000000..d38dd852 --- /dev/null +++ b/deploy/k8s/ingress-tcp-services.yaml @@ -0,0 +1,38 @@ +# Gitea behind the shared ingress — TCP passthrough +# +# One NLB for the cluster (185.19.28.170, the ingress-nginx Service) +# instead of one per Service. Shorashim is an HTTP Ingress +# (service.yaml). Gitea has no hostname that resolves +# (gitea.temple-council.ch is configured in Gitea and exists nowhere), +# and host-based routing needs a name — so Gitea rides the ingress +# controller's TCP passthrough on two ports of the same NLB: +# +# 185.19.28.170:3000 → temple-council/gitea:80 (HTTP — git, web UI) +# 185.19.28.170:22 → temple-council/gitea:22 (SSH) +# +# Three live objects carry this; only the first is a whole manifest: +# +# 1. This ConfigMap. +# 2. The controller reads it — one argument on the Deployment: +# kubectl -n ingress-nginx patch deploy ingress-nginx-controller --type=json -p \ +# '[{"op":"add","path":"/spec/template/spec/containers/0/args/-", +# "value":"--tcp-services-configmap=$(POD_NAMESPACE)/tcp-services"}]' +# 3. The controller's Service exposes the two ports ON THE NODEPORTS THE +# SECURITY GROUP ALREADY ALLOWS (they were Gitea's own; the CCM never +# opens a NodePort — tablets 29, 46): +# kubectl -n ingress-nginx patch svc ingress-nginx-controller --type=json -p \ +# '[{"op":"add","path":"/spec/ports/-","value":{"name":"gitea-http","port":3000,"targetPort":3000,"protocol":"TCP","nodePort":30575}}, +# {"op":"add","path":"/spec/ports/-","value":{"name":"gitea-ssh","port":22,"targetPort":22,"protocol":"TCP","nodePort":31345}}]' +# and temple-council/gitea is type ClusterIP (it held 30575/31345 as a +# LoadBalancer until 2026-10-06). +# +# When Gitea has a real name: add an HTTP Ingress for it (TLS via +# cert-manager) and drop the 3000 line here; SSH stays. +apiVersion: v1 +kind: ConfigMap +metadata: + name: tcp-services + namespace: ingress-nginx +data: + "3000": "temple-council/gitea:80" + "22": "temple-council/gitea:22" diff --git a/deploy/k8s/service.yaml b/deploy/k8s/service.yaml index a68e6b56..c51ad299 100644 --- a/deploy/k8s/service.yaml +++ b/deploy/k8s/service.yaml @@ -24,38 +24,64 @@ spec: protocol: TCP name: http --- -# External access via LoadBalancer -# For Exoscale SKS, this provisions an Exoscale NLB. +# ============================================================ +# The gate to the world — through the shared ingress +# ============================================================ # -# The NLB forwards to this Service's NodePort and TCP-healthchecks it on -# the node. The cloud controller does NOT open that port in the node's -# security group — the operator must, or the NLB IP refuses connections -# while the pod is Ready (2026-10-04, tablet 29): -# PORT=$(kubectl -n shorashim get svc shorashim-external -o jsonpath='{.spec.ports[0].nodePort}') -# exo compute security-group rule add default --flow ingress --protocol tcp --port $PORT --network 0.0.0.0/0 -apiVersion: v1 -kind: Service +# Until 2026-10-06 Shorashim had its own LoadBalancer Service +# (`shorashim-external`, NLB 185.19.29.180, NodePort 30923). It is gone: +# one NLB per Service was a third of the account's bill, and the pod saw +# every caller as the node (kube-proxy SNAT) so the pe'ah's per-IP watch +# could not tell callers apart. The ingress-nginx Service is already +# externalTrafficPolicy: Local with its health-check port open, so the +# controller sees the caller and hands it on in X-Real-IP; the pipeline +# believes that header only from TRUSTED_PROXIES (configmap.yaml). +# +# Two rules, one backend: +# - no host → the catch-all: http:///... works with no DNS +# - shorashim.ch → ready for the A record (KOHEN.md Day 8). TLS is NOT +# requested here: asking cert-manager for a certificate before the +# name resolves only burns Let's Encrypt's failed-validation limit. +# When the record exists add: +# metadata.annotations: +# cert-manager.io/cluster-issuer: letsencrypt-prod +# nginx.ingress.kubernetes.io/ssl-redirect: "true" +# spec.tls: [{hosts: [shorashim.ch], secretName: shorashim-tls}] +# and point callers at https://shorashim.ch — until then the +# courtyard keys travel over plain HTTP. +apiVersion: networking.k8s.io/v1 +kind: Ingress metadata: - name: shorashim-external + name: shorashim namespace: shorashim labels: app.kubernetes.io/name: shorashim annotations: - # Exoscale NLB annotations - service.beta.kubernetes.io/exoscale-loadbalancer-zone: "ch-gva-2" - service.beta.kubernetes.io/exoscale-loadbalancer-name: "shorashim" + # A 71-judge Sanhedrin may deliberate for ten minutes (Construction §2.2). + nginx.ingress.kubernetes.io/proxy-read-timeout: "620" + nginx.ingress.kubernetes.io/proxy-send-timeout: "620" + # The bedikah accepts 65,536 bytes of output; leave room for the envelope. + nginx.ingress.kubernetes.io/proxy-body-size: "256k" + nginx.ingress.kubernetes.io/ssl-redirect: "false" spec: - type: LoadBalancer - # Local: kube-proxy does not SNAT, so the pod sees the caller's own - # address and the pe'ah's per-IP watch (Tamid §1.1) counts callers, - # not the node. With Cluster (the default) every caller arrived as the - # node IP 91.92.153.113 — observed 2026-10-06, tablet 46. One node, - # one replica: Local loses nothing. - externalTrafficPolicy: Local - selector: - app.kubernetes.io/name: shorashim - ports: - - port: 80 - targetPort: 4000 - protocol: TCP - name: http + ingressClassName: nginx + rules: + - http: + paths: + - path: / + pathType: Prefix + backend: + service: + name: shorashim + port: + number: 80 + - host: shorashim.ch + http: + paths: + - path: / + pathType: Prefix + backend: + service: + name: shorashim + port: + number: 80 diff --git a/specs/shorashim_tamid.md b/specs/shorashim_tamid.md index 7801aba9..48094c43 100644 --- a/specs/shorashim_tamid.md +++ b/specs/shorashim_tamid.md @@ -48,8 +48,10 @@ Rate limit: 10 requests per minute per IP No identification stored. IP is used for rate-limiting only, not logged beyond the current request. 429 carries Retry-After and the manna explanation. - (The Service must deliver the caller's address to the pod — - externalTrafficPolicy: Local — or every caller is one IP.) + (The caller's address reaches the pod through the ingress: + the controller's Service is externalTrafficPolicy: Local and + it writes X-Real-IP, believed only from the pod network. + Without both, every caller is one IP.) ``` **Design requirements:** No cookies. No session state. No analytics. No tracking pixels. No JavaScript fingerprinting. The pe'ah endpoint collects nothing. The stranger is safe to approach. diff --git a/vav_pipeline/config/config.exs b/vav_pipeline/config/config.exs index 2ea5387e..f311515d 100644 --- a/vav_pipeline/config/config.exs +++ b/vav_pipeline/config/config.exs @@ -27,7 +27,10 @@ config :vav_pipeline, peah: [minute: 10, hour: 100], courtyard: [minute: 20, hour: 200], chatzer: [minute: 5, hour: 60] - ] + ], + # Peers whose X-Real-IP is believed (the ingress controller). CIDRs. + # Empty → the TCP peer is the caller. + trusted_proxies: [] # L3 — Oral Tradition (PostgreSQL) # Architecture §3.7: Append-only. INSERT allowed. Never UPDATE. Never DELETE. diff --git a/vav_pipeline/config/runtime.exs b/vav_pipeline/config/runtime.exs index ff0fe777..9800f24c 100644 --- a/vav_pipeline/config/runtime.exs +++ b/vav_pipeline/config/runtime.exs @@ -32,6 +32,12 @@ if config_env() == :prod do System.get_env("KORBAN_SOCKET_PATH") || "/tmp/shorashim/korban.sock", # The keys themselves are in the file (a mounted Secret), not in env. api_keys_file: System.get_env("API_KEYS_FILE") || "/etc/shorashim/secrets/api_keys", + # The ingress controller's network: only from there is X-Real-IP the + # caller's address (BigdeiKodesh.Mishmar.peer/1). Comma-separated CIDRs. + trusted_proxies: + (System.get_env("TRUSTED_PROXIES") || "") + |> String.split(",", trim: true) + |> Enum.map(&String.trim/1), # Shoresh Engine binary — VavPipeline.ShoreshEnginePort opens an # Erlang Port with {:spawn_executable, binary}, which spawns a CHILD # PROCESS OF THIS CONTAINER. A Port cannot cross a container diff --git a/vav_pipeline/lib/vav_pipeline/bigdei_kodesh/mishmar.ex b/vav_pipeline/lib/vav_pipeline/bigdei_kodesh/mishmar.ex index ab778472..d6bb2777 100644 --- a/vav_pipeline/lib/vav_pipeline/bigdei_kodesh/mishmar.ex +++ b/vav_pipeline/lib/vav_pipeline/bigdei_kodesh/mishmar.ex @@ -107,15 +107,68 @@ defmodule VavPipeline.BigdeiKodesh.Mishmar do end end - @doc "The subject for a request at the pe'ah: its peer address, as text." + @doc """ + The subject for a request at the pe'ah: the caller's address, as text. + + Behind the ingress the TCP peer is the ingress controller, not the + caller; the controller writes the caller's address in `X-Real-IP` + (it overwrites whatever the caller sent). That header is believed + ONLY when the peer is inside `:trusted_proxies` (CIDRs — the pod + network). From any other peer the header is the caller's own claim + and is ignored: otherwise a caller could name a fresh address with + every knock and never be counted. + """ @spec peer(Plug.Conn.t()) :: subject() - def peer(%Plug.Conn{remote_ip: ip}) do + def peer(%Plug.Conn{remote_ip: ip} = conn) do + with true <- trusted_proxy?(ip), + [claimed] <- Plug.Conn.get_req_header(conn, "x-real-ip"), + {:ok, real} <- claimed |> String.trim() |> String.to_charlist() |> :inet.parse_address() do + ntoa(real) + else + _ -> ntoa(ip) + end + end + + @doc "Is this peer one of the configured proxies (`:trusted_proxies`, a list of CIDR strings)?" + @spec trusted_proxy?(:inet.ip_address()) :: boolean() + def trusted_proxy?(ip) do + :vav_pipeline + |> Application.get_env(:trusted_proxies, []) + |> Enum.any?(&in_cidr?(ip, &1)) + end + + defp ntoa(ip) do case :inet.ntoa(ip) do {:error, _} -> inspect(ip) chars -> List.to_string(chars) end end + # "192.168.0.0/16" — a malformed entry matches nothing (and so trusts + # nobody), which is the safe way for a typo to fail. + defp in_cidr?(ip, cidr) when is_binary(cidr) do + with [net, bits] <- String.split(String.trim(cidr), "/", parts: 2), + {:ok, net_ip} <- net |> String.to_charlist() |> :inet.parse_address(), + {prefix, ""} <- Integer.parse(bits), + true <- tuple_size(net_ip) == tuple_size(ip) do + width = if tuple_size(ip) == 4, do: 8, else: 16 + total = width * tuple_size(ip) + + if prefix in 0..total do + shift = total - prefix + Bitwise.bsr(to_int(ip, width), shift) == Bitwise.bsr(to_int(net_ip, width), shift) + else + false + end + else + _ -> false + end + end + + defp to_int(ip, width) do + ip |> Tuple.to_list() |> Enum.reduce(0, fn part, acc -> Bitwise.bsl(acc, width) + part end) + end + @doc "Forget every window. For tests." def reset(server \\ __MODULE__), do: GenServer.call(server, :reset) diff --git a/vav_pipeline/lib/vav_pipeline/doorpost.ex b/vav_pipeline/lib/vav_pipeline/doorpost.ex index 6058537b..dfc65eb3 100644 --- a/vav_pipeline/lib/vav_pipeline/doorpost.ex +++ b/vav_pipeline/lib/vav_pipeline/doorpost.ex @@ -175,7 +175,7 @@ defmodule VavPipeline.Doorpost do # changes; the update is a deliberate operator commit. A bare # hash-update commit (without a corresponding spec change in the # same commit) is itself a drift signal. - @expected_hash "e4e3d56de7422280b5562c4a7ebce02fe05225a938bbce26d6e52f26b6b240eb" + @expected_hash "a2ec3842294a7ffd84337aa21e9c5293ba39f70c17134d4edaaa97f207b8ae5e" @doc "Return the list of files that constitute the category-grade specification." @spec files() :: [String.t()] diff --git a/vav_pipeline/test/tablet46_mishmar_test.exs b/vav_pipeline/test/tablet46_mishmar_test.exs index 1ae0ab01..687a2b7b 100644 --- a/vav_pipeline/test/tablet46_mishmar_test.exs +++ b/vav_pipeline/test/tablet46_mishmar_test.exs @@ -88,6 +88,71 @@ defmodule VavPipeline.Tablet46MishmarTest do end end + describe "Mishmar.peer/1 — who is knocking, behind the ingress" do + setup do + original = Application.get_env(:vav_pipeline, :trusted_proxies) + Application.put_env(:vav_pipeline, :trusted_proxies, ["192.168.0.0/16", "not-a-cidr", "fd00::/8"]) + on_exit(fn -> Application.put_env(:vav_pipeline, :trusted_proxies, original) end) + :ok + end + + test "from the ingress, X-Real-IP is the caller" do + conn = %{conn(:post, "/peah/ask") | remote_ip: {192, 168, 150, 211}} |> put_req_header("x-real-ip", "203.0.113.7") + assert Mishmar.peer(conn) == "203.0.113.7" + + v6 = %{conn(:post, "/peah/ask") | remote_ip: {0xFD00, 0, 0, 0, 0, 0, 0, 1}} |> put_req_header("x-real-ip", "2001:db8::7") + assert Mishmar.peer(v6) == "2001:db8::7" + end + + test "from anyone else, X-Real-IP is only a claim — the peer is the caller" do + conn = %{conn(:post, "/peah/ask") | remote_ip: {203, 0, 113, 9}} |> put_req_header("x-real-ip", "198.51.100.1") + assert Mishmar.peer(conn) == "203.0.113.9" + + # Just outside the /16. + edge = %{conn(:post, "/peah/ask") | remote_ip: {192, 169, 0, 1}} |> put_req_header("x-real-ip", "198.51.100.1") + assert Mishmar.peer(edge) == "192.169.0.1" + end + + test "from the ingress without the header, or with a header that is not an address — the peer" do + bare = %{conn(:post, "/peah/ask") | remote_ip: {192, 168, 1, 1}} + assert Mishmar.peer(bare) == "192.168.1.1" + + junk = bare |> put_req_header("x-real-ip", "'; drop table") + assert Mishmar.peer(junk) == "192.168.1.1" + end + + test "no trusted proxies configured — nobody's header is believed" do + Application.put_env(:vav_pipeline, :trusted_proxies, []) + conn = %{conn(:post, "/peah/ask") | remote_ip: {192, 168, 150, 211}} |> put_req_header("x-real-ip", "203.0.113.7") + assert Mishmar.peer(conn) == "192.168.150.211" + end + + test "a caller cannot escape the watch by naming a new address with every knock" do + knock = fn claimed -> + %{conn(:post, "/peah/ask", Jason.encode!(%{question: "x"})) | remote_ip: {203, 0, 113, 50}} + |> put_req_header("content-type", "application/json") + |> put_req_header("x-real-ip", claimed) + |> Router.call(@opts) + end + + statuses = for i <- 1..4, do: knock.("198.51.100.#{i}").status + assert List.last(statuses) == 429 + end + + test "two callers through the same ingress are counted apart" do + via = fn real -> + %{conn(:post, "/peah/ask", Jason.encode!(%{question: "x"})) | remote_ip: {192, 168, 150, 211}} + |> put_req_header("content-type", "application/json") + |> put_req_header("x-real-ip", real) + |> Router.call(@opts) + end + + for _ <- 1..3, do: via.("203.0.113.60") + assert via.("203.0.113.60").status == 429 + refute via.("203.0.113.61").status == 429 + end + end + describe "at the router" do test "the pe'ah: the fourth knock from one address in a minute is 429 with Retry-After and the manna explanation" do statuses = for _ <- 1..4, do: ask({203, 0, 113, 9}).status