One gate to the world: Shorashim and Gitea behind the shared ingress

Three NLBs became one (185.19.28.170, ingress-nginx). Shorashim: `shorashim-external` (NLB 185.19.29.180, NodePort 30923) is replaced by an Ingress — a host-less catch-all (works by IP, no DNS) plus a `shorashim.ch` rule ready for the A record; TLS deliberately not requested until the name resolves. Sanhedrin-length proxy timeouts. The watch behind the ingress (completes tablet 46): the TCP peer is now the ingress controller, so Mishmar.peer/1 takes the caller from X-Real-IP — only when the peer is inside TRUSTED_PROXIES (the pod network, configmap); from anyone else the header is the caller's own claim and is ignored. The ingress Service was already externalTrafficPolicy: Local with its health-check port open, so the caller's address arrives without the security-group change the direct route needed. Verified live: the 11th knock is 429, a forged X-Real-IP is still 429, and in the same minute the old route (pod sees the node) was a different subject. Gitea: no name of its resolves, so it rides the controller's TCP passthrough on the same NLB — :3000 → gitea:80, :22 → gitea:22 (ingress-tcp-services.yaml), on the two NodePorts the security group already allowed. gitea-mcp now reaches Gitea over the cluster network instead of out through an NLB and back. Tamid §1.1 note, KOHEN.md, AGENTS.md. Doorpost e4e3d56d… → a2ec3842…. Elixir 539/0 (L3 on and off). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-06 17:59:23 +00:00
1 parent 54ca651f37
commit e17436c4ca
@@ -25,6 +25,12 @@ data:
  # keys never are.
  API_KEYS_FILE: "/etc/shorashim/secrets/api_keys"

+  # The pod network. The ingress controller lives in it; only from here
+  # is X-Real-IP believed as the caller's address (the pe'ah's per-IP
+  # watch, Tamid §1.1). Nothing outside the cluster can reach the pod
+  # directly any more (service.yaml), so nothing outside can forge it.
+  TRUSTED_PROXIES: "192.168.0.0/16"
+
  # Shoresh Engine binary — staged onto the shared shoresh-bin volume
  # by the shoresh-engine-install init container (see deployment.yaml).
  # vav_pipeline spawns it as an Erlang Port, i.e. as a child process of
@@ -3,8 +3,11 @@
 # Deploys the official Gitea MCP server (gitea.com/gitea/gitea-mcp)
 # in HTTP mode, exposing /mcp endpoint for Streamable HTTP transport.
 #
-# The MCP server connects to the Gitea backend at 185.19.29.94
-# using a Personal Access Token stored in gitea-mcp-secrets.
+# The MCP server connects to the Gitea backend over the cluster network
+# (gitea.temple-council.svc.cluster.local) using a Personal Access Token
+# stored in gitea-mcp-secrets. Until 2026-10-06 it went out to Gitea's
+# own NLB (185.19.29.94) and back in; that NLB is gone — Gitea is behind
+# the shared ingress now (ingress-tcp-services.yaml).
 ---
 apiVersion: apps/v1
 kind: Deployment
@@ -31,7 +34,7 @@ spec:
            - "-t"
            - "http"
            - "--host"
-            - "http://185.19.29.94"
+            - "http://gitea.temple-council.svc.cluster.local"
            - "--port"
            - "8080"
          env:
@@ -41,7 +44,7 @@ spec:
                  name: gitea-mcp-secrets
                  key: GITEA_TOKEN
            - name: GITEA_HOST
-              value: "http://185.19.29.94"
+              value: "http://gitea.temple-council.svc.cluster.local"
          ports:
            - containerPort: 8080
              name: http
@@ -3,11 +3,11 @@
 #
 # Two secrets needed:
 # 1. GITEA_TOKEN: Personal Access Token from Gitea
-#    Create at: http://185.19.29.94/user/settings/applications
+#    Create at: http://185.19.28.170:3000/user/settings/applications
 #    Permissions: repo (read/write)
 #
 # 2. OAuth Client ID/Secret: Registered OAuth app in Gitea
-#    Create at: http://185.19.29.94/user/settings/applications
+#    Create at: http://185.19.28.170:3000/user/settings/applications
 #    Redirect URI: https://claude.ai/api/mcp/auth_callback
 #    (These go into claude.ai connector settings, not this secret)
 ---
@@ -0,0 +1,38 @@
+# Gitea behind the shared ingress — TCP passthrough
+#
+# One NLB for the cluster (185.19.28.170, the ingress-nginx Service)
+# instead of one per Service. Shorashim is an HTTP Ingress
+# (service.yaml). Gitea has no hostname that resolves
+# (gitea.temple-council.ch is configured in Gitea and exists nowhere),
+# and host-based routing needs a name — so Gitea rides the ingress
+# controller's TCP passthrough on two ports of the same NLB:
+#
+#   185.19.28.170:3000  →  temple-council/gitea:80   (HTTP — git, web UI)
+#   185.19.28.170:22    →  temple-council/gitea:22   (SSH)
+#
+# Three live objects carry this; only the first is a whole manifest:
+#
+# 1. This ConfigMap.
+# 2. The controller reads it — one argument on the Deployment:
+#      kubectl -n ingress-nginx patch deploy ingress-nginx-controller --type=json -p \
+#        '[{"op":"add","path":"/spec/template/spec/containers/0/args/-",
+#           "value":"--tcp-services-configmap=$(POD_NAMESPACE)/tcp-services"}]'
+# 3. The controller's Service exposes the two ports ON THE NODEPORTS THE
+#    SECURITY GROUP ALREADY ALLOWS (they were Gitea's own; the CCM never
+#    opens a NodePort — tablets 29, 46):
+#      kubectl -n ingress-nginx patch svc ingress-nginx-controller --type=json -p \
+#        '[{"op":"add","path":"/spec/ports/-","value":{"name":"gitea-http","port":3000,"targetPort":3000,"protocol":"TCP","nodePort":30575}},
+#          {"op":"add","path":"/spec/ports/-","value":{"name":"gitea-ssh","port":22,"targetPort":22,"protocol":"TCP","nodePort":31345}}]'
+#    and temple-council/gitea is type ClusterIP (it held 30575/31345 as a
+#    LoadBalancer until 2026-10-06).
+#
+# When Gitea has a real name: add an HTTP Ingress for it (TLS via
+# cert-manager) and drop the 3000 line here; SSH stays.
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: tcp-services
+  namespace: ingress-nginx
+data:
+  "3000": "temple-council/gitea:80"
+  "22": "temple-council/gitea:22"
@@ -24,38 +24,64 @@ spec:
      protocol: TCP
      name: http
 ---
-# External access via LoadBalancer
-# For Exoscale SKS, this provisions an Exoscale NLB.
+# ============================================================
+# The gate to the world — through the shared ingress
+# ============================================================
 #
-# The NLB forwards to this Service's NodePort and TCP-healthchecks it on
-# the node. The cloud controller does NOT open that port in the node's
-# security group — the operator must, or the NLB IP refuses connections
-# while the pod is Ready (2026-10-04, tablet 29):
-#   PORT=$(kubectl -n shorashim get svc shorashim-external -o jsonpath='{.spec.ports[0].nodePort}')
-#   exo compute security-group rule add default --flow ingress --protocol tcp --port $PORT --network 0.0.0.0/0
-apiVersion: v1
-kind: Service
+# Until 2026-10-06 Shorashim had its own LoadBalancer Service
+# (`shorashim-external`, NLB 185.19.29.180, NodePort 30923). It is gone:
+# one NLB per Service was a third of the account's bill, and the pod saw
+# every caller as the node (kube-proxy SNAT) so the pe'ah's per-IP watch
+# could not tell callers apart. The ingress-nginx Service is already
+# externalTrafficPolicy: Local with its health-check port open, so the
+# controller sees the caller and hands it on in X-Real-IP; the pipeline
+# believes that header only from TRUSTED_PROXIES (configmap.yaml).
+#
+# Two rules, one backend:
+#   - no host  → the catch-all: http://<ingress-ip>/... works with no DNS
+#   - shorashim.ch → ready for the A record (KOHEN.md Day 8). TLS is NOT
+#     requested here: asking cert-manager for a certificate before the
+#     name resolves only burns Let's Encrypt's failed-validation limit.
+#     When the record exists add:
+#       metadata.annotations:
+#         cert-manager.io/cluster-issuer: letsencrypt-prod
+#         nginx.ingress.kubernetes.io/ssl-redirect: "true"
+#       spec.tls: [{hosts: [shorashim.ch], secretName: shorashim-tls}]
+#     and point callers at https://shorashim.ch — until then the
+#     courtyard keys travel over plain HTTP.
+apiVersion: networking.k8s.io/v1
+kind: Ingress
 metadata:
-  name: shorashim-external
+  name: shorashim
  namespace: shorashim
  labels:
    app.kubernetes.io/name: shorashim
  annotations:
-    # Exoscale NLB annotations
-    service.beta.kubernetes.io/exoscale-loadbalancer-zone: "ch-gva-2"
-    service.beta.kubernetes.io/exoscale-loadbalancer-name: "shorashim"
+    # A 71-judge Sanhedrin may deliberate for ten minutes (Construction §2.2).
+    nginx.ingress.kubernetes.io/proxy-read-timeout: "620"
+    nginx.ingress.kubernetes.io/proxy-send-timeout: "620"
+    # The bedikah accepts 65,536 bytes of output; leave room for the envelope.
+    nginx.ingress.kubernetes.io/proxy-body-size: "256k"
+    nginx.ingress.kubernetes.io/ssl-redirect: "false"
 spec:
-  type: LoadBalancer
-  # Local: kube-proxy does not SNAT, so the pod sees the caller's own
-  # address and the pe'ah's per-IP watch (Tamid §1.1) counts callers,
-  # not the node. With Cluster (the default) every caller arrived as the
-  # node IP 91.92.153.113 — observed 2026-10-06, tablet 46. One node,
-  # one replica: Local loses nothing.
-  externalTrafficPolicy: Local
-  selector:
-    app.kubernetes.io/name: shorashim
-  ports:
-    - port: 80
-      targetPort: 4000
-      protocol: TCP
-      name: http
+  ingressClassName: nginx
+  rules:
+    - http:
+        paths:
+          - path: /
+            pathType: Prefix
+            backend:
+              service:
+                name: shorashim
+                port:
+                  number: 80
+    - host: shorashim.ch
+      http:
+        paths:
+          - path: /
+            pathType: Prefix
+            backend:
+              service:
+                name: shorashim
+                port:
+                  number: 80