builder fix for kohen Day-3 FAIL: image arch, azazel co-residence, runbook drift
ROOT CAUSE (kohen Day-3 escalation): None of the four Dockerfiles pinned --platform. Build on Apple Silicon shipped arm64-only images to a registry consumed by amd64 SKS nodes. Init container failed exec format error before producing logs. DOCKERFILE FIX (defense in depth): Pin --platform=linux/amd64 on every FROM line in all four Dockerfiles. The build artifact now self-corrects regardless of build host. AZAZEL CO-RESIDENCE (anomaly the kohen flagged): Azazel L3 migration shipped (20260428…create_azazel_records.exs) but no separate container existed. Reading the code: azazel_processor is imported by korban_processor.external_offering at runtime in the same Python process — not a separate process. Dockerfile.korban_processor now COPYs azazel_processor/src/ alongside korban_processor. Documented in the Dockerfile (AZAZEL CO-RESIDENT section) and in deployment.yaml. Both containers (vav-pipeline + korban-processor) now mount the shared tmp-shorashim volume so the Python→Elixir file bridge (azazel_ingest.ex polls /tmp/shorashim/azazel/pending_l3/) works end-to-end. CANONICAL BUILD/PUSH (scripts/build_and_push.sh): - Pins linux/amd64 by default; PLATFORMS env for multi-arch - Pre-flight checks (repo root, buildx, registry login) - Builds and pushes all four images with one command - Closes the gap KOHEN_PROMPT.md gate 0i would otherwise leave open KOHEN.md DRIFT (kohen recorded these as anomalies through Day 1-3): - Line 58 hash fragment 99bbd07a…0a0187b → 99bbd07a…4ea377 (correct) - Inheritance table: "Not yet created" claims rewritten as "Verify before assuming" — a prior partial setup may have left state, and the kohen prompt requires verifying not assuming - L1 PV row: marked NOT REQUIRED (L1 baked into images) - §Day 3a ConfigMap recipe marked vestigial (kept as comment) - §Day 3b: removed manna-volume.yaml from kubectl apply list (deployment uses emptyDir; PVC is for a future move) - §Day 3c: "5 containers" → "4 containers + 1 init" - §Day 3 fail-mode list expanded with the four most common causes, in order, including exec format error → gate 0i KOHEN_PROMPT.md: - Gate 0f corrected to four images (talmudic-engine is image #3; azazel is co-resident in korban-processor not a separate image) - Gate 0i added: image-architecture vs node-architecture verification before Day 1, with kubectl one-liner and docker manifest inspect per image. Drift handler in the gates section: 0i FAIL → escalate to builder, do not begin Day 1. NO SPEC FILES TOUCHED: Doorpost.verify_identity() still :ok — none of these files are in @spec_files. No atomic spec+hash commit needed. NEXT STEP FOR THE KOHEN: Once these images are rebuilt and pushed via scripts/build_and_push.sh (by anyone with the Harbor CLI secret + a working docker daemon), restart from Day 1. Existing DB role shorashim_app, ALTER DEFAULT PRIVILEGES, rotated DATABASE_URL, shorashim-config, and shorashim- registry secret all remain valid across the restart. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Scott KarpandClaude Opus 4.7 committed
1 parent
1df606a9c5
commit
55bc5bdc44
8 files changed
+221
-33
No files matched your search
Executable
+111
@@ -0,0 +1,111 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# build_and_push.sh — canonical builder procedure for shorashim images.
|
||||
#
|
||||
# Pin platform to linux/amd64 (Exoscale SKS) so a build on an Apple
|
||||
# Silicon workstation does not silently ship arm64 binaries that the
|
||||
# cluster cannot exec. The Dockerfiles also pin --platform=linux/amd64
|
||||
# on every FROM line as defense-in-depth; this script is the canonical
|
||||
# build/push path that operators and CI both use.
|
||||
#
|
||||
# Default: linux/amd64 only (cluster-correct, fastest path).
|
||||
# Optional: PLATFORMS=linux/amd64,linux/arm64 ./build_and_push.sh
|
||||
# — multi-arch (slower; preserves local runnability for arm64 devs).
|
||||
#
|
||||
# Prerequisites:
|
||||
# - Docker daemon running
|
||||
# - docker buildx available
|
||||
# - Logged in to exo.container-registry.com (CLI secret from
|
||||
# Harbor → User Profile → CLI Secret)
|
||||
# - Run from the repo root
|
||||
#
|
||||
# What gets pushed (four images):
|
||||
# - vav-pipeline
|
||||
# - shoresh-engine
|
||||
# - talmudic-engine
|
||||
# - korban-processor (azazel_processor is baked into this image)
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
REGISTRY="${REGISTRY:-exo.container-registry.com}"
|
||||
PROJECT="${PROJECT:-the-recognition-protocol}"
|
||||
TAG="${TAG:-latest}"
|
||||
PLATFORMS="${PLATFORMS:-linux/amd64}"
|
||||
|
||||
# Sanity: must be run from repo root (where deploy/ and l1/ live)
|
||||
if [[ ! -d "deploy/docker" || ! -d "l1/torah" ]]; then
|
||||
echo "ERROR: run from the shorashim repo root (where deploy/ and l1/ live)" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Sanity: docker buildx
|
||||
if ! docker buildx version >/dev/null 2>&1; then
|
||||
echo "ERROR: docker buildx not available — install Docker Desktop or buildx plugin" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Sanity: registry login. We probe with a HEAD against the registry
|
||||
# v2 endpoint; if 401, login is missing.
|
||||
if ! docker info 2>/dev/null | grep -q "$REGISTRY"; then
|
||||
echo "Reminder: run 'docker login $REGISTRY' first (CLI secret from Harbor User Profile)."
|
||||
echo "Continuing — push will fail clearly if not logged in."
|
||||
fi
|
||||
|
||||
# Create or use the shorashim buildx builder
|
||||
if ! docker buildx inspect shorashim-builder >/dev/null 2>&1; then
|
||||
echo ">> creating buildx builder 'shorashim-builder'"
|
||||
docker buildx create --name shorashim-builder --use
|
||||
else
|
||||
docker buildx use shorashim-builder
|
||||
fi
|
||||
docker buildx inspect --bootstrap >/dev/null
|
||||
|
||||
# Source dir → image name → Dockerfile mapping.
|
||||
# korban_processor's Dockerfile also COPYs azazel_processor (see
|
||||
# Dockerfile.korban_processor — AZAZEL CO-RESIDENT section).
|
||||
build_one() {
|
||||
local component="$1"
|
||||
local image="$2"
|
||||
local dockerfile="deploy/docker/Dockerfile.${component}"
|
||||
|
||||
if [[ ! -f "$dockerfile" ]]; then
|
||||
echo "ERROR: $dockerfile not found" >&2
|
||||
return 1
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "================================================================"
|
||||
echo ">> building ${image}:${TAG} for ${PLATFORMS}"
|
||||
echo " dockerfile: ${dockerfile}"
|
||||
echo "================================================================"
|
||||
|
||||
docker buildx build \
|
||||
--platform "$PLATFORMS" \
|
||||
--file "$dockerfile" \
|
||||
--tag "${REGISTRY}/${PROJECT}/${image}:${TAG}" \
|
||||
--push \
|
||||
.
|
||||
}
|
||||
|
||||
build_one vav_pipeline vav-pipeline
|
||||
build_one shoresh_engine shoresh-engine
|
||||
build_one talmudic_engine talmudic-engine
|
||||
build_one korban_processor korban-processor
|
||||
|
||||
echo ""
|
||||
echo "================================================================"
|
||||
echo ">> all four images built and pushed for ${PLATFORMS}"
|
||||
echo "================================================================"
|
||||
echo ""
|
||||
echo "Verify from the cluster (after kohen has kubeconfig loaded):"
|
||||
echo " kubectl get nodes -o jsonpath='{.items[0].status.nodeInfo.architecture}'"
|
||||
echo " # expected: amd64"
|
||||
echo ""
|
||||
echo "Verify image manifests:"
|
||||
echo " for img in vav-pipeline shoresh-engine talmudic-engine korban-processor; do"
|
||||
echo " docker manifest inspect \"${REGISTRY}/${PROJECT}/\$img:${TAG}\" \\"
|
||||
echo " | jq -r '.architecture // .manifests[].platform.architecture' \\"
|
||||
echo " | sort -u"
|
||||
echo " done"
|
||||
echo ""
|
||||
echo "Then re-run kohen gate 0i in KOHEN_PROMPT.md."
|
||||
Reference in new issue
Block a user