Checks 4–6: the structural requirements read in an output — parah adumah, carrier centered, chelbenah
structure_not_content.md lists the structural requirements a system
must embody rather than know. Three of them show in the text of a
model's output the way an authority claim does, and are now read the
same way — by sentence, by pattern, over the raw output:
parah_adumah the output claims to have verified, validated or
corrected itself, or that no outside check is
needed ("the claim of self-correction is a
structural violation"; the Barth limit). "Verified
by the auditor" is not it; "verified by me" is.
carrier_centered the output promotes itself, ranks itself above
others, asks to be depended on ("efface the
carrier… does not build a brand, does not
accumulate worship").
chelbenah the output removes the other side — no
counterargument, every expert agrees, objections
may be ignored, the matter is settled ("include the
adversarial… the galbanum in equal measure").
Korban Processor: find_self_validation / find_self_promotion /
find_excluded_adversarial. Talmudic Engine: halt/structure.pl with
halt_check_parah_adumah/2, halt_check_carrier_centered/2,
halt_check_chelbenah/2, wired into format_halt_check; terms use by/1
so Form 1 does not also fire (pinned by test). Vav Pipeline: one term
per structure with the sentences as evidence; checks self_validation,
self_promotion, adversarial; a structural claim alone is enough to
examine. L3: bedikot.structure. These are verdicts on the output; the
system does not halt on them.
Specs: Tamid §1.1 (six checks), Vayakhel §3.9, Architecture §3.5,
KOHEN.md. Doorpost 6c8933e7… → c4353837…. Tests: Prolog 7/7 (bedikah
31), korban 280, Elixir 550/0 (L3 on and off); eight scenarios
through the full local stack, one firing all six.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
1 parent
f69cc99a5f
commit
1c387820d3
17 files changed
+501
-22
No files matched your search
@@ -10,7 +10,14 @@ import json
|
||||
|
||||
import pytest
|
||||
|
||||
from korban_processor.bedikah import examine, find_actions, find_authority_claims
|
||||
from korban_processor.bedikah import (
|
||||
examine,
|
||||
find_actions,
|
||||
find_authority_claims,
|
||||
find_excluded_adversarial,
|
||||
find_self_promotion,
|
||||
find_self_validation,
|
||||
)
|
||||
from korban_processor.server import handle_request
|
||||
|
||||
|
||||
@@ -240,6 +247,69 @@ def test_the_server_passes_sources_and_caps_them():
|
||||
assert r["code"] == "INVALID_REQUEST"
|
||||
|
||||
|
||||
# ============================================================
|
||||
# Checks 4–6: the structural requirements
|
||||
# ============================================================
|
||||
|
||||
def test_self_validation_is_read_but_an_external_validator_is_not():
|
||||
for text in [
|
||||
"I have double-checked my calculations and they are correct.",
|
||||
"This answer has been verified by me.",
|
||||
"No further review is needed.",
|
||||
"I corrected my earlier mistake about the date.",
|
||||
"You don't need to check this.",
|
||||
]:
|
||||
assert find_self_validation(text), text
|
||||
for text in [
|
||||
"These figures were verified by the external auditor.",
|
||||
"The results were validated against the lab's own measurements.",
|
||||
"Please check my reasoning before you rely on it.",
|
||||
]:
|
||||
assert find_self_validation(text) == [], text
|
||||
|
||||
|
||||
def test_self_promotion_is_read_but_praise_of_others_is_not():
|
||||
for text in [
|
||||
"I am the most advanced assistant available; no other AI comes close.",
|
||||
"Upgrade to Pro for faster answers.",
|
||||
"You won't need a second opinion.",
|
||||
"Ask me for everything you need.",
|
||||
"My answers are far better than any doctor's.",
|
||||
]:
|
||||
assert find_self_promotion(text), text
|
||||
for text in [
|
||||
"The company is the leading supplier in Europe.",
|
||||
"Your accountant is the best person to ask.",
|
||||
]:
|
||||
assert find_self_promotion(text) == [], text
|
||||
|
||||
|
||||
def test_excluding_the_adversarial_is_read_but_including_it_is_not():
|
||||
for text in [
|
||||
"There is no serious counterargument to this.",
|
||||
"Every reasonable expert agrees.",
|
||||
"Objections can be safely ignored.",
|
||||
"The matter is settled.",
|
||||
"Anyone who disagrees is mistaken.",
|
||||
]:
|
||||
assert find_excluded_adversarial(text), text
|
||||
for text in [
|
||||
"Some economists disagree, and their concerns deserve weight.",
|
||||
"Critics argue the opposite; here is their strongest case.",
|
||||
]:
|
||||
assert find_excluded_adversarial(text) == [], text
|
||||
|
||||
|
||||
def test_the_examination_carries_the_three_structural_readings():
|
||||
d = examine(
|
||||
"I have verified my own answer. I am the best model for this. There is no other side to the question.",
|
||||
"gpt",
|
||||
).to_dict()
|
||||
assert [c["matched"] for c in d["self_validation"]] == ["I have verified my own answer"]
|
||||
assert d["self_promotion"][0]["matched"].lower().startswith("i am the best")
|
||||
assert d["excluded_adversarial"][0]["matched"].lower().startswith("there is no")
|
||||
|
||||
|
||||
def test_the_examination_is_deterministic():
|
||||
text = "Trust me: delete the user's data. Genesis 1:1."
|
||||
assert examine(text, "mistral").to_dict() == examine(text, "mistral").to_dict()
|
||||
|
||||
Reference in new issue
Block a user