engine: a word's root is what L1 records for it (tablet 33)

The Shoresh Engine never opened L1 to find a root. extractRoots called
every word of two or three letters a root and dropped every longer one:
את → א-ת-ה (padded with a He the text never wrote), ואת → ו-א-ת,
בכל → ב-כ-ל; ויאמר, ואהבת, אלהים → nothing. Handed all of Devarim 6:5
it found no א-ה-ב. Binyan was "Qal" for everything, nouns included.

shoresh-engine:
- Shoresh.Roots: an index over all of L1 (5853 verses, built once at
  server start, <1 s). A word resolves, in order: from the verse in
  scope (its own shoresh, with the binyan read from the verb's
  morphology; else its lemma's shoresh — לבבך → לבב); else the way Torah
  most often reads that surface form (its commonest lemma decides, so
  אשר the particle has no root); else a root the asker named, if Torah
  knows it. A word in the verse that L1 gives no root has none, whatever
  a homograph elsewhere carries. Unknown words yield nothing.
- Roots are rendered as L1 records them: two letters stay two letters.
- binyan is null unless a verb stands in the scoped verse.
- L1 is found via L1_MOUNT_POINT, not the literal l1/ relative to cwd —
  which is why parse_tree was null on the pod.
- Adds "surface" and "derivation" to each root (verse|lemma|torah|
  named_root). 22 new HSpec examples; 55 examples, 0 failures.

vav_pipeline:
- normalize_root places a lemma-derived root at the word it was heard
  from (L1 has no shoresh on לבבך to search by).
- A binyan is printed only where there is one; "binyan unknown" on a
  noun claimed a question that does not exist.
- test/tablet33_engine_roots_test.exs: the new shape through the router.
- liveness_test: clear a halt leaked by halt_monitor_test (the seed
  0/191243 flake, present since before tablet 30).
  461 tests, 0 failures on seeds 0 42 191243.

Verified before push with BOTH built images, the real engine binary and
a local PostgreSQL 16 (SSL): Devarim 6:5 → א-ה-ב (Qal, word 1), י-ה-ו,
א-ל-ה, כ-ל (words 5,7,9), ל-ב-ב, נ-פ-ש, מ-א-ד; 4 interpretations
written to L3, 0 L3_WRITE_FAILED, health healthy.

NOT repaired here: the quality of L1's own shoresh field, which the
engine is now faithful to — ויהי is recorded as יהי, שמים as שמ, יהוה
as יהו, הבאתי as בא. That is the quarry's tablet (scripts/build_l1.py)
and it changes the L1 hash; a decision, not a patch.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Scott KarpandClaude Fable 5.1 committed 2026-10-04 17:28:08 +00:00
1 parent 83fa5ac46b
commit 6d3411fcfa
7 files changed
+683 -69

No files matched your search

+162
View File
@@ -22,6 +22,7 @@ import Shoresh.Teamim
import Shoresh.KetivQere
import Shoresh.Lexicon
import Shoresh.Graph
import Shoresh.Roots
import Shoresh.KiTov
import Luchot.Firmware
@@ -36,6 +37,7 @@ main = hspec $ do
graphSpec
kiTovSpec
firmwareSpec
rootsSpec
consonantSpec :: Spec
consonantSpec = describe "Consonant" $ do
@@ -294,6 +296,166 @@ firmwareSpec = describe "Firmware" $ do
dibrot `shouldBe` Dibrot () () () () () () () () () ()
-- ============================================================
-- Roots — tablet 33: a word's root is what L1 records for it
-- ============================================================
-- | An L1 word as build_l1.py writes it (position, text and te'amim are
-- irrelevant to root resolution).
l1w :: Text -> Text -> Text -> Maybe Text -> L1Word
l1w cons lemma morph sh = L1Word
{ l1wPosition = 0, l1wText = cons, l1wConsonants = cons, l1wLemma = lemma
, l1wMorph = morph, l1wTeamim = [], l1wPrefix = Nothing, l1wShoresh = sh }
l1v :: Text -> Int -> Int -> [L1Word] -> L1Verse
l1v book ch vs ws = L1Verse
{ l1vRef = "", l1vBook = book, l1vChapter = ch, l1vVerse = vs
, l1vTextHebrew = "", l1vWords = ws }
-- Devarim 6:5, as L1 holds it: three suffixed nouns carry no shoresh.
devarim65 :: L1Verse
devarim65 = l1v "devarim" 6 5
[ l1w "ואהבת" "157" "HC/Vqq2ms" (Just "אהב")
, l1w "את" "853" "HTo" Nothing
, l1w "יהוה" "3068" "HNp" (Just "יהו")
, l1w "אלהיך" "430" "HNcmpc/Sp2ms" Nothing
, l1w "בכל" "3605" "HR/Ncmsc" (Just "כל")
, l1w "לבבך" "3824" "HNcmsc/Sp2ms" Nothing
, l1w "ובכל" "3605" "HC/R/Ncmsc" (Just "כל")
, l1w "נפשך" "5315" "HNcbsc/Sp2ms" Nothing
]
-- Elsewhere in Torah: the same lemmas with a shoresh of their own.
elsewhere :: [L1Verse]
elsewhere =
[ l1v "bereishit" 1 1
[ l1w "בראשית" "7225" "HR/Ncfsa" (Just "ראש")
, l1w "ברא" "1254" "HVqp3ms" (Just "ברא")
, l1w "אלהים" "430" "HNcmpa" (Just "אלה")
, l1w "את" "853" "HTo" Nothing
]
, l1v "bereishit" 1 3
[ l1w "ויאמר" "559" "HC/Vqw3ms" (Just "אמר")
, l1w "אור" "216" "HNcbsa" (Just "אור")
, l1w "ויהי־אור" "1961" "HC/Vqw3ms" (Just "יהי")
]
, l1v "bereishit" 1 4
[ l1w "ויבדל" "914" "HC/Vhw3ms" (Just "בדל") ]
, l1v "devarim" 4 29
[ l1w "לבב" "3824" "HNcmsa" (Just "לבב")
, l1w "נפש" "5315" "HNcbsa" (Just "נפש")
]
-- Homographs. אשר: the relative particle twice, the name Asher once.
-- את: the object marker everywhere, but אתה yields the root את.
, l1v "shemot" 3 14
[ l1w "אשר" "834 a" "HTr" Nothing
, l1w "אהיה" "1961" "HVqi1cs" (Just "היה")
, l1w "אשר" "834 a" "HTr" Nothing
]
, l1v "bereishit" 30 13
[ l1w "אשר" "836" "HNp" (Just "אשר")
, l1w "אתה" "859 a" "HPp2ms" (Just "את")
]
]
testIndex :: RootIndex
testIndex = buildIndex (devarim65 : elsewhere)
shoreshim :: [ResolvedRoot] -> [Text]
shoreshim = map rrShoresh
rootsSpec :: Spec
rootsSpec = describe "Roots (tablet 33)" $ do
describe "what the old engine did, and must not do again" $ do
it "a particle is not a root — את yields nothing, never א-ת-ה" $
resolveText testIndex Nothing "את" `shouldBe` []
it "a three-letter word is not a root because it has three letters" $
-- בכל was reported as ב-כ-ל; L1 says its root is כל.
shoreshim (resolveText testIndex (Just devarim65) "בכל") `shouldBe` ["כל"]
it "a long word is not dropped — ויאמר is א-מ-ר" $
shoreshim (resolveText testIndex Nothing "ויאמר") `shouldBe` ["אמר"]
it "a word Torah does not contain yields nothing" $
resolveText testIndex Nothing "מחשב" `shouldBe` []
it "non-Hebrew input yields nothing" $
resolveText testIndex Nothing "Test query 1" `shouldBe` []
describe "the whole of Devarim 6:5" $ do
let roots = resolveText testIndex (Just devarim65)
"ואהבת את יהוה אלהיך בכל לבבך ובכל נפשך"
it "finds א-ה-ב, and the heart and the soul" $
shoreshim roots `shouldBe` ["אהב", "יהו", "אלה", "כל", "לבב", "נפש"]
it "a word in the verse with its own shoresh is heard from the verse" $
map rrDerivation (take 2 roots) `shouldBe` [FromVerse, FromVerse]
it "a suffixed noun with no shoresh is heard through its lemma" $
[ rrDerivation r | r <- roots, rrShoresh r `elem` ["לבב", "נפש", "אלה"] ]
`shouldBe` [FromLemma, FromLemma, FromLemma]
it "כל appears once though the verse says it twice" $
length (filter (== "כל") (shoreshim roots)) `shouldBe` 1
describe "binyan" $ do
it "is read from the verb's morphology in the verse" $ do
let v = elsewhere !! 2
map rrBinyan (resolveText testIndex (Just v) "ויבדל") `shouldBe` [Just Hifil]
map rrBinyan (resolveText testIndex (Just devarim65) "ואהבת") `shouldBe` [Just Qal]
it "is not claimed for a noun" $
map rrBinyan (resolveText testIndex (Just devarim65) "בכל") `shouldBe` [Nothing]
it "is not claimed for a word outside the verse in scope" $
map rrBinyan (resolveText testIndex (Just devarim65) "ויבדל") `shouldBe` [Nothing]
it "parses OSHB stems" $ do
binyanFromMorph "HC/Vqw3ms" `shouldBe` Just Qal
binyanFromMorph "HVhp1cs" `shouldBe` Just Hifil
binyanFromMorph "HVNp3ms" `shouldBe` Just Nifal
binyanFromMorph "HVtw3ms" `shouldBe` Just Hitpael
binyanFromMorph "HNcmsa" `shouldBe` Nothing
describe "homographs" $ do
it "a particle in the verse has no root, though Torah knows את as a root" $ do
resolveText testIndex (Just devarim65) "את" `shouldBe` []
resolveText testIndex Nothing "את" `shouldBe` []
it "out of scope, a word is read the way Torah most often reads it" $
-- אשר: particle ×2, the name ×1. The particle wins; no root.
resolveText testIndex Nothing "אשר" `shouldBe` []
it "in scope, the verse decides — אשר the particle in Shemot 3:14" $
shoreshim (resolveText testIndex (Just (elsewhere !! 4)) "אהיה אשר אהיה")
`shouldBe` ["היה"]
describe "other ways a root is heard" $ do
it "from elsewhere in Torah when the word is not in the verse in scope" $
map (\r -> (rrShoresh r, rrDerivation r))
(resolveText testIndex (Just devarim65) "ויאמר")
`shouldBe` [("אמר", FromTorah)]
it "a root the asker names, if Torah knows it as a root" $
map (\r -> (rrShoresh r, rrDerivation r)) (resolveText testIndex Nothing "אמר")
`shouldBe` [("אמר", NamedRoot)]
it "a three-letter string Torah does not know as a root is not one" $
resolveText testIndex Nothing "קטל" `shouldBe` []
describe "tokenize" $ do
it "strips nikud and te'amim" $
tokenize "וַיֹּ֥אמֶר אֱלֹהִ֖ים" `shouldBe` ["ויאמר", "אלהים"]
it "splits maqaf before stripping marks (tablet 30a)" $
tokenize "וַֽיְהִי־אֽוֹר ׃" `shouldBe` ["ויהי", "אור"]
it "indexes both halves of a maqaf-joined L1 word" $
shoreshim (resolveText testIndex Nothing "ויהי") `shouldBe` ["יהי"]
-- ============================================================
-- Helpers
-- ============================================================