feat(curator): tool extension, generated prompt check, two pi findings

curator-tools.ts registers no schema of its own: it fetches the specs from the
backend bridge, so contracts.py stays the single owner and there is no TypeScript
copy to drift. It refuses to activate without a bridge URL and token, because an
agent that silently loses its tools still answers -- from the model's memory of
what a media library might contain.

deploy-scenario.sh now vendors listed shared/extensions modules into
.pi/extensions/_shared/. A tracked extension importing from shared/ cannot
resolve that path once installed outside the repository, so the deployed tree has
to be self-contained; this overwrites rather than merges, keeping the repository
authoritative. common.sh gains toml_list, using tomllib rather than more awk
because an array can span lines or carry comments.

verify-generated.sh checks that generated regions in tracked prompts match the
backend that generates them, and is wired into the pre-commit hook. This is
needed because of finding 22 below: the tool list has to be copied into the
prompt, and a copy drifts silently.

Two findings recorded in docs/pi-runtime-notes.md, both measured:

  21. An extension that fails to import is silent -- exit 0, empty stderr, no
      tools. A missing --extension path exits 1 with a clear message, but a
      module that throws while loading reports nothing. The agent then invented a
      complete library listing with plausible episode counts, quality and size.
      A later identical run said it had no data instead, so the failure is both
      silent and inconsistent.

  22. --system-prompt suppresses the tool list. The customPrompt branch returns
      before toolsList and guidelines are built, so promptSnippet and
      promptGuidelines are inert. The tools stay callable over the provider API,
      so tool use becomes a coin flip: one run in four looked at the library and
      three said they had not been given any results.

SYSTEM.phase3.md is staged alongside the deployed SYSTEM.md rather than replacing
it: profile.toml still describes the phase-0 configuration that is actually
running, and the live service is untouched.

scripts/deploy-scenario.sh

@@ -73,6 +73,34 @@ while IFS= read -r rel; do
   fi
 done < <(cd "$DIR/workspace" && find . -type f -printf '%P\n' | sort)
 
+# ---------------------------------------------------------------------------
+# 1b. Vendored shared extensions
+#
+# A tracked extension that imports from shared/extensions/ cannot resolve that
+# path once installed outside the repository. The listed modules are copied into
+# .pi/extensions/_shared/ so the deployed tree is self-contained, while the
+# repository stays the single source of truth: this overwrites, never merges.
+# ---------------------------------------------------------------------------
+SHARED="$(toml_list "$PROFILE" resources shared_extensions)"
+if [ -n "$SHARED" ]; then
+  while IFS= read -r mod; do
+    [ -n "$mod" ] || continue
+    src="$REPO_ROOT/shared/extensions/$mod"
+    [ -f "$src" ] || die "profile lists shared extension '$mod', which does not exist"
+    dst="$WORKSPACE/.pi/extensions/_shared/$mod"
+    if [ -f "$dst" ] && cmp -s "$src" "$dst"; then
+      continue
+    fi
+    CHANGES=$((CHANGES + 1))
+    if [ -f "$dst" ]; then info "  update  .pi/extensions/_shared/$mod"
+    else info "  vendor  .pi/extensions/_shared/$mod"; fi
+    if [ "$APPLY" -eq 1 ]; then
+      install -d -m 700 "$(dirname "$dst")"
+      install -m 600 "$src" "$dst"
+    fi
+  done < <(printf '%s\n' "$SHARED")
+fi
+
 # Executables under bin/ need the execute bit back.
 if [ "$APPLY" -eq 1 ] && [ -d "$DIR/workspace/bin" ]; then
   while IFS= read -r rel; do

scripts/lib/common.sh

@@ -64,6 +64,24 @@ toml_get() {
 
 # Print a unified diff between a tracked file and its live counterpart.
 # Returns 0 when identical, 1 when different or missing.
+# Read a TOML array of strings, one element per line.
+#
+# Uses Python's tomllib rather than more awk: an array can span lines, hold
+# comments, or be empty, and a hand-rolled parser that mostly works is worse
+# than none because it fails on the one profile that formats it differently.
+toml_list() {
+  local file="$1" section="$2" key="$3"
+  python3 - "$file" "$section" "$key" <<'PYEOF'
+import sys, tomllib, pathlib
+data = tomllib.loads(pathlib.Path(sys.argv[1]).read_text())
+value = data.get(sys.argv[2], {}).get(sys.argv[3]) or []
+if isinstance(value, str):
+    value = [value]
+for item in value:
+    print(item)
+PYEOF
+}
+
 diff_file() {
   local tracked="$1" live="$2" label="$3"
   if [ ! -f "$live" ]; then

scripts/verify-generated.sh

@@ -0,0 +1,64 @@
+#!/usr/bin/env bash
+# ---------------------------------------------------------------------------
+# Check that generated regions in tracked files are current.
+#
+# The tool list inside a scenario's system prompt is generated from the backend's
+# contracts module. It has to be generated rather than referenced because pi does
+# not put the tool list in the prompt when --system-prompt is used: the
+# customPrompt branch returns before `toolsList` is assembled, so `promptSnippet`
+# and `promptGuidelines` never reach the model.
+#
+# That makes the prompt a copy of the tool definitions, and a copy drifts. The
+# failure is quiet and bad: the model is told about a tool that no longer exists,
+# or not told about one that does, and answers from memory instead of asking.
+#
+# Run by the pre-commit hook. The backend path comes from profile.toml, so this
+# checks against the code that will actually serve the tools.
+# ---------------------------------------------------------------------------
+# shellcheck source=lib/common.sh
+. "$(dirname "${BASH_SOURCE[0]}")/lib/common.sh"
+
+FIX=0
+[ "${1:-}" = "--fix" ] && FIX=1
+
+STALE=0
+CHECKED=0
+
+while IFS= read -r profile; do
+  name="$(basename "$(dirname "$profile")")"
+  backend="$(toml_get "$profile" scenario backend)"
+  [ -n "$backend" ] || continue
+  [ -d "$backend" ] || { warn "$name: backend not found: $backend"; continue; }
+
+  while IFS= read -r prompt; do
+    [ -f "$prompt" ] || continue
+    grep -q 'BEGIN GENERATED TOOL LIST' "$prompt" || continue
+    CHECKED=$((CHECKED + 1))
+    rel="${prompt#"$REPO_ROOT"/}"
+    if PYTHONPATH="$backend" python3 - "$prompt" "$FIX" <<'PY'
+import pathlib, sys
+from curator import contracts
+path, fix = pathlib.Path(sys.argv[1]), sys.argv[2] == "1"
+text = path.read_text(encoding="utf-8")
+current = contracts.splice_tool_prose(text)
+if current == text:
+    sys.exit(0)
+if fix:
+    path.write_text(current, encoding="utf-8")
+    sys.exit(0)
+sys.exit(1)
+PY
+    then
+      if [ "$FIX" -eq 1 ]; then ok "$rel"; else ok "$rel"; fi
+    else
+      STALE=$((STALE + 1))
+      warn "stale generated region: $rel"
+    fi
+  done < <(find "$(dirname "$profile")/workspace" -name '*.md' -type f 2>/dev/null | sort)
+done < <(find "$REPO_ROOT/scenarios" -mindepth 2 -maxdepth 2 -name profile.toml | sort)
+
+if [ "$STALE" -gt 0 ]; then
+  die "$STALE generated region(s) out of date. Regenerate with:
+       scripts/verify-generated.sh --fix"
+fi
+ok "verify-generated: $CHECKED file(s) checked, all current"

scripts/verify-no-secrets.sh

@@ -117,4 +117,12 @@ MSG
 fi
 
 echo "verify-no-secrets: ${#FILES[@]} file(s) scanned, clean"
+
+# Generated regions must match the code that generates them. A stale tool list in
+# a system prompt tells the model about tools that do not exist, or hides ones
+# that do, and the symptom is an agent answering from memory.
+REPO="$(git rev-parse --show-toplevel)"
+if [ -x "$REPO/scripts/verify-generated.sh" ]; then
+  "$REPO/scripts/verify-generated.sh" || exit 1
+fi
 exit 0