Kai
eaa3f6a8a1
curator-tools.ts registers no schema of its own: it fetches the specs from the
backend bridge, so contracts.py stays the single owner and there is no TypeScript
copy to drift. It refuses to activate without a bridge URL and token, because an
agent that silently loses its tools still answers -- from the model's memory of
what a media library might contain.
deploy-scenario.sh now vendors listed shared/extensions modules into
.pi/extensions/_shared/. A tracked extension importing from shared/ cannot
resolve that path once installed outside the repository, so the deployed tree has
to be self-contained; this overwrites rather than merges, keeping the repository
authoritative. common.sh gains toml_list, using tomllib rather than more awk
because an array can span lines or carry comments.
verify-generated.sh checks that generated regions in tracked prompts match the
backend that generates them, and is wired into the pre-commit hook. This is
needed because of finding 22 below: the tool list has to be copied into the
prompt, and a copy drifts silently.
Two findings recorded in docs/pi-runtime-notes.md, both measured:
21. An extension that fails to import is silent -- exit 0, empty stderr, no
tools. A missing --extension path exits 1 with a clear message, but a
module that throws while loading reports nothing. The agent then invented a
complete library listing with plausible episode counts, quality and size.
A later identical run said it had no data instead, so the failure is both
silent and inconsistent.
22. --system-prompt suppresses the tool list. The customPrompt branch returns
before toolsList and guidelines are built, so promptSnippet and
promptGuidelines are inert. The tools stay callable over the provider API,
so tool use becomes a coin flip: one run in four looked at the library and
three said they had not been given any results.
SYSTEM.phase3.md is staged alongside the deployed SYSTEM.md rather than replacing
it: profile.toml still describes the phase-0 configuration that is actually
running, and the live service is untouched.
65 lines
2.4 KiB
Bash
Executable File
65 lines
2.4 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# ---------------------------------------------------------------------------
|
|
# Check that generated regions in tracked files are current.
|
|
#
|
|
# The tool list inside a scenario's system prompt is generated from the backend's
|
|
# contracts module. It has to be generated rather than referenced because pi does
|
|
# not put the tool list in the prompt when --system-prompt is used: the
|
|
# customPrompt branch returns before `toolsList` is assembled, so `promptSnippet`
|
|
# and `promptGuidelines` never reach the model.
|
|
#
|
|
# That makes the prompt a copy of the tool definitions, and a copy drifts. The
|
|
# failure is quiet and bad: the model is told about a tool that no longer exists,
|
|
# or not told about one that does, and answers from memory instead of asking.
|
|
#
|
|
# Run by the pre-commit hook. The backend path comes from profile.toml, so this
|
|
# checks against the code that will actually serve the tools.
|
|
# ---------------------------------------------------------------------------
|
|
# shellcheck source=lib/common.sh
|
|
. "$(dirname "${BASH_SOURCE[0]}")/lib/common.sh"
|
|
|
|
FIX=0
|
|
[ "${1:-}" = "--fix" ] && FIX=1
|
|
|
|
STALE=0
|
|
CHECKED=0
|
|
|
|
while IFS= read -r profile; do
|
|
name="$(basename "$(dirname "$profile")")"
|
|
backend="$(toml_get "$profile" scenario backend)"
|
|
[ -n "$backend" ] || continue
|
|
[ -d "$backend" ] || { warn "$name: backend not found: $backend"; continue; }
|
|
|
|
while IFS= read -r prompt; do
|
|
[ -f "$prompt" ] || continue
|
|
grep -q 'BEGIN GENERATED TOOL LIST' "$prompt" || continue
|
|
CHECKED=$((CHECKED + 1))
|
|
rel="${prompt#"$REPO_ROOT"/}"
|
|
if PYTHONPATH="$backend" python3 - "$prompt" "$FIX" <<'PY'
|
|
import pathlib, sys
|
|
from curator import contracts
|
|
path, fix = pathlib.Path(sys.argv[1]), sys.argv[2] == "1"
|
|
text = path.read_text(encoding="utf-8")
|
|
current = contracts.splice_tool_prose(text)
|
|
if current == text:
|
|
sys.exit(0)
|
|
if fix:
|
|
path.write_text(current, encoding="utf-8")
|
|
sys.exit(0)
|
|
sys.exit(1)
|
|
PY
|
|
then
|
|
if [ "$FIX" -eq 1 ]; then ok "$rel"; else ok "$rel"; fi
|
|
else
|
|
STALE=$((STALE + 1))
|
|
warn "stale generated region: $rel"
|
|
fi
|
|
done < <(find "$(dirname "$profile")/workspace" -name '*.md' -type f 2>/dev/null | sort)
|
|
done < <(find "$REPO_ROOT/scenarios" -mindepth 2 -maxdepth 2 -name profile.toml | sort)
|
|
|
|
if [ "$STALE" -gt 0 ]; then
|
|
die "$STALE generated region(s) out of date. Regenerate with:
|
|
scripts/verify-generated.sh --fix"
|
|
fi
|
|
ok "verify-generated: $CHECKED file(s) checked, all current"
|