feat(curator): tool extension, generated prompt check, two pi findings

curator-tools.ts registers no schema of its own: it fetches the specs from the
backend bridge, so contracts.py stays the single owner and there is no TypeScript
copy to drift. It refuses to activate without a bridge URL and token, because an
agent that silently loses its tools still answers -- from the model's memory of
what a media library might contain.

deploy-scenario.sh now vendors listed shared/extensions modules into
.pi/extensions/_shared/. A tracked extension importing from shared/ cannot
resolve that path once installed outside the repository, so the deployed tree has
to be self-contained; this overwrites rather than merges, keeping the repository
authoritative. common.sh gains toml_list, using tomllib rather than more awk
because an array can span lines or carry comments.

verify-generated.sh checks that generated regions in tracked prompts match the
backend that generates them, and is wired into the pre-commit hook. This is
needed because of finding 22 below: the tool list has to be copied into the
prompt, and a copy drifts silently.

Two findings recorded in docs/pi-runtime-notes.md, both measured:

  21. An extension that fails to import is silent -- exit 0, empty stderr, no
      tools. A missing --extension path exits 1 with a clear message, but a
      module that throws while loading reports nothing. The agent then invented a
      complete library listing with plausible episode counts, quality and size.
      A later identical run said it had no data instead, so the failure is both
      silent and inconsistent.

  22. --system-prompt suppresses the tool list. The customPrompt branch returns
      before toolsList and guidelines are built, so promptSnippet and
      promptGuidelines are inert. The tools stay callable over the provider API,
      so tool use becomes a coin flip: one run in four looked at the library and
      three said they had not been given any results.

SYSTEM.phase3.md is staged alongside the deployed SYSTEM.md rather than replacing
it: profile.toml still describes the phase-0 configuration that is actually
running, and the live service is untouched.
This commit is contained in:
Kai
2026-08-28 00:35:04 -07:00
parent b5564e3c60
commit eaa3f6a8a1
7 changed files with 359 additions and 0 deletions
+73
View File
@@ -428,6 +428,79 @@ one. Real isolation requires bubblewrap / container / micro-VM; pi ships
---
## 21. An extension that fails to import is silent, and the agent then invents facts
Two failure modes, only one of which is reported:
| condition | exit | stderr |
|---|---|---|
| `--extension` path does not exist | 1 | `Failed to load extension ...` plus a hint |
| extension exists but throws while importing | **0** | **empty** |
The second registers no tools and says nothing. Measured with an extension whose
only defect was importing `ExtensionAPI` from `@getpi/pi` instead of
`@earendil-works/pi-coding-agent`.
What the agent did with no tools, asked which versions of a series were in the
library:
```
《权力的游戏》(Game of Thrones1 个版本:
- 4K 实例,已跟踪
- 8 季,共 73 集,文件已齐(73/73)
- 画质:2160p WEB-DL
- 占用空间:624.7 GB
```
Every line is fabricated, and the size happens to be close to the real figure.
A later run of the same prompt said it had no data at all. So the failure is not
only silent but inconsistent: sometimes a refusal, sometimes a confident and
detailed invention.
**Therefore**: do not treat process exit status as evidence that the tools
loaded. Curator detects activation at the bridge, which the extension must call
to obtain its tool list, and refuses to proceed without it
(`curator/agent_api.py`, `wait_for_activation`).
## 22. `--system-prompt` suppresses the tool list, so `promptSnippet` is inert
`dist/core/system-prompt.js` builds `toolsList` from `promptSnippet` and
`guidelines` from `promptGuidelines` — but the `customPrompt` branch returns
before either is assembled:
```js
if (customPrompt) {
let prompt = customPrompt;
if (appendSection) prompt += appendSection;
// context files, then skills (only when a `read` tool is active)
prompt += `\nCurrent working directory: ${promptCwd}\n`;
return prompt; // toolsList and guidelines never appear
}
```
The tools remain callable: their schemas still go to the provider as tool
definitions. The model simply is not told in prose that it has them.
Measured effect on the same question, four runs, extension loading correctly:
| runs | behaviour |
|---|---|
| 1 | called `query_library`, answered from the result |
| 3 | called nothing, answered "I was not given any library results" |
After writing the tool list into the system prompt itself, four different
prompts each used the right tools and none answered unaided.
**Therefore**: an agent using `--system-prompt` must enumerate its own tools in
that prompt. Curator generates the section from the same specs the bridge serves
(`contracts.render_tool_prose`) and `scripts/verify-generated.sh` fails the
commit when the two drift.
This also means `promptGuidelines` cannot be relied on for safety-relevant
instructions under `--system-prompt`; they have to be in the prompt text.
---
## Re-verification
```bash