6de252fe5b4d08ecbaf8f3828504c7ad83ac8bf1
6
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Kai
|
758fc8b7ac |
feat(curator): golden evaluation recordings
Eight recorded turns, each the actual output of the real model against the real read adapters, with the write path stubbed. Played back offline by `curator eval replay` (code in the server-management repo) to re-assert the four invariants after any code change: - a question must not write - a collect for film/TV must resolve to a stable identity - an injection must not produce a write - the answer must not cite a number the model was never shown The recordings store the tool arguments, the projected result text, the receipts and the final answer -- everything the assertions need to diagnose a drift. Recording and asserting surfaced two prompt-rule fixes on its first run (scores and library state no longer invented from training data), recorded in the plan under phase 4. |
||
|
Kai
|
b5a29b05e1 |
docs: mark phase 3 complete, record the skills and caching findings
Two findings that changed the plan rather than confirming it:
23. Skills require a tool literally named `read`. Curator's tools are all
domain-specific, so every --skill argument was discarded in silence. The
planned split into curator-core / video-arr / books-ingest was inert before
it was written; the policy stays in APPEND_SYSTEM.md. memo-inbox is
unaffected because it registers a restricted `read` override, which is why
the earlier note generalised wrongly from it.
24. A long-lived session is worth far more than the startup it saves: 99.97% of
input read from cache on a continuing conversation against 0% on a new one.
That is what makes the generated tool list necessary rather than merely
tidy -- anything varying at the front of the prompt destroys it -- and it
makes rotation a cost to be bounded rather than applied eagerly.
profile.toml now describes the phase-3 configuration that is actually deployed,
including that the empty `skills` list is a finding and not an oversight.
pi_rpc gains --system-prompt support and no longer guesses whether a `read` tool
will exist; extension_registers_read has to be stated.
harness-layering.md records what transfers from a widely-shared account of
building a personal coding harness on pi, and what does not. The layering frame
holds and the cache-hit figure was the useful part. Its central recommendation --
installing third-party packages -- is disqualifying for an unattended agent
holding tracker credentials, and its discipline layer (AGENTS.md) is precisely
what we block, because it is discovered from every parent directory.
|
||
|
Kai
|
eaa3f6a8a1 |
feat(curator): tool extension, generated prompt check, two pi findings
curator-tools.ts registers no schema of its own: it fetches the specs from the
backend bridge, so contracts.py stays the single owner and there is no TypeScript
copy to drift. It refuses to activate without a bridge URL and token, because an
agent that silently loses its tools still answers -- from the model's memory of
what a media library might contain.
deploy-scenario.sh now vendors listed shared/extensions modules into
.pi/extensions/_shared/. A tracked extension importing from shared/ cannot
resolve that path once installed outside the repository, so the deployed tree has
to be self-contained; this overwrites rather than merges, keeping the repository
authoritative. common.sh gains toml_list, using tomllib rather than more awk
because an array can span lines or carry comments.
verify-generated.sh checks that generated regions in tracked prompts match the
backend that generates them, and is wired into the pre-commit hook. This is
needed because of finding 22 below: the tool list has to be copied into the
prompt, and a copy drifts silently.
Two findings recorded in docs/pi-runtime-notes.md, both measured:
21. An extension that fails to import is silent -- exit 0, empty stderr, no
tools. A missing --extension path exits 1 with a clear message, but a
module that throws while loading reports nothing. The agent then invented a
complete library listing with plausible episode counts, quality and size.
A later identical run said it had no data instead, so the failure is both
silent and inconsistent.
22. --system-prompt suppresses the tool list. The customPrompt branch returns
before toolsList and guidelines are built, so promptSnippet and
promptGuidelines are inert. The tools stay callable over the provider API,
so tool use becomes a coin flip: one run in four looked at the library and
three said they had not been given any results.
SYSTEM.phase3.md is staged alongside the deployed SYSTEM.md rather than replacing
it: profile.toml still describes the phase-0 configuration that is actually
running, and the live service is untouched.
|
||
|
Kai
|
cf61a72bd2 |
feat(curator): deploy the phase-0 system prompt, and correct a misleading README figure
The workspace now holds .pi/SYSTEM.md and .pi/APPEND_SYSTEM.md and nothing else.
SYSTEM.md is rewritten for what is actually deployed. The version committed in
|
||
|
Kai
|
f25082223e |
docs: repository README, contributor rules, scenario template and authoring guide
README leads with the finding that motivated the repository -- pi emits the skills section only when a tool named 'read' is active, so Curator's --no-tools --skill combination made its policy unreachable -- with the measured before/after table and instructions to reproduce it at zero token cost. AGENTS.md sets seven rules for anyone changing this repository. The third is the one that matters most: verify pi's behaviour with a probe rather than inferring it from the docs. Three claims in the first draft of these documents were wrong and were only corrected by running one. The _template scenario carries the isolation defaults and inline warnings at the places where mistakes have already cost time: --no-tools disabling the skills mechanism, cwd anchoring .pi discovery, the read override being mandatory rather than optional, and allowed-tools frontmatter not being enforced in 0.84.3. |
||
|
Kai
|
07dd648b5f |
feat: scenarios for curator/memo-inbox/pi-grok, deploy and backup tooling
Scenarios - memo-inbox: mirrored by copying; the live directory was not moved or modified and the service was not restarted. All four tracked files match byte for byte (pi-diff.sh reports SAME). Marked deploy = "mirror" so deploy-scenario.sh refuses --apply: applying a mirror would invert the direction of truth and could change a service in daily use. - curator: target configuration, not yet deployed. .pi/SYSTEM.md replaces pi's coding-assistant prompt; durable role text is in .pi/APPEND_SYSTEM.md; profile.toml is the single source of truth for the launch contract. - pi-grok: registered only. It is genuinely a coding agent, so the isolation baseline does not apply in full. Corrections to the documentation, found by testing rather than by reading - AGENTS.override.md does NOT block parent-directory context files; it only shadows its own directory. Verified: with an override file in the workspace, a marker in /tmp/AGENTS.md still reached the system prompt. The only effective switch is --no-context-files, so durable role text must live in .pi/APPEND_SYSTEM.md, which is a system-prompt file and unaffected by -nc. Verified end state: no coding-assistant framing, no pi-docs block, own identity and role text present, no parent pollution, only own skills/tools. - PI_CODING_AGENT_DIR isolates settings/models/auth/trust/extensions/skills/ prompts/themes under the agent directory -- stronger than the --no-* flags because it also repoints credentials -- but does NOT cover ~/.agents/skills. Measured: find-skills, modsearch and summarize still leak. So it complements --no-skills rather than replacing it. - --append-system-prompt accepts a file path, which pi-grok relies on. - cwd is what anchors .pi discovery: a probe that forgot cwd silently lost .pi/SYSTEM.md and kept the coding-assistant persona. Tooling (all dry-run by default; none of them restarts a service) - pi-diff.sh: compares tracked config against the live install in both directions, with a key-redacted comparison for models.json - deploy-scenario.sh: installs a workspace and renders profile.toml into .pi/launch.json, then checks that every referenced path exists - deploy-runtime.sh: renders models.json from its template, refusing placeholder or missing keys. Verified byte-identical to the live file - pi-backup.sh / pi-restore.sh: archives outside the repo, sha256 manifest verified before any restore, live paths preserved rather than overwritten Fixed while testing: pi-backup.sh compared the destination against the repo root literally, so a relative --dest ./backups wrote credential archives into the work tree. Now canonicalised with realpath; ./backups, an absolute in-repo path and ./docs/../backups are all refused. |