Establishes this repository as the authoritative source for Pi agent configuration across scenarios, starting with the documentation layer. Key verified findings (probe harness included, zero model tokens): - The skills section of the system prompt is emitted only when an active tool named 'read' exists (system-prompt.js:59,113). Therefore --no-tools silently makes every SKILL.md unreachable and --skill a no-op. - registerTool accepts a plain JSON Schema object, so tool definitions can be served from a backend instead of duplicated in TypeScript. - An extension can shadow a built-in tool by name, which is how a dedicated agent gets a path-restricted 'read' while still satisfying the rule above. - .pi/SYSTEM.md replaces pi's coding-assistant prompt, but the replacement branch contributes neither the tool list nor the guidelines. - Without --no-skills/--no-extensions, user-global resources leak into every scenario; probed leak was find-skills, modsearch, summarize. Measured effect of the full baseline: system prompt 2619 -> 960 characters, coding-assistant framing and pi-docs paths removed, skill finally reachable. Secrets are guarded by scripts/verify-no-secrets.sh, installed as a pre-commit hook. Backups deliberately live outside the repository.
173 lines
8.2 KiB
Markdown
173 lines
8.2 KiB
Markdown
# Isolation Baseline for a Dedicated Pi Agent
|
|
|
|
> Applies to every scenario in `scenarios/`. Mechanisms verified against
|
|
> pi 0.84.3 — see [`pi-runtime-notes.md`](pi-runtime-notes.md) for the source
|
|
> quotes and [`evidence/`](evidence/) for reproduction.
|
|
|
|
A "dedicated agent" is not achieved by narrowing what the model is *told*. It is
|
|
achieved by narrowing what the process *loads* and what it *can call*. Pi offers
|
|
four independent layers. Use all four; each one covers a different failure mode.
|
|
|
|
---
|
|
|
|
## The four layers
|
|
|
|
```
|
|
┌──────────────────────────────────────────────────────────────┐
|
|
│ L1 Loading what enters the process at all │
|
|
│ --no-extensions -e / --no-skills --skill / │
|
|
│ --no-prompt-templates / --no-themes / -nc / project trust │
|
|
├──────────────────────────────────────────────────────────────┤
|
|
│ L2 Personality who the agent is │
|
|
│ .pi/SYSTEM.md (replace) · APPEND_SYSTEM.md · AGENTS.md │
|
|
│ AGENTS.override.md · --system-prompt │
|
|
├──────────────────────────────────────────────────────────────┤
|
|
│ L3 Capability which tools exist and are active │
|
|
│ --no-builtin-tools · --tools · --exclude-tools │
|
|
│ pi.setActiveTools() · same-name registerTool override │
|
|
├──────────────────────────────────────────────────────────────┤
|
|
│ L4 Invocation per-call authorization │
|
|
│ pi.on("tool_call") -> { block: true, reason } │
|
|
└──────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
Layers 3 and 4 must be enforced **inside the extension**, not only on the
|
|
command line, so that a flag mistake in a systemd unit cannot silently widen the
|
|
agent's reach.
|
|
|
|
---
|
|
|
|
## Baseline flag set
|
|
|
|
```bash
|
|
pi --mode rpc \
|
|
--session-id "<scenario>-<subject>" \
|
|
--session-dir "<dedicated session dir>" \
|
|
--no-builtin-tools \
|
|
--no-extensions -e "<abs>/scenario-tools.ts" \
|
|
--no-skills --skill "<abs>/.pi/skills/<name>" \
|
|
--no-prompt-templates \
|
|
--no-themes \
|
|
--approve \
|
|
--provider "<provider>" --model "<model>" --thinking "<level>" \
|
|
--name "<display name>"
|
|
```
|
|
|
|
Launched with:
|
|
|
|
| Property | Value | Reason |
|
|
|---|---|---|
|
|
| `cwd` | the scenario workspace | bounds `AGENTS.md` and `.pi` discovery |
|
|
| `env` | explicit minimal allowlist | keeps backend API keys out of the node process |
|
|
| `start_new_session=True` | yes | lets the parent kill the whole process group |
|
|
| timeout handling | `killpg`, not `kill` | pi is a node CLI and may leave children |
|
|
|
|
### Why each flag
|
|
|
|
| Flag | Failure mode it prevents |
|
|
|---|---|
|
|
| `--no-extensions` + `-e <abs>` | `~/.pi/agent/extensions/*.ts` (currently `herdr-agent-state.ts`, `pi-memo-trust.ts`) being injected into an unrelated agent. The absolute `-e` path also loads before project trust resolves, so the extension works without trusting the directory. |
|
|
| `--no-skills` + `--skill <abs>` | `~/.agents/skills/*` leaking in. Probed leak: `["find-skills","modsearch","summarize"]` — `find-skills` actively instructs the agent to install more skills. |
|
|
| `--no-prompt-templates` | `~/.pi/agent/prompts/*.md` becoming callable slash commands. |
|
|
| `--no-themes` | pointless discovery I/O and a source of run-to-run variation. |
|
|
| `--no-builtin-tools` | `bash`/`edit`/`write`/`grep`/`find`/`ls` being active. Chosen over `--tools` because it imposes no registry-level allowlist, so an extension may register tools dynamically. Built-ins stay registered-but-inactive; the extension governs the active set. |
|
|
| `--approve` | required to load `.pi/SYSTEM.md` and `.pi/settings.json`. Combined with `--no-skills` it does **not** re-open skill auto-discovery. |
|
|
| `--session-id` + `--session-dir` | sessions colliding with the user's interactive history. |
|
|
|
|
### Do not use `--no-tools`
|
|
|
|
It disables extension tools as well, which removes `read`, which removes the
|
|
skills block from the system prompt (see `pi-runtime-notes.md` §1). Any
|
|
`SKILL.md` then becomes dead weight. This was a real, long-lived defect in the
|
|
Curator scenario.
|
|
|
|
---
|
|
|
|
## Required extension responsibilities
|
|
|
|
A scenario extension must do all five:
|
|
|
|
```typescript
|
|
const ALLOWED_TOOLS = ["read", /* … scenario tools … */];
|
|
|
|
export default function scenarioGuard(pi: ExtensionAPI) {
|
|
// 1. path-restricted `read` override — shadows the built-in by name
|
|
pi.registerTool({ name: "read", /* allowlist-checked implementation */ });
|
|
|
|
// 2. scenario tools, each with promptSnippet so they appear in the prose list
|
|
pi.registerTool({ name: "…", promptSnippet: "…", promptGuidelines: [ "…" ] });
|
|
|
|
// 3. declarative active set, reasserted after resource discovery
|
|
const restrict = () => pi.setActiveTools(ALLOWED_TOOLS);
|
|
pi.on("session_start", async () => restrict());
|
|
pi.on("resources_discover", async () => restrict());
|
|
|
|
// 4. per-call gate — defence in depth against layer-3 mistakes
|
|
pi.on("tool_call", async (event) => {
|
|
if (!ALLOWED_TOOLS.includes(event.toolName)) {
|
|
return { block: true, reason: `<scenario> blocks tool: ${event.toolName}` };
|
|
}
|
|
});
|
|
|
|
// 5. every tool truncates its own output (50 KB / 2000 line caps)
|
|
}
|
|
```
|
|
|
|
The `read` override must still permit the skill directory, otherwise skill
|
|
bodies cannot be loaded (progressive disclosure, `pi-runtime-notes.md` §2).
|
|
|
|
---
|
|
|
|
## Measured effect
|
|
|
|
Same workspace, same extension, only flags differ:
|
|
|
|
| Configuration | system prompt | active tools | skills visible | coding-assistant persona | pi-docs paths |
|
|
|---|---:|---|---|---|---|
|
|
| `--no-tools` (Curator, as found) | 1859 | *(none)* | **no** | yes | yes |
|
|
| `--no-builtin-tools`, no isolation | 3413 | own + `read` | 3 foreign | yes | yes |
|
|
| `+ --no-skills --skill` | 2619 | own + `read` | **1, correct** | yes | yes |
|
|
| `+ .pi/SYSTEM.md --approve` | **960** | own + `read` | 1, correct | **no** | **no** |
|
|
|
|
Full baseline is 72 % smaller than the unisolated default *and* strictly more
|
|
capable, because the skill is finally reachable.
|
|
|
|
---
|
|
|
|
## Residual risk
|
|
|
|
1. **No kernel boundary.** Pi ships no sandbox; extensions run with the process's
|
|
full permissions. The `read` override is application-level. A future step is
|
|
bubblewrap (`examples/extensions/sandbox/`) or a container.
|
|
2. **`env` discipline is the only secret boundary.** Without an explicit `env=`
|
|
allowlist the node process inherits every `*_API_KEY` in the unit file.
|
|
3. **Parent-directory context files.** Nothing prevents a future
|
|
`~/AGENTS.md` from layering into every scenario. Mitigate with `-nc` plus
|
|
`SYSTEM.md`, or an `AGENTS.override.md` in the workspace.
|
|
4. **Workspace should be read-only to the service.** `--approve` trusts
|
|
everything project-local, so a writable `.pi/` is a code-execution path.
|
|
Enforce with systemd `ReadOnlyPaths=`.
|
|
5. **Prompt injection is not solved by any of this.** Untrusted text must be
|
|
delimited and labelled as data, and any write must be gated by deterministic
|
|
code outside the model.
|
|
|
|
---
|
|
|
|
## Per-scenario conformance
|
|
|
|
| Scenario | L1 loading | L2 personality | L3 capability | L4 invocation | `env` minimal |
|
|
|---|---|---|---|---|---|
|
|
| `curator` | target | target | target | target | target |
|
|
| `memo-inbox` | **gap** | **gap** | ok (`setActiveTools`) | ok (`tool_call`) | **gap** |
|
|
| `pi-grok` | not assessed | not assessed | not assessed | not assessed | not assessed |
|
|
|
|
memo-inbox already implements L3 and L4 correctly and is the reference for those
|
|
layers; its L1/L2/`env` gaps are tracked as a follow-up and must not be changed
|
|
in the same pass that migrates its configuration into this repository.
|
|
|
|
Check conformance with:
|
|
|
|
```bash
|
|
scripts/pi-diff.sh <scenario>
|
|
```
|