docs: pi 0.84.3 runtime mechanics, isolation baseline, personality layering, gateway patterns
Establishes this repository as the authoritative source for Pi agent configuration across scenarios, starting with the documentation layer. Key verified findings (probe harness included, zero model tokens): - The skills section of the system prompt is emitted only when an active tool named 'read' exists (system-prompt.js:59,113). Therefore --no-tools silently makes every SKILL.md unreachable and --skill a no-op. - registerTool accepts a plain JSON Schema object, so tool definitions can be served from a backend instead of duplicated in TypeScript. - An extension can shadow a built-in tool by name, which is how a dedicated agent gets a path-restricted 'read' while still satisfying the rule above. - .pi/SYSTEM.md replaces pi's coding-assistant prompt, but the replacement branch contributes neither the tool list nor the guidelines. - Without --no-skills/--no-extensions, user-global resources leak into every scenario; probed leak was find-skills, modsearch, summarize. Measured effect of the full baseline: system prompt 2619 -> 960 characters, coding-assistant framing and pi-docs paths removed, skill finally reachable. Secrets are guarded by scripts/verify-no-secrets.sh, installed as a pre-commit hook. Backups deliberately live outside the repository.
This commit is contained in:
Kai
@@ -0,0 +1,359 @@
|
||||
# Pi Runtime Notes
|
||||
|
||||
> Verified against **pi 0.84.3** on Debian 13 / node v22.23.2, 2026-08-27.
|
||||
> Every claim below is either quoted from `dist/` source or reproduced by the
|
||||
> probe harness in [`evidence/probe-harness/`](evidence/probe-harness/).
|
||||
> Results: [`evidence/2026-08-27-isolation-probe.md`](evidence/2026-08-27-isolation-probe.md).
|
||||
|
||||
This file records behaviour that the official docs either do not state or state
|
||||
only in passing, but which determines whether a dedicated Pi agent works at all.
|
||||
Re-verify after every `pi update`.
|
||||
|
||||
---
|
||||
|
||||
## 1. The skills section requires an active `read` tool
|
||||
|
||||
`dist/core/system-prompt.js`:
|
||||
|
||||
```javascript
|
||||
const hasRead = tools.includes("read");
|
||||
// Append skills section (only if read tool is available)
|
||||
if (hasRead && skills.length > 0) {
|
||||
prompt += formatSkillsForPrompt(skills);
|
||||
}
|
||||
```
|
||||
|
||||
`tools` is the **active** tool-name list, passed as `selectedTools` from
|
||||
`agent-session.js:_rebuildSystemPrompt(this.getActiveToolNames())`.
|
||||
|
||||
**Consequence:** `--no-tools` silently removes the entire skills block.
|
||||
Any `SKILL.md` is then unreachable, and `--skill <path>` becomes a no-op.
|
||||
|
||||
Probe evidence (`--no-tools` vs `--no-builtin-tools`):
|
||||
|
||||
| Flags | `available_skills` in prompt | prompt length |
|
||||
|---|---|---|
|
||||
| `--no-tools` | **false** | 1859 |
|
||||
| `--no-builtin-tools` | true | 3413 |
|
||||
|
||||
This was a live defect in the Curator scenario: it ran `--no-tools --skill …`
|
||||
for its whole lifetime, so its 64-line media policy never reached the model.
|
||||
|
||||
## 2. Skill injection is progressive, not full text
|
||||
|
||||
`formatSkillsForPrompt` emits only `<name>`, `<description>` and `<location>`
|
||||
per skill, plus the instruction "Use the read tool to load a skill's file when
|
||||
the task matches its description."
|
||||
|
||||
So the body of a `SKILL.md` costs nothing up front but **is only ever read via
|
||||
the `read` tool**. A skill body is unreachable without a working `read`.
|
||||
|
||||
Corollary: if you override `read` with a path-restricted implementation, the
|
||||
allowlist must include the skill directory or skill bodies cannot be loaded.
|
||||
|
||||
## 3. `--skill` does not double-load an auto-discovered skill
|
||||
|
||||
`dist/core/skills.js:loadSkills()` dedupes twice — by canonical realpath, and
|
||||
by skill `name`. Defaults are added first, so on a name collision the
|
||||
**auto-discovered** skill wins and a collision diagnostic is emitted.
|
||||
Same file via symlink is skipped silently.
|
||||
|
||||
## 4. Tool gating semantics
|
||||
|
||||
| Flag | registry filter | initial active set | extension tools |
|
||||
|---|---|---|---|
|
||||
| *(none)* | none | `settings.defaultTools ?? [read,bash,edit,write]` | all activated |
|
||||
| `--tools a,b` | **strict allowlist across every source** | `[a,b]` | must be named to exist |
|
||||
| `--no-tools` | empty allowlist | `[]` | **also killed** |
|
||||
| `--no-builtin-tools` | none | `[]` | **all activated** |
|
||||
| `--exclude-tools x` | removes `x` | filtered | applies to all sources |
|
||||
|
||||
From `dist/core/agent-session.js`:
|
||||
|
||||
```javascript
|
||||
const isAllowedTool = (name) =>
|
||||
(!allowedToolNames || allowedToolNames.has(name)) && !excludedToolNames?.has(name);
|
||||
```
|
||||
|
||||
Under `--no-builtin-tools` the built-ins stay **registered but inactive**, so an
|
||||
extension can re-enable them with `pi.setActiveTools([...])`. Under `--no-tools`
|
||||
they were never registered and cannot be revived.
|
||||
|
||||
Probed: `--no-builtin-tools` + one extension →
|
||||
`ACTIVE_TOOLS=["probe_plain_schema","read"]`, while `bash/edit/write/grep/find/ls`
|
||||
remain listed as `[builtin]` and inactive.
|
||||
|
||||
## 5. `registerTool` accepts a plain JSON Schema object — verified
|
||||
|
||||
Declared type is TypeBox `TSchema`, but a raw JSON Schema object works:
|
||||
|
||||
```typescript
|
||||
pi.registerTool({
|
||||
name: "probe_plain_schema",
|
||||
parameters: { type: "object", properties: { q: { type: "string" } }, required: ["q"] } as any,
|
||||
...
|
||||
});
|
||||
```
|
||||
|
||||
Probe result: the tool appears as `probe_plain_schema[cli]` and is active.
|
||||
|
||||
**Why this matters:** a backend can serve its tool definitions as JSON Schema
|
||||
over HTTP and a generic bridge extension can register them at runtime, so the
|
||||
schema has exactly one source of truth. See `shared/extensions/`.
|
||||
|
||||
Note `--tools` is a *registry-level* allowlist, so dynamically registered tools
|
||||
must either be named in `--tools` or you must use `--no-builtin-tools`
|
||||
(no allowlist) and let the extension govern the active set.
|
||||
|
||||
## 6. An extension can override a built-in tool by name — verified
|
||||
|
||||
Registering `name: "read"` replaces the built-in in the registry: the probe
|
||||
showed `read[cli]` instead of `read[builtin]` while other built-ins kept
|
||||
`[builtin]`. This is the supported way to give an agent a path-restricted
|
||||
`read` while still satisfying the requirement in §1.
|
||||
|
||||
`memo-guard.ts` in the memo-inbox scenario has used this in production.
|
||||
|
||||
## 7. Only tools with `promptSnippet` appear in the human-readable tool list
|
||||
|
||||
`dist/core/system-prompt.js`: "A tool appears in Available tools only when the
|
||||
caller provides a one-line snippet."
|
||||
|
||||
The tool *schemas* are always sent through the provider's tool-calling API, so
|
||||
an omitted `promptSnippet` does not break invocation — but the model loses the
|
||||
prose overview. Always set `promptSnippet`, and `promptGuidelines` where useful.
|
||||
|
||||
## 8. `.pi/SYSTEM.md` replaces the default prompt; the replacement branch omits tools and guidelines
|
||||
|
||||
Default prompt (`system-prompt.js:82`) begins:
|
||||
|
||||
> You are an expert coding assistant operating inside pi, a coding agent harness.
|
||||
> You help users by reading files, executing commands, editing code, and writing new files.
|
||||
|
||||
and ends with a block of **absolute paths to pi's own README / docs / examples**
|
||||
plus "read the docs and examples, and follow .md cross-references before
|
||||
implementing".
|
||||
|
||||
For a non-coding agent this is not merely noise: it is a ready-made escalation
|
||||
path for prompt injection.
|
||||
|
||||
The `customPrompt` branch (`system-prompt.js:13-33`) concatenates only:
|
||||
|
||||
```
|
||||
SYSTEM.md → APPEND_SYSTEM.md → <project_context>(AGENTS.md) → skills section → cwd
|
||||
```
|
||||
|
||||
It does **not** include `toolsList` or `Guidelines`. A replacement `SYSTEM.md`
|
||||
must therefore carry its own tool overview and output discipline.
|
||||
|
||||
`.pi/SYSTEM.md` lives under `.pi/`, so it requires **project trust** — pass
|
||||
`--approve` in non-interactive modes.
|
||||
|
||||
Probe evidence:
|
||||
|
||||
| Combination | coding-assistant text | pi-docs block | SYSTEM.md applied | prompt length |
|
||||
|---|---|---|---|---|
|
||||
| isolation flags only | true | true | false | 2619 |
|
||||
| `+ .pi/SYSTEM.md + --approve` | **false** | **false** | **true** | **960** |
|
||||
|
||||
## 9. Context files are loaded from every parent directory
|
||||
|
||||
Load order: `~/.pi/agent/AGENTS.md`, then each parent directory walking up from
|
||||
cwd, then cwd. `AGENTS.override.md` replaces `AGENTS.md`/`CLAUDE.md` **for that
|
||||
directory only**; other directories still layer normally.
|
||||
|
||||
So a stray `~/AGENTS.md` or `~/pi-workspaces/AGENTS.md` silently contaminates
|
||||
every scenario rooted below it. Neither existed on this host as of 2026-08-27,
|
||||
but nothing prevents one from appearing. Use `-nc` plus an explicit
|
||||
`SYSTEM.md`, or an `AGENTS.override.md` in the workspace, to make the
|
||||
personality deterministic. Probed: `-nc` removes the AGENTS.md marker.
|
||||
|
||||
## 10. Project trust gates `.pi/`, and CLI `-e` bypasses it
|
||||
|
||||
Trust is required when the directory contains `.pi/settings.json`,
|
||||
`.pi/{extensions,skills,prompts,themes}`, `.pi/SYSTEM.md`,
|
||||
`.pi/APPEND_SYSTEM.md`, or a project `.agents/skills`. A bare `.pi` does not
|
||||
count.
|
||||
|
||||
Always loaded regardless of trust: `AGENTS.override.md`, `AGENTS.md`,
|
||||
`CLAUDE.md`, user/global extensions, and **CLI `-e` extensions** — the last one
|
||||
is deliberate so they can handle the `project_trust` event.
|
||||
|
||||
Non-interactive modes (`-p`, `--mode json`, `--mode rpc`) never prompt; they use
|
||||
`defaultProjectTrust` (`ask` default / `always` / `never`) unless `--approve` /
|
||||
`--no-approve` overrides for the run.
|
||||
|
||||
**Design consequence:** loading a scenario extension with an absolute
|
||||
`-e /path/to/ext.ts` avoids the trust question entirely, which is the most
|
||||
robust option for a systemd-managed gateway.
|
||||
|
||||
## 11. User-global resources leak into every scenario
|
||||
|
||||
Probed with no isolation flags, from an unrelated workspace, the skills list was:
|
||||
|
||||
```
|
||||
PROBE_SKILLNAMES=["find-skills","modsearch","summarize"]
|
||||
```
|
||||
|
||||
Those come from `~/.agents/skills/`. `find-skills` in particular instructs the
|
||||
agent to discover and install further skills — an unwanted capability surface
|
||||
for a narrow-purpose agent. `~/.pi/agent/extensions/*.ts` leak the same way.
|
||||
|
||||
With `--no-skills --skill <abs>` the list becomes exactly `["probe-skill"]`.
|
||||
|
||||
## 12. No output-schema flag; use a terminating tool with constrained sampling
|
||||
|
||||
There is no `--output-schema` / `--response-format`. Structured output is
|
||||
achieved with a tool whose `parameters` is the desired schema:
|
||||
|
||||
```typescript
|
||||
{
|
||||
name: "emit_result",
|
||||
parameters: <schema>,
|
||||
constrainedSampling: { type: "json_schema", strict: "prefer" },
|
||||
async execute(_id, params) {
|
||||
return { content: [...], details: params, terminate: true };
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`ConstrainedSamplingConfig`:
|
||||
|
||||
```typescript
|
||||
| { type: "json_schema"; strict: "prefer" | "require" }
|
||||
| { type: "grammar"; variants: Partial<Record<"openai_lark" | "openai_regex", string>> }
|
||||
```
|
||||
|
||||
`strict: "prefer"` falls back gracefully; `"require"` fails the request if the
|
||||
provider cannot honour it. `terminate: true` ends the turn only when *every*
|
||||
finalized tool result in the batch sets it.
|
||||
|
||||
This replaces regex-scraping JSON out of prose.
|
||||
|
||||
## 13. Tools must truncate their own output
|
||||
|
||||
Built-in caps are **50 KB** and **2000 lines**. Helpers exported for this:
|
||||
`truncateHead`, `truncateTail`, `truncateLine`, `formatSize`,
|
||||
`DEFAULT_MAX_BYTES`, `DEFAULT_MAX_LINES`.
|
||||
|
||||
Errors: **throw** from `execute` to set `isError: true`. Returning an
|
||||
error-shaped object does not mark the call failed.
|
||||
|
||||
`AgentToolResult.content` goes to the model; `details` is persisted in the
|
||||
session but **not** sent to the model.
|
||||
|
||||
## 14. Per-call gating hook
|
||||
|
||||
```typescript
|
||||
pi.on("tool_call", async (event) => {
|
||||
if (!ALLOWED.includes(event.toolName)) {
|
||||
return { block: true, reason: "…" };
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
`ToolCallEventResult = { block?, reason?, terminate? }`. To rewrite arguments,
|
||||
mutate `event.input` in place. Later handlers observe earlier mutations and no
|
||||
re-validation happens afterwards.
|
||||
|
||||
Combined with `pi.setActiveTools()` on `session_start` and
|
||||
`resources_discover`, this gives two independent layers of capability control
|
||||
that survive a mistake in the CLI flags.
|
||||
|
||||
## 15. RPC mode is strict JSONL and is meant to be long-lived
|
||||
|
||||
`--mode rpc`: commands as JSON objects on stdin, one per line; responses and
|
||||
events as JSON lines on stdout.
|
||||
|
||||
> Split records on `\n` only. Accept optional `\r\n` by stripping a trailing
|
||||
> `\r`. Node `readline` is **not** protocol-compliant because it also splits on
|
||||
> `U+2028`/`U+2029`, which are valid inside JSON strings.
|
||||
|
||||
33 commands, including `prompt`, `steer`, `follow_up`, `abort`, `new_session`,
|
||||
`get_state`, `set_model`, `set_thinking_level`, `compact`,
|
||||
`set_auto_compaction`, `get_session_stats`, `switch_session`, `fork`,
|
||||
`get_entries`.
|
||||
|
||||
`message_update` carries **deltas only** — no cumulative `message`, no
|
||||
`partial`. Treat `message_end.message` as authoritative.
|
||||
|
||||
There is **no** command to inject a tool result from the host. Host-side
|
||||
capabilities must be exposed as an extension tool that calls back out.
|
||||
|
||||
Probing `get_state` alone starts the agent, fires `session_start`, and exits
|
||||
without ever contacting the model — a zero-cost way to inspect the resolved
|
||||
system prompt and tool set.
|
||||
|
||||
## 16. Usage and cost are recorded per assistant message
|
||||
|
||||
Every `AssistantMessage` carries:
|
||||
|
||||
```typescript
|
||||
Usage { input, output, cacheRead, cacheWrite, totalTokens,
|
||||
cost: { input, output, cacheRead, cacheWrite, total } }
|
||||
```
|
||||
|
||||
`ToolResultMessage`, `CompactionEntry` and `BranchSummaryEntry` carry an
|
||||
optional `usage` for nested LLM work. Live equivalent: RPC
|
||||
`get_session_stats`, which also returns
|
||||
`contextUsage { tokens, contextWindow, percent }`.
|
||||
|
||||
Observability therefore needs no extra instrumentation — read the session JSONL
|
||||
or subscribe to `message_update`.
|
||||
|
||||
## 17. Sessions never rotate or expire
|
||||
|
||||
`--session-id <id>` is an idempotent "use this project session, create if
|
||||
missing" primitive (mutually exclusive with `--session`, `--continue`,
|
||||
`--resume`; validated against `^[A-Za-z0-9](?:[A-Za-z0-9._-]*[A-Za-z0-9])?$`).
|
||||
|
||||
There is no built-in TTL, rotation, size cap or pruning. Files grow
|
||||
monotonically until deleted by hand.
|
||||
|
||||
## 18. Auto-compaction rarely triggers on large-context models
|
||||
|
||||
Condition: `contextTokens > contextWindow - reserveTokens`.
|
||||
|
||||
```json
|
||||
{ "compaction": { "enabled": true, "reserveTokens": 16384, "keepRecentTokens": 20000 } }
|
||||
```
|
||||
|
||||
With a 1,050,000-token context window the threshold is ~1.03 M tokens, so
|
||||
latency and cost degrade for a very long time before compaction ever fires.
|
||||
A gateway must implement its own rotation policy — see
|
||||
[`gateway-patterns.md`](gateway-patterns.md).
|
||||
|
||||
## 19. `allowed-tools` in SKILL.md frontmatter is not enforced
|
||||
|
||||
`dist/core/skills.js:loadSkillFromFile` consumes only `name`, `description` and
|
||||
`disable-model-invocation`. `allowed-tools` is documented as experimental and is
|
||||
**not read** in 0.84.3.
|
||||
|
||||
The memo-inbox scenario declares `allowed-tools:` in its `SKILL.md`; the real
|
||||
enforcement is the `ALLOWED_TOOLS` array plus `setActiveTools` and the
|
||||
`tool_call` hook inside `memo-guard.ts`. Do not rely on the frontmatter field
|
||||
for security.
|
||||
|
||||
## 20. There is no sandbox
|
||||
|
||||
`docs/security.md`: "Pi does not include a built-in sandbox. Built-in tools can
|
||||
read files, write files, edit files, and run shell commands with the permissions
|
||||
of the pi process. Extensions are TypeScript modules that run with the same
|
||||
permissions."
|
||||
|
||||
A path-restricted `read` override is an application-level boundary, not a kernel
|
||||
one. Real isolation requires bubblewrap / container / micro-VM; pi ships
|
||||
`examples/extensions/sandbox/` (bubblewrap, `.pi/sandbox.json`) and
|
||||
`examples/extensions/gondolin/` (micro-VM) as starting points.
|
||||
|
||||
---
|
||||
|
||||
## Re-verification
|
||||
|
||||
```bash
|
||||
cd docs/evidence/probe-harness
|
||||
./collect-evidence.sh > ../$(date -u +%Y-%m-%d)-isolation-probe.md
|
||||
```
|
||||
|
||||
The harness costs zero model tokens: it sends a single `get_state` over RPC and
|
||||
reads the resolved system prompt from `session_start`.
|
||||
Reference in New Issue
Block a user