docs: mark phase 3 complete, record the skills and caching findings

Two findings that changed the plan rather than confirming it:

  23. Skills require a tool literally named `read`. Curator's tools are all
      domain-specific, so every --skill argument was discarded in silence. The
      planned split into curator-core / video-arr / books-ingest was inert before
      it was written; the policy stays in APPEND_SYSTEM.md. memo-inbox is
      unaffected because it registers a restricted `read` override, which is why
      the earlier note generalised wrongly from it.

  24. A long-lived session is worth far more than the startup it saves: 99.97% of
      input read from cache on a continuing conversation against 0% on a new one.
      That is what makes the generated tool list necessary rather than merely
      tidy -- anything varying at the front of the prompt destroys it -- and it
      makes rotation a cost to be bounded rather than applied eagerly.

profile.toml now describes the phase-3 configuration that is actually deployed,
including that the empty `skills` list is a finding and not an oversight.
pi_rpc gains --system-prompt support and no longer guesses whether a `read` tool
will exist; extension_registers_read has to be stated.

harness-layering.md records what transfers from a widely-shared account of
building a personal coding harness on pi, and what does not. The layering frame
holds and the cache-hit figure was the useful part. Its central recommendation --
installing third-party packages -- is disqualifying for an unattended agent
holding tracker credentials, and its discipline layer (AGENTS.md) is precisely
what we block, because it is discovered from every parent directory.
This commit is contained in:
Kai
2026-08-28 01:10:58 -07:00
parent eaa3f6a8a1
commit b5a29b05e1
8 changed files with 401 additions and 102 deletions
+100
View File
@@ -0,0 +1,100 @@
# Harness layering, and which parts transfer to a dedicated agent
Notes taken from a widely-shared account of building a personal *coding* harness on
pi ([@chasen_liao, 2026-08-27](https://x.com/chasen_liao/status/2092963119337476137)),
checked against what this repository actually runs. Recorded because the layering
is a good frame and because several of its specifics are actively wrong for a
service-embedded agent — and the reasons are worth stating once rather than
rediscovering.
## The frame: three layers
| layer | coding harness | this repository |
|---|---|---|
| discipline — durable, always loaded | global + project `AGENTS.md` | `.pi/SYSTEM.md` + `.pi/APPEND_SYSTEM.md` |
| capability — loaded when needed | Skills, hidden by default | tool list generated into the system prompt |
| control plane | third-party packages | one scenario extension + a loopback bridge |
The frame holds. What differs is every mechanism, and the differences are not
stylistic.
## What transfers
**Discipline belongs in a durable file, not in each request.** We reached the same
place from the opposite direction: per-request restatements of "which adapters
exist" were removed in phase 2 because they varied the prompt prefix and cost
cache hits (`pi-runtime-notes.md` §24).
**Don't pile on capabilities.** "Several dozen enabled made routing worse" matches
the reason Curator exposes five tools and not fifteen. Every tool is a branch the
model can take wrongly.
**Watch where context goes.** The article uses `pi-context-usage` for this. We take
the same numbers from `message_update.usage` in the RPC stream, which costs nothing
extra, and write them to `control_events` per turn. The article's reported 98%
cache hit is the single most useful figure in it, and it is what justified phase
3b's long-lived sessions; we measure 99.97% on a continuing conversation.
**One writer at a time.** Their rule for parallel subagents ("only one writer per
directory; reviewers may run in parallel because they are read-only") is the same
shape as Curator's: many read tools, exactly one write path through
`CuratorService`.
**No claiming done without evidence.** Their coding loop insists on it; Curator
enforces it mechanically, generating receipts from `tool_execution_end` rather than
letting the model narrate what it did.
**pi has no sandbox and runs at full privilege.** Stated as a closing caution
there; recorded as `pi-runtime-notes.md` §20 here, and the reason isolation is four
layers rather than a flag.
## What does not transfer, and why
**Third-party packages.** `pi install npm:...` is the article's main recommendation.
For a dedicated agent it is disqualifying: a package runs arbitrary code in the
same process as the agent, and Curator's whole posture is that the agent reaches
exactly five audited endpoints over loopback. The article says as much in passing
("look at the source and permissions before installing") — advice that scales to a
human's interactive tool and not to an unattended service holding tracker
credentials. `no_extensions = true` plus one reviewed extension stays.
**`AGENTS.md` as the discipline layer.** For an interactive agent in a repository
this is right. For us it is precisely the thing to block: `AGENTS.md` is discovered
from *every parent directory* of the workspace, so a file written for an unrelated
project leaks in. `--no-context-files` is the only switch that stops it, and it is
on. Discipline lives in the system-prompt files, which that flag does not touch.
**Skills as the capability layer.** Cannot work here at all: pi emits the skills
section only when a tool named `read` is active, and Curator's tools are all
domain-specific, so every `--skill` argument is discarded in silence
(`pi-runtime-notes.md` §23). The capability layer is generated into the system
prompt instead.
Beyond the mechanism, on-demand loading is *undesirable* for us: it varies the
prompt prefix, and a varying prefix is exactly what destroys the cache hit the
article is celebrating. Interactive sessions can afford it; a per-message service
cannot.
**Subagents and parallel lanes.** Curator answers one message about one library.
There is no plan to decompose. The article's own caution — "don't force the full
workflow onto a small task" — applies, and here every task is small.
**`$`-expansion, `/btw`, `/goal`, `/context`, TUI packages.** All interactive
affordances for a human at a terminal. Curator has no human at a terminal; it has
Telegram and an HTTP server.
**`pi-ask` for stopping at ambiguity.** The instinct is right and Curator has an
equivalent, but it cannot be a tool: a Telegram round-trip is not a blocking
prompt. Ambiguity is handled by the `clarify` intent, and the harder rule is that
ambiguity must resolve towards *reading*, not asking — phase 0 removed a
`clarify → library_query` rewrite that was guessing, while phase 3 keeps "when a
message is only a title, query rather than ask what the user wants".
## The transferable conclusion
The article's actual thesis is not its package list, it is that a small core plus
your own assembly beats a fixed harness. That is the same conclusion this
repository reached, with the opposite emphasis: for an unattended agent holding
real credentials, most of the assembly is deciding what *not* to load, and every
capability has to be justified against what it would cost if the model were
adversarial rather than merely wrong.
+65
View File
@@ -501,6 +501,71 @@ instructions under `--system-prompt`; they have to be in the prompt text.
---
## 23. Skills need a tool literally named `read`, so a domain-only agent cannot use them
Section 1 recorded that `--no-tools` disables skills. The real rule is narrower and
worse: `formatSkillsForPrompt` runs only when a tool **named `read`** is active.
```js
const customPromptHasRead = !selectedTools || selectedTools.includes("read");
if (customPromptHasRead && skills.length > 0) {
prompt += formatSkillsForPrompt(skills);
}
```
Measured with a probe registering three tools and loading one skill:
| active tools | `--system-prompt` | skills section | skill names |
|---|---|---|---|
| `probe_read`, `query_library`, … | yes | **true** | `[…, curator-core]` |
| `probe_read`, `query_library`, … | no | true | `[…, curator-core]` |
| `query_library`, `lookup_online`, `counts` | yes | **false** | `[]` |
| `query_library`, `lookup_online`, `counts` | no | **false** | `[]` |
So it is not about `--system-prompt` and not about `--no-tools`. An agent whose
tools are all domain-specific gets **nothing** from `--skill`, silently.
memo-inbox is unaffected because it registers a path-restricted `read` override.
Curator has no `read` and no reason to invent one, so its `--skill` arguments were
discarded and the plan's split into `curator-core` / `video-arr` / `books-ingest`
was inert before it was written. The policy stays in `APPEND_SYSTEM.md`, which is
unconditional.
`PiLaunchConfig._read_reachable` used to assume `no_builtin_tools` implied an
extension supplying `read`. It no longer guesses: `extension_registers_read` must
be set explicitly, and loading skills without it logs a warning.
A side benefit of not using skills: on-demand injection varies the prompt, and a
varying prefix defeats provider prompt caching. See section 24.
## 24. A long-lived session is worth far more than the process startup it saves
Measured on Curator's conversation path, same question, same model:
| turn | cache read / billed input |
|---|---|
| first turn of a new session | 0% |
| continuing session | **99.97%** |
The saving is not the ~1-2 s of process startup, it is that the whole system
prompt and prior history are re-read from cache instead of re-charged.
The practical constraints that follow:
- **Nothing may vary at the front of the prompt.** The tool list is generated into
the system prompt once (section 22) rather than injected per turn, and
per-request restatements of "which adapters exist" were removed.
- **Rotation is a cost.** Each rotation resets the cache, so rotate on explicit
bounds (`rotate_after_prompts`, `rotate_after_messages`) rather than eagerly.
- **Session files must persist.** Curator keys the session id on a uuid5 of the
chat id, so a service restart resumes the same session and keeps the cache warm.
- **Idle processes must still be reclaimed.** A process per chat that is never
stopped is 100-200 MB and several tasks; the set has to be swept on a TTL or it
grows until `MemoryMax` or `TasksMax` produces an unexplained failure to start a
new conversation.
---
## Re-verification
```bash
+36 -1
View File
@@ -309,7 +309,42 @@ destructive 处理**,所以漏分类会 fail closed。拒绝本身作为 `plan
**验收**Web 与 Telegram 的同一动作产生同构账本记录;`control_events` 有读者;
任一任务可查出意图、计划、执行、核验、失败。
### 阶段 3 · Curator 采用 memo-inbox 模式(约 2.5 天)
### 阶段 3 · Curator 采用工具 + 长驻 RPC —— ✅ 已完成 2026-08-28
commits`be6e1dc` 桥接 + extension + 工具提示生成 · `7bb7b03` 长驻 RPC +
工具驱动回答 + 每对话 token。测试 108 → 129。已部署并重启,生产验证见下。
**§4b 里「skills 拆三个」这一项作废,不是延后。** pi 只在有一个**名叫 `read`**
的工具激活时才渲染 skills;Curator 的工具全是领域工具,所以每个 `--skill`
都被静默丢弃(实测四种组合,见 pi-runtime-notes §23)。策略留在 `APPEND_SYSTEM.md`
memo-inbox 不受影响,因为它注册了受限的 `read` 覆盖。
**写操作需要两个独立判断一致**:模型可以调 `propose_write`,但只有 Curator
自己的意图识别(在这一轮之前跑完)也认定用户要求写,才会执行。两个判断里,
偏向「动手」的那个是模型的。授权在 `finally` 里撤销,否则后续纯对话轮会继承它。
**长驻会话的真实收益**(同一问题同一模型):延续会话 **99.97%** 输入命中缓存,
新会话首轮 0%。省的不是 1~2 秒进程启动,而是整个系统提示与历史不再重复计费。
这也是工具清单必须**生成进系统提示**而不是每轮注入的原因 —— 前缀一变,缓存就没了。
**三个跑起来才发现的问题**
1. 每个对话进程都用了池的默认 token,而默认 token 的上下文不属于任何 chat、
永远未授权。于是显式写入被拒,理由对默认上下文是真的、对这段对话是错的。
改为每对话 token,顺带修掉真实并发隐患:两个 chat 是两个线程,
共享轮次状态会让一个 chat 的写落到另一个的 job 上。
2. 无工具的结构化轮拿到了带工具清单的提示,等于告诉模型它能查库而其实不能。
现在两条路径各有提示,并有测试断言两者不同。
3. `_conversations` 从不回收,活的 node 进程数随 chat 数只增不减。
每个 100~200 MB 且占多个 task,症状会是某天「新对话起不来」而不是明显的泄漏。
改为按 TTL 惰性清扫。
**生产验证**(真实数据、真实模型):库存查询 1 次工具且答案正确;跨类型问题
3 次工具且无写入;显式加书调用 propose_write 并如实转述幂等回执(且仍区分
「已提交」与「已入库」);「值得收吗」7 次只读工具、无写入;注入不产生任何写入。
六个页面全 200,关闭后无孤儿 pi 进程,空载 19.4 MB。
### 阶段 3 原始清单(供对照)
1. `curator/agent_api.py`:仅 `127.0.0.1`,启动生成 secret 经 `env` 传给 extension
端点 `query_library` / `lookup_online` / `book_reviews` / `counts` /