docs: mark phase 3 complete, record the skills and caching findings
Two findings that changed the plan rather than confirming it:
23. Skills require a tool literally named `read`. Curator's tools are all
domain-specific, so every --skill argument was discarded in silence. The
planned split into curator-core / video-arr / books-ingest was inert before
it was written; the policy stays in APPEND_SYSTEM.md. memo-inbox is
unaffected because it registers a restricted `read` override, which is why
the earlier note generalised wrongly from it.
24. A long-lived session is worth far more than the startup it saves: 99.97% of
input read from cache on a continuing conversation against 0% on a new one.
That is what makes the generated tool list necessary rather than merely
tidy -- anything varying at the front of the prompt destroys it -- and it
makes rotation a cost to be bounded rather than applied eagerly.
profile.toml now describes the phase-3 configuration that is actually deployed,
including that the empty `skills` list is a finding and not an oversight.
pi_rpc gains --system-prompt support and no longer guesses whether a `read` tool
will exist; extension_registers_read has to be stated.
harness-layering.md records what transfers from a widely-shared account of
building a personal coding harness on pi, and what does not. The layering frame
holds and the cache-hit figure was the useful part. Its central recommendation --
installing third-party packages -- is disqualifying for an unattended agent
holding tracker credentials, and its discipline layer (AGENTS.md) is precisely
what we block, because it is discovered from every parent directory.
This commit is contained in:
@@ -0,0 +1,100 @@
|
||||
# Harness layering, and which parts transfer to a dedicated agent
|
||||
|
||||
Notes taken from a widely-shared account of building a personal *coding* harness on
|
||||
pi ([@chasen_liao, 2026-08-27](https://x.com/chasen_liao/status/2092963119337476137)),
|
||||
checked against what this repository actually runs. Recorded because the layering
|
||||
is a good frame and because several of its specifics are actively wrong for a
|
||||
service-embedded agent — and the reasons are worth stating once rather than
|
||||
rediscovering.
|
||||
|
||||
## The frame: three layers
|
||||
|
||||
| layer | coding harness | this repository |
|
||||
|---|---|---|
|
||||
| discipline — durable, always loaded | global + project `AGENTS.md` | `.pi/SYSTEM.md` + `.pi/APPEND_SYSTEM.md` |
|
||||
| capability — loaded when needed | Skills, hidden by default | tool list generated into the system prompt |
|
||||
| control plane | third-party packages | one scenario extension + a loopback bridge |
|
||||
|
||||
The frame holds. What differs is every mechanism, and the differences are not
|
||||
stylistic.
|
||||
|
||||
## What transfers
|
||||
|
||||
**Discipline belongs in a durable file, not in each request.** We reached the same
|
||||
place from the opposite direction: per-request restatements of "which adapters
|
||||
exist" were removed in phase 2 because they varied the prompt prefix and cost
|
||||
cache hits (`pi-runtime-notes.md` §24).
|
||||
|
||||
**Don't pile on capabilities.** "Several dozen enabled made routing worse" matches
|
||||
the reason Curator exposes five tools and not fifteen. Every tool is a branch the
|
||||
model can take wrongly.
|
||||
|
||||
**Watch where context goes.** The article uses `pi-context-usage` for this. We take
|
||||
the same numbers from `message_update.usage` in the RPC stream, which costs nothing
|
||||
extra, and write them to `control_events` per turn. The article's reported 98%
|
||||
cache hit is the single most useful figure in it, and it is what justified phase
|
||||
3b's long-lived sessions; we measure 99.97% on a continuing conversation.
|
||||
|
||||
**One writer at a time.** Their rule for parallel subagents ("only one writer per
|
||||
directory; reviewers may run in parallel because they are read-only") is the same
|
||||
shape as Curator's: many read tools, exactly one write path through
|
||||
`CuratorService`.
|
||||
|
||||
**No claiming done without evidence.** Their coding loop insists on it; Curator
|
||||
enforces it mechanically, generating receipts from `tool_execution_end` rather than
|
||||
letting the model narrate what it did.
|
||||
|
||||
**pi has no sandbox and runs at full privilege.** Stated as a closing caution
|
||||
there; recorded as `pi-runtime-notes.md` §20 here, and the reason isolation is four
|
||||
layers rather than a flag.
|
||||
|
||||
## What does not transfer, and why
|
||||
|
||||
**Third-party packages.** `pi install npm:...` is the article's main recommendation.
|
||||
For a dedicated agent it is disqualifying: a package runs arbitrary code in the
|
||||
same process as the agent, and Curator's whole posture is that the agent reaches
|
||||
exactly five audited endpoints over loopback. The article says as much in passing
|
||||
("look at the source and permissions before installing") — advice that scales to a
|
||||
human's interactive tool and not to an unattended service holding tracker
|
||||
credentials. `no_extensions = true` plus one reviewed extension stays.
|
||||
|
||||
**`AGENTS.md` as the discipline layer.** For an interactive agent in a repository
|
||||
this is right. For us it is precisely the thing to block: `AGENTS.md` is discovered
|
||||
from *every parent directory* of the workspace, so a file written for an unrelated
|
||||
project leaks in. `--no-context-files` is the only switch that stops it, and it is
|
||||
on. Discipline lives in the system-prompt files, which that flag does not touch.
|
||||
|
||||
**Skills as the capability layer.** Cannot work here at all: pi emits the skills
|
||||
section only when a tool named `read` is active, and Curator's tools are all
|
||||
domain-specific, so every `--skill` argument is discarded in silence
|
||||
(`pi-runtime-notes.md` §23). The capability layer is generated into the system
|
||||
prompt instead.
|
||||
|
||||
Beyond the mechanism, on-demand loading is *undesirable* for us: it varies the
|
||||
prompt prefix, and a varying prefix is exactly what destroys the cache hit the
|
||||
article is celebrating. Interactive sessions can afford it; a per-message service
|
||||
cannot.
|
||||
|
||||
**Subagents and parallel lanes.** Curator answers one message about one library.
|
||||
There is no plan to decompose. The article's own caution — "don't force the full
|
||||
workflow onto a small task" — applies, and here every task is small.
|
||||
|
||||
**`$`-expansion, `/btw`, `/goal`, `/context`, TUI packages.** All interactive
|
||||
affordances for a human at a terminal. Curator has no human at a terminal; it has
|
||||
Telegram and an HTTP server.
|
||||
|
||||
**`pi-ask` for stopping at ambiguity.** The instinct is right and Curator has an
|
||||
equivalent, but it cannot be a tool: a Telegram round-trip is not a blocking
|
||||
prompt. Ambiguity is handled by the `clarify` intent, and the harder rule is that
|
||||
ambiguity must resolve towards *reading*, not asking — phase 0 removed a
|
||||
`clarify → library_query` rewrite that was guessing, while phase 3 keeps "when a
|
||||
message is only a title, query rather than ask what the user wants".
|
||||
|
||||
## The transferable conclusion
|
||||
|
||||
The article's actual thesis is not its package list, it is that a small core plus
|
||||
your own assembly beats a fixed harness. That is the same conclusion this
|
||||
repository reached, with the opposite emphasis: for an unattended agent holding
|
||||
real credentials, most of the assembly is deciding what *not* to load, and every
|
||||
capability has to be justified against what it would cost if the model were
|
||||
adversarial rather than merely wrong.
|
||||
@@ -501,6 +501,71 @@ instructions under `--system-prompt`; they have to be in the prompt text.
|
||||
|
||||
---
|
||||
|
||||
## 23. Skills need a tool literally named `read`, so a domain-only agent cannot use them
|
||||
|
||||
Section 1 recorded that `--no-tools` disables skills. The real rule is narrower and
|
||||
worse: `formatSkillsForPrompt` runs only when a tool **named `read`** is active.
|
||||
|
||||
```js
|
||||
const customPromptHasRead = !selectedTools || selectedTools.includes("read");
|
||||
if (customPromptHasRead && skills.length > 0) {
|
||||
prompt += formatSkillsForPrompt(skills);
|
||||
}
|
||||
```
|
||||
|
||||
Measured with a probe registering three tools and loading one skill:
|
||||
|
||||
| active tools | `--system-prompt` | skills section | skill names |
|
||||
|---|---|---|---|
|
||||
| `probe_read`, `query_library`, … | yes | **true** | `[…, curator-core]` |
|
||||
| `probe_read`, `query_library`, … | no | true | `[…, curator-core]` |
|
||||
| `query_library`, `lookup_online`, `counts` | yes | **false** | `[]` |
|
||||
| `query_library`, `lookup_online`, `counts` | no | **false** | `[]` |
|
||||
|
||||
So it is not about `--system-prompt` and not about `--no-tools`. An agent whose
|
||||
tools are all domain-specific gets **nothing** from `--skill`, silently.
|
||||
|
||||
memo-inbox is unaffected because it registers a path-restricted `read` override.
|
||||
Curator has no `read` and no reason to invent one, so its `--skill` arguments were
|
||||
discarded and the plan's split into `curator-core` / `video-arr` / `books-ingest`
|
||||
was inert before it was written. The policy stays in `APPEND_SYSTEM.md`, which is
|
||||
unconditional.
|
||||
|
||||
`PiLaunchConfig._read_reachable` used to assume `no_builtin_tools` implied an
|
||||
extension supplying `read`. It no longer guesses: `extension_registers_read` must
|
||||
be set explicitly, and loading skills without it logs a warning.
|
||||
|
||||
A side benefit of not using skills: on-demand injection varies the prompt, and a
|
||||
varying prefix defeats provider prompt caching. See section 24.
|
||||
|
||||
## 24. A long-lived session is worth far more than the process startup it saves
|
||||
|
||||
Measured on Curator's conversation path, same question, same model:
|
||||
|
||||
| turn | cache read / billed input |
|
||||
|---|---|
|
||||
| first turn of a new session | 0% |
|
||||
| continuing session | **99.97%** |
|
||||
|
||||
The saving is not the ~1-2 s of process startup, it is that the whole system
|
||||
prompt and prior history are re-read from cache instead of re-charged.
|
||||
|
||||
The practical constraints that follow:
|
||||
|
||||
- **Nothing may vary at the front of the prompt.** The tool list is generated into
|
||||
the system prompt once (section 22) rather than injected per turn, and
|
||||
per-request restatements of "which adapters exist" were removed.
|
||||
- **Rotation is a cost.** Each rotation resets the cache, so rotate on explicit
|
||||
bounds (`rotate_after_prompts`, `rotate_after_messages`) rather than eagerly.
|
||||
- **Session files must persist.** Curator keys the session id on a uuid5 of the
|
||||
chat id, so a service restart resumes the same session and keeps the cache warm.
|
||||
- **Idle processes must still be reclaimed.** A process per chat that is never
|
||||
stopped is 100-200 MB and several tasks; the set has to be swept on a TTL or it
|
||||
grows until `MemoryMax` or `TasksMax` produces an unexplained failure to start a
|
||||
new conversation.
|
||||
|
||||
---
|
||||
|
||||
## Re-verification
|
||||
|
||||
```bash
|
||||
|
||||
@@ -309,7 +309,42 @@ destructive 处理**,所以漏分类会 fail closed。拒绝本身作为 `plan
|
||||
**验收**:Web 与 Telegram 的同一动作产生同构账本记录;`control_events` 有读者;
|
||||
任一任务可查出意图、计划、执行、核验、失败。
|
||||
|
||||
### 阶段 3 · Curator 采用 memo-inbox 模式(约 2.5 天)
|
||||
### 阶段 3 · Curator 采用工具 + 长驻 RPC —— ✅ 已完成 2026-08-28
|
||||
|
||||
commits:`be6e1dc` 桥接 + extension + 工具提示生成 · `7bb7b03` 长驻 RPC +
|
||||
工具驱动回答 + 每对话 token。测试 108 → 129。已部署并重启,生产验证见下。
|
||||
|
||||
**§4b 里「skills 拆三个」这一项作废,不是延后。** pi 只在有一个**名叫 `read`**
|
||||
的工具激活时才渲染 skills;Curator 的工具全是领域工具,所以每个 `--skill`
|
||||
都被静默丢弃(实测四种组合,见 pi-runtime-notes §23)。策略留在 `APPEND_SYSTEM.md`。
|
||||
memo-inbox 不受影响,因为它注册了受限的 `read` 覆盖。
|
||||
|
||||
**写操作需要两个独立判断一致**:模型可以调 `propose_write`,但只有 Curator
|
||||
自己的意图识别(在这一轮之前跑完)也认定用户要求写,才会执行。两个判断里,
|
||||
偏向「动手」的那个是模型的。授权在 `finally` 里撤销,否则后续纯对话轮会继承它。
|
||||
|
||||
**长驻会话的真实收益**(同一问题同一模型):延续会话 **99.97%** 输入命中缓存,
|
||||
新会话首轮 0%。省的不是 1~2 秒进程启动,而是整个系统提示与历史不再重复计费。
|
||||
这也是工具清单必须**生成进系统提示**而不是每轮注入的原因 —— 前缀一变,缓存就没了。
|
||||
|
||||
**三个跑起来才发现的问题**:
|
||||
|
||||
1. 每个对话进程都用了池的默认 token,而默认 token 的上下文不属于任何 chat、
|
||||
永远未授权。于是显式写入被拒,理由对默认上下文是真的、对这段对话是错的。
|
||||
改为每对话 token,顺带修掉真实并发隐患:两个 chat 是两个线程,
|
||||
共享轮次状态会让一个 chat 的写落到另一个的 job 上。
|
||||
2. 无工具的结构化轮拿到了带工具清单的提示,等于告诉模型它能查库而其实不能。
|
||||
现在两条路径各有提示,并有测试断言两者不同。
|
||||
3. `_conversations` 从不回收,活的 node 进程数随 chat 数只增不减。
|
||||
每个 100~200 MB 且占多个 task,症状会是某天「新对话起不来」而不是明显的泄漏。
|
||||
改为按 TTL 惰性清扫。
|
||||
|
||||
**生产验证**(真实数据、真实模型):库存查询 1 次工具且答案正确;跨类型问题
|
||||
3 次工具且无写入;显式加书调用 propose_write 并如实转述幂等回执(且仍区分
|
||||
「已提交」与「已入库」);「值得收吗」7 次只读工具、无写入;注入不产生任何写入。
|
||||
六个页面全 200,关闭后无孤儿 pi 进程,空载 19.4 MB。
|
||||
|
||||
### 阶段 3 原始清单(供对照)
|
||||
|
||||
1. `curator/agent_api.py`:仅 `127.0.0.1`,启动生成 secret 经 `env` 传给 extension;
|
||||
端点 `query_library` / `lookup_online` / `book_reviews` / `counts` /
|
||||
|
||||
Reference in New Issue
Block a user