diff --git a/.opencode/agents/dr-analyst.md b/.opencode/agents/dr-analyst.md index a8dc707..38db71a 100644 --- a/.opencode/agents/dr-analyst.md +++ b/.opencode/agents/dr-analyst.md @@ -17,6 +17,8 @@ permission: "*": deny "wc *": allow "python3 *": allow + "uv run python scripts/search.py *": allow + "uv run python scripts/ground.py *": allow "mkdir *": allow "grep *": allow "cat *": allow @@ -66,6 +68,15 @@ Read `projects//phase1/framework.md` to understand the chapter's positioni - Round 3: Counter-evidence (search for limitations, failures, controversies) - Round 4: Tavily/Exa/Brave for gap-filling, trace back to Tier 1-2 originals +Mandatory project search gateway: +- Literature / reviews: `uv run python scripts/search.py "" --route scholar --num-results 10 --year-low 2023` +- Patents / FTO: `uv run python scripts/search.py "" --route patents --num-results 10` +- News / transactions: `uv run python scripts/search.py "" --route news --num-results 10 --time-range m` +- Generic gap-fill: `uv run python scripts/search.py "" --route general --num-results 10` +- Fast grounded fact-check (native model web search): `uv run python scripts/ground.py "" --json` + +Record the routes used in the evidence file. Do not use Tavily / Exa / Brave MCP as the primary path for literature or patent searches. + Search in **both English and Chinese** for each direction (Chinese sources critical for China market / NMPA / CSRC disclosures). ### Step 3: Source Scoring diff --git a/.opencode/agents/dr-searcher.md b/.opencode/agents/dr-searcher.md index f6c5b43..3bc57d8 100644 --- a/.opencode/agents/dr-searcher.md +++ b/.opencode/agents/dr-searcher.md @@ -7,10 +7,14 @@ temperature: 0.1 tools: read: true webfetch: true + bash: true skill: true permission: bash: "*": deny + "uv run python scripts/search.py *": allow + "uv run python scripts/ground.py *": allow + "python3 scripts/search.py *": allow edit: deny webfetch: allow task: @@ -25,10 +29,13 @@ permission: 1. 加载 `skill:search-strategy` 了解信源优先级与检索规则 2. 加载 `skill:source-quality` 了解评分标准与黑名单 -3. 按调用方给定的关键词方向，执行 **3 轮检索**： - - 第 1 轮：英文关键词，优先 Tavily advanced 模式，锁定 Tier 1 域名 - - 第 2 轮：中文关键词，查中文专业来源 - - 第 3 轮：反方/限制性关键词（如 `limitations`, `adverse`, `failed`） +3. 按调用方给定的关键词方向，执行 **3 轮检索**，必须优先使用项目搜索网关： + - 文献：`uv run python scripts/search.py "" --route scholar --num-results 10 --year-low 2023` + - 专利：`uv run python scripts/search.py "" --route patents --num-results 10` + - 新闻/行业动态：`uv run python scripts/search.py "" --route news --num-results 10 --time-range m` + - 通用补漏：`uv run python scripts/search.py "" --route general --num-results 10` + - 快速 grounding：`uv run python scripts/ground.py "" --json` + - Tavily / Brave / Exa MCP 只能作为 gap-fill 或脚本不可用时的兜底 4. 对每条候选信源按 source-quality 评分，过滤掉评分 < 5 及黑名单 5. 整理输出，直接返回给调用方（不写文件） @@ -43,6 +50,7 @@ permission: - 英文：... - 中文：... - 反方：... +- Routes used: scholar / patents / news / general ### 信源列表（共 N 条，Tier 1-2） diff --git a/.opencode/agents/dr-verifier.md b/.opencode/agents/dr-verifier.md index c8a3b38..3a90ebe 100644 --- a/.opencode/agents/dr-verifier.md +++ b/.opencode/agents/dr-verifier.md @@ -2,7 +2,7 @@ description: Cross-model verification agent (English). Uses non-Claude model (GPT-5.4) to do counter-evidence searching and fact-check on completed chapters, avoiding same-source bias. Scheduled by dr-pm after dr-analyst finishes each chapter. mode: subagent hidden: true -model: zenmux/openai/gpt-5.4 +model: zenmux/openai/gpt-5.4-mini temperature: 0.2 tools: read: true @@ -10,12 +10,16 @@ tools: edit: false apply_patch: false webfetch: true + bash: true skill: true permission: edit: allow webfetch: allow bash: "*": deny + "uv run python scripts/search.py *": allow + "uv run python scripts/ground.py *": allow + "python3 scripts/search.py *": allow task: "*": deny --- @@ -74,6 +78,12 @@ For each core claim, search: Run 3-5 webfetch queries per claim, prioritizing Tier 1-2 sources. +Use the project search gateway before generic webfetch: +- `uv run python scripts/search.py " limitations failed controversy" --route scholar --num-results 10 --year-low 2023` +- For patent/IP claims: `uv run python scripts/search.py "" --route patents --num-results 10` +- For news or transaction claims: `uv run python scripts/search.py "" --route news --num-results 10 --time-range y` +- For rapid independent spot checks: `uv run python scripts/ground.py "" --json` + ### Step 3: Data Sanity Check Verify all numbers in the chapter: diff --git a/.opencode/commands/dr-apply-models.md b/.opencode/commands/dr-apply-models.md new file mode 100644 index 0000000..3069bdb --- /dev/null +++ b/.opencode/commands/dr-apply-models.md @@ -0,0 +1,33 @@ +--- +description: 将模型预设应用到 agent 文件。用法：/dr-apply-models +agent: dr-pm +--- + +你是 dr-pm。把模型预设应用到 agent 配置文件。 + +## 执行步骤 + +1. 如果 `$ARGUMENTS` 为空，先列出可用 profile： + +```bash +uv run python scripts/dr.py models --list +``` + +并提示用户至少选择 `simple / medium / premium` 之一。 + +2. 如果 `$ARGUMENTS` 非空，执行 dry-run： + +```bash +uv run python scripts/dr.py apply-models --profile "$ARGUMENTS" --target both --dry-run +``` + +3. 将 dry-run 结果展示给用户确认影响范围后，再执行实际应用： + +```bash +uv run python scripts/dr.py apply-models --profile "$ARGUMENTS" --target both +``` + +4. 最后输出： +- 采用的 profile +- 更新的文件数与路径 +- 下一步建议（如需同步到本机 `.codex/**`，运行 `uv run python scripts/install_codex_adapter.py --force`） diff --git a/.opencode/commands/dr-finalize.md b/.opencode/commands/dr-finalize.md index f01ea30..27a0bc8 100644 --- a/.opencode/commands/dr-finalize.md +++ b/.opencode/commands/dr-finalize.md @@ -1,11 +1,11 @@ --- -description: Phase 4 - 成稿（v0.6）。dr-editor-in-chief 写 ES/Abstract/Glossary，然后调 Python 脚本链路：translate → build_glossary → apply_glossary → polish → build_report。用法：/dr-finalize [slug] +description: Phase 4 - 成稿（v0.12）。dr-editor-in-chief 写 ES/Abstract/Glossary，然后调统一 Python pipeline（phase4_pipeline.py）。用法：/dr-finalize [slug] agent: dr-editor-in-chief --- -你是 dr-editor-in-chief。用户执行了 `/dr-finalize $ARGUMENTS`，进入 Phase 4 成稿链路（v0.6 架构）。 +你是 dr-editor-in-chief。用户执行了 `/dr-finalize $ARGUMENTS`，进入 Phase 4 成稿链路（v0.12 架构）。 -## 架构变更说明（v0.6） +## 架构变更说明（v0.12） **Phase 4 的翻译/润色/出稿已从 LLM agent 改为 Python 脚本**。原因： - LLM agent 一次性处理整篇报告（19k+ 词）会超 Sonnet output token 上限（~32k），不稳定 @@ -40,57 +40,40 @@ agent: dr-editor-in-chief - 给每章强加 SCQA 或小节标题 - 保留调度元数据（字数配额/研究员/quota 等） -## Step 3: 翻译（Python 脚本） +## Step 3: 执行统一 Phase 4 pipeline（Python 脚本） ```bash -uv run python scripts/translate.py +uv run python scripts/phase4_pipeline.py ``` -完成条件：`phase4/final_zh.md` 生成且字数 ≥ 目标字数的 90%。如未达标，`--force` 强制重跑。 +默认行为： +- 自动估算 translate / polish 并发 +- glossary 仅核查低置信度术语（`--glossary-mode low-confidence`） +- 统一串联 translate → glossary(optional) → apply_glossary → polish → build_report -## Step 4: 术语表核查（强烈推荐） +可选参数示例： ```bash -uv run python scripts/build_glossary.py --workers 4 +uv run python scripts/phase4_pipeline.py --glossary-mode full +uv run python scripts/phase4_pipeline.py --glossary-mode off ``` -完成后查看 `phase4/glossary.json`： -- `confidence == "high"` 且 `issue` 非空的条目：说明发现了错误，需要回塑到正文 -- 关注公司名 / 机构名 / 产品名类，它们最容易有拼写错误 +完成条件：`phase4/final_zh_polished.md`、PDF、DOCX 全部生成，且无致命报错。 -## Step 5: 应用术语修正（Python 脚本） +## Step 4: （可选）分步重跑 -```bash -# 先预览 -uv run python scripts/apply_glossary.py --input phase4/final_zh.md --dry-run - -# 确认无误后应用 -uv run python scripts/apply_glossary.py --input phase4/final_zh.md -``` - -这会把 glossary 中发现的拼写错误 / 错译直接替换进 `final_zh.md`。 -如润色后仍需二次复核，可手动对 `final_zh_polished.md` 再运行一次： -`uv run python scripts/apply_glossary.py --input phase4/final_zh_polished.md --dry-run`。 - -## Step 6: 润色（Python 脚本） - -```bash -uv run python scripts/polish.py -``` - -输出：`phase4/final_zh_polished.md`。查看 `phase4/polish_notes.jsonl` 了解模型标记的异常点。 - -## Step 7: 出稿（Python 脚本） - -```bash -uv run python scripts/build_report.py -``` +当你只想重跑单环节时，仍可手动调用： +- `translate.py` +- `build_glossary.py` +- `apply_glossary.py` +- `polish.py` +- `build_report.py` 自动： - 按 `manifest.report_title` 命名输出（`.pdf` + `<Title>.docx`） - PDF 自动插 TOC + 从 `phase2/sources.jsonl` 生成参考文献 -## Step 8: 更新 manifest +## Step 5: 更新 manifest ```json { @@ -115,7 +98,7 @@ uv run python scripts/build_report.py <slug> } ``` -## Step 9: 汇报 +## Step 6: 汇报 向用户展示： - 各阶段耗时和成本 diff --git a/.opencode/commands/dr-models.md b/.opencode/commands/dr-models.md new file mode 100644 index 0000000..4720489 --- /dev/null +++ b/.opencode/commands/dr-models.md @@ -0,0 +1,31 @@ +--- +description: 查看或解析模型预设。用法：/dr-models [profile] +agent: dr-pm +--- + +你是 dr-pm。目标是把当前模型预设解析成清晰结果，并给出可执行命令。 + +## 执行步骤 + +1. 如果 `$ARGUMENTS` 为空：运行 + +```bash +uv run python scripts/dr.py models +``` + +2. 如果 `$ARGUMENTS` 非空：把它当作 profile，运行 + +```bash +uv run python scripts/dr.py models --profile "$ARGUMENTS" +``` + +3. 输出结果时必须包含： +- 当前 profile 名称 +- 各角色模型映射（至少 dr_plan / dr_pm / dr_analyst / dr_verifier / translate / polish / glossary） +- 一条可复制命令，用于 Phase 4 指定该 profile： + +```bash +uv run python scripts/dr.py finalize <slug> --model-profile <profile> +``` + +4. 如果 profile 不存在，提示可用 profile 并建议 `simple / medium / premium` 三档。 diff --git a/.opencode/opencode.json b/.opencode/opencode.json index 7c789ee..a45c824 100644 --- a/.opencode/opencode.json +++ b/.opencode/opencode.json @@ -113,7 +113,7 @@ "environment": { "TAVILY_API_KEY": "{env:TAVILY_API_KEY}" }, - "enabled": true + "enabled": false }, "brave-search": { "type": "local", @@ -121,7 +121,7 @@ "environment": { "BRAVE_API_KEY": "{env:BRAVE_API_KEY}" }, - "enabled": true + "enabled": false }, "exa": { "type": "local", @@ -129,7 +129,7 @@ "environment": { "EXA_API_KEY": "{env:EXA_API_KEY}" }, - "enabled": true + "enabled": false } }, "permission": { diff --git a/.opencode/skills/search-strategy/SKILL.md b/.opencode/skills/search-strategy/SKILL.md index 27f27da..9284b4d 100644 --- a/.opencode/skills/search-strategy/SKILL.md +++ b/.opencode/skills/search-strategy/SKILL.md @@ -82,7 +82,46 @@ description: 生物医药深度研究的统一检索策略。规定信源优先 --- -## 三、API 调用顺序（技术栈，v0.8 更新） +## 三、强制工具入口（v0.12） + +所有 agent 做联网检索时，**优先调用项目内 Python 网关**，不要直接把 Tavily / Brave / Exa MCP 当成主路径： + +```bash +uv run python scripts/search.py "<query>" --route scholar --num-results 10 --year-low 2023 +uv run python scripts/search.py "<query>" --route patents --num-results 10 +uv run python scripts/search.py "<query>" --route news --num-results 10 --time-range m +uv run python scripts/search.py "<query>" --route general --num-results 10 +uv run python scripts/search.py "<query>" --profile china_market --num-results 10 --trace +uv run python scripts/ground.py "<query>" --model google/gemini-3.1-flash-lite-preview --json +``` + +也可以按研究场景跑 profile： + +```bash +uv run python scripts/search.py "<query>" --profile biomed_literature --num-results 10 --year-low 2023 +uv run python scripts/search.py "<query>" --profile patent_heavy --num-results 10 +``` + +**原因**： +- Python 网关在 repo 内，可被 OpenCode / Codex / Gemini CLI / Claude Code 共同复用。 +- `--route patents` 固定优先 Serper + Google Patents，避免专利检索被 Tavily 普通网页结果替代。 +- `--route scholar` 固定优先 Serper Scholar，避免论文检索只停留在通用网页摘要。 +- 专用 route（scholar/patents/news）默认 `--strict-specialized`，Serper 异常时应显式失败，不允许静默降级。 +- Tavily / Exa / Brave 只作为 gap-fill 或 MCP 兜底，不作为文献/专利主路径。 + +每个检索小结必须写明实际使用过的 route，例如： + +```text +Routes used: scholar, patents, general +``` + +如果由于缺 key 或 API 错误无法调用 Serper，必须在输出中明确写（且建议重新执行，不直接进入正文证据）： + +```text +Serper unavailable: <原因>; fallback used: general site:patents.google.com +``` + +## 四、API 调用顺序（技术栈，v0.11 更新） **按"查询类型"路由到最合适的 API**，而不是一律走通用搜索。 @@ -114,24 +153,23 @@ description: 生物医药深度研究的统一检索策略。规定信源优先 ### Serper（google.serper.dev）使用模板 **专利检索**： -```python -from scripts.lib.search_client import SearchClient -with SearchClient() as c: - hits = c.patents("dual-target siRNA GalNAc", num_results=10) +```bash +uv run python scripts/search.py "dual-target siRNA GalNAc" --route patents --num-results 10 ``` **学术论文**： -```python -hits = c.scholar("dual-target RNAi 2024", num_results=10, year_low=2023) -# hits[i].snippet 里包含引用数和期刊信息 +```bash +uv run python scripts/search.py "dual-target RNAi 2024" --route scholar --num-results 10 --year-low 2023 ``` **新闻（时效性）**： -```python -hits = c.news("Arrowhead ARO-DIMER-PA clinical trial", time_range="w") # 最近一周 +```bash +uv run python scripts/search.py "Arrowhead ARO-DIMER-PA clinical trial" --route news --num-results 10 --time-range w ``` -### Tavily MCP 调用模板（通用网页 - Phase 1 初扫） +### Tavily MCP 调用模板（兜底，不作为主路径） + +仅当 `scripts/search.py` 不可用，或需要 MCP 特有能力时使用。通用网页结果必须回溯到 Tier 1-2 原始来源。 ``` 工具名：tavily_search 参数： @@ -171,7 +209,7 @@ curl -s "https://api.fda.gov/drug/event.json?search=patient.drug.medicinalproduc --- -## 四、关键词策略 +## 五、关键词策略 ### 中英双语必备 - 任何生物医药主题**必须同时用中英文检索** @@ -195,7 +233,7 @@ curl -s "https://api.fda.gov/drug/event.json?search=patient.drug.medicinalproduc --- -## 五、每条信源的提取字段（标准化） +## 六、每条信源的提取字段（标准化） 任何信源进 `sources.jsonl` 必须有以下字段： @@ -225,7 +263,7 @@ curl -s "https://api.fda.gov/drug/event.json?search=patient.drug.medicinalproduc --- -## 六、失败兜底 +## 七、失败兜底 - 某个 API 限流/超时：**等 5s 重试 3 次**，仍失败则跳过并在日志标注 - 某个信源 404：在 sources.jsonl 标 `"dead_link": true`，不删除（审计用） @@ -233,7 +271,7 @@ curl -s "https://api.fda.gov/drug/event.json?search=patient.drug.medicinalproduc --- -## 七、硬规则总结 +## 八、硬规则总结 1. ✅ 每 section 至少 4 轮检索 2. ✅ 中英双语必查 @@ -241,6 +279,8 @@ curl -s "https://api.fda.gov/drug/event.json?search=patient.drug.medicinalproduc 4. ✅ 反方关键词必查 5. ✅ Tier 4 结果只做发现，不做佐证 6. ✅ 所有信源写入 sources.jsonl 并评分 -7. ❌ 不得引用 Wikipedia 做结论 -8. ❌ 不得编造数据、URL、DOI -9. ❌ 不得使用黑名单信源 +7. ✅ 文献检索必须优先 `scripts/search.py --route scholar` +8. ✅ 专利检索必须优先 `scripts/search.py --route patents` +9. ❌ 不得引用 Wikipedia 做结论 +10. ❌ 不得编造数据、URL、DOI +11. ❌ 不得使用黑名单信源 diff --git a/AGENTS.md b/AGENTS.md index 7d97645..da08fd2 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -74,8 +74,9 @@ ### Phase 4：成稿 - **驱动命令**：`/dr-finalize` -- **主导 agent**：dr-chief-editor → dr-polisher → dr-reporter -- **产出**：`final.md` + `final.pdf`（ReportLab）+ `final.docx`（Pandoc） +- **主导 agent**：dr-editor-in-chief（创作）→ `scripts/phase4_pipeline.py`（执行链路） +- **执行链路**：translate → glossary(optional) → apply_glossary → polish → build_report +- **产出**：`phase4/final_en.md` + `phase4/final_zh.md` + `phase4/final_zh_polished.md` + `phase4/*.pdf` + `phase4/*.docx` --- diff --git a/PLAN.md b/PLAN.md index cf05a20..aa00d40 100644 --- a/PLAN.md +++ b/PLAN.md @@ -546,3 +546,56 @@ OpenCode 的坑：如果只是在主会话里装样子地写"让 X agent 做"， - 安装后运行 `/debug-config` 确认 `.codex/config.toml` 被 Codex 加载。 - 自动化研究默认权限：`sandbox_mode = "workspace-write"`、`approval_policy = "never"`、`web_search = "live"`、`sandbox_workspace_write.network_access = true`。 - Tavily / Brave / Exa MCP server 在模板中默认 `enabled = true` 且 `required = false`；确认本机 key、npm 与网络可用可直接使用，某个服务异常时再单独关闭。 + +- 2026-04-24 v0.11：**项目内搜索网关与 search-strategy 强化** + + **目标**：把搜索主路径从平台 MCP 收敛到项目内 Python CLI，避免 Codex/OpenCode/Gemini/Claude Code 各自配置差异导致策略漂移。 + + **变更**： + - 新增 `scripts/search.py`：统一搜索入口，支持 `--route scholar|patents|news|general` 与 `--profile biomed_literature|patent_heavy|china_market|investment`。 + - `scripts/lib/search_client.py` 调整为 Serper / Exa / Tavily 路由：文献走 Serper Scholar，专利走 Serper + Google Patents，新闻走 Serper News，通用搜索走 Exa → Tavily。 + - `search-strategy` 明确 MCP 只做 gap-fill；文献必须优先 `scripts/search.py --route scholar`，专利必须优先 `scripts/search.py --route patents`。 + - OpenCode `dr-searcher` / `dr-analyst` / `dr-verifier` 增加搜索网关调用要求与必要 bash 权限。 + - Codex adapter 模板同步要求 `dr-run`、`dr-searcher`、`dr-analyst`、`dr-verifier` 使用搜索网关。 + +- 2026-04-29 v0.12：**三轨并行改造（搜索稳定性 + 模型配置化 + Phase 4 替代式 pipeline）** + + **目标**：并行解决三项瓶颈： + 1) 搜索工具遵循不稳定； + 2) 模型选择被硬编码锁定； + 3) Phase 4 串行链路耗时过长。 + + **Track A — 搜索路径可控化（Sprint 1）**： + - 新增 `scripts/ground.py`，统一封装 ZenMux native grounding（`web_search_options`）并输出引用 URL。 + - `scripts/lib/zenmux_client.py` 增加 `web_search` 参数透传与 `chat_complete_with_meta()`（返回 content/usage/citations/raw）。 + - `scripts/lib/search_client.py` 对 `scholar/patents/news` 默认启用 strict 模式，Serper 异常时显式失败，禁止静默降级。 + - `scripts/search.py` 增加 `--strict-specialized`、`--trace`、`china_market` 查询重写。 + - `.opencode/opencode.json` 关闭 Tavily/Brave/Exa MCP 的默认启用，收敛到项目内搜索网关。 + + **Track B — 模型配置化（Sprint 2-3）**： + - 新增统一配置 `configs/models.yaml`（`simple/medium/premium/cn_heavy/codex_native`）。 + - 新增 `scripts/lib/model_config.py`，支持 profile 解析、override（`ROLE=MODEL`）与 profile 列表。 + - `scripts/dr.py` 新增 `models`、`apply-models`，并让 `finalize` 支持 `--model-profile` 与 `--model-override`。 + - 新增 `scripts/apply_model_profile.py`，可将 profile 批量回填到 `.opencode/agents/*.md` 与 `codex_adapter_templates/codex/agents/*.toml`。 + - 新增 OpenCode 命令：`/dr-models`、`/dr-apply-models`。 + + **Track C — Phase 4 替代式重构（Sprint 4）**： + - 新增 `scripts/phase4_pipeline.py` 作为统一编排入口： + `translate -> glossary(optional) -> apply_glossary -> polish -> build_report`。 + - glossary 核查支持 `off/low-confidence/full`，默认 `low-confidence`；低置信度条目过多时自动回退 `full`，避免超长命令参数。 + - translate/polish workers 支持自动估算（`0 => auto`），降低人工调参成本。 + - `scripts/dr.py finalize` 与 `.opencode/commands/dr-finalize.md` 切换到新 pipeline。 + + **Sprint 5 回归验证**： + - 新增 `scripts/sprint5_regression.py`，覆盖模型预设解析、搜索网关 dry-run、Phase 4 finalize dry-run 三项关键回归检查。 + - 文档同步：`README.md`、`docs/model-playbook.md`、`docs/search-playbook.md`、`docs/codex-usage.md`。 + + **Sprint 6 收尾验收**： + - AGENTS.md 的 Phase 4 描述更新为 v0.12 真实链路（`dr-editor-in-chief + scripts/phase4_pipeline.py`）。 + - README 增补一键回归命令：`uv run python scripts/sprint5_regression.py <slug>`。 + - 验收口径固定： + 1) `dr.py models --list` 可列出预设； + 2) `dr.py apply-models` 可 dry-run 与落盘； + 3) `scripts/search.py` 专用路由默认 strict； + 4) `dr.py finalize --model-profile <x>` 走统一 Phase 4 pipeline； + 5) `scripts/sprint5_regression.py` 全部 PASS。 diff --git a/README.md b/README.md index b9598ba..d5bbcde 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ > 生物医药行业的 AI 驱动深度研究流水线。基于 OpenCode 多 agent 协作，以麦肯锡/德勤式方法论产出专业级研究报告（PDF + DOCX）。 -**当前状态**：v0.10 迭代中。OpenCode 全流程可用（Phase 1-4），Phase 4 已切换为 Python 脚本化流水线；Codex native adapter 正在建设为独立于 OpenCode 的并列入口。 +**当前状态**：v0.12 迭代完成。OpenCode 全流程可用（Phase 1-4），搜索网关、模型预设与 Phase 4 统一 pipeline 已落地；Codex native adapter 与 OpenCode 保持并列入口。 详见 `PLAN.md` 了解完整方案、版本记录与迭代路径。 --- @@ -117,6 +117,8 @@ source scripts/activate.sh | `/dr-research [slug]` | Phase 2：并行深度研究 | ✅ 可用 | | `/dr-review [slug]` | Phase 3：总编审校 | ✅ 可用 | | `/dr-finalize [slug]` | Phase 4：英文合稿 → 中文翻译/术语核查/润色 → PDF+DOCX | ✅ 可用 | +| `/dr-models [profile]` | 解析模型预设（simple / medium / premium 等） | ✅ 可用 | +| `/dr-apply-models <profile>` | 把模型预设写入 OpenCode/Codex agent 文件 | ✅ 可用 | | `/dr-glossary [slug]` | 术语表事实核查 | ✅ 可用 | | `/dr-status [slug]` | 查看进度 | ✅ 可用 | @@ -152,7 +154,19 @@ source scripts/activate.sh ### Phase 4 Python 流水线 -Phase 4 已不再依赖单个 LLM agent 一次性翻译整篇报告，而是由 Python 控制切块、并发、重试与断点续传： +Phase 4 已切到统一 pipeline（替代式）：由 Python 控制切块、并发、重试与断点续传： + +```bash +uv run python scripts/phase4_pipeline.py <slug> +# 等价入口（支持模型预设） +uv run python scripts/dr.py finalize <slug> --model-profile medium +``` + +默认行为： +- 自动估算 translate/polish 并发（`--translate-workers 0` / `--polish-workers 0`） +- glossary 仅核查低置信度术语（`--glossary-mode low-confidence`） + +你也可以手动分步执行： ```bash uv run python scripts/translate.py <slug> --workers 4 @@ -196,6 +210,35 @@ codex exec "$(uv run python scripts/dr.py prompt dr-run <slug-or-topic>)" - `docs/model-playbook.md` - `docs/search-playbook.md` +模型预设配置文件： +- `configs/models.yaml`（统一预设，支持 `simple / medium / premium / cn_heavy / codex_native`） + +命令行查看解析后的模型映射： + +```bash +uv run python scripts/dr.py models +uv run python scripts/dr.py models --list +uv run python scripts/dr.py models --profile premium +uv run python scripts/dr.py models --profile medium --model-override dr_verifier=zenmux/openai/gpt-5.4 + +# apply profile to agent files +uv run python scripts/dr.py apply-models --profile medium --target both --dry-run +uv run python scripts/dr.py apply-models --profile medium --target both +``` + +Sprint 5 回归检查（一键）： + +```bash +uv run python scripts/sprint5_regression.py <slug> +``` + +统一搜索入口： + +```bash +uv run python scripts/search.py "dual-target RNAi 2024" --route scholar --year-low 2023 +uv run python scripts/search.py "dual-target siRNA GalNAc" --route patents +``` + --- ## 项目结构 diff --git a/codex_adapter_templates/codex/agents/dr-analyst.toml b/codex_adapter_templates/codex/agents/dr-analyst.toml index 8b398a0..7bf8c2a 100644 --- a/codex_adapter_templates/codex/agents/dr-analyst.toml +++ b/codex_adapter_templates/codex/agents/dr-analyst.toml @@ -1,12 +1,18 @@ name = "dr-analyst" description = "Chapter deep-research agent that writes English chapter drafts and evidence matrices." -model = "gpt-5.4" +model = "zenmux-anthropic/claude-sonnet-4-6" model_reasoning_effort = "high" sandbox_mode = "workspace-write" developer_instructions = """ You are dr-analyst. Work in English. Own exactly one assigned chapter. Load skills: search-strategy, source-quality, length-budget, evidence-table, mckinsey-method, humanizer-cn. +Use the project search gateway before MCP or generic web search: +- literature/reviews: uv run python scripts/search.py "<query>" --route scholar --num-results 10 --year-low 2023 +- patents/FTO: uv run python scripts/search.py "<query>" --route patents --num-results 10 +- news/transactions: uv run python scripts/search.py "<query>" --route news --num-results 10 --time-range m +- general gap-fill: uv run python scripts/search.py "<query>" --route general --num-results 10 +Record the routes used in the evidence file. Tavily / Exa / Brave MCP are gap-fill only for literature and patent topics. Write: - projects/<slug>/phase2/drafts/chXX.md - projects/<slug>/phase2/evidence/chXX-evidence.md diff --git a/codex_adapter_templates/codex/agents/dr-chief-editor.toml b/codex_adapter_templates/codex/agents/dr-chief-editor.toml index c8292ca..1be6555 100644 --- a/codex_adapter_templates/codex/agents/dr-chief-editor.toml +++ b/codex_adapter_templates/codex/agents/dr-chief-editor.toml @@ -1,6 +1,6 @@ name = "dr-chief-editor" description = "Phase 3 read-only editorial reviewer for whole-report logic, evidence, MECE, and quality." -model = "gpt-5.4" +model = "zenmux/google/gemini-3.1-pro-preview" model_reasoning_effort = "xhigh" sandbox_mode = "read-only" developer_instructions = """ diff --git a/codex_adapter_templates/codex/agents/dr-editor-in-chief.toml b/codex_adapter_templates/codex/agents/dr-editor-in-chief.toml index 3512193..5c08c93 100644 --- a/codex_adapter_templates/codex/agents/dr-editor-in-chief.toml +++ b/codex_adapter_templates/codex/agents/dr-editor-in-chief.toml @@ -1,6 +1,6 @@ name = "dr-editor-in-chief" description = "Phase 4 lead editor for English final assembly and deterministic script orchestration." -model = "gpt-5.4" +model = "zenmux-anthropic/claude-opus-4-7" model_reasoning_effort = "xhigh" sandbox_mode = "workspace-write" developer_instructions = """ diff --git a/codex_adapter_templates/codex/agents/dr-plan.toml b/codex_adapter_templates/codex/agents/dr-plan.toml index f8e4bd2..a45daaf 100644 --- a/codex_adapter_templates/codex/agents/dr-plan.toml +++ b/codex_adapter_templates/codex/agents/dr-plan.toml @@ -1,6 +1,6 @@ name = "dr-plan" description = "Deep Research framework planner for Phase 1 interview, initial scan synthesis, and bilingual research framework." -model = "gpt-5.4" +model = "zenmux-anthropic/claude-opus-4-7" model_reasoning_effort = "high" sandbox_mode = "workspace-write" developer_instructions = """ diff --git a/codex_adapter_templates/codex/agents/dr-pm.toml b/codex_adapter_templates/codex/agents/dr-pm.toml index 5e1beb9..9132bec 100644 --- a/codex_adapter_templates/codex/agents/dr-pm.toml +++ b/codex_adapter_templates/codex/agents/dr-pm.toml @@ -1,6 +1,6 @@ name = "dr-pm" description = "Deep Research project manager for Phase 2 batching, analyst/verifier orchestration, and project status." -model = "gpt-5.4" +model = "zenmux-anthropic/claude-sonnet-4-6" model_reasoning_effort = "high" sandbox_mode = "workspace-write" developer_instructions = """ diff --git a/codex_adapter_templates/codex/agents/dr-reporter.toml b/codex_adapter_templates/codex/agents/dr-reporter.toml index 35556af..4eba862 100644 --- a/codex_adapter_templates/codex/agents/dr-reporter.toml +++ b/codex_adapter_templates/codex/agents/dr-reporter.toml @@ -1,6 +1,6 @@ name = "dr-reporter" description = "Report production agent for PDF/DOCX rendering and final output checks." -model = "gpt-5.4-mini" +model = "zenmux-anthropic/claude-sonnet-4-6" model_reasoning_effort = "medium" sandbox_mode = "workspace-write" developer_instructions = """ diff --git a/codex_adapter_templates/codex/agents/dr-searcher.toml b/codex_adapter_templates/codex/agents/dr-searcher.toml index e1ac931..168cce7 100644 --- a/codex_adapter_templates/codex/agents/dr-searcher.toml +++ b/codex_adapter_templates/codex/agents/dr-searcher.toml @@ -1,13 +1,18 @@ name = "dr-searcher" description = "Lightweight source discovery agent for initial scans and targeted source finding." -model = "gpt-5.4-mini" +model = "zenmux-anthropic/claude-haiku-4-5" model_reasoning_effort = "medium" sandbox_mode = "read-only" developer_instructions = """ You are dr-searcher. Your job is source discovery only. Do not write project files unless explicitly instructed by the parent. Load skills: search-strategy and source-quality. -Search English and Chinese keywords, prioritize Tier 1-2 sources, include counter-evidence search terms, and return concise Markdown with URLs/DOIs and source-quality scores. +Use the project search gateway before MCP or generic web search: +- literature: uv run python scripts/search.py "<query>" --route scholar --num-results 10 --year-low 2023 +- patents: uv run python scripts/search.py "<query>" --route patents --num-results 10 +- news: uv run python scripts/search.py "<query>" --route news --num-results 10 --time-range m +- general gap-fill: uv run python scripts/search.py "<query>" --route general --num-results 10 +Search English and Chinese keywords, prioritize Tier 1-2 sources, include counter-evidence search terms, report the routes used, and return concise Markdown with URLs/DOIs and source-quality scores. Do not use Wikipedia as evidence. Do not fabricate URLs, DOIs, trial IDs, patents, or source ids. """ diff --git a/codex_adapter_templates/codex/agents/dr-verifier.toml b/codex_adapter_templates/codex/agents/dr-verifier.toml index d7c56f9..4c6b4fe 100644 --- a/codex_adapter_templates/codex/agents/dr-verifier.toml +++ b/codex_adapter_templates/codex/agents/dr-verifier.toml @@ -1,12 +1,16 @@ name = "dr-verifier" description = "Independent counter-evidence and fact-checking agent for completed chapters." -model = "gpt-5.4" +model = "zenmux/openai/gpt-5.4-mini" model_reasoning_effort = "high" sandbox_mode = "workspace-write" developer_instructions = """ You are dr-verifier. Act as an independent devil's advocate. Do not protect the analyst's conclusion. Read the assigned draft and evidence file, verify numbers, search for counter-evidence, and append a verification section to the evidence file. +Use the project search gateway before generic web search: +- literature counter-evidence: uv run python scripts/search.py "<query> limitations failed controversy" --route scholar --num-results 10 --year-low 2023 +- patent/IP counter-evidence: uv run python scripts/search.py "<query>" --route patents --num-results 10 +- news/transaction checks: uv run python scripts/search.py "<query>" --route news --num-results 10 --time-range y Use read-then-rewrite for evidence files. Do not edit chapter drafts. Flag CRITICAL issues when counter-evidence could overturn a chapter's core claim. Use Chinese and English searches for China-market claims. diff --git a/configs/models.yaml b/configs/models.yaml new file mode 100644 index 0000000..f3dbd50 --- /dev/null +++ b/configs/models.yaml @@ -0,0 +1,84 @@ +version: 1 + +defaults: + profile: medium + script_models: + translate: anthropic/claude-sonnet-4.6 + glossary: anthropic/claude-haiku-4.5 + polish: anthropic/claude-sonnet-4.6 + +profiles: + simple: + description: Lower cost exploration profile for quick scoping. + roles: + dr_plan: zenmux/qwen/qwen3.6-plus + dr_pm: zenmux/qwen/qwen3.6-plus + dr_searcher: zenmux-anthropic/claude-haiku-4-5 + dr_analyst: zenmux/deepseek/deepseek-v3.2 + dr_verifier: zenmux/minimax/minimax-m2.7 + dr_chief_editor: zenmux/google/gemini-2.5-pro + dr_editor_in_chief: zenmux-anthropic/claude-sonnet-4-6 + dr_reporter: zenmux-anthropic/claude-sonnet-4-6 + translate: anthropic/claude-haiku-4.5 + glossary: anthropic/claude-haiku-4.5 + polish: anthropic/claude-haiku-4.5 + + medium: + description: Recommended default profile for most production runs. + roles: + dr_plan: zenmux-anthropic/claude-opus-4-7 + dr_pm: zenmux-anthropic/claude-sonnet-4-6 + dr_searcher: zenmux-anthropic/claude-haiku-4-5 + dr_analyst: zenmux-anthropic/claude-sonnet-4-6 + dr_verifier: zenmux/openai/gpt-5.4-mini + dr_chief_editor: zenmux/google/gemini-3.1-pro-preview + dr_editor_in_chief: zenmux-anthropic/claude-opus-4-7 + dr_reporter: zenmux-anthropic/claude-sonnet-4-6 + translate: anthropic/claude-sonnet-4.6 + glossary: anthropic/claude-haiku-4.5 + polish: anthropic/claude-sonnet-4.6 + + premium: + description: Highest quality profile for formal client-facing deliverables. + roles: + dr_plan: zenmux-anthropic/claude-opus-4-7 + dr_pm: zenmux-anthropic/claude-sonnet-4-6 + dr_searcher: zenmux-anthropic/claude-haiku-4-5 + dr_analyst: zenmux-anthropic/claude-sonnet-4-6 + dr_verifier: zenmux/openai/gpt-5.4 + dr_chief_editor: zenmux/google/gemini-3.1-pro-preview + dr_editor_in_chief: zenmux-anthropic/claude-opus-4-7 + dr_reporter: zenmux-anthropic/claude-sonnet-4-6 + translate: anthropic/claude-sonnet-4.6 + glossary: anthropic/claude-haiku-4.5 + polish: anthropic/claude-sonnet-4.6 + + cn_heavy: + description: China-market-heavy profile with stronger CN-side verification. + roles: + dr_plan: zenmux-anthropic/claude-opus-4-7 + dr_pm: zenmux-anthropic/claude-sonnet-4-6 + dr_searcher: zenmux-anthropic/claude-haiku-4-5 + dr_analyst: zenmux-anthropic/claude-sonnet-4-6 + dr_verifier: zenmux/qwen/qwen3.6-plus + dr_chief_editor: zenmux/google/gemini-3.1-pro-preview + dr_editor_in_chief: zenmux-anthropic/claude-opus-4-7 + dr_reporter: zenmux-anthropic/claude-sonnet-4-6 + translate: anthropic/claude-sonnet-4.6 + glossary: anthropic/claude-haiku-4.5 + polish: anthropic/claude-sonnet-4.6 + + codex_native: + description: OpenAI-native profile for Codex adapter runs. + roles: + dr_plan: gpt-5.4 + dr_pm: gpt-5.4 + dr_searcher: gpt-5.4-mini + dr_analyst: gpt-5.4 + dr_verifier: gpt-5.4 + dr_chief_editor: gpt-5.4 + dr_editor_in_chief: gpt-5.4 + dr_reporter: gpt-5.4-mini + translate: anthropic/claude-sonnet-4.6 + glossary: anthropic/claude-haiku-4.5 + polish: anthropic/claude-sonnet-4.6 diff --git a/docs/codex-usage.md b/docs/codex-usage.md index 124ba84..4366404 100644 --- a/docs/codex-usage.md +++ b/docs/codex-usage.md @@ -85,20 +85,33 @@ Phase 4 推荐走确定性 CLI，而不是让单个 agent 翻译整篇： ```bash uv run python scripts/dr.py finalize <slug> \ - --translate-workers 4 \ - --glossary-workers 4 \ - --polish-workers 4 + --model-profile medium ``` -网络不稳时： +等价底层入口（统一 pipeline）： + +```bash +uv run python scripts/phase4_pipeline.py <slug> +``` + +网络不稳时可显式降并发： ```bash uv run python scripts/dr.py finalize <slug> \ + --model-profile medium \ --translate-workers 1 \ --glossary-workers 3 \ --polish-workers 1 ``` +术语核查策略可选： + +```bash +uv run python scripts/dr.py finalize <slug> --model-profile medium --glossary-mode low-confidence +uv run python scripts/dr.py finalize <slug> --model-profile medium --glossary-mode full +uv run python scripts/dr.py finalize <slug> --model-profile medium --glossary-mode off +``` + ## Subagent Usage Codex 的平台限制是：subagents 不会仅因为 `.codex/agents/*.toml` 存在就自动启动，必须由当前主线程明确要求。`dr-run` 已把这个要求写进 PM prompt：Phase 1 会调度 `dr-plan` / `dr-searcher`，Phase 2 会调度 `dr-analyst` / `dr-verifier`，Phase 3 会调度 `dr-chief-editor`。 diff --git a/docs/model-playbook.md b/docs/model-playbook.md index 86325ba..3d38d56 100644 --- a/docs/model-playbook.md +++ b/docs/model-playbook.md @@ -1,6 +1,6 @@ # Model Playbook -> v0.9 起，本文件作为模型选择攻略本。`.opencode/opencode.json` 仍是 OpenCode 的模型白名单，`configs/model_profiles.yaml` 是跨平台策略参考。 +> v0.12 起，本文件作为模型选择攻略本。`.opencode/opencode.json` 仍是 OpenCode 的模型白名单，`configs/models.yaml` 是跨平台策略参考。 ## Profiles diff --git a/docs/search-playbook.md b/docs/search-playbook.md index 910aa55..4bdebd1 100644 --- a/docs/search-playbook.md +++ b/docs/search-playbook.md @@ -2,6 +2,22 @@ > v0.9 起，本文件作为搜索 API 选择攻略本。搜索返回本身多为发现入口，结论支撑仍以 AGENTS.md 的 Tier 1-2 信源为准。 +## Default Pattern + +v0.12 起，默认搜索路径收敛到项目内 Python 网关： + +```bash +uv run python scripts/search.py "<query>" --route scholar --num-results 10 --year-low 2023 +uv run python scripts/search.py "<query>" --route patents --num-results 10 +uv run python scripts/search.py "<query>" --route news --num-results 10 --time-range m +uv run python scripts/search.py "<query>" --route general --num-results 10 +uv run python scripts/ground.py "<query>" --json +``` + +其中 `scholar / patents / news` 默认走严格模式（Serper 失败不静默降级）；需要容错时显式加 `--no-strict-specialized`。 + +MCP server 只作为交互式补漏和特殊工具能力，不作为文献、专利、新闻检索主路径。这样 OpenCode、Codex、Gemini CLI、Claude Code 都能复用同一套路由，减少每个平台单独配置 Tavily/Exa/Brave MCP 的依赖。 + ## Search Sources ### Tavily @@ -27,6 +43,7 @@ - 优点：Google Search / Scholar / News 代理，免费额度较高。 - 用法：Google Scholar、Google Patents、新闻时效检索。 - 风险：专利是 `site:patents.google.com` 技巧，不等同官方专利库。 +- 项目内调用：`scripts/search.py --route scholar|patents|news`。 ### PubMed / NCBI @@ -56,11 +73,11 @@ ### biomed_literature -PubMed / NCBI → ClinicalTrials → FDA/EMA/NMPA → Serper Scholar → Tavily/Exa 补漏。 +PubMed / NCBI → ClinicalTrials → FDA/EMA/NMPA → `scripts/search.py --route scholar` → Tavily/Exa 补漏。 ### patent_heavy -Google Patents/Serper → USPTO/EPO/CNIPA → 公司年报/招股书 → Tavily/Exa 补同族专利线索。 +`scripts/search.py --route patents` → USPTO/EPO/CNIPA → 公司年报/招股书 → Tavily/Exa 补同族专利线索。 ### china_market diff --git a/pyproject.toml b/pyproject.toml index 55895c0..af67d60 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "deep-research" -version = "0.3.0" +version = "0.12.0" description = "生物医药 Deep Research 系统 - OpenCode 多 agent 协作研究流水线" requires-python = ">=3.10" readme = "README.md" diff --git a/scripts/apply_model_profile.py b/scripts/apply_model_profile.py new file mode 100644 index 0000000..7347b09 --- /dev/null +++ b/scripts/apply_model_profile.py @@ -0,0 +1,129 @@ +#!/usr/bin/env python3 +"""Apply a model profile to agent definition files. + +Supports: +- OpenCode YAML frontmatter agents in .opencode/agents/*.md +- Codex TOML agents in codex_adapter_templates/codex/agents/*.toml +""" + +from __future__ import annotations + +import argparse +import re +import sys +from pathlib import Path + + +REPO_ROOT = Path(__file__).resolve().parent.parent +if str(REPO_ROOT) not in sys.path: + sys.path.insert(0, str(REPO_ROOT)) + +from scripts.lib.model_config import ( + ModelConfigError, + parse_model_overrides, + resolve_model_profile, +) + +OPENCODE_ROLE_TO_FILE = { + "dr_plan": ".opencode/agents/dr-plan.md", + "dr_pm": ".opencode/agents/dr-pm.md", + "dr_searcher": ".opencode/agents/dr-searcher.md", + "dr_analyst": ".opencode/agents/dr-analyst.md", + "dr_verifier": ".opencode/agents/dr-verifier.md", + "dr_chief_editor": ".opencode/agents/dr-chief-editor.md", + "dr_editor_in_chief": ".opencode/agents/dr-editor-in-chief.md", + "dr_reporter": ".opencode/agents/dr-reporter.md", +} + +CODEX_ROLE_TO_FILE = { + "dr_plan": "codex_adapter_templates/codex/agents/dr-plan.toml", + "dr_pm": "codex_adapter_templates/codex/agents/dr-pm.toml", + "dr_searcher": "codex_adapter_templates/codex/agents/dr-searcher.toml", + "dr_analyst": "codex_adapter_templates/codex/agents/dr-analyst.toml", + "dr_verifier": "codex_adapter_templates/codex/agents/dr-verifier.toml", + "dr_chief_editor": "codex_adapter_templates/codex/agents/dr-chief-editor.toml", + "dr_editor_in_chief": "codex_adapter_templates/codex/agents/dr-editor-in-chief.toml", + "dr_reporter": "codex_adapter_templates/codex/agents/dr-reporter.toml", +} + + +def replace_opencode_model(path: Path, model: str) -> bool: + text = path.read_text(encoding="utf-8") + new_text, count = re.subn(r"(?m)^model:\s*.+$", f"model: {model}", text, count=1) + if count == 0: + raise SystemExit(f"failed to locate model field: {path}") + if new_text == text: + return False + path.write_text(new_text, encoding="utf-8") + return True + + +def replace_codex_model(path: Path, model: str) -> bool: + text = path.read_text(encoding="utf-8") + new_text, count = re.subn(r'(?m)^model\s*=\s*"[^"]+"$', f'model = "{model}"', text, count=1) + if count == 0: + raise SystemExit(f"failed to locate model field: {path}") + if new_text == text: + return False + path.write_text(new_text, encoding="utf-8") + return True + + +def main() -> int: + parser = argparse.ArgumentParser(description="Apply model profile to agent files") + parser.add_argument("--profile", required=True, help="Profile name from configs/models.yaml") + parser.add_argument("--target", choices=["opencode", "codex", "both"], default="both") + parser.add_argument( + "--model-override", + action="append", + default=[], + metavar="ROLE=MODEL", + help="Override one role model, repeatable", + ) + parser.add_argument("--dry-run", action="store_true") + args = parser.parse_args() + + try: + resolved = resolve_model_profile( + profile=args.profile, + overrides=parse_model_overrides(args.model_override), + ) + except ModelConfigError as exc: + raise SystemExit(f"model profile resolution failed: {exc}") from exc + + roles = resolved["roles"] + changed: list[str] = [] + + def apply_map(mapping: dict[str, str], mode: str) -> None: + for role, rel_path in mapping.items(): + model = roles.get(role) + if not model: + continue + file_path = REPO_ROOT / rel_path + if not file_path.exists(): + continue + if args.dry_run: + changed.append(f"{mode}:{rel_path} -> {model}") + continue + did_change = replace_opencode_model(file_path, model) if mode == "opencode" else replace_codex_model(file_path, model) + if did_change: + changed.append(f"{mode}:{rel_path} -> {model}") + + if args.target in ("opencode", "both"): + apply_map(OPENCODE_ROLE_TO_FILE, "opencode") + if args.target in ("codex", "both"): + apply_map(CODEX_ROLE_TO_FILE, "codex") + + print(f"Profile applied: {resolved['profile']}") + print(f"Target: {args.target}") + if changed: + print("Updated:") + for item in changed: + print(f" - {item}") + else: + print("No files changed.") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/dr.py b/scripts/dr.py index 0b94c1c..95f674d 100644 --- a/scripts/dr.py +++ b/scripts/dr.py @@ -16,6 +16,17 @@ from pathlib import Path REPO_ROOT = Path(__file__).resolve().parent.parent +if str(REPO_ROOT) not in sys.path: + sys.path.insert(0, str(REPO_ROOT)) + +from scripts.lib.model_config import ( + ModelConfigError, + list_model_profiles, + parse_model_overrides, + resolve_model_profile, +) + + PROJECTS_DIR = REPO_ROOT / "projects" CODEX_COMMANDS_DIR = REPO_ROOT / ".codex" / "commands" CODEX_COMMAND_TEMPLATES_DIR = REPO_ROOT / "codex_adapter_templates" / "codex" / "commands" @@ -152,48 +163,82 @@ def cmd_glossary(args: argparse.Namespace) -> int: def cmd_finalize(args: argparse.Namespace) -> int: project_root = resolve_project(args.project) - steps = [ - [ - sys.executable, - str(REPO_ROOT / "scripts" / "translate.py"), - str(project_root), - "--workers", - str(args.translate_workers), - ], - [ - sys.executable, - str(REPO_ROOT / "scripts" / "build_glossary.py"), - str(project_root), - "--workers", - str(args.glossary_workers), - ], - [ - sys.executable, - str(REPO_ROOT / "scripts" / "apply_glossary.py"), - str(project_root), - "--input", - "phase4/final_zh.md", - ], - [ - sys.executable, - str(REPO_ROOT / "scripts" / "polish.py"), - str(project_root), - "--workers", - str(args.polish_workers), - ], - [ - sys.executable, - str(REPO_ROOT / "scripts" / "build_report.py"), - str(project_root), - ], + try: + resolved = resolve_model_profile( + profile=args.model_profile, + overrides=parse_model_overrides(args.model_override), + ) + except ModelConfigError as exc: + raise SystemExit(f"model profile resolution failed: {exc}") from exc + roles = resolved["roles"] + + cmd = [ + sys.executable, + str(REPO_ROOT / "scripts" / "phase4_pipeline.py"), + str(project_root), + "--translate-workers", + str(args.translate_workers), + "--glossary-workers", + str(args.glossary_workers), + "--polish-workers", + str(args.polish_workers), + "--translate-model", + roles.get("translate", "anthropic/claude-sonnet-4.6"), + "--glossary-model", + roles.get("glossary", "anthropic/claude-haiku-4.5"), + "--polish-model", + roles.get("polish", "anthropic/claude-sonnet-4.6"), + "--glossary-mode", + args.glossary_mode, ] - for step in steps: - rc = run_cmd(step, dry_run=args.dry_run) - if rc != 0: - return rc + if args.dry_run: + cmd.append("--dry-run") + return run_cmd(cmd, dry_run=False) + + +def cmd_models(args: argparse.Namespace) -> int: + if args.list: + for name in list_model_profiles(): + print(name) + return 0 + + try: + resolved = resolve_model_profile( + profile=args.profile, + overrides=parse_model_overrides(args.model_override), + ) + except ModelConfigError as exc: + raise SystemExit(f"model profile resolution failed: {exc}") from exc + + if args.json: + print(json.dumps(resolved, ensure_ascii=False, indent=2)) + return 0 + + print(f"Profile: {resolved['profile']}") + if resolved["description"]: + print(f"Description: {resolved['description']}") + print("Roles:") + for role in sorted(resolved["roles"]): + print(f" {role}: {resolved['roles'][role]}") return 0 +def cmd_apply_models(args: argparse.Namespace) -> int: + cmd = [ + sys.executable, + str(REPO_ROOT / "scripts" / "apply_model_profile.py"), + "--profile", + args.profile, + "--target", + args.target, + ] + for item in args.model_override: + cmd += ["--model-override", item] + if args.dry_run: + cmd.append("--dry-run") + return run_cmd(cmd, dry_run=False) + + def build_parser() -> argparse.ArgumentParser: parser = argparse.ArgumentParser(description="Deep Research platform-neutral CLI") sub = parser.add_subparsers(dest="cmd", required=True) @@ -219,12 +264,51 @@ def build_parser() -> argparse.ArgumentParser: finalize = sub.add_parser("finalize", help="Run Phase 4 deterministic pipeline") finalize.add_argument("project", help="Project slug or path") - finalize.add_argument("--translate-workers", type=int, default=4) + finalize.add_argument("--translate-workers", type=int, default=0) finalize.add_argument("--glossary-workers", type=int, default=4) - finalize.add_argument("--polish-workers", type=int, default=4) + finalize.add_argument("--polish-workers", type=int, default=0) + finalize.add_argument( + "--glossary-mode", + choices=["off", "low-confidence", "full"], + default="low-confidence", + ) + finalize.add_argument("--model-profile", help="Model profile name from configs/models.yaml") + finalize.add_argument( + "--model-override", + action="append", + default=[], + metavar="ROLE=MODEL", + help="Override one role model, repeatable", + ) finalize.add_argument("--dry-run", action="store_true") finalize.set_defaults(func=cmd_finalize) + models = sub.add_parser("models", help="Resolve and print model profile") + models.add_argument("--profile", help="Profile name from configs/models.yaml") + models.add_argument("--list", action="store_true", help="List available profiles") + models.add_argument( + "--model-override", + action="append", + default=[], + metavar="ROLE=MODEL", + help="Override one role model, repeatable", + ) + models.add_argument("--json", action="store_true", help="Emit JSON") + models.set_defaults(func=cmd_models) + + apply_models = sub.add_parser("apply-models", help="Apply profile to agent files") + apply_models.add_argument("--profile", required=True, help="Profile name from configs/models.yaml") + apply_models.add_argument("--target", choices=["opencode", "codex", "both"], default="both") + apply_models.add_argument( + "--model-override", + action="append", + default=[], + metavar="ROLE=MODEL", + help="Override one role model, repeatable", + ) + apply_models.add_argument("--dry-run", action="store_true") + apply_models.set_defaults(func=cmd_apply_models) + return parser diff --git a/scripts/ground.py b/scripts/ground.py new file mode 100644 index 0000000..f2eaa6a --- /dev/null +++ b/scripts/ground.py @@ -0,0 +1,68 @@ +#!/usr/bin/env python3 +"""Native web grounding wrapper via ZenMux chat completions. + +Use this when you need reproducible, model-native web search (grounding) and +machine-readable citations. +""" + +from __future__ import annotations + +import argparse +import json +from pathlib import Path + +from scripts.lib.zenmux_client import ZenMuxClient, load_secrets + + +def build_parser() -> argparse.ArgumentParser: + parser = argparse.ArgumentParser(description="Grounded web query via ZenMux") + parser.add_argument("query", help="Question or search prompt") + parser.add_argument("--model", default="google/gemini-3.1-flash-lite-preview") + parser.add_argument("--max-tokens", type=int, default=2400) + parser.add_argument("--temperature", type=float, default=0.2) + parser.add_argument("--json", action="store_true", help="Emit JSON envelope") + parser.add_argument("--log-file", help="Optional JSONL call log path") + parser.add_argument("--system", default=( + "You are a research assistant. Use web grounding when helpful. " + "Return concise facts with explicit source-backed statements." + )) + return parser + + +def main() -> int: + args = build_parser().parse_args() + load_secrets() + log_file = Path(args.log_file) if args.log_file else None + + with ZenMuxClient(log_file=log_file) as client: + result = client.chat_complete_with_meta( + model=args.model, + system=args.system, + user=args.query, + temperature=args.temperature, + max_tokens=args.max_tokens, + web_search=True, + web_search_options={}, + tag="ground", + ) + + if args.json: + payload = { + "query": args.query, + "model": args.model, + "content": result["content"], + "citations": result["citations"], + "usage": result["usage"], + } + print(json.dumps(payload, ensure_ascii=False, indent=2)) + else: + print(result["content"]) + if result["citations"]: + print("\nCitations:") + for idx, url in enumerate(result["citations"], start=1): + print(f"{idx}. {url}") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/lib/model_config.py b/scripts/lib/model_config.py new file mode 100644 index 0000000..d82f188 --- /dev/null +++ b/scripts/lib/model_config.py @@ -0,0 +1,81 @@ +"""Model profile loading and resolution utilities.""" + +from __future__ import annotations + +from pathlib import Path +from typing import Any + +import yaml + + +REPO_ROOT = Path(__file__).resolve().parents[2] +DEFAULT_MODEL_CONFIG = REPO_ROOT / "configs" / "models.yaml" +LEGACY_MODEL_CONFIG = REPO_ROOT / "configs" / "model_profiles.yaml" + + +class ModelConfigError(RuntimeError): + pass + + +def load_model_config(path: Path | None = None) -> dict[str, Any]: + cfg_path = path or DEFAULT_MODEL_CONFIG + if not cfg_path.exists() and LEGACY_MODEL_CONFIG.exists(): + cfg_path = LEGACY_MODEL_CONFIG + if not cfg_path.exists(): + raise ModelConfigError(f"model config not found: {cfg_path}") + try: + data = yaml.safe_load(cfg_path.read_text(encoding="utf-8")) or {} + except Exception as exc: + raise ModelConfigError(f"invalid YAML in {cfg_path}: {exc}") from exc + if not isinstance(data, dict): + raise ModelConfigError(f"invalid model config shape in {cfg_path}") + return data + + +def resolve_model_profile( + *, + profile: str | None = None, + overrides: dict[str, str] | None = None, + path: Path | None = None, +) -> dict[str, Any]: + cfg = load_model_config(path) + profiles = cfg.get("profiles") or {} + defaults = cfg.get("defaults") or {} + selected = profile or defaults.get("profile") + if not selected: + raise ModelConfigError("no model profile provided and no defaults.profile set") + if selected not in profiles: + raise ModelConfigError(f"unknown model profile: {selected}") + + roles = dict((profiles[selected] or {}).get("roles") or {}) + if defaults.get("script_models"): + for role, model in (defaults.get("script_models") or {}).items(): + roles.setdefault(role, model) + for role, model in (overrides or {}).items(): + roles[role] = model + + return { + "profile": selected, + "description": (profiles[selected] or {}).get("description", ""), + "roles": roles, + } + + +def list_model_profiles(path: Path | None = None) -> list[str]: + cfg = load_model_config(path) + profiles = cfg.get("profiles") or {} + return sorted(profiles.keys()) + + +def parse_model_overrides(items: list[str] | None) -> dict[str, str]: + out: dict[str, str] = {} + for item in items or []: + if "=" not in item: + raise ModelConfigError(f"invalid override '{item}', expected role=model") + role, model = item.split("=", 1) + role = role.strip() + model = model.strip() + if not role or not model: + raise ModelConfigError(f"invalid override '{item}', expected role=model") + out[role] = model + return out diff --git a/scripts/lib/search_client.py b/scripts/lib/search_client.py index 4fe3e88..af38e7f 100644 --- a/scripts/lib/search_client.py +++ b/scripts/lib/search_client.py @@ -1,11 +1,12 @@ -"""通用搜索客户端（Exa 优先，Tavily fallback）。 +"""通用搜索客户端（Serper / Exa / Tavily 路由）。 为 build_glossary.py 这类术语核查场景服务。 关键设计： - `trust_env=False` 绕开系统 socks 代理（Clash on macOS 配 socks5 时 httpx 会 TLS EOF） -- Exa 优先：LinkedIn / 官网 / 百度百科返回质量最高 -- 遇到配额问题自动降级到 Tavily 或返回 empty +- 专利 / Scholar / News 优先 Serper，保证 Google Patents / Google Scholar 路径被真正调用 +- 通用网页 Exa 优先，Tavily fallback +- 遇到配额问题自动降级或返回 empty - 不做深度 crawl，只要摘要 """ @@ -125,10 +126,11 @@ class SearchClient: 所有客户端都延迟导入 serper_client，避免没装 SERPAPI_KEY 时 import 炸。 """ - def __init__(self) -> None: + def __init__(self, *, strict_specialized: bool = True) -> None: self._exa: ExaClient | None = None self._tavily: TavilyClient | None = None self._serper = None # 惰性实例化 + self.strict_specialized = strict_specialized try: self._exa = ExaClient() except SearchError: @@ -137,10 +139,9 @@ class SearchClient: self._tavily = TavilyClient() except SearchError: pass - if not (self._exa or self._tavily): - raise SearchError( - "neither EXA_API_KEY nor TAVILY_API_KEY available" - ) + self._has_serper_key = bool(os.environ.get("SERPER_API_KEY") or os.environ.get("SERPAPI_KEY")) + if not (self._exa or self._tavily or self._has_serper_key): + raise SearchError("no search API key available: set SERPER_API_KEY, SERPAPI_KEY, EXA_API_KEY, or TAVILY_API_KEY") def _get_serper(self): """惰性创建 SerperClient。没 key 时返回 None。""" @@ -190,9 +191,12 @@ class SearchClient: try: hits = serper.patents(query, num_results=num_results) return [SearchHit(h.title, h.url, h.snippet) for h in hits] - except Exception: - pass + except Exception as exc: + if self.strict_specialized: + raise SearchError(f"serper patents failed: {exc}") from exc # 降级：通用搜索加 site 限定 + if self.strict_specialized: + raise SearchError("serper unavailable for patents route; refusing silent fallback") return self.search(f"site:patents.google.com {query}", num_results=num_results) def scholar( @@ -215,8 +219,11 @@ class SearchClient: ) for h in hits ] - except Exception: - pass + except Exception as exc: + if self.strict_specialized: + raise SearchError(f"serper scholar failed: {exc}") from exc + if self.strict_specialized: + raise SearchError("serper unavailable for scholar route; refusing silent fallback") return self.search(query, num_results=num_results) def news( @@ -239,8 +246,11 @@ class SearchClient: ) for h in hits ] - except Exception: - pass + except Exception as exc: + if self.strict_specialized: + raise SearchError(f"serper news failed: {exc}") from exc + if self.strict_specialized: + raise SearchError("serper unavailable for news route; refusing silent fallback") return self.search(query, num_results=num_results) diff --git a/scripts/lib/zenmux_client.py b/scripts/lib/zenmux_client.py index 42fc79e..db2e124 100644 --- a/scripts/lib/zenmux_client.py +++ b/scripts/lib/zenmux_client.py @@ -141,6 +141,8 @@ class ZenMuxClient: temperature: float = 0.3, max_tokens: int = 16000, extra_messages: list[dict[str, str]] | None = None, + web_search: bool = False, + web_search_options: dict[str, Any] | None = None, tag: str = "", ) -> str: """一次非流式对话补全。 @@ -167,6 +169,8 @@ class ZenMuxClient: "temperature": temperature, "max_tokens": max_tokens, } + if web_search: + body["web_search_options"] = web_search_options or {} headers = { "Authorization": f"Bearer {self.api_key}", "Content-Type": "application/json", @@ -233,6 +237,116 @@ class ZenMuxClient: self.usage.failed_calls += 1 raise ZenMuxError(f"max retries exhausted. last error: {last_error}") + def chat_complete_with_meta( + self, + model: str, + system: str, + user: str, + *, + temperature: float = 0.3, + max_tokens: int = 16000, + extra_messages: list[dict[str, str]] | None = None, + web_search: bool = False, + web_search_options: dict[str, Any] | None = None, + tag: str = "", + ) -> dict[str, Any]: + """Return content plus metadata from one completion call.""" + messages: list[dict[str, str]] = [{"role": "system", "content": system}] + if extra_messages: + messages.extend(extra_messages) + messages.append({"role": "user", "content": user}) + + body: dict[str, Any] = { + "model": model, + "messages": messages, + "temperature": temperature, + "max_tokens": max_tokens, + } + if web_search: + body["web_search_options"] = web_search_options or {} + + headers = { + "Authorization": f"Bearer {self.api_key}", + "Content-Type": "application/json", + } + url = f"{self.base_url}/chat/completions" + + last_error = "" + for attempt in range(MAX_RETRIES): + t0 = time.time() + try: + resp = self._client.post(url, json=body, headers=headers) + elapsed = time.time() - t0 + except httpx.RequestError as e: + last_error = f"network: {e}" + elapsed = time.time() - t0 + self._log({"tag": tag, "attempt": attempt, "elapsed": elapsed, "error": last_error}) + time.sleep(2 ** attempt) + continue + + if resp.status_code != 200: + retryable = resp.status_code in RETRYABLE_STATUSES + last_error = f"HTTP {resp.status_code}: {resp.text[:500]}" + self._log({ + "tag": tag, + "attempt": attempt, + "elapsed": round(elapsed, 2), + "status": resp.status_code, + "error": last_error, + "retryable": retryable, + }) + if not retryable: + self.usage.failed_calls += 1 + raise ZenMuxError(last_error) + sleep_for = min(60, (2 ** attempt) + (attempt * 0.5)) + time.sleep(sleep_for) + continue + + try: + data = resp.json() + except Exception as e: + raise ZenMuxError(f"invalid JSON from zenmux: {e}; body={resp.text[:500]}") + + usage = data.get("usage", {}) or {} + with self._usage_lock: + self.usage.add(model, usage) + + message = ((data.get("choices") or [{}])[0].get("message") or {}) + content = message.get("content") or "" + annotations = message.get("annotations") or [] + urls: list[str] = [] + for ann in annotations: + if not isinstance(ann, dict): + continue + citation = ann.get("url_citation") or {} + url_item = citation.get("url") + if url_item: + urls.append(url_item) + self._log({ + "tag": tag, + "model": model, + "attempt": attempt, + "elapsed": round(elapsed, 2), + "usage": usage, + "out_chars": len(content), + "status": 200, + "web_search": web_search, + "citations": len(urls), + }) + if not content.strip(): + last_error = "empty content" + time.sleep(2 ** attempt) + continue + return { + "content": content, + "usage": usage, + "citations": urls, + "raw": data, + } + + self.usage.failed_calls += 1 + raise ZenMuxError(f"max retries exhausted. last error: {last_error}") + def load_secrets(env_path: Path | None = None) -> None: """从 secrets.env 把 key 塞到 os.environ，便于脚本直接运行。 diff --git a/scripts/phase4_pipeline.py b/scripts/phase4_pipeline.py new file mode 100644 index 0000000..0559b28 --- /dev/null +++ b/scripts/phase4_pipeline.py @@ -0,0 +1,183 @@ +#!/usr/bin/env python3 +"""Phase 4 replacement pipeline orchestrator. + +Default flow: +1) translate.py +2) optional glossary verification (low-confidence/full/off) +3) apply_glossary.py +4) polish.py +5) build_report.py +""" + +from __future__ import annotations + +import argparse +import json +import os +import subprocess +import sys +import time +from pathlib import Path + + +REPO_ROOT = Path(__file__).resolve().parent.parent +if str(REPO_ROOT) not in sys.path: + sys.path.insert(0, str(REPO_ROOT)) + +from scripts.lib.markdown_chunker import split_by_headers + + +def resolve_project(arg: str) -> Path: + p = Path(arg) + if p.is_dir(): + return p.resolve() + cand = REPO_ROOT / "projects" / arg + if cand.is_dir(): + return cand.resolve() + raise SystemExit(f"project not found: {arg}") + + +def run_step(cmd: list[str], *, dry_run: bool) -> int: + print("$ " + " ".join(cmd)) + if dry_run: + return 0 + return subprocess.run(cmd, cwd=REPO_ROOT, check=False).returncode + + +def infer_workers(source_file: Path, fallback: int, cap: int = 8) -> int: + if not source_file.exists(): + return fallback + text = source_file.read_text(encoding="utf-8") + blocks = split_by_headers(text, max_level=2) + if not blocks: + return fallback + cpu_cap = max(2, min(cap, (os.cpu_count() or 4))) + suggested = max(2, min(cpu_cap, (len(blocks) + 5) // 6)) + return max(1, suggested if fallback <= 0 else min(max(fallback, 1), cpu_cap)) + + +def low_confidence_terms(glossary_path: Path) -> list[str]: + if not glossary_path.exists(): + return [] + try: + glossary = json.loads(glossary_path.read_text(encoding="utf-8")) + except Exception: + return [] + out: list[str] = [] + for term, entry in glossary.items(): + if not isinstance(entry, dict): + out.append(term) + continue + conf = str(entry.get("confidence", "")).lower() + verified = bool(entry.get("verified_at")) + if conf != "high" or not verified: + out.append(term) + return sorted(set(out)) + + +def main() -> int: + parser = argparse.ArgumentParser(description="Phase 4 replacement pipeline") + parser.add_argument("project", help="Project slug or full path") + parser.add_argument("--translate-model", default="anthropic/claude-sonnet-4.6") + parser.add_argument("--glossary-model", default="anthropic/claude-haiku-4.5") + parser.add_argument("--polish-model", default="anthropic/claude-sonnet-4.6") + parser.add_argument("--translate-workers", type=int, default=0, help="0 means auto") + parser.add_argument("--glossary-workers", type=int, default=4) + parser.add_argument("--polish-workers", type=int, default=0, help="0 means auto") + parser.add_argument( + "--glossary-mode", + choices=["off", "low-confidence", "full"], + default="low-confidence", + help="off: skip, low-confidence: verify only low-confidence terms, full: verify all", + ) + parser.add_argument("--dry-run", action="store_true") + args = parser.parse_args() + + project_root = resolve_project(args.project) + phase4 = project_root / "phase4" + src_en = phase4 / "final_en.md" + if not src_en.exists(): + raise SystemExit(f"missing source: {src_en}") + + tw = infer_workers(src_en, args.translate_workers) + zh = phase4 / "final_zh.md" + pw = infer_workers(zh if zh.exists() else src_en, args.polish_workers) + + print(f"Project: {project_root.name}") + print(f"Translate workers: {tw} | Polish workers: {pw}") + print(f"Glossary mode: {args.glossary_mode}") + print() + + t0 = time.time() + steps: list[list[str]] = [ + [ + sys.executable, + str(REPO_ROOT / "scripts" / "translate.py"), + str(project_root), + "--workers", + str(tw), + "--model", + args.translate_model, + ] + ] + + if args.glossary_mode != "off": + gcmd = [ + sys.executable, + str(REPO_ROOT / "scripts" / "build_glossary.py"), + str(project_root), + "--workers", + str(args.glossary_workers), + "--model", + args.glossary_model, + ] + if args.glossary_mode == "low-confidence": + terms = low_confidence_terms(phase4 / "glossary.json") + if terms: + if len(terms) > 80: + print(f"[info] low-confidence terms={len(terms)} is large; fallback to full glossary verify") + else: + gcmd += ["--only", ",".join(terms)] + else: + print("[info] no low-confidence glossary terms found; skipping glossary step") + gcmd = [] + if gcmd: + steps.append(gcmd) + + steps.extend( + [ + [ + sys.executable, + str(REPO_ROOT / "scripts" / "apply_glossary.py"), + str(project_root), + "--input", + "phase4/final_zh.md", + ], + [ + sys.executable, + str(REPO_ROOT / "scripts" / "polish.py"), + str(project_root), + "--workers", + str(pw), + "--model", + args.polish_model, + ], + [ + sys.executable, + str(REPO_ROOT / "scripts" / "build_report.py"), + str(project_root), + ], + ] + ) + + for cmd in steps: + rc = run_step(cmd, dry_run=args.dry_run) + if rc != 0: + return rc + + print(f"\nPhase 4 pipeline done in {time.time() - t0:.1f}s") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/search.py b/scripts/search.py new file mode 100644 index 0000000..27dfbc6 --- /dev/null +++ b/scripts/search.py @@ -0,0 +1,183 @@ +#!/usr/bin/env python3 +"""Unified search gateway for Deep Research agents. + +This script is the stable project-owned entrypoint that agents should call +instead of vendor MCP tools. MCP search remains optional, while this gateway +keeps routing behavior reproducible across OpenCode, Codex, and future +adapters. +""" + +from __future__ import annotations + +import argparse +import json +import sys +from dataclasses import asdict +from pathlib import Path + + +REPO_ROOT = Path(__file__).resolve().parent.parent +if str(REPO_ROOT) not in sys.path: + sys.path.insert(0, str(REPO_ROOT)) + +from scripts.lib.search_client import SearchClient, SearchError, SearchHit +from scripts.lib.zenmux_client import load_secrets + + +ROUTE_HELP = { + "general": "Exa -> Tavily generic web discovery", + "scholar": "Serper Scholar -> generic fallback", + "patents": "Serper Google Patents -> site:patents.google.com fallback", + "news": "Serper News -> generic fallback", +} + +PROFILE_ROUTES = { + "biomed_literature": ["scholar", "general"], + "patent_heavy": ["patents", "general"], + "china_market": ["news", "general"], + "investment": ["news", "general"], +} + +PROFILE_QUERY_PREFIX = { + "china_market": "(China OR Chinese OR 中国 OR 国内)", +} + + +def search_route(client: SearchClient, route: str, query: str, args: argparse.Namespace) -> list[SearchHit]: + if route == "general": + return client.search(query, num_results=args.num_results) + if route == "scholar": + return client.scholar(query, num_results=args.num_results, year_low=args.year_low) + if route == "patents": + return client.patents(query, num_results=args.num_results) + if route == "news": + return client.news(query, num_results=args.num_results, time_range=args.time_range) + raise SystemExit(f"unknown route: {route}") + + +def emit_markdown(route_hits: list[tuple[str, list[SearchHit]]], query: str) -> None: + print(f"# Search Results: {query}") + for route, hits in route_hits: + print() + print(f"## Route: {route} ({ROUTE_HELP[route]})") + if not hits: + print("No results.") + continue + for i, hit in enumerate(hits, start=1): + print(f"{i}. {hit.title or '(untitled)'}") + print(f" - URL: {hit.url}") + if hit.snippet: + print(f" - Snippet: {hit.snippet}") + + +def emit_json(route_hits: list[tuple[str, list[SearchHit]]], query: str) -> None: + data = { + "query": query, + "routes": [ + { + "route": route, + "route_help": ROUTE_HELP[route], + "results": [asdict(hit) for hit in hits], + } + for route, hits in route_hits + ], + } + print(json.dumps(data, ensure_ascii=False, indent=2)) + + +def emit_trace_markdown(route_trace: list[dict[str, str]]) -> None: + print() + print("## Route Trace") + for item in route_trace: + print(f"- {item['route']}: {item['status']} ({item['detail']})") + + +def build_parser() -> argparse.ArgumentParser: + parser = argparse.ArgumentParser(description="Deep Research search gateway") + parser.add_argument("query", help="Search query") + parser.add_argument( + "--route", + choices=sorted(ROUTE_HELP), + default="general", + help="Single search route to run", + ) + parser.add_argument( + "--profile", + choices=sorted(PROFILE_ROUTES), + help="Run a strategy profile instead of a single route", + ) + parser.add_argument("--num-results", type=int, default=10) + parser.add_argument("--year-low", type=int, help="Lower year bound for scholar searches") + parser.add_argument("--time-range", choices=["d", "w", "m", "y"], help="Serper news time range") + parser.add_argument("--json", action="store_true", help="Emit JSON instead of Markdown") + parser.add_argument("--dry-run", action="store_true", help="Show planned routes without calling APIs") + parser.add_argument( + "--strict-specialized", + action=argparse.BooleanOptionalAction, + default=True, + help="Fail fast if scholar/news/patents cannot use Serper", + ) + parser.add_argument("--trace", action="store_true", help="Include route execution trace") + return parser + + +def main() -> int: + parser = build_parser() + args = parser.parse_args() + load_secrets() + + routes = PROFILE_ROUTES[args.profile] if args.profile else [args.route] + query = args.query + if args.profile in PROFILE_QUERY_PREFIX: + query = f"{PROFILE_QUERY_PREFIX[args.profile]} {query}" + + if args.dry_run: + for route in routes: + print(f"{route}: {ROUTE_HELP[route]}") + if query != args.query: + print(f"query_rewritten: {query}") + return 0 + + try: + with SearchClient(strict_specialized=args.strict_specialized) as client: + route_hits = [] + route_trace: list[dict[str, str]] = [] + for route in routes: + try: + hits = search_route(client, route, query, args) + route_hits.append((route, hits)) + route_trace.append({"route": route, "status": "ok", "detail": f"hits={len(hits)}"}) + except SearchError as exc: + route_hits.append((route, [])) + route_trace.append({"route": route, "status": "failed", "detail": str(exc)}) + if route != "general": + continue + raise + except SearchError as exc: + raise SystemExit(f"search failed: {exc}") from exc + + if args.json: + data = { + "query": query, + "original_query": args.query, + "strict_specialized": args.strict_specialized, + "routes": [ + { + "route": route, + "route_help": ROUTE_HELP[route], + "results": [asdict(hit) for hit in hits], + } + for route, hits in route_hits + ], + "trace": route_trace if args.trace else [], + } + print(json.dumps(data, ensure_ascii=False, indent=2)) + else: + emit_markdown(route_hits, query) + if args.trace: + emit_trace_markdown(route_trace) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/sprint5_regression.py b/scripts/sprint5_regression.py new file mode 100644 index 0000000..fec558f --- /dev/null +++ b/scripts/sprint5_regression.py @@ -0,0 +1,102 @@ +#!/usr/bin/env python3 +"""Sprint 5 regression checks for v0.12 changes. + +Checks are non-destructive and default to dry-run behavior. +""" + +from __future__ import annotations + +import argparse +import subprocess +import sys +from pathlib import Path + + +REPO_ROOT = Path(__file__).resolve().parent.parent + + +def run(cmd: list[str]) -> tuple[int, str]: + proc = subprocess.run( + cmd, + cwd=REPO_ROOT, + check=False, + capture_output=True, + text=True, + ) + out = (proc.stdout or "") + (proc.stderr or "") + return proc.returncode, out + + +def check(name: str, cmd: list[str], must_contain: list[str] | None = None) -> bool: + print(f"[check] {name}") + print(" $ " + " ".join(cmd)) + rc, out = run(cmd) + if rc != 0: + print(f" FAIL: exit={rc}") + if out.strip(): + print(" output:") + print(" " + out.strip().replace("\n", "\n ")) + return False + for token in must_contain or []: + if token not in out: + print(f" FAIL: missing token '{token}'") + return False + print(" PASS") + return True + + +def main() -> int: + parser = argparse.ArgumentParser(description="Run Sprint 5 regression checks") + parser.add_argument("project", help="Project slug or path for finalize dry-run") + args = parser.parse_args() + + checks = [ + ( + "model profiles list", + ["uv", "run", "python", "scripts/dr.py", "models", "--list"], + ["medium", "premium", "simple"], + ), + ( + "search gateway dry-run", + [ + "uv", + "run", + "python", + "scripts/search.py", + "GLP-1 obesity", + "--profile", + "china_market", + "--dry-run", + ], + ["news:", "general:", "query_rewritten:"], + ), + ( + "phase4 finalize dry-run", + [ + "uv", + "run", + "python", + "scripts/dr.py", + "finalize", + args.project, + "--model-profile", + "medium", + "--dry-run", + ], + ["Phase 4 pipeline done"], + ), + ] + + ok = True + for name, cmd, tokens in checks: + ok = check(name, cmd, tokens) and ok + + if not ok: + print("\nSprint 5 regression: FAILED") + return 1 + print("\nSprint 5 regression: PASSED") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main())