v0.9: parallelize phase4 and add model/search playbooks
This commit is contained in:
@@ -62,13 +62,15 @@ uv run python scripts/build_glossary.py <slug> --workers 4
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
# 先预览
|
# 先预览
|
||||||
uv run python scripts/apply_glossary.py <slug> --dry-run
|
uv run python scripts/apply_glossary.py <slug> --input phase4/final_zh.md --dry-run
|
||||||
|
|
||||||
# 确认无误后应用
|
# 确认无误后应用
|
||||||
uv run python scripts/apply_glossary.py <slug>
|
uv run python scripts/apply_glossary.py <slug> --input phase4/final_zh.md
|
||||||
```
|
```
|
||||||
|
|
||||||
这会把 glossary 中发现的拼写错误 / 错译直接替换进 `final_zh.md`。
|
这会把 glossary 中发现的拼写错误 / 错译直接替换进 `final_zh.md`。
|
||||||
|
如润色后仍需二次复核,可手动对 `final_zh_polished.md` 再运行一次:
|
||||||
|
`uv run python scripts/apply_glossary.py <slug> --input phase4/final_zh_polished.md --dry-run`。
|
||||||
|
|
||||||
## Step 6: 润色(Python 脚本)
|
## Step 6: 润色(Python 脚本)
|
||||||
|
|
||||||
|
|||||||
@@ -1,8 +1,8 @@
|
|||||||
# Deep Research 系统方案(OpenCode 实现)
|
# Deep Research 系统方案(OpenCode 实现)
|
||||||
|
|
||||||
> 本文件是整套方案的**单一真实源**,中断后续接时从此文件恢复上下文。
|
> 本文件是整套方案的**单一真实源**,中断后续接时从此文件恢复上下文。
|
||||||
> 最后更新:2026-04-20
|
> 最后更新:2026-04-24
|
||||||
> 实施阶段:路径 2 — 最小可用先行(MVP)
|
> 实施阶段:v0.9 — OpenCode 全流程 + Phase 4 并发化 + Codex 第一阶段兼容
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -492,3 +492,32 @@ OpenCode 的坑:如果只是在主会话里装样子地写"让 X agent 做",
|
|||||||
- 反驳证据段标题规范化(建议从"反方证据/Counter-Evidence"改为观点化标题如"另一种声音")
|
- 反驳证据段标题规范化(建议从"反方证据/Counter-Evidence"改为观点化标题如"另一种声音")
|
||||||
- build_glossary 默认放到 Phase 2 阶段运行,在源头拦截错误
|
- build_glossary 默认放到 Phase 2 阶段运行,在源头拦截错误
|
||||||
- 提示 dr-analyst 加强对公司名/机构名的搜索验证流程
|
- 提示 dr-analyst 加强对公司名/机构名的搜索验证流程
|
||||||
|
|
||||||
|
- 2026-04-24 v0.9:**Phase 4 并发提速 + 模型/搜索攻略本 + Codex 兼容**
|
||||||
|
|
||||||
|
**目标**:在不破坏 OpenCode 主流程的前提下,把 v0.6 Python 化 Phase 4 进一步提速,并补齐跨平台使用说明。OpenCode 仍是主适配器;Codex 第一阶段只复用 `AGENTS.md` 与 Python 脚本,不复刻 OpenCode subagent。
|
||||||
|
|
||||||
|
**Phase 4 并发化**:
|
||||||
|
- `scripts/translate.py` 新增 `--workers`,默认 4;设为 1 时回退串行。
|
||||||
|
- 翻译阶段改为"稳定术语表快照 + 并发 chunk 翻译 + 事后统一合并 glossary patch",避免多线程同时写 `glossary.json`。
|
||||||
|
- `scripts/polish.py` 新增 `--workers`,默认 4;润色块彼此独立,按完成顺序写 chunk,最终按原始 order 合并。
|
||||||
|
- `scripts/lib/zenmux_client.py` 增加日志与 usage 聚合锁,避免并发 JSONL 日志交错或 token 统计竞争。
|
||||||
|
|
||||||
|
**流程修正**:
|
||||||
|
- 修正 `apply_glossary.py` 默认输入,从 `phase4/final_zh_polished.md` 改为 `phase4/final_zh.md`。
|
||||||
|
- `/dr-finalize` 明确默认顺序:`translate.py → build_glossary.py → apply_glossary.py --input phase4/final_zh.md → polish.py → build_report.py`。
|
||||||
|
- 保留二次修正选项:润色后可手动对 `final_zh_polished.md` 再跑一次 `apply_glossary.py --input phase4/final_zh_polished.md --dry-run`。
|
||||||
|
|
||||||
|
**模型与搜索攻略本**:
|
||||||
|
- 新增 `docs/model-playbook.md`:定义 premium / balanced / budget / cn-heavy / verifier 五套模型策略。
|
||||||
|
- 新增 `docs/search-playbook.md`:说明 Tavily / Exa / Brave / Serper / PubMed / ClinicalTrials / FDA/EMA/NMPA / Patents 的使用边界。
|
||||||
|
- 新增 `configs/model_profiles.yaml` 与 `configs/search_profiles.yaml`,作为跨平台、人类和 agent 共用的策略配置参考;当前不强制重构 `.opencode/agents` 自动读取。
|
||||||
|
|
||||||
|
**Codex 兼容**:
|
||||||
|
- 新增 `docs/codex-usage.md`,说明 Codex 下如何遵循 `AGENTS.md`、运行 Phase 4 Python 流水线、检查 git staging,避免误提交 `projects/**` 研究产物。
|
||||||
|
- Codex v1 定位为"审阅/规划/修补/执行脚本";确定性编排继续放在 Python 脚本,OpenCode subagent 调度暂不移植。
|
||||||
|
|
||||||
|
**Git 管理要求**:
|
||||||
|
- 本轮迭代应在独立分支推送到 Gitea。
|
||||||
|
- 提交范围仅限系统文件和文档:`README.md`、`PLAN.md`、`scripts/**`、`docs/**`、`configs/**`、必要的 `.opencode/commands/**`。
|
||||||
|
- 不提交 `projects/**`、生成的 PDF/DOCX/TXT、一次性研究产物或本地临时脚本。
|
||||||
|
|||||||
@@ -2,8 +2,8 @@
|
|||||||
|
|
||||||
> 生物医药行业的 AI 驱动深度研究流水线。基于 OpenCode 多 agent 协作,以麦肯锡/德勤式方法论产出专业级研究报告(PDF + DOCX)。
|
> 生物医药行业的 AI 驱动深度研究流水线。基于 OpenCode 多 agent 协作,以麦肯锡/德勤式方法论产出专业级研究报告(PDF + DOCX)。
|
||||||
|
|
||||||
**当前状态**:MVP(路径 2 — 最小可用先行),仅实现 Phase 1 能力。
|
**当前状态**:v0.9 核心迭代。OpenCode 全流程可用(Phase 1-4),Phase 4 已切换为 Python 脚本化流水线,并新增 Codex 第一阶段兼容说明。
|
||||||
详见 `PLAN.md` 了解完整方案与迭代路径。
|
详见 `PLAN.md` 了解完整方案、版本记录与迭代路径。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -108,22 +108,23 @@ source scripts/activate.sh
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## MVP 可用命令
|
## 可用命令
|
||||||
|
|
||||||
| 命令 | 功能 | 状态 |
|
| 命令 | 功能 | 状态 |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| `/dr-init <主题>` | 初始化新研究,启动访谈 | ✅ MVP |
|
| `/dr-init <主题>` | 初始化新研究,启动访谈 | ✅ 可用 |
|
||||||
| `/dr-frame [slug]` | Phase 1:生成 8-15 章研究框架 | ✅ MVP |
|
| `/dr-frame [slug]` | Phase 1:生成 8-15 章双语研究框架 | ✅ 可用 |
|
||||||
| `/dr-research` | Phase 2:深度研究(并行) | ⏳ 下一阶段 |
|
| `/dr-research [slug]` | Phase 2:并行深度研究 | ✅ 可用 |
|
||||||
| `/dr-review` | Phase 3:总编审校 | ⏳ 下一阶段 |
|
| `/dr-review [slug]` | Phase 3:总编审校 | ✅ 可用 |
|
||||||
| `/dr-finalize` | Phase 4:成稿 PDF+DOCX | ⏳ 下一阶段 |
|
| `/dr-finalize [slug]` | Phase 4:英文合稿 → 中文翻译/术语核查/润色 → PDF+DOCX | ✅ 可用 |
|
||||||
| `/dr-status` | 查看进度 | ⏳ 下一阶段 |
|
| `/dr-glossary [slug]` | 术语表事实核查 | ✅ 可用 |
|
||||||
|
| `/dr-status [slug]` | 查看进度 | ✅ 可用 |
|
||||||
|
|
||||||
### 典型 MVP 流程
|
### 典型流程
|
||||||
|
|
||||||
```
|
```
|
||||||
1. /dr-init GLP-1 减重药物市场
|
1. /dr-init GLP-1 减重药物市场
|
||||||
→ dr-plan 向你提 6-8 个访谈问题(研究类型、受众、时间范围等)
|
→ dr-plan 向你提 8 个访谈问题(研究类型、受众、时间范围等)
|
||||||
→ 你回答后,生成 projects/glp1-obesity-market-2026/manifest.json
|
→ 你回答后,生成 projects/glp1-obesity-market-2026/manifest.json
|
||||||
|
|
||||||
2. /dr-frame
|
2. /dr-frame
|
||||||
@@ -135,9 +136,45 @@ source scripts/activate.sh
|
|||||||
3. 你审核框架,或提修改意见,或直接确认
|
3. 你审核框架,或提修改意见,或直接确认
|
||||||
→ 确认后,manifest.phase1.approved = true
|
→ 确认后,manifest.phase1.approved = true
|
||||||
|
|
||||||
4. (后续)/dr-research 触发 Phase 2 深研 — 目前未实现
|
4. /dr-research
|
||||||
|
→ dr-pm 分批并行调度 dr-analyst 深研
|
||||||
|
→ dr-verifier 做反方验证
|
||||||
|
→ 产出 phase2/drafts、evidence、sources.jsonl
|
||||||
|
|
||||||
|
5. /dr-review
|
||||||
|
→ dr-chief-editor 通读审校,产出 phase3/critique.md
|
||||||
|
|
||||||
|
6. /dr-finalize
|
||||||
|
→ dr-editor-in-chief 合并英文终稿
|
||||||
|
→ Python 脚本执行 translate → glossary → apply_glossary → polish → build_report
|
||||||
|
→ 产出 final_zh_polished.md、PDF、DOCX
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Phase 4 Python 流水线
|
||||||
|
|
||||||
|
Phase 4 已不再依赖单个 LLM agent 一次性翻译整篇报告,而是由 Python 控制切块、并发、重试与断点续传:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
uv run python scripts/translate.py <slug> --workers 4
|
||||||
|
uv run python scripts/build_glossary.py <slug> --workers 4
|
||||||
|
uv run python scripts/apply_glossary.py <slug> --input phase4/final_zh.md --dry-run
|
||||||
|
uv run python scripts/apply_glossary.py <slug> --input phase4/final_zh.md
|
||||||
|
uv run python scripts/polish.py <slug> --workers 4
|
||||||
|
uv run python scripts/build_report.py <slug>
|
||||||
|
```
|
||||||
|
|
||||||
|
网络不稳或 API 限流时,把 `--workers` 降到 `3` 或 `1` 即可断点续跑。
|
||||||
|
|
||||||
|
### 多平台兼容
|
||||||
|
|
||||||
|
- OpenCode:主适配器,使用 `.opencode/agents` 与 `.opencode/commands`。
|
||||||
|
- Codex:第一阶段兼容,使用 `AGENTS.md` + Python 脚本链路,详见 `docs/codex-usage.md`。
|
||||||
|
- Gemini CLI / Claude Code:暂不做强适配,后续以同一套脚本与配置为基础扩展。
|
||||||
|
|
||||||
|
模型与搜索 API 选择见:
|
||||||
|
- `docs/model-playbook.md`
|
||||||
|
- `docs/search-playbook.md`
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 项目结构
|
## 项目结构
|
||||||
|
|||||||
@@ -0,0 +1,62 @@
|
|||||||
|
profiles:
|
||||||
|
premium:
|
||||||
|
description: "Highest quality for formal deliverables."
|
||||||
|
roles:
|
||||||
|
dr_plan: "zenmux-anthropic/claude-opus-4-7"
|
||||||
|
dr_pm: "zenmux-anthropic/claude-sonnet-4-6"
|
||||||
|
dr_analyst: "zenmux-anthropic/claude-sonnet-4-6"
|
||||||
|
dr_chief_editor: "zenmux/google/gemini-3.1-pro-preview"
|
||||||
|
dr_editor_in_chief: "zenmux-anthropic/claude-opus-4-7"
|
||||||
|
dr_verifier: "zenmux/openai/gpt-5.4"
|
||||||
|
translate: "anthropic/claude-sonnet-4.6"
|
||||||
|
polish: "anthropic/claude-sonnet-4.6"
|
||||||
|
glossary: "anthropic/claude-haiku-4.5"
|
||||||
|
balanced:
|
||||||
|
description: "Default profile for most reports."
|
||||||
|
roles:
|
||||||
|
dr_plan: "zenmux-anthropic/claude-opus-4-7"
|
||||||
|
dr_pm: "zenmux-anthropic/claude-sonnet-4-6"
|
||||||
|
dr_analyst: "zenmux-anthropic/claude-sonnet-4-6"
|
||||||
|
dr_chief_editor: "zenmux/google/gemini-3.1-pro-preview"
|
||||||
|
dr_editor_in_chief: "zenmux-anthropic/claude-opus-4-7"
|
||||||
|
dr_verifier: "zenmux/qwen/qwen3.6-plus"
|
||||||
|
translate: "anthropic/claude-sonnet-4.6"
|
||||||
|
polish: "anthropic/claude-sonnet-4.6"
|
||||||
|
glossary: "anthropic/claude-haiku-4.5"
|
||||||
|
budget:
|
||||||
|
description: "Lower-cost exploratory profile; requires stronger review."
|
||||||
|
roles:
|
||||||
|
dr_plan: "zenmux/qwen/qwen3.6-plus"
|
||||||
|
dr_pm: "zenmux/qwen/qwen3.6-plus"
|
||||||
|
dr_analyst: "zenmux/deepseek/deepseek-v3.2"
|
||||||
|
dr_chief_editor: "zenmux/google/gemini-2.5-pro"
|
||||||
|
dr_editor_in_chief: "zenmux-anthropic/claude-sonnet-4-6"
|
||||||
|
dr_verifier: "zenmux/minimax/minimax-m2.7"
|
||||||
|
translate: "anthropic/claude-haiku-4.5"
|
||||||
|
polish: "anthropic/claude-haiku-4.5"
|
||||||
|
glossary: "anthropic/claude-haiku-4.5"
|
||||||
|
cn_heavy:
|
||||||
|
description: "China-market-heavy research with more Chinese-source verification."
|
||||||
|
roles:
|
||||||
|
dr_plan: "zenmux-anthropic/claude-opus-4-7"
|
||||||
|
dr_pm: "zenmux-anthropic/claude-sonnet-4-6"
|
||||||
|
dr_analyst: "zenmux-anthropic/claude-sonnet-4-6"
|
||||||
|
dr_chief_editor: "zenmux/google/gemini-3.1-pro-preview"
|
||||||
|
dr_editor_in_chief: "zenmux-anthropic/claude-opus-4-7"
|
||||||
|
dr_verifier: "zenmux/qwen/qwen3.6-plus"
|
||||||
|
translate: "anthropic/claude-sonnet-4.6"
|
||||||
|
polish: "anthropic/claude-sonnet-4.6"
|
||||||
|
glossary: "anthropic/claude-haiku-4.5"
|
||||||
|
verifier:
|
||||||
|
description: "Cross-model verification profile; avoid Claude for verifier."
|
||||||
|
roles:
|
||||||
|
dr_verifier_primary: "zenmux/openai/gpt-5.4"
|
||||||
|
dr_verifier_backup_a: "zenmux/qwen/qwen3.6-plus"
|
||||||
|
dr_verifier_backup_b: "zenmux/minimax/minimax-m2.7"
|
||||||
|
dr_verifier_backup_c: "zenmux/moonshotai/kimi-k2.5"
|
||||||
|
|
||||||
|
defaults:
|
||||||
|
profile: "balanced"
|
||||||
|
translate_workers: 4
|
||||||
|
polish_workers: 4
|
||||||
|
glossary_workers: 4
|
||||||
@@ -0,0 +1,60 @@
|
|||||||
|
profiles:
|
||||||
|
biomed_literature:
|
||||||
|
description: "Mechanism, clinical, systematic review, and biomedical evidence."
|
||||||
|
order:
|
||||||
|
- "pubmed_ncbi"
|
||||||
|
- "clinicaltrials"
|
||||||
|
- "fda_ema_nmpa"
|
||||||
|
- "serper_scholar"
|
||||||
|
- "tavily_exa_gap_fill"
|
||||||
|
patent_heavy:
|
||||||
|
description: "IP landscape, freedom-to-operate, and process-route research."
|
||||||
|
order:
|
||||||
|
- "google_patents_via_serper"
|
||||||
|
- "uspto_epo_cnipa"
|
||||||
|
- "company_disclosures"
|
||||||
|
- "exa_tavily_family_discovery"
|
||||||
|
china_market:
|
||||||
|
description: "China regulatory, company, supply-chain, and market research."
|
||||||
|
order:
|
||||||
|
- "nmpa_cde"
|
||||||
|
- "exchange_disclosures"
|
||||||
|
- "serper_brave_chinese"
|
||||||
|
- "exa_company_pages"
|
||||||
|
- "tavily_gap_fill"
|
||||||
|
investment:
|
||||||
|
description: "Public-company, market-size, and transaction-oriented research."
|
||||||
|
order:
|
||||||
|
- "sec_exchange_filings"
|
||||||
|
- "consulting_and_database_reports"
|
||||||
|
- "company_announcements"
|
||||||
|
- "serper_news"
|
||||||
|
- "tavily_gap_fill"
|
||||||
|
|
||||||
|
apis:
|
||||||
|
tavily:
|
||||||
|
best_for: ["initial_scan", "gap_fill", "llm_friendly_snippets"]
|
||||||
|
evidence_role: "discovery_only_unless_original_source"
|
||||||
|
exa:
|
||||||
|
best_for: ["company_pages", "terminology_check", "long_tail_professional_pages"]
|
||||||
|
evidence_role: "discovery_or_secondary"
|
||||||
|
brave:
|
||||||
|
best_for: ["cross_check", "counter_evidence", "mixed_language_search"]
|
||||||
|
evidence_role: "discovery_only_unless_original_source"
|
||||||
|
serper:
|
||||||
|
best_for: ["google_scholar", "google_patents", "news"]
|
||||||
|
evidence_role: "discovery_or_secondary"
|
||||||
|
pubmed_ncbi:
|
||||||
|
best_for: ["tier1_literature", "systematic_reviews", "meta_analysis"]
|
||||||
|
evidence_role: "tier1_or_tier2"
|
||||||
|
clinicaltrials:
|
||||||
|
best_for: ["trial_registration", "pipeline_status", "endpoint_design"]
|
||||||
|
evidence_role: "tier1"
|
||||||
|
regulatory:
|
||||||
|
best_for: ["approval_status", "label", "safety", "review_documents"]
|
||||||
|
evidence_role: "tier1"
|
||||||
|
|
||||||
|
defaults:
|
||||||
|
glossary_workers: 4
|
||||||
|
unstable_network_workers: 3
|
||||||
|
trace_back_to_tier_1_2: true
|
||||||
@@ -0,0 +1,85 @@
|
|||||||
|
# Codex Usage
|
||||||
|
|
||||||
|
> v0.9 的 Codex 兼容是第一阶段:不复刻 OpenCode subagent,而是沿用 AGENTS.md 的研究规则和 Python 脚本流水线。Codex 负责规划、审阅、修补、执行脚本;确定性编排交给脚本。
|
||||||
|
|
||||||
|
## Setup
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /Users/tankai/Documents/Projects/deep_research
|
||||||
|
source scripts/activate.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
Codex 会读取项目根目录的 `AGENTS.md`,研究方法、信源分级、输出规范仍与 OpenCode 一致。
|
||||||
|
|
||||||
|
## Recommended Codex Workflow
|
||||||
|
|
||||||
|
### Phase 1-3
|
||||||
|
|
||||||
|
OpenCode 仍是主入口,因为 `.opencode/commands` 与 subagent 调度已经稳定:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
opencode
|
||||||
|
/dr-init <topic>
|
||||||
|
/dr-frame <slug>
|
||||||
|
/dr-research <slug>
|
||||||
|
/dr-review <slug>
|
||||||
|
```
|
||||||
|
|
||||||
|
Codex 可用于:
|
||||||
|
|
||||||
|
- 审阅 `framework.md` 和 `critique.md`
|
||||||
|
- 修正文档、脚本和配置
|
||||||
|
- 运行静态检查、引用检查和出稿脚本
|
||||||
|
- 对某个章节做人工式复核建议
|
||||||
|
|
||||||
|
### Phase 4
|
||||||
|
|
||||||
|
Codex 可直接运行 Python 化流水线:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
uv run python scripts/translate.py <slug> --workers 4
|
||||||
|
uv run python scripts/build_glossary.py <slug> --workers 4
|
||||||
|
uv run python scripts/apply_glossary.py <slug> --input phase4/final_zh.md --dry-run
|
||||||
|
uv run python scripts/apply_glossary.py <slug> --input phase4/final_zh.md
|
||||||
|
uv run python scripts/polish.py <slug> --workers 4
|
||||||
|
uv run python scripts/build_report.py <slug>
|
||||||
|
```
|
||||||
|
|
||||||
|
网络不稳时:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
uv run python scripts/translate.py <slug> --workers 1
|
||||||
|
uv run python scripts/polish.py <slug> --workers 1
|
||||||
|
uv run python scripts/build_glossary.py <slug> --workers 3
|
||||||
|
```
|
||||||
|
|
||||||
|
## Git Hygiene
|
||||||
|
|
||||||
|
本仓库常有大量 `projects/**` 研究产物处于修改状态。Codex 提交系统迭代时必须只 stage 相关系统文件:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add README.md PLAN.md docs configs scripts .opencode/commands/dr-finalize.md
|
||||||
|
git diff --staged --name-only
|
||||||
|
```
|
||||||
|
|
||||||
|
提交前确认 staged 列表不包含:
|
||||||
|
|
||||||
|
- `projects/**`
|
||||||
|
- 已生成 PDF/DOCX/TXT
|
||||||
|
- 临时检查脚本或一次性研究产物
|
||||||
|
|
||||||
|
## What Codex Should Not Do Yet
|
||||||
|
|
||||||
|
- 不直接替代 OpenCode 的 `/dr-research` 多 agent 调度。
|
||||||
|
- 不在 Codex 内复刻 dr-analyst/dr-verifier 的长流程并行。
|
||||||
|
- 不把普通网页当作 Tier 1-2 证据。
|
||||||
|
- 不在未检查 staged 列表前 commit。
|
||||||
|
|
||||||
|
## Future Platform Adapters
|
||||||
|
|
||||||
|
后续 Gemini CLI / Claude Code 兼容应沿用同一原则:
|
||||||
|
|
||||||
|
1. 共享 `AGENTS.md` 方法论。
|
||||||
|
2. 共享 `scripts/` 的确定性流水线。
|
||||||
|
3. 平台只提供交互、审阅、调度或修补能力。
|
||||||
|
4. 不为每个平台复制一套研究逻辑。
|
||||||
@@ -0,0 +1,80 @@
|
|||||||
|
# Model Playbook
|
||||||
|
|
||||||
|
> v0.9 起,本文件作为模型选择攻略本。`.opencode/opencode.json` 仍是 OpenCode 的模型白名单,`configs/model_profiles.yaml` 是跨平台策略参考。
|
||||||
|
|
||||||
|
## Profiles
|
||||||
|
|
||||||
|
### premium
|
||||||
|
|
||||||
|
用于高价值、长上下文、强叙事任务。成本最高,但最稳。
|
||||||
|
|
||||||
|
- dr-plan:Claude Opus 4.7(cache)
|
||||||
|
- dr-editor-in-chief:Claude Opus 4.7(cache)
|
||||||
|
- dr-pm / dr-analyst:Claude Sonnet 4.6(cache)
|
||||||
|
- dr-chief-editor:Gemini 3.1 Pro Preview
|
||||||
|
- dr-verifier:GPT-5.4
|
||||||
|
- translate / polish:Claude Sonnet 4.6
|
||||||
|
|
||||||
|
适用:投资报告、管理层决策报告、需要交付 PDF/DOCX 的正式项目。
|
||||||
|
|
||||||
|
### balanced
|
||||||
|
|
||||||
|
默认推荐。保留关键环节的强模型,把机械任务交给更便宜模型。
|
||||||
|
|
||||||
|
- dr-plan:Claude Opus 4.7(cache)
|
||||||
|
- dr-editor-in-chief:Claude Opus 4.7(cache)
|
||||||
|
- dr-pm / dr-analyst:Claude Sonnet 4.6(cache)
|
||||||
|
- dr-searcher:Claude Haiku 4.5
|
||||||
|
- dr-verifier:Qwen3.6 Plus 或 GPT-5.4
|
||||||
|
- translate / polish:Claude Sonnet 4.6,`--workers 4`
|
||||||
|
|
||||||
|
适用:大多数 10,000-35,000 中文字报告。
|
||||||
|
|
||||||
|
### budget
|
||||||
|
|
||||||
|
用于探索性研究或内部草稿。牺牲部分风格和深度,换取成本控制。
|
||||||
|
|
||||||
|
- dr-plan:Claude Sonnet 4.6 或 Qwen3.6 Plus
|
||||||
|
- dr-editor-in-chief:Claude Sonnet 4.6
|
||||||
|
- dr-analyst:Qwen3.6 Plus / DeepSeek V3.2
|
||||||
|
- dr-verifier:MiniMax M2.7 / Kimi K2.5
|
||||||
|
- translate / polish:Qwen3.6 Plus 或 Claude Haiku 4.5(需要人工抽检)
|
||||||
|
|
||||||
|
适用:快速扫盘、预研、内部初稿。不建议直接用于最终交付。
|
||||||
|
|
||||||
|
### cn-heavy
|
||||||
|
|
||||||
|
用于中国市场、NMPA、企业工商/中文媒体密集场景。
|
||||||
|
|
||||||
|
- dr-analyst:Claude Sonnet 4.6 + Qwen3.6 Plus 交叉
|
||||||
|
- dr-verifier:Qwen3.6 Plus
|
||||||
|
- dr-searcher:Haiku + Serper/Brave 中文关键词
|
||||||
|
- translate / polish:Claude Sonnet 4.6
|
||||||
|
|
||||||
|
适用:中国药企、国产替代、供应链、政策环境、中文专有名词较多的研究。
|
||||||
|
|
||||||
|
### verifier
|
||||||
|
|
||||||
|
用于交叉模型验证,避免同模型同偏差。
|
||||||
|
|
||||||
|
- 首选:GPT-5.4
|
||||||
|
- 备用:Qwen3.6 Plus
|
||||||
|
- 低成本:MiniMax M2.7 / Kimi K2.5
|
||||||
|
|
||||||
|
规则:dr-verifier 不使用 Claude 家族,除非用户明确接受同源偏差风险。
|
||||||
|
|
||||||
|
## Switching Rules
|
||||||
|
|
||||||
|
- 高层结构、最终合稿、Executive Summary:优先 Opus。
|
||||||
|
- 章节深研:Sonnet 是默认;预算敏感时用 Qwen/DeepSeek,但必须加强 verifier。
|
||||||
|
- 事实核查与术语核查:优先 Haiku + 搜索 API;低置信度条目人工复核。
|
||||||
|
- 翻译与润色:默认 Sonnet;并发提速优先调 `--workers`,不要盲目换便宜模型。
|
||||||
|
- 限流时:先降并发,再换模型。
|
||||||
|
|
||||||
|
## Operational Defaults
|
||||||
|
|
||||||
|
```bash
|
||||||
|
uv run python scripts/translate.py <slug> --workers 4 --model anthropic/claude-sonnet-4.6
|
||||||
|
uv run python scripts/polish.py <slug> --workers 4 --model anthropic/claude-sonnet-4.6
|
||||||
|
uv run python scripts/build_glossary.py <slug> --workers 4 --model anthropic/claude-haiku-4.5
|
||||||
|
```
|
||||||
@@ -0,0 +1,78 @@
|
|||||||
|
# Search API Playbook
|
||||||
|
|
||||||
|
> v0.9 起,本文件作为搜索 API 选择攻略本。搜索返回本身多为发现入口,结论支撑仍以 AGENTS.md 的 Tier 1-2 信源为准。
|
||||||
|
|
||||||
|
## Search Sources
|
||||||
|
|
||||||
|
### Tavily
|
||||||
|
|
||||||
|
- 优点:LLM 友好,摘要质量稳定,适合快速发现方向。
|
||||||
|
- 用法:初扫、普通网页、报告线索、交叉补漏。
|
||||||
|
- 风险:不能把普通网页当结论支撑,必须追溯原始来源。
|
||||||
|
|
||||||
|
### Exa
|
||||||
|
|
||||||
|
- 优点:neural search,对官网、公司页、长尾专业内容召回好。
|
||||||
|
- 用法:术语核查、公司/产品名纠错、专业网页发现。
|
||||||
|
- 风险:macOS 代理环境容易 TLS EOF,项目内 `SearchClient` 已使用 `trust_env=False` 绕开系统代理。
|
||||||
|
|
||||||
|
### Brave
|
||||||
|
|
||||||
|
- 优点:独立搜索引擎,适合与 Tavily/Exa 交叉验证。
|
||||||
|
- 用法:Phase 1 初扫、反方证据、中文/英文混合搜索。
|
||||||
|
- 风险:结果质量波动,需要人工筛 Tier。
|
||||||
|
|
||||||
|
### Serper
|
||||||
|
|
||||||
|
- 优点:Google Search / Scholar / News 代理,免费额度较高。
|
||||||
|
- 用法:Google Scholar、Google Patents、新闻时效检索。
|
||||||
|
- 风险:专利是 `site:patents.google.com` 技巧,不等同官方专利库。
|
||||||
|
|
||||||
|
### PubMed / NCBI
|
||||||
|
|
||||||
|
- 优点:生物医药论文的一手入口。
|
||||||
|
- 用法:机制、临床、系统综述、meta 分析。
|
||||||
|
- 风险:无 API key 限流较低;摘要不足以替代全文判断。
|
||||||
|
|
||||||
|
### ClinicalTrials.gov / ChiCTR
|
||||||
|
|
||||||
|
- 优点:临床试验注册的一手来源。
|
||||||
|
- 用法:管线、适应症、试验阶段、终点设计、入组状态。
|
||||||
|
- 风险:注册信息不等于结果;需要结合论文、公司披露、监管文件。
|
||||||
|
|
||||||
|
### openFDA / FDA / EMA / NMPA
|
||||||
|
|
||||||
|
- 优点:监管公告与标签信息,Tier 1。
|
||||||
|
- 用法:批准状态、安全性、适应症、审评文件。
|
||||||
|
- 风险:不同监管地区口径不同,必须注明地区与日期。
|
||||||
|
|
||||||
|
### Patents
|
||||||
|
|
||||||
|
- 优点:IP 与工艺路线研究的核心证据。
|
||||||
|
- 用法:Google Patents、USPTO、EPO、CNIPA。
|
||||||
|
- 风险:专利文本难读,权利要求和实施例要分开判断。
|
||||||
|
|
||||||
|
## Recommended Profiles
|
||||||
|
|
||||||
|
### biomed_literature
|
||||||
|
|
||||||
|
PubMed / NCBI → ClinicalTrials → FDA/EMA/NMPA → Serper Scholar → Tavily/Exa 补漏。
|
||||||
|
|
||||||
|
### patent_heavy
|
||||||
|
|
||||||
|
Google Patents/Serper → USPTO/EPO/CNIPA → 公司年报/招股书 → Tavily/Exa 补同族专利线索。
|
||||||
|
|
||||||
|
### china_market
|
||||||
|
|
||||||
|
NMPA/CDE → 港交所/上交所/深交所披露 → 中文专业数据库/媒体 → Brave/Serper 中文搜索。
|
||||||
|
|
||||||
|
### investment
|
||||||
|
|
||||||
|
SEC/交易所披露 → Evaluate/IQVIA/咨询报告 → 公司公告 → 新闻仅作时效入口。
|
||||||
|
|
||||||
|
## Failure Handling
|
||||||
|
|
||||||
|
- 大量 SSL/TLS 错误:先把 workers 降到 3,再重跑。
|
||||||
|
- API 限流:保留缓存结果,断点续跑,不要强制 `--force`。
|
||||||
|
- 搜索返回普通网页:只做线索,继续追原始论文、监管、专利或公司披露。
|
||||||
|
- 中英文译名冲突:写入 glossary,标 medium/low confidence,交人工复核。
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
#!/usr/bin/env python3
|
#!/usr/bin/env python3
|
||||||
"""根据 glossary 的核查结果,在 final_zh_polished.md 上做精确的文本替换。
|
"""根据 glossary 的核查结果,在 final_zh.md 上做精确的文本替换。
|
||||||
|
|
||||||
原理:
|
原理:
|
||||||
- build_glossary.py 会给每条术语标 `zh`(正确中文)和 `issue`(有发现问题)
|
- build_glossary.py 会给每条术语标 `zh`(正确中文)和 `issue`(有发现问题)
|
||||||
@@ -178,7 +178,7 @@ def main() -> int:
|
|||||||
parser = argparse.ArgumentParser(description="用 glossary 修正正文术语")
|
parser = argparse.ArgumentParser(description="用 glossary 修正正文术语")
|
||||||
parser.add_argument("project", help="项目 slug 或路径")
|
parser.add_argument("project", help="项目 slug 或路径")
|
||||||
parser.add_argument(
|
parser.add_argument(
|
||||||
"--input", default="phase4/final_zh_polished.md",
|
"--input", default="phase4/final_zh.md",
|
||||||
help="待修正的 Markdown",
|
help="待修正的 Markdown",
|
||||||
)
|
)
|
||||||
parser.add_argument(
|
parser.add_argument(
|
||||||
|
|||||||
@@ -22,7 +22,7 @@
|
|||||||
|
|
||||||
用法:
|
用法:
|
||||||
uv run python scripts/build_glossary.py <project_slug>
|
uv run python scripts/build_glossary.py <project_slug>
|
||||||
uv run python scripts/build_glossary.py <project_slug> --workers 6
|
uv run python scripts/build_glossary.py <project_slug> --workers 4
|
||||||
uv run python scripts/build_glossary.py <project_slug> --only "Mabwell,Maywavee"
|
uv run python scripts/build_glossary.py <project_slug> --only "Mabwell,Maywavee"
|
||||||
uv run python scripts/build_glossary.py <project_slug> --force
|
uv run python scripts/build_glossary.py <project_slug> --force
|
||||||
|
|
||||||
@@ -184,7 +184,7 @@ def main() -> int:
|
|||||||
)
|
)
|
||||||
parser.add_argument("--model", default=DEFAULT_MODEL, help="LLM 模型")
|
parser.add_argument("--model", default=DEFAULT_MODEL, help="LLM 模型")
|
||||||
parser.add_argument(
|
parser.add_argument(
|
||||||
"--workers", type=int, default=6, help="并发度(默认 6,Exa 限速 5 QPS)"
|
"--workers", type=int, default=4, help="并发度(默认 4;网络不稳时建议降到 3)"
|
||||||
)
|
)
|
||||||
parser.add_argument(
|
parser.add_argument(
|
||||||
"--force", action="store_true",
|
"--force", action="store_true",
|
||||||
|
|||||||
@@ -15,6 +15,7 @@ from __future__ import annotations
|
|||||||
import json
|
import json
|
||||||
import os
|
import os
|
||||||
import sys
|
import sys
|
||||||
|
import threading
|
||||||
import time
|
import time
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
@@ -111,6 +112,8 @@ class ZenMuxClient:
|
|||||||
self.log_file = log_file
|
self.log_file = log_file
|
||||||
self.usage = UsageStats()
|
self.usage = UsageStats()
|
||||||
self._client = httpx.Client(timeout=timeout)
|
self._client = httpx.Client(timeout=timeout)
|
||||||
|
self._log_lock = threading.Lock()
|
||||||
|
self._usage_lock = threading.Lock()
|
||||||
|
|
||||||
def close(self) -> None:
|
def close(self) -> None:
|
||||||
self._client.close()
|
self._client.close()
|
||||||
@@ -125,6 +128,7 @@ class ZenMuxClient:
|
|||||||
if not self.log_file:
|
if not self.log_file:
|
||||||
return
|
return
|
||||||
self.log_file.parent.mkdir(parents=True, exist_ok=True)
|
self.log_file.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
with self._log_lock:
|
||||||
with self.log_file.open("a", encoding="utf-8") as f:
|
with self.log_file.open("a", encoding="utf-8") as f:
|
||||||
f.write(json.dumps(payload, ensure_ascii=False) + "\n")
|
f.write(json.dumps(payload, ensure_ascii=False) + "\n")
|
||||||
|
|
||||||
@@ -191,6 +195,7 @@ class ZenMuxClient:
|
|||||||
except Exception as e:
|
except Exception as e:
|
||||||
raise ZenMuxError(f"invalid JSON from zenmux: {e}; body={resp.text[:500]}")
|
raise ZenMuxError(f"invalid JSON from zenmux: {e}; body={resp.text[:500]}")
|
||||||
usage = data.get("usage", {}) or {}
|
usage = data.get("usage", {}) or {}
|
||||||
|
with self._usage_lock:
|
||||||
self.usage.add(model, usage)
|
self.usage.add(model, usage)
|
||||||
content = ""
|
content = ""
|
||||||
choices = data.get("choices") or []
|
choices = data.get("choices") or []
|
||||||
|
|||||||
+52
-13
@@ -19,6 +19,7 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import argparse
|
import argparse
|
||||||
|
import concurrent.futures
|
||||||
import json
|
import json
|
||||||
import sys
|
import sys
|
||||||
import time
|
import time
|
||||||
@@ -141,6 +142,12 @@ def main() -> int:
|
|||||||
default=None,
|
default=None,
|
||||||
help="最多润色前 N 个未缓存的块(调试用)",
|
help="最多润色前 N 个未缓存的块(调试用)",
|
||||||
)
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--workers",
|
||||||
|
type=int,
|
||||||
|
default=4,
|
||||||
|
help="并发润色 worker 数(默认 4;设为 1 回退串行)",
|
||||||
|
)
|
||||||
args = parser.parse_args()
|
args = parser.parse_args()
|
||||||
|
|
||||||
load_secrets()
|
load_secrets()
|
||||||
@@ -167,27 +174,34 @@ def main() -> int:
|
|||||||
total_cn = sum(count_chinese_chars(b.content) for b in blocks)
|
total_cn = sum(count_chinese_chars(b.content) for b in blocks)
|
||||||
print(f"Source: {src_path.relative_to(project_root)}")
|
print(f"Source: {src_path.relative_to(project_root)}")
|
||||||
print(f"Blocks: {len(blocks)} | total Chinese chars: {total_cn:,}")
|
print(f"Blocks: {len(blocks)} | total Chinese chars: {total_cn:,}")
|
||||||
print(f"Model: {args.model} | temperature: {args.temperature}")
|
workers = max(1, args.workers)
|
||||||
|
print(f"Model: {args.model} | temperature: {args.temperature} | workers: {workers}")
|
||||||
print()
|
print()
|
||||||
|
|
||||||
start = time.time()
|
start = time.time()
|
||||||
translated_this_run = 0
|
todo: list[MarkdownBlock] = []
|
||||||
|
cached_count = 0
|
||||||
notes_records: list[dict] = []
|
notes_records: list[dict] = []
|
||||||
with ZenMuxClient(log_file=log_file) as client:
|
|
||||||
for b in blocks:
|
for b in blocks:
|
||||||
chunk_path = chunks_dir / f"{b.order:03d}-{b.anchor}.md"
|
chunk_path = chunks_dir / f"{b.order:03d}-{b.anchor}.md"
|
||||||
if only_orders is not None and b.order not in only_orders:
|
if only_orders is not None and b.order not in only_orders:
|
||||||
continue
|
continue
|
||||||
if chunk_path.exists() and not args.force:
|
if chunk_path.exists() and not args.force:
|
||||||
|
cached_count += 1
|
||||||
print(f" [ok ] #{b.order:03d} {b.short_title} (cached)")
|
print(f" [ok ] #{b.order:03d} {b.short_title} (cached)")
|
||||||
continue
|
continue
|
||||||
if args.limit is not None and translated_this_run >= args.limit:
|
if args.limit is not None and len(todo) >= args.limit:
|
||||||
continue
|
continue
|
||||||
|
todo.append(b)
|
||||||
|
|
||||||
label = f"#{b.order:03d} L{b.level} {count_chinese_chars(b.content):>4}字 {b.short_title}"
|
if todo:
|
||||||
print(f" [... ] {label} ", end="", flush=True)
|
print(f"To polish this run: {len(todo)} blocks | cached: {cached_count}")
|
||||||
|
print()
|
||||||
|
|
||||||
|
with ZenMuxClient(log_file=log_file) as client:
|
||||||
|
def run_one(b: MarkdownBlock) -> tuple[MarkdownBlock, str, str, float]:
|
||||||
|
chunk_path = chunks_dir / f"{b.order:03d}-{b.anchor}.md"
|
||||||
t0 = time.time()
|
t0 = time.time()
|
||||||
try:
|
|
||||||
polished, notes = polish_block(
|
polished, notes = polish_block(
|
||||||
client,
|
client,
|
||||||
b,
|
b,
|
||||||
@@ -195,22 +209,47 @@ def main() -> int:
|
|||||||
system_prompt=system_prompt,
|
system_prompt=system_prompt,
|
||||||
temperature=args.temperature,
|
temperature=args.temperature,
|
||||||
)
|
)
|
||||||
|
chunk_path.write_text(polished + "\n", encoding="utf-8")
|
||||||
|
return b, polished, notes, time.time() - t0
|
||||||
|
|
||||||
|
if workers == 1:
|
||||||
|
for b in todo:
|
||||||
|
label = f"#{b.order:03d} L{b.level} {count_chinese_chars(b.content):>4}字 {b.short_title}"
|
||||||
|
print(f" [... ] {label} ", end="", flush=True)
|
||||||
|
try:
|
||||||
|
block, polished, notes, elapsed = run_one(b)
|
||||||
except (ZenMuxError, RuntimeError) as e:
|
except (ZenMuxError, RuntimeError) as e:
|
||||||
print(f"\n [FAIL] {label}\n {e}")
|
print(f"\n [FAIL] {label}\n {e}")
|
||||||
continue
|
continue
|
||||||
elapsed = time.time() - t0
|
|
||||||
|
|
||||||
chunk_path.write_text(polished + "\n", encoding="utf-8")
|
|
||||||
cn = count_chinese_chars(polished)
|
cn = count_chinese_chars(polished)
|
||||||
before_cn = count_chinese_chars(b.content)
|
before_cn = count_chinese_chars(block.content)
|
||||||
delta = cn - before_cn
|
delta = cn - before_cn
|
||||||
sign = "+" if delta >= 0 else ""
|
sign = "+" if delta >= 0 else ""
|
||||||
translated_this_run += 1
|
|
||||||
if notes:
|
if notes:
|
||||||
notes_records.append(
|
notes_records.append(
|
||||||
{"order": b.order, "anchor": b.anchor, "title": b.short_title, "notes": notes}
|
{"order": block.order, "anchor": block.anchor, "title": block.short_title, "notes": notes}
|
||||||
)
|
)
|
||||||
print(f"\r [done] {label} → {cn}字 ({sign}{delta}, {elapsed:4.1f}s)")
|
print(f"\r [done] {label} → {cn}字 ({sign}{delta}, {elapsed:4.1f}s)")
|
||||||
|
else:
|
||||||
|
with concurrent.futures.ThreadPoolExecutor(max_workers=workers) as pool:
|
||||||
|
futures = {pool.submit(run_one, b): b for b in todo}
|
||||||
|
for fut in concurrent.futures.as_completed(futures):
|
||||||
|
b = futures[fut]
|
||||||
|
label = f"#{b.order:03d} L{b.level} {count_chinese_chars(b.content):>4}字 {b.short_title}"
|
||||||
|
try:
|
||||||
|
block, polished, notes, elapsed = fut.result()
|
||||||
|
except (ZenMuxError, RuntimeError) as e:
|
||||||
|
print(f" [FAIL] {label}\n {e}")
|
||||||
|
continue
|
||||||
|
cn = count_chinese_chars(polished)
|
||||||
|
before_cn = count_chinese_chars(block.content)
|
||||||
|
delta = cn - before_cn
|
||||||
|
sign = "+" if delta >= 0 else ""
|
||||||
|
if notes:
|
||||||
|
notes_records.append(
|
||||||
|
{"order": block.order, "anchor": block.anchor, "title": block.short_title, "notes": notes}
|
||||||
|
)
|
||||||
|
print(f" [done] {label} → {cn}字 ({sign}{delta}, {elapsed:4.1f}s)")
|
||||||
|
|
||||||
# 汇总
|
# 汇总
|
||||||
merged: list[str] = []
|
merged: list[str] = []
|
||||||
|
|||||||
+71
-22
@@ -6,12 +6,12 @@
|
|||||||
# 或:
|
# 或:
|
||||||
uv run python scripts/translate.py projects/dual-target-rnai-pipeline-2026
|
uv run python scripts/translate.py projects/dual-target-rnai-pipeline-2026
|
||||||
|
|
||||||
断点续传:每块翻译完立即写入 `phase4/zh_chunks/<anchor>.md` 和术语表 patch。
|
断点续传:每块翻译完立即写入 `phase4/zh_chunks/<anchor>.md`,术语表 patch 在本轮结束后统一合并。
|
||||||
重跑时已存在的块直接跳过,只译缺的。
|
重跑时已存在的块直接跳过,只译缺的。
|
||||||
|
|
||||||
设计要点:
|
设计要点:
|
||||||
1. 切块按 H2 粒度,单块一般 <600 英文词,单次 API 调用远低于 Sonnet output token 上限
|
1. 切块按 H2 粒度,单块一般 <600 英文词,单次 API 调用远低于 Sonnet output token 上限
|
||||||
2. 术语表累积式更新:每块调用传入当前已知术语,译完回写 patch,保证全文一致
|
2. 并发翻译使用稳定术语表快照,译完后统一合并 glossary patch,避免多线程写冲突
|
||||||
3. 失败不会污染最终产物:块级文件独立,可重跑;汇总步骤独立
|
3. 失败不会污染最终产物:块级文件独立,可重跑;汇总步骤独立
|
||||||
4. 日志完整:每次 API 调用写 `phase4/logs/translate.jsonl`
|
4. 日志完整:每次 API 调用写 `phase4/logs/translate.jsonl`
|
||||||
"""
|
"""
|
||||||
@@ -19,6 +19,7 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import argparse
|
import argparse
|
||||||
|
import concurrent.futures
|
||||||
import json
|
import json
|
||||||
import sys
|
import sys
|
||||||
import time
|
import time
|
||||||
@@ -29,7 +30,6 @@ sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
|||||||
from scripts.lib.markdown_chunker import (
|
from scripts.lib.markdown_chunker import (
|
||||||
MarkdownBlock,
|
MarkdownBlock,
|
||||||
count_chinese_chars,
|
count_chinese_chars,
|
||||||
merge_blocks,
|
|
||||||
split_by_headers,
|
split_by_headers,
|
||||||
)
|
)
|
||||||
from scripts.lib.zenmux_client import ZenMuxClient, ZenMuxError, load_secrets
|
from scripts.lib.zenmux_client import ZenMuxClient, ZenMuxError, load_secrets
|
||||||
@@ -112,12 +112,19 @@ def save_glossary(path: Path, glossary: dict[str, str]) -> None:
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _glossary_value(value) -> str:
|
||||||
|
if isinstance(value, dict):
|
||||||
|
return str(value.get("zh") or value.get("中文") or "")
|
||||||
|
return str(value)
|
||||||
|
|
||||||
|
|
||||||
def build_user_prompt(block: MarkdownBlock, glossary: dict[str, str]) -> str:
|
def build_user_prompt(block: MarkdownBlock, glossary: dict[str, str]) -> str:
|
||||||
glossary_hint = (
|
glossary_lines = (
|
||||||
"\n".join(f"{en} || {zh}" for en, zh in sorted(glossary.items()))
|
f"{en} || {_glossary_value(zh)}"
|
||||||
if glossary
|
for en, zh in sorted(glossary.items())
|
||||||
else "(none yet)"
|
if _glossary_value(zh)
|
||||||
)
|
)
|
||||||
|
glossary_hint = "\n".join(glossary_lines) or "(none yet)"
|
||||||
level_hint = (
|
level_hint = (
|
||||||
f"H{block.level}" if block.level >= 1 else "frontmatter (no heading)"
|
f"H{block.level}" if block.level >= 1 else "frontmatter (no heading)"
|
||||||
)
|
)
|
||||||
@@ -195,6 +202,12 @@ def main() -> int:
|
|||||||
default=None,
|
default=None,
|
||||||
help="最多翻译前 N 个未缓存的块(调试用)",
|
help="最多翻译前 N 个未缓存的块(调试用)",
|
||||||
)
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--workers",
|
||||||
|
type=int,
|
||||||
|
default=4,
|
||||||
|
help="并发翻译 worker 数(默认 4;设为 1 回退串行)",
|
||||||
|
)
|
||||||
args = parser.parse_args()
|
args = parser.parse_args()
|
||||||
|
|
||||||
load_secrets()
|
load_secrets()
|
||||||
@@ -223,26 +236,34 @@ def main() -> int:
|
|||||||
print(f"Source: {src_path.relative_to(project_root)}")
|
print(f"Source: {src_path.relative_to(project_root)}")
|
||||||
print(f"Blocks: {len(blocks)} | total English words: {total_en_words:,}")
|
print(f"Blocks: {len(blocks)} | total English words: {total_en_words:,}")
|
||||||
print(f"Glossary loaded: {len(glossary)} terms")
|
print(f"Glossary loaded: {len(glossary)} terms")
|
||||||
print(f"Model: {args.model} | temperature: {args.temperature}")
|
workers = max(1, args.workers)
|
||||||
|
print(f"Model: {args.model} | temperature: {args.temperature} | workers: {workers}")
|
||||||
print()
|
print()
|
||||||
|
|
||||||
start = time.time()
|
start = time.time()
|
||||||
translated_this_run = 0
|
todo: list[MarkdownBlock] = []
|
||||||
with ZenMuxClient(log_file=log_file) as client:
|
cached_count = 0
|
||||||
for b in blocks:
|
for b in blocks:
|
||||||
chunk_path = chunks_dir / f"{b.order:03d}-{b.anchor}.md"
|
chunk_path = chunks_dir / f"{b.order:03d}-{b.anchor}.md"
|
||||||
if only_orders is not None and b.order not in only_orders:
|
if only_orders is not None and b.order not in only_orders:
|
||||||
continue
|
continue
|
||||||
if chunk_path.exists() and not args.force:
|
if chunk_path.exists() and not args.force:
|
||||||
|
cached_count += 1
|
||||||
print(f" [ok ] #{b.order:03d} {b.short_title} (cached)")
|
print(f" [ok ] #{b.order:03d} {b.short_title} (cached)")
|
||||||
continue
|
continue
|
||||||
if args.limit is not None and translated_this_run >= args.limit:
|
if args.limit is not None and len(todo) >= args.limit:
|
||||||
continue
|
continue
|
||||||
|
todo.append(b)
|
||||||
|
|
||||||
label = f"#{b.order:03d} L{b.level} {b.word_count:>4}w {b.short_title}"
|
if todo:
|
||||||
print(f" [... ] {label} ", end="", flush=True)
|
print(f"To translate this run: {len(todo)} blocks | cached: {cached_count}")
|
||||||
|
print()
|
||||||
|
|
||||||
|
successful_patches: list[dict[str, str]] = []
|
||||||
|
with ZenMuxClient(log_file=log_file) as client:
|
||||||
|
def run_one(b: MarkdownBlock) -> tuple[MarkdownBlock, str, dict[str, str], float]:
|
||||||
|
chunk_path = chunks_dir / f"{b.order:03d}-{b.anchor}.md"
|
||||||
t0 = time.time()
|
t0 = time.time()
|
||||||
try:
|
|
||||||
translation, patch = translate_block(
|
translation, patch = translate_block(
|
||||||
client,
|
client,
|
||||||
b,
|
b,
|
||||||
@@ -251,19 +272,47 @@ def main() -> int:
|
|||||||
system_prompt=system_prompt,
|
system_prompt=system_prompt,
|
||||||
temperature=args.temperature,
|
temperature=args.temperature,
|
||||||
)
|
)
|
||||||
|
chunk_path.write_text(translation + "\n", encoding="utf-8")
|
||||||
|
return b, translation, patch, time.time() - t0
|
||||||
|
|
||||||
|
if workers == 1:
|
||||||
|
for b in todo:
|
||||||
|
label = f"#{b.order:03d} L{b.level} {b.word_count:>4}w {b.short_title}"
|
||||||
|
print(f" [... ] {label} ", end="", flush=True)
|
||||||
|
try:
|
||||||
|
_block, translation, patch, elapsed = run_one(b)
|
||||||
except (ZenMuxError, RuntimeError) as e:
|
except (ZenMuxError, RuntimeError) as e:
|
||||||
print(f"\n [FAIL] {label}\n {e}")
|
print(f"\n [FAIL] {label}\n {e}")
|
||||||
continue
|
continue
|
||||||
elapsed = time.time() - t0
|
successful_patches.append(patch)
|
||||||
|
|
||||||
chunk_path.write_text(translation + "\n", encoding="utf-8")
|
|
||||||
if patch:
|
|
||||||
for k, v in patch.items():
|
|
||||||
glossary.setdefault(k, v)
|
|
||||||
save_glossary(glossary_path, glossary)
|
|
||||||
cn = count_chinese_chars(translation)
|
cn = count_chinese_chars(translation)
|
||||||
translated_this_run += 1
|
|
||||||
print(f"\r [done] {label} → {cn:>4}字 ({elapsed:4.1f}s, +{len(patch)} terms)")
|
print(f"\r [done] {label} → {cn:>4}字 ({elapsed:4.1f}s, +{len(patch)} terms)")
|
||||||
|
else:
|
||||||
|
with concurrent.futures.ThreadPoolExecutor(max_workers=workers) as pool:
|
||||||
|
futures = {pool.submit(run_one, b): b for b in todo}
|
||||||
|
for fut in concurrent.futures.as_completed(futures):
|
||||||
|
b = futures[fut]
|
||||||
|
label = f"#{b.order:03d} L{b.level} {b.word_count:>4}w {b.short_title}"
|
||||||
|
try:
|
||||||
|
_block, translation, patch, elapsed = fut.result()
|
||||||
|
except (ZenMuxError, RuntimeError) as e:
|
||||||
|
print(f" [FAIL] {label}\n {e}")
|
||||||
|
continue
|
||||||
|
successful_patches.append(patch)
|
||||||
|
cn = count_chinese_chars(translation)
|
||||||
|
print(f" [done] {label} → {cn:>4}字 ({elapsed:4.1f}s, +{len(patch)} terms)")
|
||||||
|
|
||||||
|
glossary_conflicts: list[tuple[str, str, str]] = []
|
||||||
|
for patch in successful_patches:
|
||||||
|
for k, v in patch.items():
|
||||||
|
if k not in glossary:
|
||||||
|
glossary[k] = v
|
||||||
|
elif _glossary_value(glossary[k]) != v:
|
||||||
|
glossary_conflicts.append((k, _glossary_value(glossary[k]), v))
|
||||||
|
if successful_patches:
|
||||||
|
save_glossary(glossary_path, glossary)
|
||||||
|
if glossary_conflicts:
|
||||||
|
print(f" [WARN] glossary patch 冲突 {len(glossary_conflicts)} 条,保留既有译法")
|
||||||
|
|
||||||
# 汇总:按 order 拼接所有 chunk
|
# 汇总:按 order 拼接所有 chunk
|
||||||
merged: list[str] = []
|
merged: list[str] = []
|
||||||
|
|||||||
Reference in New Issue
Block a user