v0.6-wip: Python-based Phase 4 translation pipeline
架构变更:把 dr-translator 从 opencode agent 降级为 Python 脚本编排下的 LLM
调用。根本原因是 agent 一次性处理 19k 英文词整文,单次 output token 接近
Sonnet 4.6 上限(~32k),多次重跑都卡在同一个坑里——问题是架构本身,不是
prompt。
新架构:
scripts/lib/zenmux_client.py HTTP 客户端,指数退避重试、token 统计
JSONL 日志、secrets.env 自动加载
scripts/lib/markdown_chunker.py 按 H1/H2 切块,稳定 anchor ID(order+title
sha1),支持合并/统计
scripts/prompts/translate_system.txt 英译中 prompt,用自定义 <<<TRANSLATION>>>
分隔符格式(规避 Markdown-in-JSON 问题)
scripts/prompts/polish_system.txt 中文润色 prompt(留给下一步 polish.py)
scripts/translate.py 主入口:章节级切块 → 逐块翻译 → 拼接
关键设计:
- 0 依赖 LLM 遵从性:Python 控制切块/循环/重试,LLM 只做单块翻译
- 断点续传:每块翻译完立即写 phase4/zh_chunks/<order>-<anchor>.md
- 术语表累积:每块的 glossary_patch 合并回 phase4/glossary.json
- 失败隔离:单块失败不影响其他块,重跑只补缺
- 调试友好:--only N,M / --limit K / --force
实测(dual-target-rnai-pipeline-2026):
- 63 块全部成功,17 分钟,$1.70
- 33,441 中文字(符合"研究类 ≥30,000 字"硬标准)
- 310 条双语术语
- 翻译质量:接近母语咨询分析师写作
下一步:polish.py(按 H2 section 润色)、merge_chapters.py(从 phase2/drafts
合并生成 final_en.md)、重构 dr-editor-in-chief 调度脚本、更新 /dr-finalize。
Co-authored-by: User <human>
This commit is contained in:
@@ -365,7 +365,7 @@ version = "0.3.0"
|
||||
source = { virtual = "." }
|
||||
dependencies = [
|
||||
{ name = "biopython" },
|
||||
{ name = "httpx" },
|
||||
{ name = "httpx", extra = ["socks"] },
|
||||
{ name = "lxml" },
|
||||
{ name = "markdown-it-py" },
|
||||
{ name = "matplotlib" },
|
||||
@@ -390,7 +390,7 @@ dev = [
|
||||
[package.metadata]
|
||||
requires-dist = [
|
||||
{ name = "biopython", specifier = ">=1.83" },
|
||||
{ name = "httpx", specifier = ">=0.27.0" },
|
||||
{ name = "httpx", extras = ["socks"], specifier = ">=0.27.0" },
|
||||
{ name = "lxml", specifier = ">=5.1.0" },
|
||||
{ name = "markdown-it-py", specifier = ">=3.0.0" },
|
||||
{ name = "matplotlib", specifier = ">=3.8.0" },
|
||||
@@ -513,6 +513,11 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/2a/39/e50c7c3a983047577ee07d2a9e53faf5a69493943ec3f6a384bdc792deb2/httpx-0.28.1-py3-none-any.whl", hash = "sha256:d909fcccc110f8c7faf814ca82a9a4d816bc5a6dbfea25d6591d6985b8ba59ad", size = 73517, upload-time = "2024-12-06T15:37:21.509Z" },
|
||||
]
|
||||
|
||||
[package.optional-dependencies]
|
||||
socks = [
|
||||
{ name = "socksio" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "idna"
|
||||
version = "3.11"
|
||||
@@ -1465,6 +1470,15 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/b7/ce/149a00dd41f10bc29e5921b496af8b574d8413afcd5e30dfa0ed46c2cc5e/six-1.17.0-py2.py3-none-any.whl", hash = "sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274", size = 11050, upload-time = "2024-12-04T17:35:26.475Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "socksio"
|
||||
version = "1.0.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/f8/5c/48a7d9495be3d1c651198fd99dbb6ce190e2274d0f28b9051307bdec6b85/socksio-1.0.0.tar.gz", hash = "sha256:f88beb3da5b5c38b9890469de67d0cb0f9d494b78b106ca1845f96c10b91c4ac", size = 19055, upload-time = "2020-04-17T15:50:34.664Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/37/c3/6eeb6034408dac0fa653d126c9204ade96b819c936e136c5e8a6897eee9c/socksio-1.0.0-py3-none-any.whl", hash = "sha256:95dc1f15f9b34e8d7b16f06d74b8ccf48f609af32ab33c608d08761c5dcbb1f3", size = 12763, upload-time = "2020-04-17T15:50:31.878Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "tomli"
|
||||
version = "2.4.1"
|
||||
|
||||
Reference in New Issue
Block a user