kai/deep_research
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
6 Commits 3 Branches 7 Tags
a86010e9a7a4a2561d2fb6cbe84d8d338a54d695
Commit Graph
6 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
kaiandUser <human> a86010e9a7 v0.6-wip: polish pipeline + PDF template fixes 
Phase 4 \u6da6\u8272\u5c42\u4e0e PDF \u6a21\u677f\u4fee\u590d\uff0c\u63a5\u7740\u4e0a\u4e00\u4e2a commit\u3002

polish.py\uff08\u65b0\u589e\uff09\uff1a
- \u548c translate.py \u5bf9\u79f0\uff0c\u6309 H2 section \u5207\u5757 \u2192 \u5faa\u73af\u6da6\u8272 \u2192 \u62fc\u63a5
- \u4f7f\u7528 <<<POLISHED>>>/<<<NOTES>>> \u5206\u9694\u7b26 prompt\uff08\u907f\u5f00 Markdown-in-JSON \u95ee\u9898\uff09
- \u65ad\u70b9\u7eed\u4f20\u3001\u6a21\u578b\u81ea\u8bc4\u6ce8\u8bb0\u843d\u76d8 polish_notes.jsonl
- \u5728\u53cc\u9776\u70b9 RNAi \u9879\u76ee\u8dd1\u901a\uff1a60 \u5757\u5168\u6210\u529f\uff0c10.7 \u5206\u949f\uff0c$1.20\uff0c\u5b57\u6570 -0.2%

report-template.py\uff08\u5927\u6539\u4e00\u6279 P0 bug\uff09\uff1a
- \u5b57\u4f53\u6ce8\u518c\u652f\u6301 fonts/ttf/ \u5b50\u76ee\u5f55\uff08\u89e3\u51b3 OTF PostScript outlines \u4e0d\u517c\u5bb9\uff09
- \u5220\u9664 build_disclaimer \u91cd\u590d\u8c03\u7528\uff08\u514d\u8d23\u58f0\u660e\u4ece Markdown \u8bfb\uff0cmanifest \u4e0d\u518d\u91cd\u590d\uff09
- build_body \u81ea\u52a8\u8df3\u8fc7\u6b63\u6587\u9996\u4e2a H1+\u5c01\u9762\u5143\u4fe1\u606f\u6bb5\uff08\u4e0e\u5c01\u9762\u91cd\u590d\uff09
- \u5360\u4f4d\u7b26 \u201c\u76ee\u5f55\u5c06\u5728\u6700\u7ec8\u6e32\u67d3\u65f6\u81ea\u52a8\u751f\u6210\u201d \u2192 \u81ea\u52a8\u751f\u6210 TOC
- \u5360\u4f4d\u7b26 \u201c\u5b8c\u6574\u7f16\u53f7\u53c2\u8003\u6587\u732e\u5217\u8868\u2026\u201d \u2192 \u4ece phase2/sources.jsonl \u81ea\u52a8\u751f\u6210 GB/T 7714 \u683c\u5f0f\u5f15\u6587
- src \u4e0a\u6807\u6b63\u5219\u6269\u5c55\uff1a\u652f\u6301 src_A14 / src_B-18 \u7b49\u5b57\u6bcd+\u6570\u5b57\u7ec4\u5408 ID\uff08\u539f\u53ea\u652f\u6301 src_\d+\uff09
- Unicode \u4e0a/\u4e0b\u6807\u8f6c <super>/<sub>\uff1a10\u2076 \u2192 10<super>6</super>\uff08\u601d\u6e90\u5b57\u4f53\u5b50\u96c6\u4e0d\u542b\u4e0a\u6807\u5b57\u5f62\uff0c\u5426\u5219\u6e32\u67d3\u65b9\u6846\uff09
- \u4e2d\u82f1\u6df7\u6392\u81ea\u52a8\u52a0\u7a7a\u683c\uff08CJK \u2194 [A-Za-z0-9] \u8fb9\u754c\uff09
- \u8868\u683c\u6837\u5f0f\u91cd\u505a\uff1atable-header/table-cell/table-cell-center\uff1b\u5782\u76f4\u5c45\u4e2d\uff1b\u77ed cell\uff08\u7eaf\u6570\u5b57/\u77ed\u6807\u7b7e\uff09\u6c34\u5e73\u5c45\u4e2d\uff1b\u957f cell \u81ea\u52a8 CJK \u6362\u884c
- TOC \u672b\u5c3e PageBreak\uff08\u76ee\u5f55\u72ec\u5360\u6574\u9875\uff09

\u5df2\u77e5\u672a\u4fee\u590d\uff1a
- Maywavee \u662f LLM \u5728 dr-analyst \u9636\u6bb5\u7f16\u9020\uff0c\u6b63\u786e\u4e3a Mabwell\uff08\u8fc8\u5a01\u751f\u7269\uff09\u3002\u4fe1\u6e90\u4fa7 bug\uff0c\u9700\u5728\u540e\u7eed build_glossary.py \u4e2d\u505a\u4e8b\u5b9e\u6838\u67e5\u3002
- \u6b63\u6587 101 \u4e2a src_id\u3001sources.jsonl \u53ea\u670947 \u4e2a\u3001\u4ea4\u96c6 39 \u4e2a\u2014\u2014\u662f v0.4 \u9057\u7559\u7684\u6ce8\u5165 bug\uff0cbuild_references \u73b0\u5728\u4f1a\u5217\u51fa\u7f3a\u5931\u7684 id \u4f9b\u4eba\u5de5\u6838\u5bf9
- \u53cd\u9a73\u8bc1\u636e\u6bb5\u683c\u5f0f\u4e0d\u7edf\u4e00\u662f dr-analyst/skill \u89c4\u8303\u95ee\u9898\uff0c\u4e0b\u4e00\u6279\u6539 skill

Co-authored-by: User <human>
 2026-04-22 12:58:07 +08:00
kaiandUser <human> 1b47b50d3c v0.6-wip: Python-based Phase 4 translation pipeline 
架构变更：把 dr-translator 从 opencode agent 降级为 Python 脚本编排下的 LLM
调用。根本原因是 agent 一次性处理 19k 英文词整文，单次 output token 接近
Sonnet 4.6 上限（~32k），多次重跑都卡在同一个坑里——问题是架构本身，不是
prompt。

新架构：

scripts/lib/zenmux_client.py     HTTP 客户端，指数退避重试、token 统计
                                  JSONL 日志、secrets.env 自动加载
scripts/lib/markdown_chunker.py   按 H1/H2 切块，稳定 anchor ID（order+title
                                  sha1），支持合并/统计
scripts/prompts/translate_system.txt  英译中 prompt，用自定义 <<<TRANSLATION>>>
                                       分隔符格式（规避 Markdown-in-JSON 问题）
scripts/prompts/polish_system.txt     中文润色 prompt（留给下一步 polish.py）
scripts/translate.py              主入口：章节级切块 → 逐块翻译 → 拼接

关键设计：
- 0 依赖 LLM 遵从性：Python 控制切块/循环/重试，LLM 只做单块翻译
- 断点续传：每块翻译完立即写 phase4/zh_chunks/<order>-<anchor>.md
- 术语表累积：每块的 glossary_patch 合并回 phase4/glossary.json
- 失败隔离：单块失败不影响其他块，重跑只补缺
- 调试友好：--only N,M / --limit K / --force

实测（dual-target-rnai-pipeline-2026）：
- 63 块全部成功，17 分钟，$1.70
- 33,441 中文字（符合"研究类 ≥30,000 字"硬标准）
- 310 条双语术语
- 翻译质量：接近母语咨询分析师写作

下一步：polish.py（按 H2 section 润色）、merge_chapters.py（从 phase2/drafts
合并生成 final_en.md）、重构 dr-editor-in-chief 调度脚本、更新 /dr-finalize。

Co-authored-by: User <human>
 2026-04-22 10:43:43 +08:00
kai 701bc1887e v0.5.2: dr-translator chunked translation protocol 
Root cause: dr-translator was trying to write entire final_zh.md in one
write call, hitting Sonnet 4-6 output token limit for long reports
(~19k English words → ~27k Chinese chars → blown past 32k token cap).

Fix: explicit chunk-and-append protocol
- Split final_en.md by H1 (# ) then H2 (## ) boundaries
- Each chunk ≤ 2,500 English words
- First chunk uses write to create final_zh.md
- Subsequent chunks use edit or read+write to append
- Per-chunk Chinese output kept under ~5,000 characters (safe margin)
- Preserves glossary.json updates across chunks
 2026-04-21 23:10:10 +08:00
kai 333b7bb8d5 v0.5.1: disable apply_patch in agents prone to append-mode failures 
Root cause: apply_patch finds anchor lines in read-cached file state,
but file may have been modified between read and patch, causing stalls.

Changes:
- dr-verifier: disable apply_patch AND edit; force read-then-write protocol for evidence file appends
- dr-analyst: document write-preferred protocol for sources.jsonl appends
- dr-polisher: disable apply_patch; keep edit for small string replacements
- dr-editor-in-chief / dr-translator: disable apply_patch

Recovery procedure documented in dr-verifier for write failures.
 2026-04-21 14:44:03 +08:00
kai a092af4398 v0.5: deep quality refactor (P0+P1+P2) 
- Split dr-chief-editor (Phase 3 read-only) vs new dr-editor-in-chief (Opus, Phase 4 lead)
- New dr-translator (en->zh) and new humanizer-cn / output-hygiene / en-zh-translation skills
- Switch to English working language (Phase 2-3), final Chinese translation (Phase 4)
- /dr-init: add report title proposals + word budget mode
- /dr-frame: bilingual framework
- /dr-finalize: new chain editor->translator->polisher->reporter
- report-template.py: widows/orphans/keepWithNext, 3-color hierarchy, confidentiality banner
- dr-reporter: mandatory citations backfill + output hygiene check
- dr-pm: batch-level context compression via manifest.batches_summary
- mckinsey-method: SCQA only for Executive Summary + chapter intros (no explicit labels)
- length-budget: 4 word-budget modes + en/zh 1:1.4 ratio
 2026-04-21 13:02:54 +08:00
kai 4a38f6bed1 snapshot before v0.5 refactor 2026-04-21 12:31:58 +08:00