4 Commits
Author SHA1 Message Date
kai c444007f04 v0.8: Serper 集成 + H1 章节标题双行居中 + 反方证据观点化 + 术语核查命令
新增:Google 系检索(SerpAPI → Serper.dev)

- scripts/lib/serper_client.py:封装 serper.dev 的 Google Search / Scholar / News / Patents
- 专利检索用 site:patents.google.com 技巧,serper.dev 没专用 endpoint 但效果很好
- Scholar 带引用数、年份、期刊信息,便于权威信源识别
- News 支持 time_range(d/w/m/y)时效性过滤

- scripts/lib/search_client.py 扩展为多路由门面:
  - search() 通用:Exa → Tavily
  - patents() 专利:Serper(Google Patents)→ 通用搜索 + site: 兜底
  - scholar() 论文:Serper Scholar → 通用搜索兜底
  - news() 新闻:Serper News → 通用搜索兜底
- 所有 httpx 客户端 trust_env=False,绕过系统 socks5 代理(v0.6 修过的 TLS EOF)

- .opencode/skills/search-strategy/SKILL.md §三重写:按查询类型路由,明确何时用哪个 API

H1 章节标题:两行居中 + 装饰横线

- 新增 ParagraphStyle: h1-chapter-num / h1-chapter-title
- 新增 parse_chapter_title() 支持中文/阿拉伯/混合空格章号:
  "第一章" / "第 9 章" / "第6章" / "Chapter 1" 全覆盖
- 分隔符支持: em dash — / en dash – / - / : / :
- 新增 build_chapter_header():章号小字居中 + 章名大字深蓝居中 + HRFlowable 3cm 装饰线
- 只对正文章节(_title_kind == "chapter")启用;前置件(免责声明/执行摘要/术语表/目录)
  仍用单行 h1 样式

反方证据段规范化(用户反馈 v0.7 问题 #5)

- skill:evidence-table 新增 §"正文中反方证据段落的写作规范":
  - 禁止机械标题"反驳证据" / "Counter-Evidence" / "反方观点"
  - 必须观点化,包含具体判断(如"另一种声音:管线虚胖还是真实进展?")
  - 用 H2 或 H3,禁止加粗段冒充标题
  - 给出段落结构模板(1-2 句过渡 → 列表型反方论点 → 整合判断)
- dr-analyst.md Hard Rules #3 改为引用该规范

术语表事实核查前置(新 command /dr-glossary)

- 新增 .opencode/commands/dr-glossary.md,支持 --from phase1|phase2|phase4 三个时机
- Phase 1 末 / Phase 2 初:从 framework.md 抽取专有名词种子表,在 dr-analyst 起草前
  预先核查公司名/产品名/技术名拼写,避免编造错误(Mabwell → Maywavee 这类)
- Phase 4:维持当前用法,对 glossary.json 全量核查

实测:dual-target-rnai-pipeline-2026 重生 PDF 55 页,所有 10 章标题双行居中正确渲染
(第一章/第二章/... 第十章 / 第 6 章 / 第 9 章 多种形式都识别)。
2026-04-22 17:04:05 +08:00
kai 09f681beb5 v0.7.2: 前置件排版重构 + emoji 禁令 + 引文核查
用户反馈 7 个 bug 修复:

1. 禁止 LLM 使用 emoji(全链路)
   - scripts/prompts/translate_system.txt 增加规则 12
   - scripts/prompts/polish_system.txt 增加规则 7
   - .opencode/agents/dr-analyst.md Hard Rules 增加第 10 条(同时把 prompt 自身的  改为 MUST / MUST NOT)
   - .opencode/agents/dr-editor-in-chief.md 禁止事项加入 emoji 条款
   - .opencode/skills/output-hygiene/SKILL.md 新增 §J emoji 强制禁用

2. 术语表位置错误(应在目录之后)
   重构 build_body 为两阶段:
   (a) 扫描所有前置件(第一个正文 H1 前的所有 H1/H2),按 title_kind 分组收集
   (b) 按固定顺序渲染:免责声明 → 执行摘要 → 目录 → 术语表 → 正文 → 参考文献
   无论 Markdown 原文顺序如何,排版都一致。

3. 执行摘要/术语表提升为一级标题 + 分页空页 bug
   统一所有独立章节(disclaimer/executive_summary/toc/glossary/references)用 h1 样式,
   章节前 PageBreak;但第一个独立章节不 PageBreak(封面后已换页,避免空白)。
   去掉 build_toc 内部末尾 PageBreak(原双 PageBreak 夹出空白页)。

4. 参考文献分页
   已作为独立章节自动分页。

5. 附录章节自动删除
   _title_kind 识别 "appendix" / "version_history" / "abstract" 全部跳过。
   正文中若写了这些章节,模板直接丢弃。

6. 信源完整性核查
   新增 scripts/check_citations.py:
   - 孤立引用(正文有 sources 无)检测
   - 孤岛信源(sources 有正文无)检测
   - emoji 扫描
   - 实测发现项目中 61 条孤立引用(dr-analyst 编造的占位符)+ 5 条孤岛信源

7. git commit message 中文转义 bug
   之前 commit 用 shell 双引号 + 反斜杠导致 \uXXXX 字面保留。
   本 commit 用 heredoc 保证中文以 UTF-8 直接写入。
   已 push 的历史不改,之后都用本 commit 的写法。

PDF 验证结果:55 页,0 空白页。
章节起始页:封面(1) - 免责声明(2) - 执行摘要(3) - 目录(5) - 术语表(7) -
第一章(12) - 第十章(48) - 参考文献(52)。
2026-04-22 16:31:01 +08:00
kaiandUser <human> 1394d98346 v0.7.1: \u4fee\u590d emoji \u65b9\u6846 + \u5206\u9875\u95ee\u9898 + \u5220\u9664\u6458\u8981\u91cd\u590d
\u7528\u6237\u53cd\u9988\u4e09\u4e2a P0 bug\uff1a

1. \u8868\u683c\u4e2d\u7684 \u2705/\u274c/\ud83d\udd36 \u6e32\u67d3\u4e3a\u65b9\u6846
   \u539f\u56e0\uff1a\u601d\u6e90\u5b57\u4f53\u5b50\u96c6\u4e0d\u542b\u5f69\u8272 emoji\u7684 glyph\u3002\u9664\u975e\u6362\u5b57\u4f53\uff08\u8d85\u51fa\u9879\u76ee\u8303\u56f4\uff09\uff0c
   \u5426\u5219\u53ea\u80fd\u66ff\u6362\u6210\u5b57\u4f53\u6709\u7684\u7b26\u53f7\u3002
   \u65b0\u589e _EMOJI_FALLBACK \u6620\u5c04\u8868\uff0c\u6db5\u76d6 20+ \u5e38\u7528 emoji\uff08\u5bf9\u5e94 CHECK/CROSS/DIAMOND/CIRCLE/STAR \u7b49\uff09\u3002
   \u7528 fontTools \u9a8c\u8bc1\u8fc7\u5b57\u6bcd\u6709\u65e0\uff1a\u2713/\u00d7/\u25c6/\u25c7/\u25cf/\u25cb/\u2605/\u203b \u90fd\u6709\u3001\u2717/\u2611/\u2610 \u6ca1\u6709\uff0c\u5df2\u8c03\u6574\u76f8\u5e94\u66ff\u6362\u503c\u3002

2. \u76ee\u5f55/\u672f\u8bed\u8868\u4e0e\u4e0b\u4e00\u7ae0\u540c\u9875 / \u4e2d\u95f4\u7a7a\u9875
   \u539f\u56e0\uff1abuild_toc \u672b\u5c3e\u52a0 PageBreak\uff0c\u7136\u540e\u4e0b\u4e00\u4e2a\u7ae0\u8282 H1 \u53c8 PageBreak\uff0c\u4e24\u4e2a PageBreak \u5939\u51fa\u7a7a\u9875\u3002
   \u4fee\u590d\uff1abuild_toc / \u672f\u8bed\u8868\u6e32\u67d3\u672b\u5c3e\u53bb\u6389 PageBreak\uff0c\u4ec5\u9760\u4e0b\u4e2a H1 \u7684 PageBreak\u3002
   \u5f00\u5934\u7684 PageBreak \u4fdd\u7559\uff0c\u4fdd\u8bc1\u201c\u76ee\u5f55 / \u672f\u8bed\u8868\u72ec\u7acb\u7b2c\u4e00\u9875\u5f00\u59cb\u201d\u3002

3. \u201c\u6267\u884c\u6458\u8981\u201d\u548c\u201c\u6458\u8981\u201d\u91cd\u590d
   \u6839\u636e\u7528\u6237\u9009\u62e9\uff1a\u4fdd\u7559\u6267\u884c\u6458\u8981\uff08Executive Summary\uff09\uff0c\u5220\u6389\u6458\u8981\uff08Abstract\uff09\u3002
   build_body \u65b0\u589e\u201c\u8df3\u8fc7\u6574\u4e2a\u7ae0\u8282\u201d\u903b\u8f91\uff1a\u770b\u5230 "## \u6458\u8981" / "# \u6458\u8981" / "## Abstract" \u5c31\u8df3\u5230\u4e0b\u4e00\u4e2a H1/H2\u3002

\u91cd\u6784\u6e05\u7406\uff1a
- \u63d0\u53d6 _render_generic_block \u5e2e\u52a9\u51fd\u6570\uff0c\u4ee3\u66ff build_body \u5185\u8054\u7684 p/quote/bullet/hr/image/table \u6e32\u67d3\u4ee3\u7801
- \u672f\u8bed\u8868\u90e8\u5206\u5faa\u73af\u590d\u7528\u8be5\u51fd\u6570

\u9a8c\u8bc1\u7ed3\u679c\uff1aPDF \u4ece 56 \u9875 \u2192 54 \u9875\uff08\u5220\u6389\u91cd\u590d\u7684\u6458\u8981 + \u6d88\u9664\u7a7a\u9875\uff09\u3002
\u5c55\u5f00\u7ae0\u8282\u8d77\u59cb\u9875\u987a\u5e8f\uff1a\u5c01\u9762(1) - \u514d\u8d23(2) - \u6267\u884c\u6458\u8981(2) - \u672f\u8bed\u8868(4) - \u76ee\u5f55(9) - \u6b63\u6587\u7b2c\u4e00\u7ae0(11) - \u7b2c\u5341\u7ae0(47) - \u53c2\u8003\u6587\u732e\u3002

remote \u914d\u7f6e\u4e3a http://192.168.50.45:3000/kai/deep_research.git\u3002

Co-authored-by: User <human>
2026-04-22 16:05:45 +08:00
kaiandUser <human> c88da4a20f v0.7: \u4fee\u590d PDF \u5f15\u6587\u7f16\u53f7\u4e0d\u5bf9\u5e94 + \u5c01\u9762\u91cd\u590d
\u4e24\u4e2a P0 bug \u4fee\u590d\uff1a

1. \u5f15\u6587\u7f16\u53f7\u5931\u914d
   \u5148\u524d\u7b56\u7565\uff1a\u53c2\u8003\u6587\u732e\u533a\u6309\u6b63\u6587\u51fa\u73b0\u987a\u5e8f\u91cd\u7f16\u53f7\u4e3a [1]/[2]/...\uff0c\u5bfc\u81f4\u6b63\u6587\u4e2d\u4e0a\u6807\u7684 [src_E43]
   \u4e0e\u53c2\u8003\u6587\u732e\u533a\u7684 [27] \u5b8c\u5168\u5bf9\u4e0d\u4e0a\u3002
   \u65b0\u7b56\u7565\uff1a\u53c2\u8003\u6587\u732e\u6761\u76ee\u76f4\u63a5\u7528\u539f\u59cb src_id \u4f5c\u7f16\u53f7\uff08\u5982 [src_E43] ...\uff09\uff0c
   \u6309\u5b57\u6bcd\u6570\u5b57\u6392\u5e8f\u5206\u7ec4\u5c55\u793a\u3002\u6b63\u6587\u548c\u53c2\u8003\u6587\u732e\u540c key\uff0c\u4e00\u773c\u5bf9\u5e94\u3002
   \u540c\u65f6\u628a\u7f3a\u5931\u7684 src_id\uff08sources.jsonl \u91cc\u6ca1\u7684\uff09\u5355\u72ec\u5217\u5728\u300c\u672a\u627e\u5230\u6765\u6e90\u300d
   \u7ae0\u8282\uff0c\u6a59\u8272\u8b66\u793a\uff0c\u8868\u660e\u662f\u539f\u59cb\u62a5\u544a\u7684\u8d28\u91cf\u95ee\u9898\u3001\u800c\u975e\u6e32\u67d3\u95ee\u9898\u3002
   \u9876\u90e8\u65b0\u589e\u300c\u5f15\u6587\u5065\u5eb7\u72b6\u6001\u300d\u5c0f\u7ed3\uff08\u6b63\u6587\u5f15\u7528X\u3001\u6536\u5f55Y\u3001\u7f3a\u5931Z\uff09\u3002

2. \u5c01\u9762\u91cd\u590d\uff08\u622a\u56fe\uff1a\u526f\u6807\u9898 + Confidentiality/Date/Version \u4ecd\u5728\u6b63\u6587\u9996\u9875\uff09
   \u539f\u56e0\uff1a\u539f\u8df3\u8fc7\u903b\u8f91\u662f\u201c\u8df3\u9996\u4e2a H1 \u2192 \u8df3\u5339\u914d is_cover_frontmatter \u7684 p\u201d\u3002
   \u526f\u6807\u9898\u662f\u52a0\u7c97\u6bb5\uff08**...**\uff09\uff0c\u4e0d\u542b "Confidentiality/Date" \u7b49\u5173\u952e\u8bcd\uff0c
   \u5339\u914d\u4e0d\u4e0a\u5c31\u89e6\u53d1\u300c\u5c01\u9762\u7ed3\u675f\u300d\u903b\u8f91\uff0c\u540e\u7eed\u5143\u4fe1\u606f\u6bb5\u4e5f\u6240\u4ee5\u5c31\u6f0f\u5305\u4e86\u3002
   \u65b0\u7b56\u7565\uff1a\u7b80\u5316\u4e3a\u300c\u6253\u8868\u4ece\u7b2c\u4e00\u4e2a H2/H3 \u5f00\u59cb\u8fed\u4ee3\u300d\uff0c\u524d\u9762\u7684 block \u5168\u90e8\u4e22\u6389\u3002
   \u7406\u7531\uff1a\u5c01\u9762\u5df2\u7531 build_cover \u4ece manifest \u72ec\u7acb\u751f\u6210\uff0c\u6b63\u6587\u5f00\u5934\u5728\u7b2c\u4e00\u4e2a H2
   \uff08\u201c## \u514d\u8d23\u58f0\u660e\u201d\uff09\u524d\u7684\u4efb\u4f55\u5185\u5bb9\u90fd\u662f\u5197\u4f59\u7684\u5c01\u9762\u5143\u4fe1\u606f\u3002

\u9a8c\u8bc1\uff1a\u91cd\u8dd1 PDF\uff0c\u7528 pypdf \u63d0\u53d6\u7b2c 1-2 \u9875\u548c\u53c2\u8003\u6587\u732e\u9875\u786e\u8ba4\u4e24\u4e2a bug \u90fd\u5df2\u6d88\u5931\u3002

\u9879\u76ee\u65b0\u589e pypdf \u4f9d\u8d56\uff08\u5de5\u5177\u7c7b\uff0c\u9a8c\u8bc1 PDF \u6587\u672c\u5185\u5bb9\u7528\uff09\u3002

Co-authored-by: User <human>
2026-04-22 15:41:56 +08:00
14 changed files with 1079 additions and 199 deletions
+10 -9
View File
@@ -170,12 +170,13 @@ Follow `skill:humanizer-cn` §1-26 strictly:
## Hard Rules
1. Every claim has `[src_xxx]` citation
2. Every numerical fact has a source
3. Counter-evidence section is mandatory (not optional)
4. Word count ≥85% of quota, or continue searching
5. No scheduling metadata in body text (no "P0 core", "quota: X", "researcher: dr-analyst")
6. No SCQA labels (not even implicitly suggested by structure)
7. ❌ Never fabricate data, URLs, DOIs
8. ❌ Never use Chinese words for claims (English working language)
9. ❌ Never delegate to other agents
1. MUST: Every claim has `[src_xxx]` citation
2. MUST: Every numerical fact has a source
3. MUST: Counter-evidence paragraph is mandatory at chapter end. Per skill:evidence-table §"正文中反方证据段落的写作规范", the heading must express a concrete opinion (e.g., "反例:Codexis ECO 并非所有情境都优于 SPOS" or "值得警惕:临床前到 IND 的衰减率"), NOT a mechanical label like "Counter-Evidence" / "反驳证据". Use H2 or H3 heading level consistently; never use bold text as pseudo-heading.
4. MUST: Word count ≥85% of quota, or continue searching
5. MUST: No scheduling metadata in body text (no "P0 core", "quota: X", "researcher: dr-analyst")
6. MUST: No SCQA labels (not even implicitly suggested by structure)
7. MUST NOT: Fabricate data, URLs, DOIs
8. MUST NOT: Use Chinese words for claims (English working language)
9. MUST NOT: Delegate to other agents
10. MUST NOT: **Use emoji anywhere in the draft** (no ✅ ❌ 🔶 🔷 ⭐ 🟢 🔴 ⚠️ 💡 📌 🔑 📊 etc.). The PDF font has no glyphs for colored emoji; they render as empty boxes. Use plain text equivalents (e.g., "✓", "×", "注:", "警告:", or descriptive words like "advantages / limitations / example").
+6 -5
View File
@@ -299,8 +299,9 @@ Phase 4 成稿完成
## 禁止事项
- 改写 dr-analyst 已完成的章节正文
- 给每章开头强加 "**Situation**:" "**Complication**:" 等标注
- 在正文里保留"章节定位 / P0 核心章 / 字数配额 / 研究员"
- 参考文献用占位符了事,要确保 dr-reporter 把它填满
- 中途调用 dr-chief-editor(它只管 Phase 3
- 改写 dr-analyst 已完成的章节正文
- 给每章开头强加 "**Situation**:" "**Complication**:" 等标注
- 在正文里保留"章节定位 / P0 核心章 / 字数配额 / 研究员"
- 参考文献用占位符了事,要确保 dr-reporter 把它填满
- 中途调用 dr-chief-editor(它只管 Phase 3
- **在正文中使用 emoji**(✅ ❌ 🔶 🔷 ⭐ 🟢 🔴 ⚠️ 💡 📌 🔑 📊 等彩色符号)。PDF 字体无法渲染,会变成方框。用文字或简单符号(✓ × 注: 警告:)代替。
+60
View File
@@ -0,0 +1,60 @@
---
description: 术语表事实核查。用法 /dr-glossary [slug] [--from phase1|phase2|phase4]。用 Haiku + Serper/Exa 核查每个术语的英文全称和中文译名,识别拼写错误与误译。可在 Phase 2 前或 Phase 4 中运行。
agent: dr-pm
---
你是 dr-pm(项目经理),用户执行了 `/dr-glossary $ARGUMENTS`
## 何时运行
术语核查可以在三个时机运行,意义不同:
- **Phase 1 末 / Phase 2 初**:基于 framework.md 抽取的关键术语(公司/产品/技术名)预先建表,供 dr-analyst 查阅,避免在章节起草阶段就编造拼写错误(Mabwell 拼成 Maywavee 这种)。
- **Phase 2 中**:对 dr-analyst 产出的 ch01-chN.md 里出现的所有专有名词做核查。
- **Phase 4**:对 translate.py 累积的 glossary.json 做核查(当前默认时机)。
## 执行步骤
### Step 1: 定位项目 + 解析模式
```bash
slug="$ARGUMENTS"
# 默认 slug 从最近修改时间选;默认阶段 phase4
```
### Step 2: 准备术语种子列表
根据阶段选择输入:
- `--from phase1`:从 `projects/<slug>/phase1/framework.md` 抽取所有 H2/H3 标题 + 表格中的英文机构名、技术名、药物名。保存到 `projects/<slug>/phase2/terms.txt`(一行一个)。
- `--from phase2`:从 `projects/<slug>/phase2/drafts/*.md` 抽取所有高频英文专有名词(首字母大写、不在常见词词典)。
- `--from phase4`(默认):直接用 `projects/<slug>/phase4/glossary.json`
### Step 3: 调用 build_glossary.py
```bash
uv run python scripts/build_glossary.py <slug> --workers 4 \
--input <输入 glossary 或 terms.txt>
```
观察输出,关注:
- `[✓]` high confidence(自动可信)
- `[~]` medium(人工复核)
- `[?]` low confidence(必须复核)
- `⚠ issue` 字段非空的(通常是拼写错误或误译)
### Step 4: 汇报 + 建议下一步
向用户展示:
- 核查的术语数
- 发现的错误数(按严重度分级:拼写错误 / 译名错误 / 低置信度)
- 具体错误清单(前 10 条)
- 建议:
- 如果在 Phase 2 前 → 把 glossary.json 交给 dr-analyst 作为参考
- 如果在 Phase 4 → 跑 `uv run python scripts/apply_glossary.py <slug>` 把修正回塗到正文
## 注意事项
- build_glossary 需要网络访问,Serper 和 Exa 都需要代理。如遇 SSL EOF,降 `--workers 3`
- Haiku 对极专业或极新兴术语(如 "muRNA", "SBS linker")的判断可能不准,低置信度的条目必须人工复核。
- 别对通用缩写(PDE/ASGPR/LNP)作自动替换——apply_glossary 已有黑名单,但核查结果在 glossary.json 里仍会体现。
+43 -1
View File
@@ -81,11 +81,53 @@ description: 证据矩阵规范。规定每条核心结论必须有对应的证
- 处理建议:保留并注明争议 / 修改措辞 / 删除
<!-- 如有重大挑战 -->
🚨 CRITICAL<说明>
CRITICAL: <说明>
```
---
## 正文中反方证据段落的写作规范(v0.8 新)
### 标题必须观点化,不能叫 "反驳证据 / Counter-Evidence"
**问题诊断**v0.7 发现每章末尾 dr-analyst 会机械地写 `## 反驳证据`,标题重复而空洞,读者看了没有信息增益。
**新规则**:正文反方证据段落的标题必须:
1. **用二级 H2 或三级 H3 标题**(统一层级,禁止用加粗段冒充标题)
2. **包含具体判断**,不要用"反驳证据" / "反方证据" / "Counter-Evidence" 这种模板化命名
3. 至少要回答:**"对前述论点的哪一方面提出了什么挑战?"**
### 可接受的命名示例
| ✗ 不推荐 | ✓ 推荐 |
|---|---|
| 反驳证据 | 另一种声音:管线虚胖还是真实进展? |
| Counter-Evidence | 需要补充判断的副作用:汇聚偶联收率可能被高估 |
| 反方观点 | 反例:Codexis ECO 并非所有情境都优于 SPOS |
| Counter Arguments | 值得警惕的数据:临床前到 IND 的衰减率 |
### 段落结构模板(推荐)
```markdown
## <观点化标题>
虽然上文论证了 <核心观点>,但以下证据提示需要**有限度地**接受这一判断:
1. **<反方论点 1>**<具体数据或案例> [src_xxx]。影响评估:<说明>
2. **<反方论点 2>**<具体数据或案例> [src_xxx]。影响评估:<说明>
综合而言,核心结论仍成立,但需在 <某个具体维度> 上留出缓冲。
```
### 禁止的写法
- 单独用 **加粗段** 冒充反方证据标题(`**反方证据:** ...`
- 反方证据后不做整合判断,只是堆数据
- 在每个小节末尾都加反方证据(只在章末加一次即可;若小节级别有重大挑战,写在小节正文里即可)
---
## 置信度分级标准
| 置信度 | 条件 | 正文处理方式 |
+28
View File
@@ -101,6 +101,34 @@ SCQA 要写得隐式融合(见 mckinsey-method skill)。
例外:行内英文术语、代码、URL、数据单位前后保持英文标点合理。
### J. Emoji(强制禁用)
**正文与表格中严禁使用任何 emoji / 彩色符号**
禁用清单(但不限于):
`✅ ❌ ✔ ✖ 🔶 🔷 ⭐ 🟢 🔴 🟡 🟠 ⚠️ ⚠ 💡 📌 🔑 📊 📈 📉 🔥 ✨ 🎯 🎉 ➔ ➜`
**原因**:PDF 使用的思源字体子集不包含这些字符的 glyph,渲染为空白方框(□)。
**替代写法**
- 表格标记"有/无":用 `✓` `×`(思源字体支持)或中文字 `是` / `否`
- 强调状态:用 `◆` `●` 等几何符号(字体支持)
- 警示:用 `注:` `警告:` `※` 等文字前缀
- 重点:用 **粗体** 或引用块,不用 emoji
扫描命令:
```bash
python3 -c "
import re
txt = open('final_zh_polished.md').read()
pat = re.compile(r'[\u2700-\u27BF]|[\U0001F300-\U0001F9FF]|[\u2B00-\u2BFF]')
hits = [(i, m.group()) for i, m in enumerate(pat.finditer(txt))]
print(f'emoji 命中:{len(hits)} 处')
for i, c in hits[:10]:
print(f' 位置 {i}: {c!r} (U+{ord(c):04X})')
"
```
---
## 三、参考文献完整性校验(最关键)
+57 -22
View File
@@ -82,39 +82,74 @@ description: 生物医药深度研究的统一检索策略。规定信源优先
---
## 三、API 调用顺序(技术栈)
## 三、API 调用顺序(技术栈v0.8 更新
**按"查询类型"路由到最合适的 API**,而不是一律走通用搜索。
```
┌─────────────────────────────────────────────┐
Phase 1 初扫(dr-searcher 用)
│ ├── tavily (MCP) — 快速宽扫 │
├── brave (MCP) — 交叉验证
└── exa (MCP) — neural search
│ │
Phase 2 深研(dr-analyst 用)
├── pubmed esearch/efetch (bash+curl)
├── clinicaltrials.gov API (bash+curl)
├── openfda API (bash+curl)
├── patentsview / google patents (bash)
└── + Phase 1 的 3 个 MCP 继续用
Phase 2 反验(dr-verifier 用)
└── 与 analyst 相同但查反向关键词
└─────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────────
查询类型 首选 备选
├──────────────────────────────────────────────────────────────┤
专利 Serper Patents Google Patents 爬页
(site:patents.google.com)
学术论文 Serper Scholar PubMed E-utils
(带引用数/年份/期刊) (原始数据库)
新闻 / 行业动态 Serper News Tavily
(时效性敏感)
临床试验 ClinicalTrials.gov ChiCTR
(原始 API)
│ 监管公告 openFDA / FDA 搜索 EMA / NMPA 官网 │
│ │
│ 公司/机构识别 Exa Tavily │
│ (LinkedIn/官网召回强) │
│ │
│ 通用网页 Exa Tavily / Brave │
└──────────────────────────────────────────────────────────────┘
```
### Tavily MCP 调用模板
### Serpergoogle.serper.dev)使用模板
**专利检索**
```python
from scripts.lib.search_client import SearchClient
with SearchClient() as c:
hits = c.patents("dual-target siRNA GalNAc", num_results=10)
```
**学术论文**
```python
hits = c.scholar("dual-target RNAi 2024", num_results=10, year_low=2023)
# hits[i].snippet 里包含引用数和期刊信息
```
**新闻(时效性)**
```python
hits = c.news("Arrowhead ARO-DIMER-PA clinical trial", time_range="w") # 最近一周
```
### Tavily MCP 调用模板(通用网页 - Phase 1 初扫)
```
工具名:tavily_search
参数:
query: "<关键词>"
search_depth: "advanced" # 默认 basic,深度研究用 advanced
search_depth: "advanced"
max_results: 10
include_domains: ["pubmed.ncbi.nlm.nih.gov", "nejm.org", "lancet.com"] # 锁 Tier 1
exclude_domains: ["baijiahao.baidu.com", "toutiao.com"] # 排黑名单
time_range: "year" # 或 "month"
include_domains: ["pubmed.ncbi.nlm.nih.gov", "nejm.org", "lancet.com"]
exclude_domains: ["baijiahao.baidu.com", "toutiao.com"]
time_range: "year"
```
### 何时用哪个
- **专利相关问题**"谁拥有技术 X"、"FTO 自由度分析")→ `c.patents()`**永远先走 Google Patents**
- **需要引用数排序的论文**("找影响力最大的 N 篇")→ `c.scholar()`
- **近期新闻**"2025 年 Q4 交易"、"上月 IND 批准")→ `c.news(..., time_range="m")`
- **快速概念验证 / 术语理解** → `c.search()`(通用)
### PubMed E-utilsbash 调用示例)
```bash
# 1. esearch 拿 PMID 列表
+423 -148
View File
@@ -49,6 +49,7 @@ try:
Table,
TableStyle,
)
from reportlab.platypus.flowables import HRFlowable
except ImportError:
print("ERROR: missing reportlab. Run: uv sync", file=sys.stderr)
sys.exit(1)
@@ -159,7 +160,7 @@ def build_styles() -> StyleSheet1:
allowOrphans=0,
))
# H1 (chapter) - page break before, deep blue
# H1 (chapter) - page break before, deep blue (used for 前置件标题如"免责声明")
ss.add(ParagraphStyle(
name="h1",
fontName="SrcSans-Bold",
@@ -173,6 +174,35 @@ def build_styles() -> StyleSheet1:
wordWrap="CJK",
))
# H1 章号(正文章节第一行):小字号居中,浅色
ss.add(ParagraphStyle(
name="h1-chapter-num",
fontName="SrcSans-Medium",
fontSize=13,
leading=20,
alignment=TA_CENTER,
spaceBefore=18,
spaceAfter=6,
textColor=colors.HexColor("#6b7280"),
letterSpacing=3, # 章号加字距,视觉更稳
keepWithNext=1,
wordWrap="CJK",
))
# H1 章名(正文章节第二行):大字号居中加粗深蓝
ss.add(ParagraphStyle(
name="h1-chapter-title",
fontName="SrcSans-Bold",
fontSize=20,
leading=32,
alignment=TA_CENTER,
spaceBefore=0,
spaceAfter=12,
textColor=colors.HexColor("#1e3a8a"),
keepWithNext=1,
wordWrap="CJK",
))
# H2 (section) - blue, no page break, keep with next
ss.add(ParagraphStyle(
name="h2",
@@ -528,6 +558,37 @@ _UNICODE_SUB = {
_SUPER_CHARS_RE = re.compile(f"([{''.join(_UNICODE_SUPER)}]+)")
_SUB_CHARS_RE = re.compile(f"([{''.join(_UNICODE_SUB)}]+)")
# 彩色 emoji / 特殊符号 → 文字替代。思源字体子集不含这些字形,直接放会渲染成方框。
# 替换为字体里**实际存在**的符号(经过 fontTools 验证)。
# 验证命令见 scripts/lib/verify_font_glyphs.py
_EMOJI_FALLBACK = {
"": "", # U+2705 → U+2713 CHECK MARK(思源有)
"": "×", # U+274C → U+00D7 MULTIPLICATION SIGN(思源有,✗ U+2717 思源没有)
"": "×",
"": "×",
"🔶": "", # U+1F536 → U+25C6 BLACK DIAMOND(思源有)
"🔷": "", # U+25C7 WHITE DIAMOND(思源有)
"🟢": "", # U+25CF BLACK CIRCLE(思源有)
"🔴": "",
"🟡": "", # U+25CB WHITE CIRCLE
"🟠": "",
"": "", # U+2605 BLACK STAR(思源有)
"": "",
"": "[✓]", # U+2611 思源没有,用方括号包围替代
"": "[×]",
"": "[ ]",
"": "",
"": "",
"⚠️": "", # U+203B REFERENCE MARK(思源有)
"": "",
"💡": "",
"📌": "",
"🔑": "",
"📊": "",
"📈": "",
"📉": "",
}
def _replace_unicode_superscripts(text: str) -> str:
"""把连续的 Unicode 上标字符替换为 ReportLab <super> 标签。
@@ -549,10 +610,20 @@ def _replace_unicode_superscripts(text: str) -> str:
return text
def _replace_emoji(text: str) -> str:
"""把字体里没有的 emoji 替换为字体里有的等价符号。"""
for emoji, fallback in _EMOJI_FALLBACK.items():
if emoji in text:
text = text.replace(emoji, fallback)
return text
def md_inline_to_rl(text: str, *, add_cjk_space: bool = True) -> str:
"""Markdown inline → ReportLab mini HTML."""
# 先做 Unicode 上/下标归一(字体子集不含这些字形,否则渲染为方框)
text = _replace_unicode_superscripts(text)
# emoji 替换为字体里有的符号
text = _replace_emoji(text)
# 然后在中英交界处加空格
if add_cjk_space:
text = _add_cjk_spaces(text)
@@ -571,6 +642,62 @@ def md_inline_to_rl(text: str, *, add_cjk_space: bool = True) -> str:
return text
# H1 正文章节标题解析:拆成(章号, 章名)
# 支持:
# 第一章 — 为何... → ("第一章", "为何...")
# 第 9 章:四大监管向量... → ("第 9 章", "四大监管向量...")
# 第6章 — 固定化... → ("第6章", "固定化...")
# 第十章 — 制造体系... → ("第十章", "制造体系...")
# Chapter 1: Why the Second... → ("Chapter 1", "Why the Second...")
# 分隔符:—(em dash / –(en dash / - / : / / 空白多于一处
_CHAPTER_HEAD_RE = re.compile(
r"^\s*"
r"(?P<num>(?:第\s*[一二三四五六七八九十百零〇两廿卅\d]+\s*章)|(?:Chapter\s+\d+))"
r"\s*[—–\-:]\s*"
r"(?P<title>.+?)\s*$",
re.IGNORECASE,
)
def parse_chapter_title(raw: str) -> tuple[str, str] | None:
"""解析章节标题。命中返回 (章号, 章名),否则 None。"""
m = _CHAPTER_HEAD_RE.match(raw.strip())
if not m:
return None
num = m.group("num").strip()
title = m.group("title").strip()
if not title:
return None
# 规范化章号空白:"第 6 章" 保留"第 6 章""第6章"保留"第6章"
num = re.sub(r"\s+", " ", num)
return num, title
def build_chapter_header(raw_title: str, styles: StyleSheet1) -> list:
"""生成正文章节标题:两行居中 + 装饰横线。
解析失败时 fallback 到普通 h1 样式。
"""
parsed = parse_chapter_title(raw_title)
if parsed is None:
return [Paragraph(md_inline_to_rl(raw_title), styles["h1"])]
num, title = parsed
return [
Paragraph(md_inline_to_rl(num), styles["h1-chapter-num"]),
Paragraph(md_inline_to_rl(title), styles["h1-chapter-title"]),
# 装饰横线:居中、宽度约 3cm(视觉重量跟两行标题平衡)
HRFlowable(
width=3 * cm,
thickness=1.2,
color=colors.HexColor("#1e3a8a"),
spaceBefore=2,
spaceAfter=18,
hAlign="CENTER",
),
]
# ============================================================
# Document builders
# ============================================================
@@ -711,8 +838,8 @@ def collect_toc_entries(blocks: List[Block]) -> List[tuple[int, str]]:
def build_toc(blocks: List[Block], styles: StyleSheet1) -> List:
"""生成目录条目。
目录末尾 PageBreak 让后续内容独立成页。开头不 PageBreak
调用方(H1 分支)已经负责在 H1 前另起一页
不在本函数内部 PageBreak——前面由调用方(H1/H2 分支)插入 PageBreak
后面靠下一个章节的 H1 PageBreak 自然起作用。避免"连续 PageBreak 产生空页"
"""
story: list = []
story.append(Paragraph("目录", styles["h1"]))
@@ -720,7 +847,6 @@ def build_toc(blocks: List[Block], styles: StyleSheet1) -> List:
for level, title in collect_toc_entries(blocks):
style_name = "toc-h1" if level == 1 else "toc-h2"
story.append(Paragraph(md_inline_to_rl(title), styles[style_name]))
story.append(PageBreak())
return story
@@ -812,6 +938,19 @@ def format_gb7714(rec: dict) -> str:
return body
def _sort_src_id(sid: str) -> tuple:
"""为 src_id 生成排序键:按字母段分组(A/B/C/E/...),组内按数字升序。"""
m = re.match(r"src_([A-Za-z]+)?(\d+)?([A-Za-z0-9_\-]*)", sid)
if not m:
return ("~", 0, sid)
alpha, num, rest = m.group(1) or "", m.group(2) or "0", m.group(3) or ""
try:
num_int = int(num)
except ValueError:
num_int = 0
return (alpha, num_int, rest)
def build_references(
blocks: List[Block],
sources_path: Optional[Path],
@@ -819,7 +958,12 @@ def build_references(
) -> List:
"""生成参考文献段落。
引用顺序:按正文首次出现的先后排列(GB/T 7714 顺序编码制)。
v0.7 改变:**不再按出现顺序重编号**(之前会导致正文中 `[src_E43]` 和参考文献
区的 `[27]` 对不上)。改为:
- 参考文献条目直接用原始 `src_id` 作为编号(如 `[src_E43] Alnylam..., 2025.`
- 按 src_id 字母数字排序分组
- 缺失的 src_id 单独一段列出,明显标注供人工核查
- 顶部给一条"引文健康状态"小结
"""
story: list = []
story.append(Paragraph("参考文献", styles["h1"]))
@@ -835,31 +979,67 @@ def build_references(
))
return story
if not sources:
# 至少列出所有被引用的 ID,供人工回填
cited_set = set(cited_ids)
matched = [sid for sid in cited_ids if sid in sources]
missing = [sid for sid in cited_ids if sid not in sources]
# sources.jsonl 里有但正文没引用的——列为"备选"不展示,只统计
unused = [sid for sid in sources if sid not in cited_set]
# 头部健康状态
health = (
f"正文引用 <b>{len(cited_set)}</b> 条独立标识符;"
f"sources.jsonl 收录 <b>{len(sources)}</b> 条,"
f"<b>{len(matched)}</b> 条可对应,"
f"<b>{len(missing)}</b> 条在 sources.jsonl 中未找到。"
)
if unused:
health += f" 另有 {len(unused)} 条收录来源未在正文中引用,已省略展示。"
story.append(Paragraph(
f"<font color='#6b7280' size=8>引文健康状态:{health}</font>",
styles["caption"],
))
story.append(Spacer(1, 0.3 * cm))
# 主列表:按 src_id 字母数字排序
if matched:
story.append(Paragraph(
f"(未找到 sources.jsonl 或其内容为空。以下为正文出现的 {len(cited_ids)} 个引用标识符)",
"<b>收录来源</b>",
styles["h3"],
))
for sid in sorted(matched, key=_sort_src_id):
rec = sources[sid]
text = format_gb7714(rec)
# 编号就是原始 sid,便于和正文中的 [src_E43] 上标对应
entry = f"<b>[{sid}]</b> {text}"
story.append(Paragraph(entry, styles["footnote"]))
# 缺失列表:明显标注
if missing:
story.append(Spacer(1, 0.4 * cm))
story.append(Paragraph(
f"<b>未找到来源({len(missing)} 条)</b>",
styles["h3"],
))
story.append(Paragraph(
"<font color='#b45309' size=8>"
"以下标识符在正文中出现但未在 <code>sources.jsonl</code> 中找到对应记录。"
"可能是编写阶段的占位符未回填,或原始研究员引用不规范,请核查后补充。"
"</font>",
styles["caption"],
))
for i, sid in enumerate(cited_ids, 1):
story.append(Paragraph(f"[{i}] {sid}", styles["footnote"]))
return story
missing: list[str] = []
for i, sid in enumerate(cited_ids, 1):
rec = sources.get(sid)
if not rec:
missing.append(sid)
# 按字母数字排序分组展示,一行三个,节省篇幅
sorted_missing = sorted(missing, key=_sort_src_id)
# 每 4 个一行
row_size = 4
for k in range(0, len(sorted_missing), row_size):
chunk = sorted_missing[k : k + row_size]
row_text = "  ".join(f"[{sid}]" for sid in chunk)
story.append(Paragraph(
f"[{i}] {sid}(来源记录缺失,请核查 sources.jsonl",
f"<font color='#b45309'>{row_text}</font>",
styles["footnote"],
))
continue
text = format_gb7714(rec)
# 前面加序号,后面追加 [sid] 便于正文回溯
entry = f"[{i}] {text} <font color='#6b7280' size=7>【{sid}】</font>"
story.append(Paragraph(entry, styles["footnote"]))
# 打印到 stderr
if missing:
print(
f"WARNING: {len(missing)} cited src_ids not found in sources.jsonl: "
@@ -939,6 +1119,49 @@ def render_table(md_table: str, styles: StyleSheet1) -> Table:
return table
def _render_generic_block(block: Block, story: list, base_dir: Path, styles: StyleSheet1, *, in_summary: bool) -> None:
"""渲染一个非 H1/H2 的 blockp/quote/bullet/hr/image/table/h3)。
提取出来的帮助函数,给术语表内部循环和主循环复用。
"""
if block.kind == "h3":
story.append(Paragraph(md_inline_to_rl(block.content), styles["h3"]))
elif block.kind == "p":
if _TOC_PLACEHOLDER_RE.search(block.content) or _REF_PLACEHOLDER_RE.search(block.content):
return
style = styles["summary"] if in_summary else styles["body"]
story.append(Paragraph(md_inline_to_rl(block.content), style))
elif block.kind == "quote":
story.append(Paragraph(md_inline_to_rl(block.content), styles["quote"]))
elif block.kind == "bullet":
story.append(Paragraph("" + md_inline_to_rl(block.content), styles["bullet"]))
elif block.kind == "hr":
story.append(Spacer(1, 0.3 * cm))
elif block.kind == "image":
img_path = base_dir / block.content
if img_path.exists():
try:
img = Image(str(img_path), width=15 * cm, height=10 * cm, kind="proportional")
story.append(img)
if block.meta and block.meta.get("caption"):
story.append(Paragraph(block.meta["caption"], styles["caption"]))
except Exception as e:
story.append(Paragraph(
f"[图片加载失败:{block.content}{e}]",
styles["caption"],
))
else:
story.append(Paragraph(
f"[图片未找到:{block.content}]",
styles["caption"],
))
elif block.kind == "table":
try:
story.append(render_table(block.content, styles))
except Exception as e:
story.append(Paragraph(f"[表格渲染失败: {e}]", styles["caption"]))
def build_body(
blocks: List[Block],
base_dir: Path,
@@ -955,143 +1178,195 @@ def build_body(
- H1 triggers PageBreakH2/H3 keepWithNext;表格 splitByRow
"""
story: list = []
first_h1_seen = False # 是否已跳过正文首个 H1
skipping_cover_meta = False # 是否在吞掉封面元信息段
# Summary 样式
in_summary = False
# 准备跳过标志:标题级别下一个 "目录""参考文献" 见到时替换掉它(包含其下紧跟的占位段)
# 采用简单索引遍历以便向前看
i = 0
# 第一步:跳过"封面块"——从正文开头一直跳到第一个 H2/H3 前。
# 封面块 = 首个 H1(主标题) + 副标题(加粗 p) + 元信息段(Confidentiality/Date/Version + 分隔线(hr
# 这些已由 build_cover 从 manifest 独立生成,正文里再出现就是重复。
# 规则简单可靠:跳过所有 block 直到遇到第一个 H2/H3(如 "## 免责声明")。
n = len(blocks)
first_section_idx = n
for k, b in enumerate(blocks):
if b.kind in ("h2", "h3"):
first_section_idx = k
break
i = first_section_idx # 从第一个 section 开始处理
def _title_kind(title_raw: str) -> str:
"""识别标题的"语义类别"。无论原文 H1 或 H2,统一归类。
返回:
'abstract' — 摘要(将被跳过)
'appendix' — 附录(将被跳过)
'version_history' — 版本历史(将被跳过)
'toc' — 目录(自动生成)
'references' — 参考文献(自动生成)
'glossary' — 术语表(独立章节)
'disclaimer' — 免责声明(独立章节)
'executive_summary' — 执行摘要(独立章节)
'chapter' — 正文章节(默认)
"""
t = title_raw.strip().lower()
# 跳过类
if t in ("摘要", "abstract"):
return "abstract"
if t.startswith("附录") or t.startswith("appendix"):
return "appendix"
if t in ("版本历史", "version history", "版本"):
return "version_history"
# 自动生成类
if t in ("目录", "table of contents"):
return "toc"
if t in ("参考文献", "references", "bibliography"):
return "references"
# 识别类(带 PageBreak 独立成章)
if t in ("术语表", "glossary"):
return "glossary"
if t in ("免责声明", "disclaimer"):
return "disclaimer"
if t in ("执行摘要", "executive summary", "管理层摘要"):
return "executive_summary"
return "chapter"
def _consume_until_next_section(start: int) -> int:
"""从 start 开始收集内容(非 H1/H2 的 block),返回下一个 H1/H2 的索引。"""
j = start
while j < n and blocks[j].kind not in ("h1", "h2"):
_render_generic_block(blocks[j], story, base_dir, styles, in_summary=in_summary)
j += 1
return j
def _skip_until_next_section(start: int) -> int:
"""从 start 开始跳过内容,返回下一个 H1/H2 的索引。"""
j = start
while j < n and blocks[j].kind not in ("h1", "h2"):
j += 1
return j
# 收集前置件(在第一个"正文 H1 章节"之前的所有内容)按 title_kind 分组。
# 然后按固定顺序输出:免责声明 → 执行摘要 → 目录 → 术语表 → 正文 → 参考文献。
# 这样无论 Markdown 里写的顺序如何,最终排版都一致(目录在术语表之前)。
first_h1_idx = n
for k in range(i, n):
if blocks[k].kind == "h1" and _title_kind(blocks[k].content) == "chapter":
first_h1_idx = k
break
# 收集"前置件段":从 i 到 first_h1_idx
front_sections: dict[str, list[Block]] = {}
def _collect_section(start: int, until: int) -> tuple[str, list[Block], int]:
"""从 start 处的 H1/H2 开始,收集这一 section 直到下一个 H1/H2(或 until)。
返回 (kind, blocks 列表, 下一个 section 的起始索引)。
"""
head = blocks[start]
kind = _title_kind(head.content)
sec = [head]
k = start + 1
while k < until and blocks[k].kind not in ("h1", "h2"):
sec.append(blocks[k])
k += 1
return kind, sec, k
# 在前置件区域内遍历
k = i
while k < first_h1_idx:
b = blocks[k]
if b.kind not in ("h1", "h2"):
k += 1
continue
kind, sec, next_k = _collect_section(k, first_h1_idx)
if kind in ("abstract", "appendix", "version_history"):
pass # 丢弃
elif kind in front_sections:
# 重复出现:保留第一份
pass
else:
front_sections[kind] = sec
k = next_k
# 前置件输出顺序(固定)
front_order = [
"disclaimer", # 免责声明
"executive_summary", # 执行摘要
"toc", # 目录
"glossary", # 术语表
]
def _render_head_section(kind: str, sec: list[Block]) -> None:
"""渲染一个前置件 section。sec[0] 是标题,其余是正文。"""
nonlocal in_summary
# 独立章节前加 PageBreak(但第一个除外,避免封面后空白页)
if len(story) > 0:
story.append(PageBreak())
in_summary = (kind == "executive_summary")
head = sec[0]
# TOC 和 references 调用专门的生成器
if kind == "toc":
story.extend(build_toc(blocks, styles))
return
if kind == "references":
story.extend(build_references(blocks, sources_path, styles))
return
# 其它前置件:H1 样式渲染标题 + 内容
story.append(Paragraph(md_inline_to_rl(head.content), styles["h1"]))
for sub in sec[1:]:
# 跳过占位符段
if sub.kind == "p" and (
_TOC_PLACEHOLDER_RE.search(sub.content)
or _REF_PLACEHOLDER_RE.search(sub.content)
):
continue
_render_generic_block(sub, story, base_dir, styles, in_summary=in_summary)
for kind in front_order:
if kind in front_sections:
_render_head_section(kind, front_sections[kind])
in_summary = False
# 现在输出正文(从 first_h1_idx 开始)
i = first_h1_idx
while i < n:
block = blocks[i]
# --- 跳过正文首个 H1(封面标题)+ 紧跟的元信息/hr ---
if not first_h1_seen and block.kind == "h1":
first_h1_seen = True
skipping_cover_meta = True
if block.kind not in ("h1", "h2"):
_render_generic_block(block, story, base_dir, styles, in_summary=in_summary)
i += 1
continue
if skipping_cover_meta:
# 吞掉 p(元信息)、hr、quote(副标题可能被当成加粗段)
# 遇到 h1/h2/h3 就停止吞
if block.kind in ("h1", "h2", "h3"):
skipping_cover_meta = False
# 不 continue,让当前 block 正常处理
elif block.kind == "p" and is_cover_frontmatter(block.content):
i += 1
continue
elif block.kind in ("hr", "quote", "p", "bullet"):
# 第一个 hr 标记封面结束
if block.kind == "hr":
skipping_cover_meta = False
i += 1
continue
# 普通段落:如果不是封面元信息,就认为封面已结束
if block.kind == "p":
skipping_cover_meta = False
# fall through to normal handling
else:
i += 1
continue
else:
i += 1
continue
# --- H1 处理(非首个)---
kind = _title_kind(block.content)
# 跳过类
if kind in ("abstract", "appendix", "version_history"):
i = _skip_until_next_section(i + 1)
continue
# 参考文献:自动生成
if kind == "references":
story.append(PageBreak())
story.extend(build_references(blocks, sources_path, styles))
j = i + 1
while j < n and blocks[j].kind == "p" and _REF_PLACEHOLDER_RE.search(blocks[j].content):
j += 1
i = j
continue
# 目录/术语表不应在正文中出现(已经作为前置件处理)
# 如果原文里在正文中又写了一遍目录/术语表,则跳过
if kind in ("toc", "glossary", "disclaimer", "executive_summary"):
i = _skip_until_next_section(i + 1)
continue
# H1 正文章节(chapter):PageBreak + 章号/章名双行居中 + 装饰线
if block.kind == "h1":
story.append(PageBreak())
content = block.content
if any(k in content for k in ("执行摘要", "Executive Summary", "管理层摘要")):
in_summary = True
else:
in_summary = False
# 目录 / 参考文献:替换为自动生成的内容
title_low = content.strip().lower()
if any(s in title_low for s in ("目录", "table of contents")):
story.extend(build_toc(blocks, styles))
# 跳过紧随其后的占位段
j = i + 1
while j < n and blocks[j].kind == "p" and _TOC_PLACEHOLDER_RE.search(blocks[j].content):
j += 1
i = j
continue
if any(s in title_low for s in ("参考文献", "references")):
story.extend(build_references(blocks, sources_path, styles))
j = i + 1
while j < n and blocks[j].kind == "p" and _REF_PLACEHOLDER_RE.search(blocks[j].content):
j += 1
i = j
continue
story.append(Paragraph(md_inline_to_rl(content), styles["h1"]))
i += 1
continue
# --- H2 同样检测占位符 ---
if block.kind == "h2":
title_low = block.content.strip().lower()
if any(s in title_low for s in ("目录", "table of contents")):
story.extend(build_toc(blocks, styles))
j = i + 1
while j < n and blocks[j].kind == "p" and _TOC_PLACEHOLDER_RE.search(blocks[j].content):
j += 1
i = j
continue
if any(s in title_low for s in ("参考文献", "references")):
story.extend(build_references(blocks, sources_path, styles))
j = i + 1
while j < n and blocks[j].kind == "p" and _REF_PLACEHOLDER_RE.search(blocks[j].content):
j += 1
i = j
continue
story.extend(build_chapter_header(block.content, styles))
else:
# H2 正文小节:h2 样式(不分页)
story.append(Paragraph(md_inline_to_rl(block.content), styles["h2"]))
i += 1
continue
if block.kind == "h3":
story.append(Paragraph(md_inline_to_rl(block.content), styles["h3"]))
elif block.kind == "p":
# 跳过已识别但没有标题的孤立占位符(防御性)
if _TOC_PLACEHOLDER_RE.search(block.content) or _REF_PLACEHOLDER_RE.search(block.content):
i += 1
continue
style = styles["summary"] if in_summary else styles["body"]
story.append(Paragraph(md_inline_to_rl(block.content), style))
elif block.kind == "quote":
story.append(Paragraph(md_inline_to_rl(block.content), styles["quote"]))
elif block.kind == "bullet":
story.append(Paragraph("" + md_inline_to_rl(block.content), styles["bullet"]))
elif block.kind == "hr":
story.append(Spacer(1, 0.3 * cm))
elif block.kind == "image":
img_path = base_dir / block.content
if img_path.exists():
try:
img = Image(str(img_path), width=15 * cm, height=10 * cm, kind="proportional")
story.append(img)
if block.meta and block.meta.get("caption"):
story.append(Paragraph(block.meta["caption"], styles["caption"]))
except Exception as e:
story.append(Paragraph(
f"[图片加载失败:{block.content}{e}]",
styles["caption"],
))
else:
story.append(Paragraph(
f"[图片未找到:{block.content}]",
styles["caption"],
))
elif block.kind == "table":
try:
story.append(render_table(block.content, styles))
except Exception as e:
story.append(Paragraph(f"[表格渲染失败: {e}]", styles["caption"]))
i += 1
return story
+1
View File
@@ -19,6 +19,7 @@ dependencies = [
"python-dateutil>=2.9.0",
"PyYAML>=6.0.1",
"rich>=13.7.0",
"pypdf>=6.10.2",
]
[project.optional-dependencies]
+146
View File
@@ -0,0 +1,146 @@
#!/usr/bin/env python3
"""引文完整性核查。
检查 final_zh_polished.md或其它正文中的 [src_xxx] 引用与 sources.jsonl 是否一致
- 孤立引用正文有但 sources.jsonl 需要 dr-analyst 补信源或删这处引用
- 孤岛信源sources.jsonl 有但正文无 polish 或润色误删了上下文 dr-analyst 收集了
但没用上
- emoji 扫描正文里不该有 emoji
- 编号格式检查 src_xxx 是否符合规范
用法
uv run python scripts/check_citations.py <project_slug>
uv run python scripts/check_citations.py <project_slug> --md phase4/final_zh.md
"""
from __future__ import annotations
import argparse
import json
import re
import sys
from pathlib import Path
EMOJI_RE = re.compile(
r"[\U0001F000-\U0001FFFF]" # Supplementary Plane emoji
r"|[\u2700-\u27BF]" # Dingbats (✅ ❌)
r"|[\u2600-\u26FF]" # Misc symbols (⭐ ⚠ ☀)
r"|[\u2B00-\u2BFF]" # Misc symbols and arrows
)
# 允许的符号(字体支持)
ALLOWED_SYMBOLS = {
"", "×", "", "", "", "", "", "",
"", "", "", "",
}
SRC_ID_RE = re.compile(r"\[(src_[A-Za-z0-9_\-]+(?:\s*,\s*src_[A-Za-z0-9_\-]+)*)\]")
def resolve_project(arg: str) -> Path:
p = Path(arg)
if p.is_dir():
return p
cand = Path.cwd() / "projects" / arg
if cand.is_dir():
return cand
raise SystemExit(f"project not found: {arg}")
def main() -> int:
parser = argparse.ArgumentParser(description="引文完整性核查")
parser.add_argument("project", help="项目 slug 或路径")
parser.add_argument("--md", default="phase4/final_zh_polished.md")
parser.add_argument("--sources", default="phase2/sources.jsonl")
args = parser.parse_args()
project = resolve_project(args.project)
md_path = project / args.md
src_path = project / args.sources
if not md_path.exists():
raise SystemExit(f"找不到正文:{md_path}")
text = md_path.read_text(encoding="utf-8")
# 1. 收集正文引用(保序去重)
cited: list[str] = []
cited_set: set[str] = set()
for m in SRC_ID_RE.finditer(text):
for sid in m.group(1).split(","):
sid = sid.strip()
if sid and sid not in cited_set:
cited_set.add(sid)
cited.append(sid)
# 2. 收集 sources.jsonl 中的 ID
sources: dict[str, dict] = {}
if src_path.exists():
for line in src_path.read_text(encoding="utf-8").splitlines():
line = line.strip()
if not line:
continue
try:
r = json.loads(line)
if sid := r.get("id"):
sources[sid] = r
except Exception:
continue
orphan_cites = [s for s in cited if s not in sources]
island_sources = [s for s in sources if s not in cited_set]
matched = [s for s in cited if s in sources]
print(f"=== 引文统计 ===")
print(f" 正文引用(独立 ID):{len(cited_set)}")
print(f" sources.jsonl 收录:{len(sources)}")
print(f" 匹配:{len(matched)}")
print(f" 孤立引用(正文有 sources 无):{len(orphan_cites)}")
print(f" 孤岛信源(sources 有正文无):{len(island_sources)}")
if orphan_cites:
print(f"\n=== 孤立引用(前 20 条)===")
for s in orphan_cites[:20]:
print(f" {s}")
if len(orphan_cites) > 20:
print(f" ...还有 {len(orphan_cites) - 20}")
print(f"\n 处理建议:")
print(f" (a) 如果是 dr-analyst 编造的占位符 → 在正文中删除该引用")
print(f" (b) 如果是信源未收录 → 补到 sources.jsonl")
if island_sources:
print(f"\n=== 孤岛信源(前 20 条)===")
for s in island_sources[:20]:
print(f" {s}{sources[s].get('title', '')[:80]}")
if len(island_sources) > 20:
print(f" ...还有 {len(island_sources) - 20}")
print(f"\n 处理建议:")
print(f" (a) 如果是 polish 阶段误删了使用该信源的段落 → 检查 polish diff")
print(f" (b) 如果是收集多余信源 → 可以保留(build_references 会自动忽略)")
# 3. Emoji 扫描
emoji_hits = []
for m in EMOJI_RE.finditer(text):
c = m.group()
if c not in ALLOWED_SYMBOLS:
line = text[:m.start()].count("\n") + 1
emoji_hits.append((line, c))
if emoji_hits:
print(f"\n=== ⚠ 发现 {len(emoji_hits)} 个 emoji(不允许出现在正文)===")
seen = {}
for line, c in emoji_hits:
seen.setdefault(c, []).append(line)
for c, lines in seen.items():
print(f" U+{ord(c):04X} {c!r}{lines[:5]} 行 等 {len(lines)}")
print(f" 建议用 python3 替换:sed -i '' 's/{list(seen.keys())[0]}//g' {md_path}")
# 返回码:有问题返回非零便于 CI 使用
if orphan_cites or emoji_hits:
return 1
return 0
if __name__ == "__main__":
sys.exit(main())
+99 -14
View File
@@ -115,11 +115,20 @@ class TavilyClient:
class SearchClient:
"""统一搜索门面:先用 Exa,失败/配额问题降级 Tavily。"""
"""统一搜索门面,支持多路由:
- `search(query)`通用网页搜索优先 Exa 降级 Tavily
- `patents(query)`专利检索 SerperGoogle Patents失败则通用搜索补刀
- `scholar(query)`学术论文 Serper Scholar失败则通用搜索补刀
- `news(query)`新闻检索 Serper News失败则通用搜索补刀
所有客户端都延迟导入 serper_client避免没装 SERPAPI_KEY import
"""
def __init__(self) -> None:
self._exa: ExaClient | None = None
self._tavily: TavilyClient | None = None
self._serper = None # 惰性实例化
try:
self._exa = ExaClient()
except SearchError:
@@ -133,11 +142,26 @@ class SearchClient:
"neither EXA_API_KEY nor TAVILY_API_KEY available"
)
def _get_serper(self):
"""惰性创建 SerperClient。没 key 时返回 None。"""
if self._serper is False:
return None
if self._serper is None:
try:
from scripts.lib.serper_client import SerperClient
self._serper = SerperClient()
except Exception:
self._serper = False
return None
return self._serper
def close(self) -> None:
if self._exa:
self._exa.close()
if self._tavily:
self._tavily.close()
if self._serper and self._serper is not False:
self._serper.close()
def __enter__(self) -> "SearchClient":
return self
@@ -146,18 +170,12 @@ class SearchClient:
self.close()
def search(self, query: str, *, num_results: int = 5) -> list[SearchHit]:
# 优先 Exa
"""通用网页搜索。Exa 首选,Tavily 备选。"""
if self._exa:
try:
return self._exa.search(query, num_results=num_results)
except SearchError as e:
msg = str(e).lower()
if "exceed" in msg or "quota" in msg or "429" in msg or "402" in msg:
# 降级
pass
else:
# 其它错误继续往下试
pass
except SearchError:
pass
if self._tavily:
try:
return self._tavily.search(query, num_results=num_results)
@@ -165,13 +183,80 @@ class SearchClient:
pass
return []
def patents(self, query: str, *, num_results: int = 10) -> list[SearchHit]:
"""专利检索:Serper 走 Google Patents 最准。降级到通用搜索 + site 限定。"""
serper = self._get_serper()
if serper:
try:
hits = serper.patents(query, num_results=num_results)
return [SearchHit(h.title, h.url, h.snippet) for h in hits]
except Exception:
pass
# 降级:通用搜索加 site 限定
return self.search(f"site:patents.google.com {query}", num_results=num_results)
def scholar(
self,
query: str,
*,
num_results: int = 10,
year_low: int | None = None,
) -> list[SearchHit]:
"""学术论文:Serper Scholar 带引用数。降级到通用搜索。"""
serper = self._get_serper()
if serper:
try:
hits = serper.scholar(query, num_results=num_results, year_low=year_low)
return [
SearchHit(
title=h.title,
url=h.url,
snippet=f"{h.snippet} | {h.source} | 引用 {h.cited_by}" if h.cited_by else h.snippet,
)
for h in hits
]
except Exception:
pass
return self.search(query, num_results=num_results)
def news(
self,
query: str,
*,
num_results: int = 10,
time_range: str | None = None,
) -> list[SearchHit]:
"""新闻检索:Serper News。降级到通用搜索。"""
serper = self._get_serper()
if serper:
try:
hits = serper.news(query, num_results=num_results, time_range=time_range)
return [
SearchHit(
title=h.title,
url=h.url,
snippet=f"{h.snippet} | {h.source} | {h.date}" if h.date else h.snippet,
)
for h in hits
]
except Exception:
pass
return self.search(query, num_results=num_results)
if __name__ == "__main__":
from scripts.lib.zenmux_client import load_secrets
load_secrets()
with SearchClient() as c:
hits = c.search("Mabwell 迈威生物 biopharmaceutical", num_results=3)
for i, h in enumerate(hits, 1):
print(f"[{i}] {h.title[:80]}")
print("--- 通用: Mabwell 迈威生物 ---")
for h in c.search("Mabwell 迈威生物 biopharmaceutical", num_results=3):
print(f" {h.title[:80]}")
print(f" {h.url}")
print("\n--- 专利: dual-target siRNA ---")
for h in c.patents("dual-target siRNA GalNAc", num_results=3):
print(f" {h.title[:80]}")
print(f" {h.url}")
print("\n--- Scholar: dual-target RNAi 2024 ---")
for h in c.scholar("dual-target RNAi drug", num_results=3, year_low=2023):
print(f" {h.title[:80]}")
print(f" {h.url}")
print(f" {h.snippet[:160]}")
+190
View File
@@ -0,0 +1,190 @@
"""Serper.dev 客户端(Google Search API 代理)。
为什么用 Serper
- 2500 次免费额度远超 SerpAPI 100/
- 支持 Google SearchScholarNewsImagesMaps
- Google Patents 无专用 endpoint但可用 `site:patents.google.com` 技巧
- 价格比 SerpAPI 便宜 3-5×
用途
- 专利检索通用 search + `site:patents.google.com`
- 学术论文/scholar endpoint
- 新闻/news endpoint时效性敏感的行业动态
httpx 客户端使用 trust_env=False 绕过系统 socks 代理macOS Clash 会导致 TLS EOF
"""
from __future__ import annotations
import os
from dataclasses import dataclass
from typing import Any, Literal
import httpx
SERPER_BASE = "https://google.serper.dev"
@dataclass
class SerperHit:
title: str
url: str
snippet: str
source: str = "" # 论文出处 / 新闻媒体
date: str = "" # 发表日期(如 scholar / news 返回的话)
cited_by: int = 0 # 学术论文的引用数(仅 scholar)
class SerperError(RuntimeError):
pass
class SerperClient:
def __init__(self, api_key: str | None = None, timeout: float = 30.0) -> None:
self.api_key = api_key or os.environ.get("SERPAPI_KEY") or os.environ.get("SERPER_API_KEY")
if not self.api_key:
raise SerperError("SERPAPI_KEY / SERPER_API_KEY not set")
self._client = httpx.Client(trust_env=False, timeout=timeout)
def close(self) -> None:
self._client.close()
def __enter__(self) -> "SerperClient":
return self
def __exit__(self, *_args: Any) -> None:
self.close()
def _post(self, path: str, body: dict) -> dict:
try:
r = self._client.post(
f"{SERPER_BASE}{path}",
json=body,
headers={
"X-API-KEY": self.api_key,
"Content-Type": "application/json",
},
)
except httpx.RequestError as e:
raise SerperError(f"network error: {e}")
if r.status_code != 200:
raise SerperError(f"HTTP {r.status_code}: {r.text[:300]}")
try:
return r.json()
except Exception as e:
raise SerperError(f"invalid JSON: {e}")
def search(
self,
query: str,
*,
num_results: int = 10,
gl: str = "us",
hl: str = "en",
) -> list[SerperHit]:
"""通用 Google 搜索。支持 site: / filetype: / 引号短语等 Google 高级语法。"""
data = self._post("/search", {
"q": query,
"num": num_results,
"gl": gl,
"hl": hl,
})
hits: list[SerperHit] = []
for item in (data.get("organic") or [])[:num_results]:
hits.append(SerperHit(
title=(item.get("title") or "")[:200],
url=item.get("link") or "",
snippet=(item.get("snippet") or "")[:600],
date=item.get("date") or "",
))
return hits
def scholar(
self,
query: str,
*,
num_results: int = 10,
year_low: int | None = None,
year_high: int | None = None,
) -> list[SerperHit]:
"""Google Scholar 搜索——学术论文首选。
返回带引用数发表年份等元数据权威信源识别更准确
"""
body: dict[str, Any] = {"q": query, "num": num_results}
if year_low is not None:
body["tbs"] = f"cdr:1,cd_min:{year_low}" + (f",cd_max:{year_high}" if year_high else "")
data = self._post("/scholar", body)
hits: list[SerperHit] = []
for item in (data.get("organic") or [])[:num_results]:
hits.append(SerperHit(
title=(item.get("title") or "")[:200],
url=item.get("link") or "",
snippet=(item.get("snippet") or "")[:600],
source=(item.get("publicationInfo") or "")[:200],
year=item.get("year") or "",
cited_by=item.get("citedBy") or 0,
) if False else SerperHit(
title=(item.get("title") or "")[:200],
url=item.get("link") or "",
snippet=(item.get("snippet") or "")[:600],
source=(item.get("publicationInfo") or "")[:200],
date=str(item.get("year") or ""),
cited_by=item.get("citedBy") or 0,
))
return hits
def patents(
self,
query: str,
*,
num_results: int = 10,
) -> list[SerperHit]:
"""Google Patents 检索——用 site: 技巧走通用搜索。
serper.dev 没有专门的 patents endpoint `site:patents.google.com` 效果很好
"""
combined = f"site:patents.google.com {query}"
return self.search(combined, num_results=num_results)
def news(
self,
query: str,
*,
num_results: int = 10,
time_range: Literal["d", "w", "m", "y"] | None = None,
) -> list[SerperHit]:
"""Google News 搜索——时效敏感行业动态。
time_range: d=24h, w=7d, m=30d, y=1y
"""
body: dict[str, Any] = {"q": query, "num": num_results}
if time_range:
body["tbs"] = f"qdr:{time_range}"
data = self._post("/news", body)
hits: list[SerperHit] = []
for item in (data.get("news") or [])[:num_results]:
hits.append(SerperHit(
title=(item.get("title") or "")[:200],
url=item.get("link") or "",
snippet=(item.get("snippet") or "")[:600],
source=(item.get("source") or "")[:200],
date=item.get("date") or "",
))
return hits
if __name__ == "__main__":
from scripts.lib.zenmux_client import load_secrets
load_secrets()
with SerperClient() as c:
print("=== Patents: dual-target siRNA ===")
for h in c.patents("dual-target siRNA GalNAc conjugate", num_results=3):
print(f" {h.title[:70]}")
print(f" {h.url}")
print("\n=== Scholar: dual-target RNAi ===")
for h in c.scholar("dual-target RNAi drug 2024", num_results=3):
print(f" {h.title[:70]} [引用 {h.cited_by}] ({h.date})")
print(f" {h.url}")
print(f" 源: {h.source[:80]}")
+1
View File
@@ -8,6 +8,7 @@
4. **保留段落数量**:不要合并或拆分段落。每段原文对应一段输出。
5. **专有名词首次出现保持"中文(English)"格式**;如果译文里这个术语已经这样标了就别改,也不要删掉。
6. **不改变论点、结论、数据、案例**。只改语言表达。
7. **严禁使用 emoji**(✅ ❌ 🔶 🔷 ⭐ 🟢 🔴 ⚠️ 💡 📌 🔑 📊 等彩色符号)。如果原文里有 emoji,替换为字体支持的符号(✓ × ◆ ● ★ * 注 等)或直接删除。这些 emoji 在 PDF 里渲染为方框。
## 要去掉的"AI 味/翻译腔"表征
+1
View File
@@ -13,6 +13,7 @@ You are a senior English-to-Chinese biomedical translator and editor. You do NOT
9. Do NOT collapse or merge consecutive paragraphs — preserve paragraph breaks.
10. Output Chinese-style punctuation inside Chinese text: `,。;:?!""()`. Keep English punctuation inside parenthetical English phrases.
11. Do NOT add separator lines (`---`) or blank lines that weren't in the source. If the source ends with `---`, keep it; if it doesn't, don't add one.
12. **NEVER use emoji** (✅ ❌ 🔶 🔷 ⭐ 🟢 🔴 ⚠️ 💡 📌 🔑 📊 etc.). If the source contains emoji, replace with plain text or punctuation equivalents (✓ × ◆ ● ★ * 注 等). These do not render in the PDF (font has no glyphs).
## Style rules (aim for native-Chinese feel)
Generated
+14
View File
@@ -374,6 +374,7 @@ dependencies = [
{ name = "pandas", version = "2.3.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" },
{ name = "pandas", version = "3.0.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" },
{ name = "pillow" },
{ name = "pypdf" },
{ name = "python-dateutil" },
{ name = "pyyaml" },
{ name = "reportlab" },
@@ -397,6 +398,7 @@ requires-dist = [
{ name = "numpy", specifier = ">=1.26.0" },
{ name = "pandas", specifier = ">=2.1.0" },
{ name = "pillow", specifier = ">=10.0.0" },
{ name = "pypdf", specifier = ">=6.10.2" },
{ name = "pytest", marker = "extra == 'dev'", specifier = ">=7.4.0" },
{ name = "python-dateutil", specifier = ">=2.9.0" },
{ name = "pyyaml", specifier = ">=6.0.1" },
@@ -1292,6 +1294,18 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/10/bd/c038d7cc38edc1aa5bf91ab8068b63d4308c66c4c8bb3cbba7dfbc049f9c/pyparsing-3.3.2-py3-none-any.whl", hash = "sha256:850ba148bd908d7e2411587e247a1e4f0327839c40e2e5e6d05a007ecc69911d", size = 122781, upload-time = "2026-01-21T03:57:55.912Z" },
]
[[package]]
name = "pypdf"
version = "6.10.2"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "typing-extensions", marker = "python_full_version < '3.11'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/7b/3f/9f2167401c2e94833ca3b69535bad89e533b5de75fefe4197a2c224baec2/pypdf-6.10.2.tar.gz", hash = "sha256:7d09ce108eff6bf67465d461b6ef352dcb8d84f7a91befc02f904455c6eea11d", size = 5315679, upload-time = "2026-04-15T16:37:36.978Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/0c/d6/1d5c60cc17bbdf37c1552d9c03862fc6d32c5836732a0415b2d637edc2d0/pypdf-6.10.2-py3-none-any.whl", hash = "sha256:aa53be9826655b51c96741e5d7983ca224d898ac0a77896e64636810517624aa", size = 336308, upload-time = "2026-04-15T16:37:34.851Z" },
]
[[package]]
name = "pytest"
version = "9.0.3"