release: v0.20 Codex-ready skill-driven core
This commit is contained in:
@@ -49,6 +49,9 @@ env_vars = ["TAVILY_API_KEY"]
|
|||||||
enabled = true
|
enabled = true
|
||||||
required = false
|
required = false
|
||||||
|
|
||||||
|
[mcp_servers.tavily.tools.tavily_search]
|
||||||
|
approval_mode = "approve"
|
||||||
|
|
||||||
[mcp_servers.brave_search]
|
[mcp_servers.brave_search]
|
||||||
command = "npx"
|
command = "npx"
|
||||||
args = ["-y", "@modelcontextprotocol/server-brave-search"]
|
args = ["-y", "@modelcontextprotocol/server-brave-search"]
|
||||||
|
|||||||
@@ -138,7 +138,32 @@ dr-reporter 从 sources.jsonl 生成参考文献列表时,按以下格式:
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 五、引用完整性检查(dr-chief-editor 用)
|
## 五、脚注使用边界
|
||||||
|
|
||||||
|
脚注不是行内引用的替代品,也不用于重复输入材料中已经被正文自然承载的事实。脚注只在以下场景使用:
|
||||||
|
|
||||||
|
- **法规原文或条款定位**:正文需要引用法规要求,但完整条款会打断叙事时,脚注写明法规名称、章节/条款和关键原文。
|
||||||
|
- **关键资料原文**:原文措辞本身会影响判断强度,且正文只保留管理结论时,脚注可放短摘录。
|
||||||
|
- **补充背景或术语解释**:正文读者可能需要额外背景,但展开会破坏行文节奏。
|
||||||
|
- **版权或使用限制说明**:图表、第三方材料、内部材料使用边界需要单独说明时。
|
||||||
|
|
||||||
|
禁止事项:
|
||||||
|
|
||||||
|
- 不要把“某份输入材料说过什么”机械搬到脚注;这类事实应通过正文和数字引用解决。
|
||||||
|
- 不要为每个本地材料引用都加脚注;脚注应少而精,优先服务关键判断。
|
||||||
|
- 不要用脚注堆砌证据,核心证据仍应进入正文或证据表。
|
||||||
|
|
||||||
|
推荐格式:
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
正文关键判断<sup>[12]</sup>。[^1]
|
||||||
|
|
||||||
|
[^1]: ICH Q10《Pharmaceutical Quality System》第 4.1 节要求管理评审输入覆盖“results of regulatory inspections and findings, audits and commitments”,并纳入 CAPA、变更以及上次管理评审行动。
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 六、引用完整性检查(dr-chief-editor 用)
|
||||||
|
|
||||||
审校时检查:
|
审校时检查:
|
||||||
1. 正文中所有 [src_xxx] 都在 sources.jsonl 里有对应记录
|
1. 正文中所有 [src_xxx] 都在 sources.jsonl 里有对应记录
|
||||||
|
|||||||
@@ -0,0 +1,62 @@
|
|||||||
|
---
|
||||||
|
name: search-gateway
|
||||||
|
description: Use when Deep Research agents or subagents need web, scholar, patent, news, regulatory, or source-discovery search without using platform MCP tools or browser search directly.
|
||||||
|
---
|
||||||
|
|
||||||
|
# Search Gateway
|
||||||
|
|
||||||
|
## Rule
|
||||||
|
|
||||||
|
Use the project Python search gateway as the only default search interface. Do not call Tavily MCP, browser MCP, generic web tools, or platform-native search from a subagent unless the user explicitly asks for that escape hatch.
|
||||||
|
|
||||||
|
## Commands
|
||||||
|
|
||||||
|
Run searches from the repository root:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
uv run python scripts/search.py "<query>" --route general --json --trace
|
||||||
|
uv run python scripts/search.py "<query>" --route evidence --json --trace
|
||||||
|
uv run python scripts/search.py "<query>" --route scholar --year-low 2020 --json --trace
|
||||||
|
uv run python scripts/search.py "<query>" --route news --time-range y --json --trace
|
||||||
|
uv run python scripts/search.py "<query>" --route patents --json --trace
|
||||||
|
uv run python scripts/search.py "<query>" --profile biomed_literature --json --trace
|
||||||
|
```
|
||||||
|
|
||||||
|
If `uv` cannot use the user cache in a sandbox, set a local cache:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
UV_CACHE_DIR=/private/tmp/deep_research_uv_cache uv run python scripts/search.py "<query>" --route general --json --trace
|
||||||
|
```
|
||||||
|
|
||||||
|
## Routing
|
||||||
|
|
||||||
|
- `general`: Tavily first, Exa fallback, Brave fallback; use for broad discovery and gap filling.
|
||||||
|
- `evidence`: Exa highlights first, Tavily fallback, Brave fallback; use when a task card needs concise, source-level candidate evidence for an evidence packet.
|
||||||
|
- `scholar`: Serper Scholar first; use for papers, reviews, technical literature, and academic validation only.
|
||||||
|
- `news`: Serper News first; use for recent industry/current information.
|
||||||
|
- `patents`: Serper Google Patents first.
|
||||||
|
- `biomed_literature`: scholar plus general discovery.
|
||||||
|
|
||||||
|
Serper is not the default general web search source. Keep it mainly for Scholar, Google Patents, News, and targeted `site:` searches where Google coverage matters.
|
||||||
|
|
||||||
|
Tavily Research is a phase-level scan tool, not a packet-writing shortcut. Use it for Phase 1 initial landscape scans, Phase 2 gap-fill after a chapter is thin, or Phase 3回炉补证据;its output must be saved, source-scored, deduplicated, and converted into candidate evidence before citation.
|
||||||
|
|
||||||
|
Exa is the preferred controlled evidence discovery route for agents because it can return short highlights/text per URL. Treat Exa hits as candidate sources unless the URL itself is an original Tier 1-2 source.
|
||||||
|
|
||||||
|
API keys are loaded from `secrets.env` by `scripts/search.py`; do not ask the user to authorize MCP calls when the env keys are available.
|
||||||
|
|
||||||
|
## Subagent Protocol
|
||||||
|
|
||||||
|
For evidence packets:
|
||||||
|
|
||||||
|
1. Search through `scripts/search.py`, save or summarize the returned JSON in the packet’s `raw_quotes_or_notes`.
|
||||||
|
2. Use search hits only as candidate sources; whenever possible, cite the original regulator, guideline, paper, or official document.
|
||||||
|
3. Put every used source in `sources` with `id`, `title`, `url`, `tier`, and `score`.
|
||||||
|
4. Do not write a final chapter during search; produce structured evidence only.
|
||||||
|
5. For repeatedly used Tier 1-2 sources, run `uv run python scripts/dr.py sources cache <project>` so later phases can cite a local Markdown snapshot rather than only a URL.
|
||||||
|
|
||||||
|
For chapter assembly:
|
||||||
|
|
||||||
|
1. Do not search. Use only `phase2/chapter_briefs`, `phase2/packets`, `phase2/sources.jsonl`, `phase0/extracted`, and `phase1/framework.md`.
|
||||||
|
2. Do not create new `source_id`.
|
||||||
|
3. If evidence is thin, mark the chapter as needing Phase 2 enrichment instead of filling with generic prose.
|
||||||
@@ -75,10 +75,12 @@ description: 生物医药深度研究的统一检索策略。规定信源优先
|
|||||||
- 例:研究"GLP-1 成为减重首选"→ 反方要搜 "GLP-1 limitations" "semaglutide side effects" "discontinuation rate"
|
- 例:研究"GLP-1 成为减重首选"→ 反方要搜 "GLP-1 limitations" "semaglutide side effects" "discontinuation rate"
|
||||||
- 至少 3-5 条反方证据
|
- 至少 3-5 条反方证据
|
||||||
|
|
||||||
### 第 4 轮:Tavily/Brave/Exa 补漏
|
### 第 4 轮:Exa/Tavily/Brave 补漏
|
||||||
- 仅用于发现前 3 轮遗漏的 URL
|
- 仅用于发现前 3 轮遗漏的 URL
|
||||||
- 发现后**必须**回溯到原始 Tier 1-2 来源(论文 DOI、监管公告原文)
|
- 发现后**必须**回溯到原始 Tier 1-2 来源(论文 DOI、监管公告原文)
|
||||||
- 不得直接引用搜索返回的二次报道
|
- 不得直接引用搜索返回的二次报道
|
||||||
|
- 章节级 evidence packet 优先用 `scripts/search.py --route evidence`,让 Exa highlights 进入 source-quality 和 evidence-table。
|
||||||
|
- Tavily Research 只用于 Phase 1 初扫、薄弱章节补证据和 Phase 3 回炉;输出必须存盘、评分、去重后再转成 candidate evidence。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -88,6 +90,7 @@ description: 生物医药深度研究的统一检索策略。规定信源优先
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
uv run python scripts/search.py "<query>" --route scholar --num-results 10 --year-low 2023
|
uv run python scripts/search.py "<query>" --route scholar --num-results 10 --year-low 2023
|
||||||
|
uv run python scripts/search.py "<query>" --route evidence --num-results 10 --json --trace
|
||||||
uv run python scripts/search.py "<query>" --route patents --num-results 10
|
uv run python scripts/search.py "<query>" --route patents --num-results 10
|
||||||
uv run python scripts/search.py "<query>" --route news --num-results 10 --time-range m
|
uv run python scripts/search.py "<query>" --route news --num-results 10 --time-range m
|
||||||
uv run python scripts/search.py "<query>" --route general --num-results 10
|
uv run python scripts/search.py "<query>" --route general --num-results 10
|
||||||
@@ -107,7 +110,7 @@ uv run python scripts/search.py "<query>" --profile patent_heavy --num-results 1
|
|||||||
- `--route patents` 固定优先 Serper + Google Patents,避免专利检索被 Tavily 普通网页结果替代。
|
- `--route patents` 固定优先 Serper + Google Patents,避免专利检索被 Tavily 普通网页结果替代。
|
||||||
- `--route scholar` 固定优先 Serper Scholar,避免论文检索只停留在通用网页摘要。
|
- `--route scholar` 固定优先 Serper Scholar,避免论文检索只停留在通用网页摘要。
|
||||||
- 专用 route(scholar/patents/news)默认 `--strict-specialized`,Serper 异常时应显式失败,不允许静默降级。
|
- 专用 route(scholar/patents/news)默认 `--strict-specialized`,Serper 异常时应显式失败,不允许静默降级。
|
||||||
- Tavily / Exa / Brave 只作为 gap-fill 或 MCP 兜底,不作为文献/专利主路径。
|
- Exa evidence route 是 packet 候选证据发现主路径;Tavily / Brave 只作为 gap-fill 或 MCP 兜底,不作为文献/专利主路径。
|
||||||
|
|
||||||
每个检索小结必须写明实际使用过的 route,例如:
|
每个检索小结必须写明实际使用过的 route,例如:
|
||||||
|
|
||||||
|
|||||||
@@ -49,7 +49,7 @@ try:
|
|||||||
Table,
|
Table,
|
||||||
TableStyle,
|
TableStyle,
|
||||||
)
|
)
|
||||||
from reportlab.platypus.flowables import HRFlowable
|
from reportlab.platypus.flowables import Flowable, HRFlowable
|
||||||
except ImportError:
|
except ImportError:
|
||||||
print("ERROR: missing reportlab. Run: uv sync", file=sys.stderr)
|
print("ERROR: missing reportlab. Run: uv sync", file=sys.stderr)
|
||||||
sys.exit(1)
|
sys.exit(1)
|
||||||
@@ -279,6 +279,23 @@ def build_styles() -> StyleSheet1:
|
|||||||
allowOrphans=0,
|
allowOrphans=0,
|
||||||
))
|
))
|
||||||
|
|
||||||
|
# Inline evidence footnotes placed by number_citations / finalization.
|
||||||
|
ss.add(ParagraphStyle(
|
||||||
|
name="evidence-footnote",
|
||||||
|
fontName="SrcSerif",
|
||||||
|
fontSize=7.4,
|
||||||
|
leading=10,
|
||||||
|
alignment=TA_JUSTIFY,
|
||||||
|
leftIndent=18,
|
||||||
|
firstLineIndent=-18,
|
||||||
|
spaceBefore=0,
|
||||||
|
spaceAfter=2,
|
||||||
|
textColor=colors.HexColor("#4b5563"),
|
||||||
|
wordWrap="CJK",
|
||||||
|
allowWidows=0,
|
||||||
|
allowOrphans=0,
|
||||||
|
))
|
||||||
|
|
||||||
# Table cell - no first-line indent, smaller font, CJK wrap for auto line break
|
# Table cell - no first-line indent, smaller font, CJK wrap for auto line break
|
||||||
ss.add(ParagraphStyle(
|
ss.add(ParagraphStyle(
|
||||||
name="table-cell",
|
name="table-cell",
|
||||||
@@ -422,6 +439,23 @@ def build_styles() -> StyleSheet1:
|
|||||||
bulletIndent=8,
|
bulletIndent=8,
|
||||||
))
|
))
|
||||||
|
|
||||||
|
# Ordered list. Keep references flush-left; do not prepend decorative bullets.
|
||||||
|
ss.add(ParagraphStyle(
|
||||||
|
name="ordered",
|
||||||
|
fontName="SrcSerif",
|
||||||
|
fontSize=9,
|
||||||
|
leading=13,
|
||||||
|
alignment=TA_JUSTIFY,
|
||||||
|
firstLineIndent=0,
|
||||||
|
leftIndent=0,
|
||||||
|
spaceBefore=0,
|
||||||
|
spaceAfter=4,
|
||||||
|
textColor=colors.HexColor("#374151"),
|
||||||
|
wordWrap="CJK",
|
||||||
|
allowWidows=0,
|
||||||
|
allowOrphans=0,
|
||||||
|
))
|
||||||
|
|
||||||
return ss
|
return ss
|
||||||
|
|
||||||
|
|
||||||
@@ -436,6 +470,28 @@ class Block:
|
|||||||
meta: Optional[dict] = None
|
meta: Optional[dict] = None
|
||||||
|
|
||||||
|
|
||||||
|
class FootnoteFlowable(Flowable):
|
||||||
|
"""Zero-height anchor that registers a footnote for the current PDF page."""
|
||||||
|
|
||||||
|
def __init__(self, label: str, content: str):
|
||||||
|
super().__init__()
|
||||||
|
self.label = label
|
||||||
|
self.content = content
|
||||||
|
self.width = 0
|
||||||
|
self.height = 0
|
||||||
|
|
||||||
|
def wrap(self, availWidth, availHeight):
|
||||||
|
return 0, 0
|
||||||
|
|
||||||
|
def draw(self):
|
||||||
|
page = self.canv.getPageNumber()
|
||||||
|
notes = getattr(self.canv, "_dr_footnotes", None)
|
||||||
|
if notes is None:
|
||||||
|
notes = {}
|
||||||
|
setattr(self.canv, "_dr_footnotes", notes)
|
||||||
|
notes.setdefault(page, []).append((self.label, self.content))
|
||||||
|
|
||||||
|
|
||||||
def parse_markdown(md_text: str) -> List[Block]:
|
def parse_markdown(md_text: str) -> List[Block]:
|
||||||
blocks: List[Block] = []
|
blocks: List[Block] = []
|
||||||
lines = md_text.split("\n")
|
lines = md_text.split("\n")
|
||||||
@@ -483,6 +539,17 @@ def parse_markdown(md_text: str) -> List[Block]:
|
|||||||
blocks.append(Block(kind="quote", content="\n".join(quote_lines)))
|
blocks.append(Block(kind="quote", content="\n".join(quote_lines)))
|
||||||
continue
|
continue
|
||||||
|
|
||||||
|
# Markdown footnote definition: [^1]: 原文摘录...
|
||||||
|
m = re.match(r"^\[\^([A-Za-z0-9_-]+)\]:\s*(.+)$", stripped)
|
||||||
|
if m:
|
||||||
|
blocks.append(Block(
|
||||||
|
kind="footnote",
|
||||||
|
content=m.group(2).strip(),
|
||||||
|
meta={"label": m.group(1)},
|
||||||
|
))
|
||||||
|
i += 1
|
||||||
|
continue
|
||||||
|
|
||||||
# Unordered list
|
# Unordered list
|
||||||
if re.match(r"^[-*+]\s+", stripped):
|
if re.match(r"^[-*+]\s+", stripped):
|
||||||
while i < len(lines) and re.match(r"^[-*+]\s+", lines[i].strip()):
|
while i < len(lines) and re.match(r"^[-*+]\s+", lines[i].strip()):
|
||||||
@@ -493,12 +560,12 @@ def parse_markdown(md_text: str) -> List[Block]:
|
|||||||
|
|
||||||
# Ordered list
|
# Ordered list
|
||||||
if re.match(r"^\d+\.\s+", stripped):
|
if re.match(r"^\d+\.\s+", stripped):
|
||||||
idx = 1
|
|
||||||
while i < len(lines) and re.match(r"^\d+\.\s+", lines[i].strip()):
|
while i < len(lines) and re.match(r"^\d+\.\s+", lines[i].strip()):
|
||||||
item = re.sub(r"^\d+\.\s+", "", lines[i].strip())
|
m = re.match(r"^(\d+)\.\s+(.+)$", lines[i].strip())
|
||||||
blocks.append(Block(kind="bullet", content=f"{idx}. {item}"))
|
if not m:
|
||||||
|
break
|
||||||
|
blocks.append(Block(kind="ordered", content=f"{m.group(1)}. {m.group(2)}"))
|
||||||
i += 1
|
i += 1
|
||||||
idx += 1
|
|
||||||
continue
|
continue
|
||||||
|
|
||||||
# Table
|
# Table
|
||||||
@@ -518,6 +585,7 @@ def parse_markdown(md_text: str) -> List[Block]:
|
|||||||
while i < len(lines) and lines[i].strip() and not (
|
while i < len(lines) and lines[i].strip() and not (
|
||||||
lines[i].strip().startswith(("#", ">", "-", "*", "+", "!"))
|
lines[i].strip().startswith(("#", ">", "-", "*", "+", "!"))
|
||||||
or re.match(r"^\d+\.\s+", lines[i].strip())
|
or re.match(r"^\d+\.\s+", lines[i].strip())
|
||||||
|
or re.match(r"^\[\^[A-Za-z0-9_-]+\]:", lines[i].strip())
|
||||||
or "|" in lines[i]
|
or "|" in lines[i]
|
||||||
):
|
):
|
||||||
para_lines.append(lines[i])
|
para_lines.append(lines[i])
|
||||||
@@ -638,6 +706,17 @@ def md_inline_to_rl(text: str, *, add_cjk_space: bool = True) -> str:
|
|||||||
+ ']</font></super>',
|
+ ']</font></super>',
|
||||||
text,
|
text,
|
||||||
)
|
)
|
||||||
|
text = re.sub(
|
||||||
|
r"<sup>(.*?)</sup>",
|
||||||
|
r"<super><font size=7>\1</font></super>",
|
||||||
|
text,
|
||||||
|
flags=re.IGNORECASE,
|
||||||
|
)
|
||||||
|
text = re.sub(
|
||||||
|
r"\[\^([A-Za-z0-9_-]+)\]",
|
||||||
|
lambda m: f"<super><font size=7>注{m.group(1)}</font></super>",
|
||||||
|
text,
|
||||||
|
)
|
||||||
text = re.sub(r"\[([^\]]+)\]\(([^)]+)\)", r"\1", text)
|
text = re.sub(r"\[([^\]]+)\]\(([^)]+)\)", r"\1", text)
|
||||||
return text
|
return text
|
||||||
|
|
||||||
@@ -746,9 +825,27 @@ def make_page_decorator(manifest: Manifest):
|
|||||||
canvas.setLineWidth(0.5)
|
canvas.setLineWidth(0.5)
|
||||||
canvas.line(2 * cm, A4[1] - 1.4 * cm, A4[0] - 2 * cm, A4[1] - 1.4 * cm)
|
canvas.line(2 * cm, A4[1] - 1.4 * cm, A4[0] - 2 * cm, A4[1] - 1.4 * cm)
|
||||||
|
|
||||||
|
# Page-bottom evidence footnotes.
|
||||||
|
notes = getattr(canvas, "_dr_footnotes", {}).get(doc.page, [])
|
||||||
|
if notes:
|
||||||
|
width = A4[0] - 4.4 * cm
|
||||||
|
x = 2.2 * cm
|
||||||
|
y = 3.45 * cm
|
||||||
|
canvas.setStrokeColor(colors.HexColor("#cbd5e1"))
|
||||||
|
canvas.setLineWidth(0.45)
|
||||||
|
canvas.line(x, y + 0.16 * cm, x + 6.8 * cm, y + 0.16 * cm)
|
||||||
|
footnote_style = build_styles()["evidence-footnote"]
|
||||||
|
for label, content in notes:
|
||||||
|
text = f"注{label}:{content}"
|
||||||
|
para = Paragraph(md_inline_to_rl(text), footnote_style)
|
||||||
|
_, h = para.wrap(width, 1.3 * cm)
|
||||||
|
y -= h
|
||||||
|
para.drawOn(canvas, x, y)
|
||||||
|
y -= 0.04 * cm
|
||||||
|
|
||||||
# Footer: page number centered
|
# Footer: page number centered
|
||||||
canvas.setFont("SrcSans-Light", 8)
|
canvas.setFont("SrcSans-Light", 8)
|
||||||
canvas.drawCentredString(A4[0] / 2, 1.2 * cm, f"— {doc.page} —")
|
canvas.drawCentredString(A4[0] / 2, 1.0 * cm, f"— {doc.page} —")
|
||||||
|
|
||||||
canvas.restoreState()
|
canvas.restoreState()
|
||||||
|
|
||||||
@@ -830,6 +927,8 @@ def collect_toc_entries(blocks: List[Block]) -> List[tuple[int, str]]:
|
|||||||
title = b.content.strip()
|
title = b.content.strip()
|
||||||
if any(s in title.lower() for s in skip_titles_substr):
|
if any(s in title.lower() for s in skip_titles_substr):
|
||||||
continue
|
continue
|
||||||
|
if b.kind == "h1" and not re.match(r"^第\s*\d+\s*章\b", title):
|
||||||
|
continue
|
||||||
level = 1 if b.kind == "h1" else 2
|
level = 1 if b.kind == "h1" else 2
|
||||||
entries.append((level, title))
|
entries.append((level, title))
|
||||||
return entries
|
return entries
|
||||||
@@ -1196,8 +1295,13 @@ def _render_generic_block(block: Block, story: list, base_dir: Path, styles: Sty
|
|||||||
story.append(Paragraph(md_inline_to_rl(block.content), style))
|
story.append(Paragraph(md_inline_to_rl(block.content), style))
|
||||||
elif block.kind == "quote":
|
elif block.kind == "quote":
|
||||||
story.append(Paragraph(md_inline_to_rl(block.content), styles["quote"]))
|
story.append(Paragraph(md_inline_to_rl(block.content), styles["quote"]))
|
||||||
|
elif block.kind == "footnote":
|
||||||
|
label = block.meta.get("label") if block.meta else ""
|
||||||
|
story.append(FootnoteFlowable(str(label), block.content))
|
||||||
elif block.kind == "bullet":
|
elif block.kind == "bullet":
|
||||||
story.append(Paragraph("• " + md_inline_to_rl(block.content), styles["bullet"]))
|
story.append(Paragraph("• " + md_inline_to_rl(block.content), styles["bullet"]))
|
||||||
|
elif block.kind == "ordered":
|
||||||
|
story.append(Paragraph(md_inline_to_rl(block.content), styles["ordered"]))
|
||||||
elif block.kind == "hr":
|
elif block.kind == "hr":
|
||||||
story.append(Spacer(1, 0.3 * cm))
|
story.append(Spacer(1, 0.3 * cm))
|
||||||
elif block.kind == "image":
|
elif block.kind == "image":
|
||||||
@@ -1349,6 +1453,11 @@ def build_body(
|
|||||||
front_sections[kind] = sec
|
front_sections[kind] = sec
|
||||||
k = next_k
|
k = next_k
|
||||||
|
|
||||||
|
# If the markdown does not contain a TOC marker, still insert a generated TOC.
|
||||||
|
# This keeps PDF output stable when Phase 4 emits a clean markdown body.
|
||||||
|
if "toc" not in front_sections:
|
||||||
|
front_sections["toc"] = [Block(kind="h2", content="目录")]
|
||||||
|
|
||||||
# 前置件输出顺序(固定)
|
# 前置件输出顺序(固定)
|
||||||
front_order = [
|
front_order = [
|
||||||
"disclaimer", # 免责声明
|
"disclaimer", # 免责声明
|
||||||
@@ -1408,13 +1517,26 @@ def build_body(
|
|||||||
i = _skip_until_next_section(i + 1)
|
i = _skip_until_next_section(i + 1)
|
||||||
continue
|
continue
|
||||||
|
|
||||||
# 参考文献:自动生成
|
# 参考文献:如果正文仍使用 [src_xxx],则从 sources.jsonl 自动生成;
|
||||||
|
# 如果正文已被 number_citations.py 转为数字编号,则保留 Markdown 内的编号清单。
|
||||||
if kind == "references":
|
if kind == "references":
|
||||||
story.append(PageBreak())
|
story.append(PageBreak())
|
||||||
story.extend(build_references(blocks, sources_path, styles))
|
|
||||||
j = i + 1
|
j = i + 1
|
||||||
while j < n and blocks[j].kind == "p" and _REF_PLACEHOLDER_RE.search(blocks[j].content):
|
has_src_citations = bool(collect_cited_src_ids(blocks))
|
||||||
|
has_ref_placeholder = (
|
||||||
|
j < n and blocks[j].kind == "p" and _REF_PLACEHOLDER_RE.search(blocks[j].content)
|
||||||
|
)
|
||||||
|
if has_src_citations or has_ref_placeholder:
|
||||||
|
story.extend(build_references(blocks, sources_path, styles))
|
||||||
|
while j < n and blocks[j].kind not in ("h1", "h2"):
|
||||||
j += 1
|
j += 1
|
||||||
|
else:
|
||||||
|
story.append(Paragraph(md_inline_to_rl(block.content), styles["h1"]))
|
||||||
|
while j < n and blocks[j].kind not in ("h1", "h2"):
|
||||||
|
_render_generic_block(blocks[j], story, base_dir, styles, in_summary=False)
|
||||||
|
j += 1
|
||||||
|
i = j
|
||||||
|
continue
|
||||||
i = j
|
i = j
|
||||||
continue
|
continue
|
||||||
|
|
||||||
@@ -1497,7 +1619,7 @@ def main():
|
|||||||
leftMargin=2.2 * cm,
|
leftMargin=2.2 * cm,
|
||||||
rightMargin=2.2 * cm,
|
rightMargin=2.2 * cm,
|
||||||
topMargin=2 * cm,
|
topMargin=2 * cm,
|
||||||
bottomMargin=2 * cm,
|
bottomMargin=4.0 * cm,
|
||||||
title=manifest.report_title,
|
title=manifest.report_title,
|
||||||
author=manifest.author,
|
author=manifest.author,
|
||||||
subject=manifest.type,
|
subject=manifest.type,
|
||||||
@@ -1510,14 +1632,14 @@ def main():
|
|||||||
id="cover",
|
id="cover",
|
||||||
)
|
)
|
||||||
normal_frame = Frame(
|
normal_frame = Frame(
|
||||||
2.2 * cm, 2 * cm,
|
2.2 * cm, 4.0 * cm,
|
||||||
A4[0] - 4.4 * cm, A4[1] - 4 * cm,
|
A4[0] - 4.4 * cm, A4[1] - 6.0 * cm,
|
||||||
id="normal",
|
id="normal",
|
||||||
)
|
)
|
||||||
decorator = make_page_decorator(manifest)
|
decorator = make_page_decorator(manifest)
|
||||||
doc.addPageTemplates([
|
doc.addPageTemplates([
|
||||||
PageTemplate(id="cover", frames=[cover_frame]),
|
PageTemplate(id="cover", frames=[cover_frame]),
|
||||||
PageTemplate(id="normal", frames=[normal_frame], onPage=decorator),
|
PageTemplate(id="normal", frames=[normal_frame], onPageEnd=decorator),
|
||||||
])
|
])
|
||||||
|
|
||||||
# Assemble story
|
# Assemble story
|
||||||
|
|||||||
@@ -193,6 +193,7 @@ uv run python scripts/dr.py methods list
|
|||||||
- 真实并发由 `scripts/runtime/workers.py` 的 worker pool 执行。
|
- 真实并发由 `scripts/runtime/workers.py` 的 worker pool 执行。
|
||||||
- 真实模型选择由 `configs/models.yaml` 和 `scripts/runtime/roles.py` 执行。
|
- 真实模型选择由 `configs/models.yaml` 和 `scripts/runtime/roles.py` 执行。
|
||||||
- 信息检索默认走 `scripts/search.py` / `SearchClient` / `search-gateway` skill;不得把 Tavily MCP、browser MCP 或平台 web search 作为默认路径,除非用户明确授权。
|
- 信息检索默认走 `scripts/search.py` / `SearchClient` / `search-gateway` skill;不得把 Tavily MCP、browser MCP 或平台 web search 作为默认路径,除非用户明确授权。
|
||||||
|
- 搜索路由必须按任务类型选择:`evidence`=Exa highlights 受控证据发现,`fda/scholar/patents/news`=专用信源路径,`general`=宽泛发现和兜底;Tavily Research 只能作为阶段性 scan/enrichment/rework 输入,不能直接替代 evidence packet 或章节正文。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@@ -701,3 +701,17 @@ OpenCode 的坑:如果只是在主会话里装样子地写"让 X agent 做",
|
|||||||
- `phase2/enrichment_rounds/roundXX/coverage_gap.json` 还未实现;下一步应先做 deterministic coverage evaluator,再让补充 task cards 从 gap 生成。
|
- `phase2/enrichment_rounds/roundXX/coverage_gap.json` 还未实现;下一步应先做 deterministic coverage evaluator,再让补充 task cards 从 gap 生成。
|
||||||
- Phase3 evaluator rubrics 仍是计划项;当前 deterministic review 已能抓部分 draft 质量问题,但还没有分维度评分与 finalize gate。
|
- Phase3 evaluator rubrics 仍是计划项;当前 deterministic review 已能抓部分 draft 质量问题,但还没有分维度评分与 finalize gate。
|
||||||
- DOCX/PPTX/图片批量 OCR、表格抽取、材料 source registry 仍放入后续资料导入增强。
|
- DOCX/PPTX/图片批量 OCR、表格抽取、材料 source registry 仍放入后续资料导入增强。
|
||||||
|
|
||||||
|
- 2026-05-07 v0.20/v0.21-alpha search routing refinement:**Exa evidence discovery + Tavily Research 边界定锚**
|
||||||
|
|
||||||
|
**设计结论**:
|
||||||
|
- Exa 更适合作为 Phase2 的受控 evidence discovery:优先返回 highlights/text,便于进入 source-quality、evidence-table 和 packet schema。
|
||||||
|
- Tavily Research 更适合作为 Phase1 初扫、薄弱章节补证据、Phase3 回炉扫描;其综合报告不得直接替代 evidence packet 或章节正文。
|
||||||
|
- Serper 继续承担 Scholar、Google Patents、News 与 Google-specific `site:` 检索;Brave 用于交叉验证和混合语种 fallback。
|
||||||
|
|
||||||
|
**已落地**:
|
||||||
|
- `scripts/search.py` 新增 `--route evidence` 与 `--exa-category`,profile 路由加入 `evidence`。
|
||||||
|
- `scripts/lib/search_client.py` 新增 `SearchClient.evidence()`,优先调用 Exa highlights/text,失败后降级 Tavily/Brave。
|
||||||
|
- `scripts/runtime/tasks.py` 把 `evidence` 纳入合法 search route,并更新主要 task axes 的默认路由。
|
||||||
|
- `scripts/runtime/workers.py` 的 `ProjectSearchProvider` 支持 `evidence` route。
|
||||||
|
- `skills/search-gateway`、`skills/search-strategy`、`docs/search-playbook.md`、`README.md`、`AGENTS.md` 同步记录搜索分工,避免后续又回到 Tavily MCP 或中文长句搜索。
|
||||||
|
|||||||
@@ -173,9 +173,12 @@ uv run python scripts/sprint5_regression.py <slug>
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
uv run python scripts/search.py "dual-target RNAi 2024" --route scholar --year-low 2023
|
uv run python scripts/search.py "dual-target RNAi 2024" --route scholar --year-low 2023
|
||||||
|
uv run python scripts/search.py "FDA warning letter CAPA deviation change control pharmaceutical" --route evidence --json --trace
|
||||||
uv run python scripts/search.py "dual-target siRNA GalNAc" --route patents
|
uv run python scripts/search.py "dual-target siRNA GalNAc" --route patents
|
||||||
```
|
```
|
||||||
|
|
||||||
|
v0.20 搜索分工:`evidence` 用 Exa highlights 做受控候选证据发现;`scholar/patents/news/fda` 保留专用路由;`general` 只做宽泛发现和兜底;Tavily Research 作为 Phase1 初扫、薄弱章节补证据和 Phase3 回炉工具,结果必须存盘、评分、去重后再进入 evidence packet。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 项目结构
|
## 项目结构
|
||||||
|
|||||||
+1
-1
@@ -78,7 +78,7 @@ profiles:
|
|||||||
polish: anthropic/claude-sonnet-4.6
|
polish: anthropic/claude-sonnet-4.6
|
||||||
|
|
||||||
codex_native:
|
codex_native:
|
||||||
description: OpenAI-native profile for Codex adapter runs.
|
description: Deprecated/misleading name. These are OpenAI models through the external Python API client, not Codex App built-in models.
|
||||||
roles:
|
roles:
|
||||||
dr_plan: gpt-5.4
|
dr_plan: gpt-5.4
|
||||||
dr_pm: gpt-5.4
|
dr_pm: gpt-5.4
|
||||||
|
|||||||
@@ -16,6 +16,12 @@ methods:
|
|||||||
- patents
|
- patents
|
||||||
- market
|
- market
|
||||||
- counter
|
- counter
|
||||||
|
integrated_lanes:
|
||||||
|
- literature evidence
|
||||||
|
- regulatory pathway
|
||||||
|
- patent/IP position
|
||||||
|
- market and competitor evidence
|
||||||
|
- counter-evidence and uncertainty
|
||||||
framework_sections:
|
framework_sections:
|
||||||
- central_thesis
|
- central_thesis
|
||||||
- chapter_outline
|
- chapter_outline
|
||||||
@@ -124,6 +130,13 @@ methods:
|
|||||||
- capa_roadmap
|
- capa_roadmap
|
||||||
- verification_evidence
|
- verification_evidence
|
||||||
- counter
|
- counter
|
||||||
|
integrated_lanes:
|
||||||
|
- site audit and recap findings
|
||||||
|
- official regulatory and guideline baseline
|
||||||
|
- enforcement precedents and warning letters
|
||||||
|
- quality/manufacturing/operations gap analysis
|
||||||
|
- remediation actions, ownership, verification evidence
|
||||||
|
- counter-evidence and boundary conditions
|
||||||
framework_sections:
|
framework_sections:
|
||||||
- material_evidence_map
|
- material_evidence_map
|
||||||
- regulatory_and_best_practice_baseline
|
- regulatory_and_best_practice_baseline
|
||||||
|
|||||||
@@ -6,14 +6,16 @@ profiles:
|
|||||||
- "clinicaltrials"
|
- "clinicaltrials"
|
||||||
- "fda_ema_nmpa"
|
- "fda_ema_nmpa"
|
||||||
- "serper_scholar"
|
- "serper_scholar"
|
||||||
- "tavily_exa_gap_fill"
|
- "exa_evidence_discovery"
|
||||||
|
- "tavily_research_scan_if_needed"
|
||||||
patent_heavy:
|
patent_heavy:
|
||||||
description: "IP landscape, freedom-to-operate, and process-route research."
|
description: "IP landscape, freedom-to-operate, and process-route research."
|
||||||
order:
|
order:
|
||||||
- "google_patents_via_serper"
|
- "google_patents_via_serper"
|
||||||
- "uspto_epo_cnipa"
|
- "uspto_epo_cnipa"
|
||||||
- "company_disclosures"
|
- "company_disclosures"
|
||||||
- "exa_tavily_family_discovery"
|
- "exa_family_discovery"
|
||||||
|
- "tavily_gap_fill"
|
||||||
china_market:
|
china_market:
|
||||||
description: "China regulatory, company, supply-chain, and market research."
|
description: "China regulatory, company, supply-chain, and market research."
|
||||||
order:
|
order:
|
||||||
@@ -21,6 +23,7 @@ profiles:
|
|||||||
- "exchange_disclosures"
|
- "exchange_disclosures"
|
||||||
- "serper_brave_chinese"
|
- "serper_brave_chinese"
|
||||||
- "exa_company_pages"
|
- "exa_company_pages"
|
||||||
|
- "exa_evidence_discovery"
|
||||||
- "tavily_gap_fill"
|
- "tavily_gap_fill"
|
||||||
investment:
|
investment:
|
||||||
description: "Public-company, market-size, and transaction-oriented research."
|
description: "Public-company, market-size, and transaction-oriented research."
|
||||||
@@ -29,15 +32,16 @@ profiles:
|
|||||||
- "consulting_and_database_reports"
|
- "consulting_and_database_reports"
|
||||||
- "company_announcements"
|
- "company_announcements"
|
||||||
- "serper_news"
|
- "serper_news"
|
||||||
|
- "exa_evidence_discovery"
|
||||||
- "tavily_gap_fill"
|
- "tavily_gap_fill"
|
||||||
|
|
||||||
apis:
|
apis:
|
||||||
tavily:
|
tavily:
|
||||||
best_for: ["initial_scan", "gap_fill", "llm_friendly_snippets"]
|
best_for: ["phase1_research_scan", "phase3_gap_fill", "llm_friendly_snippets"]
|
||||||
evidence_role: "discovery_only_unless_original_source"
|
evidence_role: "discovery_only_unless_original_source"
|
||||||
exa:
|
exa:
|
||||||
best_for: ["company_pages", "terminology_check", "long_tail_professional_pages"]
|
best_for: ["evidence_discovery", "company_pages", "terminology_check", "long_tail_professional_pages", "agent_highlights"]
|
||||||
evidence_role: "discovery_or_secondary"
|
evidence_role: "candidate_source_until_scored"
|
||||||
brave:
|
brave:
|
||||||
best_for: ["cross_check", "counter_evidence", "mixed_language_search"]
|
best_for: ["cross_check", "counter_evidence", "mixed_language_search"]
|
||||||
evidence_role: "discovery_only_unless_original_source"
|
evidence_role: "discovery_only_unless_original_source"
|
||||||
|
|||||||
+17
-6
@@ -8,13 +8,14 @@ v0.12 起,默认搜索路径收敛到项目内 Python 网关:
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
uv run python scripts/search.py "<query>" --route scholar --num-results 10 --year-low 2023
|
uv run python scripts/search.py "<query>" --route scholar --num-results 10 --year-low 2023
|
||||||
|
uv run python scripts/search.py "<query>" --route evidence --num-results 10 --json --trace
|
||||||
uv run python scripts/search.py "<query>" --route patents --num-results 10
|
uv run python scripts/search.py "<query>" --route patents --num-results 10
|
||||||
uv run python scripts/search.py "<query>" --route news --num-results 10 --time-range m
|
uv run python scripts/search.py "<query>" --route news --num-results 10 --time-range m
|
||||||
uv run python scripts/search.py "<query>" --route general --num-results 10
|
uv run python scripts/search.py "<query>" --route general --num-results 10
|
||||||
uv run python scripts/ground.py "<query>" --json
|
uv run python scripts/ground.py "<query>" --json
|
||||||
```
|
```
|
||||||
|
|
||||||
其中 `scholar / patents / news` 默认走严格模式(Serper 失败不静默降级);需要容错时显式加 `--no-strict-specialized`。
|
其中 `scholar / patents / news` 默认走严格模式(Serper 失败不静默降级);需要容错时显式加 `--no-strict-specialized`。`evidence` 是 v0.20.1 之后新增的受控证据发现路由,优先用 Exa highlights/text 为 evidence packet 提供候选来源。
|
||||||
|
|
||||||
MCP server 只作为交互式补漏和特殊工具能力,不作为文献、专利、新闻检索主路径。这样 OpenCode、Codex、Gemini CLI、Claude Code 都能复用同一套路由,减少每个平台单独配置 Tavily/Exa/Brave MCP 的依赖。
|
MCP server 只作为交互式补漏和特殊工具能力,不作为文献、专利、新闻检索主路径。这样 OpenCode、Codex、Gemini CLI、Claude Code 都能复用同一套路由,减少每个平台单独配置 Tavily/Exa/Brave MCP 的依赖。
|
||||||
|
|
||||||
@@ -23,13 +24,14 @@ MCP server 只作为交互式补漏和特殊工具能力,不作为文献、专
|
|||||||
### Tavily
|
### Tavily
|
||||||
|
|
||||||
- 优点:LLM 友好,摘要质量稳定,适合快速发现方向。
|
- 优点:LLM 友好,摘要质量稳定,适合快速发现方向。
|
||||||
- 用法:初扫、普通网页、报告线索、交叉补漏。
|
- 用法:初扫、普通网页、报告线索、交叉补漏;`research()` 更适合 Phase 1 初步扫描、薄弱章节补证据、Phase 3 回炉。
|
||||||
- 风险:不能把普通网页当结论支撑,必须追溯原始来源。
|
- 风险:不能把普通网页当结论支撑,必须追溯原始来源。
|
||||||
|
- 规则:Tavily Research 输出必须保存为过程文件,并经过 source-quality 评分、去重和 source_id 归一化;不要直接把 Tavily 的综合报告当作章节正文或最终证据。
|
||||||
|
|
||||||
### Exa
|
### Exa
|
||||||
|
|
||||||
- 优点:neural search,对官网、公司页、长尾专业内容召回好。
|
- 优点:neural/agent search,对官网、公司页、长尾专业内容召回好;highlights/text 适合喂给 agent 做证据筛选。
|
||||||
- 用法:术语核查、公司/产品名纠错、专业网页发现。
|
- 用法:`scripts/search.py --route evidence`、术语核查、公司/产品名纠错、专业网页发现、章节证据补强。
|
||||||
- 风险:macOS 代理环境容易 TLS EOF,项目内 `SearchClient` 已使用 `trust_env=False` 绕开系统代理。
|
- 风险:macOS 代理环境容易 TLS EOF,项目内 `SearchClient` 已使用 `trust_env=False` 绕开系统代理。
|
||||||
|
|
||||||
### Brave
|
### Brave
|
||||||
@@ -71,13 +73,22 @@ MCP server 只作为交互式补漏和特殊工具能力,不作为文献、专
|
|||||||
|
|
||||||
## Recommended Profiles
|
## Recommended Profiles
|
||||||
|
|
||||||
|
## v0.20 Routing Decision
|
||||||
|
|
||||||
|
- Phase 1 初步扫描:Tavily Research + Exa evidence,目标是形成假设、反证方向、章节任务切分。
|
||||||
|
- Phase 2 evidence packet:优先 `fda/scholar/patents/news` 等专用路由;需要补充候选证据时用 `evidence`,不要只用 `general`。
|
||||||
|
- Phase 3 回炉:按 critique 中的证据缺口定向调用 Tavily Research 或 Exa evidence,输出仍需进入 packet/schema。
|
||||||
|
- General route:只做宽泛发现和兜底,不作为“默认最佳搜索”。
|
||||||
|
|
||||||
|
## Recommended Profiles
|
||||||
|
|
||||||
### biomed_literature
|
### biomed_literature
|
||||||
|
|
||||||
PubMed / NCBI → ClinicalTrials → FDA/EMA/NMPA → `scripts/search.py --route scholar` → Tavily/Exa 补漏。
|
PubMed / NCBI → ClinicalTrials → FDA/EMA/NMPA → `scripts/search.py --route scholar` → `scripts/search.py --route evidence` → Tavily/Brave 补漏。
|
||||||
|
|
||||||
### patent_heavy
|
### patent_heavy
|
||||||
|
|
||||||
`scripts/search.py --route patents` → USPTO/EPO/CNIPA → 公司年报/招股书 → Tavily/Exa 补同族专利线索。
|
`scripts/search.py --route patents` → USPTO/EPO/CNIPA → 公司年报/招股书 → Exa/Tavily 补同族专利线索。
|
||||||
|
|
||||||
### china_market
|
### china_market
|
||||||
|
|
||||||
|
|||||||
@@ -6,7 +6,7 @@
|
|||||||
- 可选:--extra terms.txt(每行一个英文术语,补充进来一起核查)
|
- 可选:--extra terms.txt(每行一个英文术语,补充进来一起核查)
|
||||||
|
|
||||||
流程(每个术语独立可并行):
|
流程(每个术语独立可并行):
|
||||||
1. 用 SearchClient(Exa > Tavily)搜一次(query = "<term> <domain hint>")
|
1. 用 SearchClient(Tavily > Exa > Brave)搜一次(query = "<term> <domain hint>")
|
||||||
2. 把 top 3-5 snippet 喂给 Haiku,让模型返回 {zh, en_full, confidence, issue}
|
2. 把 top 3-5 snippet 喂给 Haiku,让模型返回 {zh, en_full, confidence, issue}
|
||||||
3. 合并回 glossary,字段扩展:
|
3. 合并回 glossary,字段扩展:
|
||||||
{
|
{
|
||||||
|
|||||||
+32
-1
@@ -243,6 +243,14 @@ def prepare_qmd(
|
|||||||
if end != -1:
|
if end != -1:
|
||||||
md_text = md_text[end + 4:].lstrip("\n")
|
md_text = md_text[end + 4:].lstrip("\n")
|
||||||
|
|
||||||
|
# Quarto already renders the title from YAML; drop a duplicated leading H1.
|
||||||
|
md_text = re.sub(
|
||||||
|
rf"^#\s+{re.escape(title)}\s*\n+",
|
||||||
|
"",
|
||||||
|
md_text,
|
||||||
|
count=1,
|
||||||
|
)
|
||||||
|
|
||||||
# Replace TOC placeholder
|
# Replace TOC placeholder
|
||||||
md_text = re.sub(
|
md_text = re.sub(
|
||||||
r"\[TOC will be generated.*?\]",
|
r"\[TOC will be generated.*?\]",
|
||||||
@@ -276,13 +284,36 @@ def prepare_qmd(
|
|||||||
"\\usepackage{longtable}\n"
|
"\\usepackage{longtable}\n"
|
||||||
"\\usepackage{booktabs}\n"
|
"\\usepackage{booktabs}\n"
|
||||||
"\\usepackage{array}\n"
|
"\\usepackage{array}\n"
|
||||||
|
"\\usepackage{xcolor}\n"
|
||||||
|
"\\usepackage{titlesec}\n"
|
||||||
|
"\\definecolor{DRBlue}{HTML}{1E3A8A}\n"
|
||||||
|
"\\definecolor{DRSlate}{HTML}{374151}\n"
|
||||||
|
"\\definecolor{DRMuted}{HTML}{6B7280}\n"
|
||||||
# Use lscape instead of pdflscape to avoid \LS@makefcolumn recursion
|
# Use lscape instead of pdflscape to avoid \LS@makefcolumn recursion
|
||||||
# which exhausts TeX param_size on large longtables.
|
# which exhausts TeX param_size on large longtables.
|
||||||
# lscape rotates content without changing page media box (reader must rotate).
|
# lscape rotates content without changing page media box (reader must rotate).
|
||||||
"\\usepackage{lscape}\n"
|
"\\usepackage{lscape}\n"
|
||||||
"\\setlength{\\LTpre}{6pt}\n"
|
"\\setlength{\\LTpre}{6pt}\n"
|
||||||
"\\setlength{\\LTpost}{6pt}\n"
|
"\\setlength{\\LTpost}{6pt}\n"
|
||||||
"\\setlength{\\tabcolsep}{3pt}\n",
|
"\\setlength{\\tabcolsep}{3pt}\n"
|
||||||
|
"\\linespread{1.18}\n"
|
||||||
|
"\\setlength{\\parindent}{2em}\n"
|
||||||
|
"\\setlength{\\parskip}{0.25em}\n"
|
||||||
|
"\\newcommand{\\sectionbreak}{\\clearpage}\n"
|
||||||
|
"\\titleformat{\\section}[display]\n"
|
||||||
|
" {\\centering\\Large\\bfseries\\sffamily\\color{DRBlue}}\n"
|
||||||
|
" {}{0pt}{}\n"
|
||||||
|
"\\titlespacing*{\\section}{0pt}{0pt}{1.1em}\n"
|
||||||
|
"\\titleformat{\\subsection}\n"
|
||||||
|
" {\\large\\bfseries\\sffamily\\color{DRBlue}}\n"
|
||||||
|
" {}{0pt}{}\n"
|
||||||
|
"\\titlespacing*{\\subsection}{0pt}{1.1em}{0.45em}\n"
|
||||||
|
"\\titleformat{\\subsubsection}\n"
|
||||||
|
" {\\normalsize\\bfseries\\sffamily\\color{DRSlate}}\n"
|
||||||
|
" {}{0pt}{}\n"
|
||||||
|
"\\titlespacing*{\\subsubsection}{0pt}{0.9em}{0.35em}\n"
|
||||||
|
"\\renewcommand{\\contentsname}{目录}\n"
|
||||||
|
"\\setcounter{tocdepth}{2}\n",
|
||||||
encoding="utf-8",
|
encoding="utf-8",
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|||||||
+165
-10
@@ -30,11 +30,12 @@ from scripts.runtime.assembly import build_chapter_briefs, build_compressed_find
|
|||||||
from scripts.runtime.orchestrator import create_phase2_task_cards, write_placeholder_packets
|
from scripts.runtime.orchestrator import create_phase2_task_cards, write_placeholder_packets
|
||||||
from scripts.runtime.methods import ResearchMethodRegistry
|
from scripts.runtime.methods import ResearchMethodRegistry
|
||||||
from scripts.runtime.phase1 import create_project, render_framework, write_material_brief
|
from scripts.runtime.phase1 import create_project, render_framework, write_material_brief
|
||||||
from scripts.runtime.review import build_phase3_critique
|
from scripts.runtime.review import build_phase3_critique, build_phase3_model_critique
|
||||||
from scripts.runtime.roles import resolve_runtime_profile
|
from scripts.runtime.roles import resolve_runtime_profile
|
||||||
|
from scripts.runtime.source_cache import cache_sources
|
||||||
from scripts.runtime.sources import rebuild_sources_from_packets
|
from scripts.runtime.sources import rebuild_sources_from_packets
|
||||||
from scripts.runtime.skills import SkillRegistry, default_adapter_skill_dirs
|
from scripts.runtime.skills import SkillRegistry, default_adapter_skill_dirs
|
||||||
from scripts.runtime.tasks import TaskCard
|
from scripts.runtime.tasks import TaskCard, load_task_cards, validate_packet, write_task_cards
|
||||||
from scripts.runtime.workers import run_packet_workers
|
from scripts.runtime.workers import run_packet_workers
|
||||||
|
|
||||||
|
|
||||||
@@ -156,7 +157,12 @@ def cmd_frame(args: argparse.Namespace) -> int:
|
|||||||
print(f"Research method: {method.key}")
|
print(f"Research method: {method.key}")
|
||||||
print(f"Chapters: {args.chapters}")
|
print(f"Chapters: {args.chapters}")
|
||||||
return 0
|
return 0
|
||||||
path = render_framework(project_root, method_key=args.method, chapter_count=args.chapters)
|
path = render_framework(
|
||||||
|
project_root,
|
||||||
|
method_key=args.method,
|
||||||
|
chapter_count=args.chapters,
|
||||||
|
preserve_existing_outline=args.preserve_existing_outline,
|
||||||
|
)
|
||||||
print(f"Project: {project_root.name}")
|
print(f"Project: {project_root.name}")
|
||||||
print(f"Wrote: {path.relative_to(project_root)}")
|
print(f"Wrote: {path.relative_to(project_root)}")
|
||||||
print("Pause: review and approve the framework before Phase 2.")
|
print("Pause: review and approve the framework before Phase 2.")
|
||||||
@@ -222,7 +228,57 @@ def cmd_research(args: argparse.Namespace) -> int:
|
|||||||
"Phase 1 is not approved. Review phase1/material_brief.md and phase1/framework.md, "
|
"Phase 1 is not approved. Review phase1/material_brief.md and phase1/framework.md, "
|
||||||
"then run `uv run python scripts/dr.py approve <project>` or pass --force."
|
"then run `uv run python scripts/dr.py approve <project>` or pass --force."
|
||||||
)
|
)
|
||||||
runtime = resolve_runtime_profile(profile=args.profile)
|
if args.profile == "codex_native" and (args.execute_packets or args.assemble_chapters):
|
||||||
|
raise SystemExit(
|
||||||
|
"`codex_native` cannot be used for Python-core model execution: scripts/dr.py currently calls external "
|
||||||
|
"API clients, not Codex App built-in models. Use a clearly external profile such as `medium`, or run "
|
||||||
|
"Codex-native execution through the surface adapter/manual task workflow."
|
||||||
|
)
|
||||||
|
runtime = resolve_runtime_profile(
|
||||||
|
profile=args.profile,
|
||||||
|
overrides=parse_model_overrides(args.model_override),
|
||||||
|
)
|
||||||
|
if args.append_task_cards:
|
||||||
|
generated = create_phase2_task_cards(
|
||||||
|
project_root,
|
||||||
|
axes=args.axis,
|
||||||
|
dry_run=True,
|
||||||
|
)
|
||||||
|
existing_path = project_root / "phase2" / "task_cards.json"
|
||||||
|
existing_cards = load_task_cards(existing_path) if existing_path.exists() else []
|
||||||
|
seen = {card.task_id for card in existing_cards}
|
||||||
|
appended_cards = [TaskCard(**item) for item in generated if item["task_id"] not in seen]
|
||||||
|
runnable_existing_cards: list[TaskCard] = []
|
||||||
|
if args.execute_packets and args.axis:
|
||||||
|
axis_set = set(args.axis)
|
||||||
|
for card in existing_cards:
|
||||||
|
if card.topic_axis not in axis_set:
|
||||||
|
continue
|
||||||
|
packet_path = project_root / card.output_packet
|
||||||
|
try:
|
||||||
|
validate_packet(json.loads(packet_path.read_text(encoding="utf-8")))
|
||||||
|
except Exception:
|
||||||
|
runnable_existing_cards.append(card)
|
||||||
|
merged_cards = [*existing_cards, *appended_cards]
|
||||||
|
if not args.dry_run:
|
||||||
|
write_task_cards(existing_path, merged_cards)
|
||||||
|
phase2 = manifest.setdefault("phase2", {})
|
||||||
|
phase2.update(
|
||||||
|
{
|
||||||
|
"status": "in_progress",
|
||||||
|
"runtime": "python-core-v0.20",
|
||||||
|
"task_cards_path": "phase2/task_cards.json",
|
||||||
|
"task_cards_total": len(merged_cards),
|
||||||
|
"task_cards_appended": len(appended_cards),
|
||||||
|
"updated_at": datetime.now(timezone.utc).replace(microsecond=0).isoformat(),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
(project_root / "manifest.json").write_text(
|
||||||
|
json.dumps(manifest, ensure_ascii=False, indent=2) + "\n",
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
card_dicts = [card.to_dict() for card in [*appended_cards, *runnable_existing_cards]]
|
||||||
|
else:
|
||||||
card_dicts = create_phase2_task_cards(
|
card_dicts = create_phase2_task_cards(
|
||||||
project_root,
|
project_root,
|
||||||
axes=args.axis,
|
axes=args.axis,
|
||||||
@@ -350,7 +406,9 @@ def cmd_run(args: argparse.Namespace) -> int:
|
|||||||
workers=args.workers,
|
workers=args.workers,
|
||||||
axis=None,
|
axis=None,
|
||||||
profile=args.profile,
|
profile=args.profile,
|
||||||
|
model_override=[],
|
||||||
execute_packets=False,
|
execute_packets=False,
|
||||||
|
append_task_cards=False,
|
||||||
allow_search_fallback=False,
|
allow_search_fallback=False,
|
||||||
build_briefs=False,
|
build_briefs=False,
|
||||||
assemble_chapters=False,
|
assemble_chapters=False,
|
||||||
@@ -362,6 +420,25 @@ def cmd_run(args: argparse.Namespace) -> int:
|
|||||||
|
|
||||||
def cmd_review(args: argparse.Namespace) -> int:
|
def cmd_review(args: argparse.Namespace) -> int:
|
||||||
project_root = resolve_project(args.project)
|
project_root = resolve_project(args.project)
|
||||||
|
if args.model_review:
|
||||||
|
if args.dry_run:
|
||||||
|
print(f"Project: {project_root.name}")
|
||||||
|
print("Phase 3 model review plan:")
|
||||||
|
print(f" model: {args.model}")
|
||||||
|
print(" context: phase3/review_context_opus_4_7.md")
|
||||||
|
print(" output: phase3/critique.md")
|
||||||
|
return 0
|
||||||
|
from scripts.lib.zenmux_client import ZenMuxClient, load_secrets
|
||||||
|
|
||||||
|
load_secrets()
|
||||||
|
with ZenMuxClient(log_file=project_root / "phase3" / "logs" / "review.jsonl") as client:
|
||||||
|
path = build_phase3_model_critique(
|
||||||
|
project_root,
|
||||||
|
client=client,
|
||||||
|
model=args.model,
|
||||||
|
max_context_chars=args.max_context_chars,
|
||||||
|
)
|
||||||
|
else:
|
||||||
path = build_phase3_critique(project_root)
|
path = build_phase3_critique(project_root)
|
||||||
print(f"Project: {project_root.name}")
|
print(f"Project: {project_root.name}")
|
||||||
print(f"Wrote: {path.relative_to(project_root)}")
|
print(f"Wrote: {path.relative_to(project_root)}")
|
||||||
@@ -482,12 +559,31 @@ def cmd_finalize(args: argparse.Namespace) -> int:
|
|||||||
roles = resolved["roles"]
|
roles = resolved["roles"]
|
||||||
|
|
||||||
if not args.legacy_translate:
|
if not args.legacy_translate:
|
||||||
|
final_input = args.input
|
||||||
|
if args.number_citations and not args.dry_run:
|
||||||
|
rc = run_cmd(
|
||||||
|
[
|
||||||
|
sys.executable,
|
||||||
|
str(REPO_ROOT / "scripts" / "number_citations.py"),
|
||||||
|
str(project_root),
|
||||||
|
"--input",
|
||||||
|
args.input,
|
||||||
|
"--output",
|
||||||
|
args.numbered_output,
|
||||||
|
],
|
||||||
|
dry_run=False,
|
||||||
|
)
|
||||||
|
if rc != 0:
|
||||||
|
return rc
|
||||||
|
final_input = args.numbered_output
|
||||||
|
elif args.number_citations:
|
||||||
|
final_input = args.numbered_output
|
||||||
cmd: list[str] = [
|
cmd: list[str] = [
|
||||||
sys.executable,
|
sys.executable,
|
||||||
str(REPO_ROOT / "scripts" / "build_report.py"),
|
str(REPO_ROOT / "scripts" / "build_report.py"),
|
||||||
str(project_root),
|
str(project_root),
|
||||||
"--input",
|
"--input",
|
||||||
args.input,
|
final_input,
|
||||||
]
|
]
|
||||||
if args.report_engine:
|
if args.report_engine:
|
||||||
cmd += ["--engine", args.report_engine]
|
cmd += ["--engine", args.report_engine]
|
||||||
@@ -497,6 +593,21 @@ def cmd_finalize(args: argparse.Namespace) -> int:
|
|||||||
cmd.append("--no-pdf")
|
cmd.append("--no-pdf")
|
||||||
if args.dry_run:
|
if args.dry_run:
|
||||||
print("Chinese-native finalize plan:")
|
print("Chinese-native finalize plan:")
|
||||||
|
if args.number_citations:
|
||||||
|
print(
|
||||||
|
"$ "
|
||||||
|
+ " ".join(
|
||||||
|
[
|
||||||
|
sys.executable,
|
||||||
|
str(REPO_ROOT / "scripts" / "number_citations.py"),
|
||||||
|
str(project_root),
|
||||||
|
"--input",
|
||||||
|
args.input,
|
||||||
|
"--output",
|
||||||
|
args.numbered_output,
|
||||||
|
]
|
||||||
|
)
|
||||||
|
)
|
||||||
print("$ " + " ".join(cmd))
|
print("$ " + " ".join(cmd))
|
||||||
if args.polish:
|
if args.polish:
|
||||||
print(
|
print(
|
||||||
@@ -506,8 +617,8 @@ def cmd_finalize(args: argparse.Namespace) -> int:
|
|||||||
sys.executable,
|
sys.executable,
|
||||||
str(REPO_ROOT / "scripts" / "polish.py"),
|
str(REPO_ROOT / "scripts" / "polish.py"),
|
||||||
str(project_root),
|
str(project_root),
|
||||||
"--input",
|
"--source",
|
||||||
args.input,
|
final_input,
|
||||||
"--workers",
|
"--workers",
|
||||||
str(args.polish_workers),
|
str(args.polish_workers),
|
||||||
"--model",
|
"--model",
|
||||||
@@ -522,8 +633,8 @@ def cmd_finalize(args: argparse.Namespace) -> int:
|
|||||||
sys.executable,
|
sys.executable,
|
||||||
str(REPO_ROOT / "scripts" / "polish.py"),
|
str(REPO_ROOT / "scripts" / "polish.py"),
|
||||||
str(project_root),
|
str(project_root),
|
||||||
"--input",
|
"--source",
|
||||||
args.input,
|
final_input,
|
||||||
"--workers",
|
"--workers",
|
||||||
str(args.polish_workers),
|
str(args.polish_workers),
|
||||||
"--model",
|
"--model",
|
||||||
@@ -633,6 +744,30 @@ def cmd_models(args: argparse.Namespace) -> int:
|
|||||||
return 0
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_sources(args: argparse.Namespace) -> int:
|
||||||
|
project_root = resolve_project(args.project)
|
||||||
|
if args.sources_cmd == "cache":
|
||||||
|
if args.dry_run:
|
||||||
|
print(f"Project: {project_root.name}")
|
||||||
|
print(f"Would cache sources from: {args.sources}")
|
||||||
|
print(f"Important only: {not args.all}")
|
||||||
|
print(f"Limit: {args.limit}")
|
||||||
|
return 0
|
||||||
|
results = cache_sources(
|
||||||
|
project_root,
|
||||||
|
sources_rel=args.sources,
|
||||||
|
important_only=not args.all,
|
||||||
|
limit=args.limit,
|
||||||
|
force=args.force,
|
||||||
|
)
|
||||||
|
print(f"Project: {project_root.name}")
|
||||||
|
print(f"Cached source snapshots: {len(results)}")
|
||||||
|
print("Wrote: phase2/source_cache/md/*.md")
|
||||||
|
print("Updated: phase2/sources.jsonl")
|
||||||
|
return 0
|
||||||
|
raise SystemExit(f"unknown sources command: {args.sources_cmd}")
|
||||||
|
|
||||||
|
|
||||||
def cmd_apply_models(args: argparse.Namespace) -> int:
|
def cmd_apply_models(args: argparse.Namespace) -> int:
|
||||||
cmd = [
|
cmd = [
|
||||||
sys.executable,
|
sys.executable,
|
||||||
@@ -672,6 +807,7 @@ def build_parser() -> argparse.ArgumentParser:
|
|||||||
frame.add_argument("project", help="Project slug or path")
|
frame.add_argument("project", help="Project slug or path")
|
||||||
frame.add_argument("--method", help="Override research method key")
|
frame.add_argument("--method", help="Override research method key")
|
||||||
frame.add_argument("--chapters", type=int, default=10)
|
frame.add_argument("--chapters", type=int, default=10)
|
||||||
|
frame.add_argument("--preserve-existing-outline", action="store_true", help="Keep current framework chapter titles and enrich Phase 1 planning")
|
||||||
frame.add_argument("--dry-run", action="store_true")
|
frame.add_argument("--dry-run", action="store_true")
|
||||||
frame.set_defaults(func=cmd_frame)
|
frame.set_defaults(func=cmd_frame)
|
||||||
|
|
||||||
@@ -694,7 +830,9 @@ def build_parser() -> argparse.ArgumentParser:
|
|||||||
research.add_argument("--workers", type=int, default=6)
|
research.add_argument("--workers", type=int, default=6)
|
||||||
research.add_argument("--axis", action="append", help="Restrict generated task axes; repeatable")
|
research.add_argument("--axis", action="append", help="Restrict generated task axes; repeatable")
|
||||||
research.add_argument("--profile", help="Model profile name from configs/models.yaml")
|
research.add_argument("--profile", help="Model profile name from configs/models.yaml")
|
||||||
|
research.add_argument("--model-override", action="append", default=[], metavar="ROLE=MODEL", help="Override a role model for this run; repeatable")
|
||||||
research.add_argument("--execute-packets", action="store_true", help="Call model workers to fill evidence packets")
|
research.add_argument("--execute-packets", action="store_true", help="Call model workers to fill evidence packets")
|
||||||
|
research.add_argument("--append-task-cards", action="store_true", help="Append newly generated task cards instead of replacing phase2/task_cards.json")
|
||||||
research.add_argument("--allow-search-fallback", action="store_true", help="Allow generic search fallback for specialized routes")
|
research.add_argument("--allow-search-fallback", action="store_true", help="Allow generic search fallback for specialized routes")
|
||||||
research.add_argument("--build-briefs", action="store_true", help="Aggregate packets into chapter briefs")
|
research.add_argument("--build-briefs", action="store_true", help="Aggregate packets into chapter briefs")
|
||||||
research.add_argument("--assemble-chapters", action="store_true", help="Call model workers to write Chinese chapter drafts")
|
research.add_argument("--assemble-chapters", action="store_true", help="Call model workers to write Chinese chapter drafts")
|
||||||
@@ -721,8 +859,12 @@ def build_parser() -> argparse.ArgumentParser:
|
|||||||
status.add_argument("project", nargs="?", help="Project slug or path")
|
status.add_argument("project", nargs="?", help="Project slug or path")
|
||||||
status.set_defaults(func=cmd_status)
|
status.set_defaults(func=cmd_status)
|
||||||
|
|
||||||
review = sub.add_parser("review", help="Run deterministic Phase 3 review")
|
review = sub.add_parser("review", help="Run Phase 3 review")
|
||||||
review.add_argument("project", help="Project slug or path")
|
review.add_argument("project", help="Project slug or path")
|
||||||
|
review.add_argument("--model-review", action="store_true", help="Run independent model-based Phase 3 review")
|
||||||
|
review.add_argument("--model", default="zenmux-anthropic/claude-opus-4-7", help="Model for --model-review")
|
||||||
|
review.add_argument("--max-context-chars", type=int, default=650_000, help="Bounded context size for model review")
|
||||||
|
review.add_argument("--dry-run", action="store_true")
|
||||||
review.set_defaults(func=cmd_review)
|
review.set_defaults(func=cmd_review)
|
||||||
|
|
||||||
prompt = sub.add_parser("prompt", help="Print a Codex command prompt template")
|
prompt = sub.add_parser("prompt", help="Print a Codex command prompt template")
|
||||||
@@ -745,6 +887,8 @@ def build_parser() -> argparse.ArgumentParser:
|
|||||||
finalize.add_argument("--input", default="phase4/final_zh.md", help="Chinese Markdown source for default v0.20 finalization")
|
finalize.add_argument("--input", default="phase4/final_zh.md", help="Chinese Markdown source for default v0.20 finalization")
|
||||||
finalize.add_argument("--legacy-translate", action="store_true", help="Use legacy final_en -> translate -> polish pipeline")
|
finalize.add_argument("--legacy-translate", action="store_true", help="Use legacy final_en -> translate -> polish pipeline")
|
||||||
finalize.add_argument("--polish", action="store_true", help="Run optional Chinese polish step before rendering")
|
finalize.add_argument("--polish", action="store_true", help="Run optional Chinese polish step before rendering")
|
||||||
|
finalize.add_argument("--number-citations", action="store_true", help="Convert [src_xxx] citations to numeric references before rendering")
|
||||||
|
finalize.add_argument("--numbered-output", default="phase4/final_zh_numbered.md", help="Output path for numeric citation Markdown")
|
||||||
finalize.add_argument("--report-engine", choices=["reportlab", "quarto"], default=None)
|
finalize.add_argument("--report-engine", choices=["reportlab", "quarto"], default=None)
|
||||||
finalize.add_argument("--no-docx", action="store_true")
|
finalize.add_argument("--no-docx", action="store_true")
|
||||||
finalize.add_argument("--no-pdf", action="store_true")
|
finalize.add_argument("--no-pdf", action="store_true")
|
||||||
@@ -781,6 +925,17 @@ def build_parser() -> argparse.ArgumentParser:
|
|||||||
models.add_argument("--json", action="store_true", help="Emit JSON")
|
models.add_argument("--json", action="store_true", help="Emit JSON")
|
||||||
models.set_defaults(func=cmd_models)
|
models.set_defaults(func=cmd_models)
|
||||||
|
|
||||||
|
sources = sub.add_parser("sources", help="Manage source snapshots and source registry")
|
||||||
|
sources_sub = sources.add_subparsers(dest="sources_cmd", required=True)
|
||||||
|
sources_cache = sources_sub.add_parser("cache", help="Cache important sources as local Markdown snapshots")
|
||||||
|
sources_cache.add_argument("project", help="Project slug or path")
|
||||||
|
sources_cache.add_argument("--sources", default="phase2/sources.jsonl", help="Source registry path relative to project")
|
||||||
|
sources_cache.add_argument("--all", action="store_true", help="Cache all remote sources, not only important official/Tier 1 sources")
|
||||||
|
sources_cache.add_argument("--limit", type=int, help="Maximum sources to cache in this run")
|
||||||
|
sources_cache.add_argument("--force", action="store_true", help="Refetch even if cached_text_path already exists")
|
||||||
|
sources_cache.add_argument("--dry-run", action="store_true")
|
||||||
|
sources_cache.set_defaults(func=cmd_sources)
|
||||||
|
|
||||||
apply_models = sub.add_parser("apply-models", help="Apply profile to agent files")
|
apply_models = sub.add_parser("apply-models", help="Apply profile to agent files")
|
||||||
apply_models.add_argument("--profile", required=True, help="Profile name from configs/models.yaml")
|
apply_models.add_argument("--profile", required=True, help="Profile name from configs/models.yaml")
|
||||||
apply_models.add_argument("--target", choices=["opencode", "codex", "both"], default="both")
|
apply_models.add_argument("--target", choices=["opencode", "codex", "both"], default="both")
|
||||||
|
|||||||
+165
-10
@@ -1,11 +1,12 @@
|
|||||||
"""通用搜索客户端(Serper / Exa / Tavily 路由)。
|
"""通用搜索客户端(Tavily / Exa / Brave / Serper 路由)。
|
||||||
|
|
||||||
为 build_glossary.py 这类术语核查场景服务。
|
为 build_glossary.py 这类术语核查场景服务。
|
||||||
|
|
||||||
关键设计:
|
关键设计:
|
||||||
- `trust_env=False` 绕开系统 socks 代理(Clash on macOS 配 socks5 时 httpx 会 TLS EOF)
|
- `trust_env=False` 绕开系统 socks 代理(Clash on macOS 配 socks5 时 httpx 会 TLS EOF)
|
||||||
- 专利 / Scholar / News 优先 Serper,保证 Google Patents / Google Scholar 路径被真正调用
|
- 专利 / Scholar / News 优先 Serper,保证 Google Patents / Google Scholar 路径被真正调用
|
||||||
- 通用网页 Exa 优先,Tavily fallback
|
- 通用网页 Tavily 优先,Exa/Brave fallback
|
||||||
|
- 证据发现 Exa 优先,用 highlights/text 摘录喂给 evidence packet
|
||||||
- 遇到配额问题自动降级或返回 empty
|
- 遇到配额问题自动降级或返回 empty
|
||||||
- 不做深度 crawl,只要摘要
|
- 不做深度 crawl,只要摘要
|
||||||
"""
|
"""
|
||||||
@@ -47,12 +48,28 @@ class ExaClient:
|
|||||||
def __exit__(self, *_args: Any) -> None:
|
def __exit__(self, *_args: Any) -> None:
|
||||||
self.close()
|
self.close()
|
||||||
|
|
||||||
def search(self, query: str, *, num_results: int = 5) -> list[SearchHit]:
|
def search(
|
||||||
|
self,
|
||||||
|
query: str,
|
||||||
|
*,
|
||||||
|
num_results: int = 5,
|
||||||
|
search_type: str = "auto",
|
||||||
|
category: str | None = None,
|
||||||
|
use_highlights: bool = False,
|
||||||
|
max_characters: int = 800,
|
||||||
|
) -> list[SearchHit]:
|
||||||
body = {
|
body = {
|
||||||
"query": query,
|
"query": query,
|
||||||
"numResults": num_results,
|
"numResults": num_results,
|
||||||
"type": "auto",
|
"type": search_type,
|
||||||
"contents": {"text": {"maxCharacters": 800}},
|
"contents": {"text": {"maxCharacters": max_characters}},
|
||||||
|
}
|
||||||
|
if category:
|
||||||
|
body["category"] = category
|
||||||
|
if use_highlights:
|
||||||
|
body["contents"]["highlights"] = {
|
||||||
|
"numSentences": 2,
|
||||||
|
"highlightsPerUrl": 3,
|
||||||
}
|
}
|
||||||
r = self._client.post(
|
r = self._client.post(
|
||||||
"https://api.exa.ai/search",
|
"https://api.exa.ai/search",
|
||||||
@@ -64,11 +81,15 @@ class ExaClient:
|
|||||||
data = r.json()
|
data = r.json()
|
||||||
out: list[SearchHit] = []
|
out: list[SearchHit] = []
|
||||||
for item in data.get("results", [])[:num_results]:
|
for item in data.get("results", [])[:num_results]:
|
||||||
|
highlights = item.get("highlights") or []
|
||||||
|
text = item.get("text") or item.get("snippet") or ""
|
||||||
|
if highlights:
|
||||||
|
text = " | ".join(str(h).strip() for h in highlights if str(h).strip())
|
||||||
out.append(
|
out.append(
|
||||||
SearchHit(
|
SearchHit(
|
||||||
title=(item.get("title") or "")[:200],
|
title=(item.get("title") or "")[:200],
|
||||||
url=item.get("url") or "",
|
url=item.get("url") or "",
|
||||||
snippet=(item.get("text") or item.get("snippet") or "")[:600],
|
snippet=text[:1000],
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
return out
|
return out
|
||||||
@@ -115,10 +136,51 @@ class TavilyClient:
|
|||||||
return out
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
class BraveClient:
|
||||||
|
def __init__(self, api_key: str | None = None, timeout: float = 30.0) -> None:
|
||||||
|
self.api_key = api_key or os.environ.get("BRAVE_API_KEY")
|
||||||
|
if not self.api_key:
|
||||||
|
raise SearchError("BRAVE_API_KEY not set")
|
||||||
|
self._client = httpx.Client(trust_env=False, timeout=timeout)
|
||||||
|
|
||||||
|
def close(self) -> None:
|
||||||
|
self._client.close()
|
||||||
|
|
||||||
|
def __enter__(self) -> "BraveClient":
|
||||||
|
return self
|
||||||
|
|
||||||
|
def __exit__(self, *_args: Any) -> None:
|
||||||
|
self.close()
|
||||||
|
|
||||||
|
def search(self, query: str, *, num_results: int = 5) -> list[SearchHit]:
|
||||||
|
r = self._client.get(
|
||||||
|
"https://api.search.brave.com/res/v1/web/search",
|
||||||
|
params={"q": query, "count": min(max(num_results, 1), 20)},
|
||||||
|
headers={
|
||||||
|
"X-Subscription-Token": self.api_key,
|
||||||
|
"Accept": "application/json",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
if r.status_code != 200:
|
||||||
|
raise SearchError(f"Brave HTTP {r.status_code}: {r.text[:200]}")
|
||||||
|
data = r.json()
|
||||||
|
out: list[SearchHit] = []
|
||||||
|
for item in (data.get("web") or {}).get("results", [])[:num_results]:
|
||||||
|
out.append(
|
||||||
|
SearchHit(
|
||||||
|
title=(item.get("title") or "")[:200],
|
||||||
|
url=item.get("url") or "",
|
||||||
|
snippet=(item.get("description") or "")[:600],
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
class SearchClient:
|
class SearchClient:
|
||||||
"""统一搜索门面,支持多路由:
|
"""统一搜索门面,支持多路由:
|
||||||
|
|
||||||
- `search(query)`:通用网页搜索,优先 Exa → 降级 Tavily
|
- `search(query)`:通用网页搜索,优先 Tavily → Exa → Brave
|
||||||
|
- `evidence(query)`:证据发现,优先 Exa highlights → Tavily → Brave
|
||||||
- `patents(query)`:专利检索,走 Serper(Google Patents);失败则通用搜索补刀
|
- `patents(query)`:专利检索,走 Serper(Google Patents);失败则通用搜索补刀
|
||||||
- `scholar(query)`:学术论文,走 Serper Scholar;失败则通用搜索补刀
|
- `scholar(query)`:学术论文,走 Serper Scholar;失败则通用搜索补刀
|
||||||
- `news(query)`:新闻检索,走 Serper News;失败则通用搜索补刀
|
- `news(query)`:新闻检索,走 Serper News;失败则通用搜索补刀
|
||||||
@@ -129,6 +191,7 @@ class SearchClient:
|
|||||||
def __init__(self, *, strict_specialized: bool = True) -> None:
|
def __init__(self, *, strict_specialized: bool = True) -> None:
|
||||||
self._exa: ExaClient | None = None
|
self._exa: ExaClient | None = None
|
||||||
self._tavily: TavilyClient | None = None
|
self._tavily: TavilyClient | None = None
|
||||||
|
self._brave: BraveClient | None = None
|
||||||
self._serper = None # 惰性实例化
|
self._serper = None # 惰性实例化
|
||||||
self.strict_specialized = strict_specialized
|
self.strict_specialized = strict_specialized
|
||||||
try:
|
try:
|
||||||
@@ -139,9 +202,13 @@ class SearchClient:
|
|||||||
self._tavily = TavilyClient()
|
self._tavily = TavilyClient()
|
||||||
except SearchError:
|
except SearchError:
|
||||||
pass
|
pass
|
||||||
|
try:
|
||||||
|
self._brave = BraveClient()
|
||||||
|
except SearchError:
|
||||||
|
pass
|
||||||
self._has_serper_key = bool(os.environ.get("SERPER_API_KEY") or os.environ.get("SERPAPI_KEY"))
|
self._has_serper_key = bool(os.environ.get("SERPER_API_KEY") or os.environ.get("SERPAPI_KEY"))
|
||||||
if not (self._exa or self._tavily or self._has_serper_key):
|
if not (self._exa or self._tavily or self._brave or self._has_serper_key):
|
||||||
raise SearchError("no search API key available: set SERPER_API_KEY, SERPAPI_KEY, EXA_API_KEY, or TAVILY_API_KEY")
|
raise SearchError("no search API key available: set SERPER_API_KEY, SERPAPI_KEY, EXA_API_KEY, TAVILY_API_KEY, or BRAVE_API_KEY")
|
||||||
|
|
||||||
def _get_serper(self):
|
def _get_serper(self):
|
||||||
"""惰性创建 SerperClient。没 key 时返回 None。"""
|
"""惰性创建 SerperClient。没 key 时返回 None。"""
|
||||||
@@ -161,6 +228,8 @@ class SearchClient:
|
|||||||
self._exa.close()
|
self._exa.close()
|
||||||
if self._tavily:
|
if self._tavily:
|
||||||
self._tavily.close()
|
self._tavily.close()
|
||||||
|
if self._brave:
|
||||||
|
self._brave.close()
|
||||||
if self._serper and self._serper is not False:
|
if self._serper and self._serper is not False:
|
||||||
self._serper.close()
|
self._serper.close()
|
||||||
|
|
||||||
@@ -171,17 +240,60 @@ class SearchClient:
|
|||||||
self.close()
|
self.close()
|
||||||
|
|
||||||
def search(self, query: str, *, num_results: int = 5) -> list[SearchHit]:
|
def search(self, query: str, *, num_results: int = 5) -> list[SearchHit]:
|
||||||
"""通用网页搜索。Exa 首选,Tavily 备选。"""
|
"""通用网页搜索。Tavily 首选,Exa/Brave 备选。"""
|
||||||
|
if self._tavily:
|
||||||
|
try:
|
||||||
|
return self._tavily.search(query, num_results=num_results)
|
||||||
|
except SearchError:
|
||||||
|
pass
|
||||||
if self._exa:
|
if self._exa:
|
||||||
try:
|
try:
|
||||||
return self._exa.search(query, num_results=num_results)
|
return self._exa.search(query, num_results=num_results)
|
||||||
except SearchError:
|
except SearchError:
|
||||||
pass
|
pass
|
||||||
|
if self._brave:
|
||||||
|
try:
|
||||||
|
return self._brave.search(query, num_results=num_results)
|
||||||
|
except SearchError:
|
||||||
|
pass
|
||||||
|
return []
|
||||||
|
|
||||||
|
def evidence(
|
||||||
|
self,
|
||||||
|
query: str,
|
||||||
|
*,
|
||||||
|
num_results: int = 10,
|
||||||
|
category: str | None = None,
|
||||||
|
) -> list[SearchHit]:
|
||||||
|
"""Evidence discovery route.
|
||||||
|
|
||||||
|
Exa is better suited for agent-facing evidence discovery because it can
|
||||||
|
return concise highlights/text per URL. Results are still candidate
|
||||||
|
sources only; downstream packets must score and trace important hits
|
||||||
|
back to original Tier 1-2 sources before making final claims.
|
||||||
|
"""
|
||||||
|
if self._exa:
|
||||||
|
try:
|
||||||
|
return self._exa.search(
|
||||||
|
query,
|
||||||
|
num_results=num_results,
|
||||||
|
search_type="auto",
|
||||||
|
category=category,
|
||||||
|
use_highlights=True,
|
||||||
|
max_characters=1200,
|
||||||
|
)
|
||||||
|
except SearchError:
|
||||||
|
pass
|
||||||
if self._tavily:
|
if self._tavily:
|
||||||
try:
|
try:
|
||||||
return self._tavily.search(query, num_results=num_results)
|
return self._tavily.search(query, num_results=num_results)
|
||||||
except SearchError:
|
except SearchError:
|
||||||
pass
|
pass
|
||||||
|
if self._brave:
|
||||||
|
try:
|
||||||
|
return self._brave.search(query, num_results=num_results)
|
||||||
|
except SearchError:
|
||||||
|
pass
|
||||||
return []
|
return []
|
||||||
|
|
||||||
def patents(self, query: str, *, num_results: int = 10) -> list[SearchHit]:
|
def patents(self, query: str, *, num_results: int = 10) -> list[SearchHit]:
|
||||||
@@ -253,6 +365,49 @@ class SearchClient:
|
|||||||
raise SearchError("serper unavailable for news route; refusing silent fallback")
|
raise SearchError("serper unavailable for news route; refusing silent fallback")
|
||||||
return self.search(query, num_results=num_results)
|
return self.search(query, num_results=num_results)
|
||||||
|
|
||||||
|
def fda(self, query: str, *, num_results: int = 10) -> list[SearchHit]:
|
||||||
|
"""FDA-focused discovery for warning letters and meeting records.
|
||||||
|
|
||||||
|
FDA enforcement examples are often more useful for GMP remediation than
|
||||||
|
generic web pages, so this route biases discovery toward warning
|
||||||
|
letters, inspection/enforcement pages, and meeting materials/minutes.
|
||||||
|
"""
|
||||||
|
def fda_only(hits: list[SearchHit]) -> list[SearchHit]:
|
||||||
|
return [hit for hit in hits if "fda.gov" in (hit.url or "").lower()]
|
||||||
|
|
||||||
|
focused_queries = [
|
||||||
|
f'site:fda.gov "Warning Letter" GMP pharmaceutical {query}',
|
||||||
|
f'site:fda.gov/inspections-compliance-enforcement-and-criminal-investigations "Warning Letter" {query}',
|
||||||
|
f'site:fda.gov "meeting materials" "pharmaceutical quality" {query}',
|
||||||
|
f'site:fda.gov "meeting minutes" FDA pharmaceutical quality {query}',
|
||||||
|
]
|
||||||
|
hits: list[SearchHit] = []
|
||||||
|
seen: set[str] = set()
|
||||||
|
per_query = max(2, min(num_results, 4))
|
||||||
|
for focused_query in focused_queries:
|
||||||
|
route_hits: list[SearchHit] = []
|
||||||
|
serper = self._get_serper()
|
||||||
|
if serper:
|
||||||
|
try:
|
||||||
|
route_hits = [
|
||||||
|
SearchHit(h.title, h.url, h.snippet)
|
||||||
|
for h in serper.search(focused_query, num_results=per_query)
|
||||||
|
]
|
||||||
|
except Exception as exc:
|
||||||
|
if self.strict_specialized:
|
||||||
|
raise SearchError(f"serper FDA search failed: {exc}") from exc
|
||||||
|
if not route_hits and not self.strict_specialized:
|
||||||
|
route_hits = self.search(focused_query, num_results=per_query)
|
||||||
|
for hit in fda_only(route_hits):
|
||||||
|
key = hit.url or hit.title
|
||||||
|
if not key or key in seen:
|
||||||
|
continue
|
||||||
|
seen.add(key)
|
||||||
|
hits.append(hit)
|
||||||
|
if len(hits) >= num_results:
|
||||||
|
return hits
|
||||||
|
return hits
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
from scripts.lib.zenmux_client import load_secrets
|
from scripts.lib.zenmux_client import load_secrets
|
||||||
|
|||||||
@@ -0,0 +1,188 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Convert Deep Research source IDs into numeric citations for final output."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
|
||||||
|
SRC_CITE_RE = re.compile(r"\[((?:src_[A-Za-z0-9_-]+)(?:\s*,\s*src_[A-Za-z0-9_-]+)*)\]")
|
||||||
|
|
||||||
|
|
||||||
|
def load_sources(path: Path) -> dict[str, dict[str, Any]]:
|
||||||
|
sources: dict[str, dict[str, Any]] = {}
|
||||||
|
if not path.exists():
|
||||||
|
return sources
|
||||||
|
for line in path.read_text(encoding="utf-8").splitlines():
|
||||||
|
if not line.strip():
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
obj = json.loads(line)
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
continue
|
||||||
|
sid = obj.get("id") or obj.get("source_id")
|
||||||
|
if sid:
|
||||||
|
sources[str(sid)] = obj
|
||||||
|
return sources
|
||||||
|
|
||||||
|
|
||||||
|
def extract_ordered_source_ids(text: str) -> list[str]:
|
||||||
|
ordered: list[str] = []
|
||||||
|
seen: set[str] = set()
|
||||||
|
for match in SRC_CITE_RE.finditer(text):
|
||||||
|
for sid in [item.strip() for item in match.group(1).split(",")]:
|
||||||
|
if sid and sid not in seen:
|
||||||
|
seen.add(sid)
|
||||||
|
ordered.append(sid)
|
||||||
|
return ordered
|
||||||
|
|
||||||
|
|
||||||
|
def _canonical_source_key(sid: str, source: dict[str, Any] | None) -> str:
|
||||||
|
"""Return a stable de-duplication key for a source record.
|
||||||
|
|
||||||
|
Phase 2 often creates chapter-local source IDs for the same local PDF or
|
||||||
|
official guideline. Final references should cite the underlying source
|
||||||
|
once, while citation_map.json keeps the full src_id traceability.
|
||||||
|
"""
|
||||||
|
if not source:
|
||||||
|
return f"missing:{sid}"
|
||||||
|
title = re.sub(r"\s+", " ", str(source.get("title") or source.get("name") or sid)).strip().lower()
|
||||||
|
title = title.removesuffix(" ocr").removesuffix(".ocr").strip()
|
||||||
|
doi = str(source.get("doi") or "").strip().lower()
|
||||||
|
if doi:
|
||||||
|
return f"doi:{doi}"
|
||||||
|
path = str(source.get("path") or "").strip()
|
||||||
|
url = str(source.get("url") or "").strip()
|
||||||
|
if title and ("phase0/extracted/" in path or "phase0/extracted/" in url):
|
||||||
|
return f"local-material:{title}"
|
||||||
|
for field in ("url", "path"):
|
||||||
|
value = str(source.get(field) or "").strip()
|
||||||
|
if value:
|
||||||
|
return f"{field}:{value.rstrip('/').lower()}"
|
||||||
|
return f"title:{title or sid}"
|
||||||
|
|
||||||
|
|
||||||
|
def build_numeric_mapping(
|
||||||
|
ordered_ids: list[str],
|
||||||
|
sources: dict[str, dict[str, Any]],
|
||||||
|
) -> tuple[dict[str, int], list[dict[str, Any]]]:
|
||||||
|
mapping: dict[str, int] = {}
|
||||||
|
records: list[dict[str, Any]] = []
|
||||||
|
seen_keys: dict[str, int] = {}
|
||||||
|
record_by_number: dict[int, dict[str, Any]] = {}
|
||||||
|
for sid in ordered_ids:
|
||||||
|
source = sources.get(sid)
|
||||||
|
key = _canonical_source_key(sid, source)
|
||||||
|
if key in seen_keys:
|
||||||
|
number = seen_keys[key]
|
||||||
|
mapping[sid] = number
|
||||||
|
record_by_number[number].setdefault("source_ids", []).append(sid)
|
||||||
|
continue
|
||||||
|
number = len(records) + 1
|
||||||
|
seen_keys[key] = number
|
||||||
|
mapping[sid] = number
|
||||||
|
record = {
|
||||||
|
"number": number,
|
||||||
|
"source_id": sid,
|
||||||
|
"source_ids": [sid],
|
||||||
|
"source": source or {},
|
||||||
|
"dedupe_key": key,
|
||||||
|
}
|
||||||
|
records.append(record)
|
||||||
|
record_by_number[number] = record
|
||||||
|
return mapping, records
|
||||||
|
|
||||||
|
|
||||||
|
def format_reference(number: int, sid: str, source: dict[str, Any] | None) -> str:
|
||||||
|
if not source:
|
||||||
|
return f"{number}. {sid}. (sources.jsonl 未找到该来源)"
|
||||||
|
authors = ", ".join(source.get("authors", [])) if source.get("authors") else ""
|
||||||
|
year = source.get("year") or source.get("date") or ""
|
||||||
|
title = source.get("title") or source.get("name") or sid
|
||||||
|
title = re.sub(r"(?i)(?:\s+OCR|\.ocr)$", "", str(title)).strip()
|
||||||
|
publisher = source.get("publisher") or source.get("venue") or source.get("source") or ""
|
||||||
|
url = source.get("url") or source.get("path") or ""
|
||||||
|
parts = [f"{number}. "]
|
||||||
|
if authors:
|
||||||
|
parts.append(f"{authors}. ")
|
||||||
|
if year:
|
||||||
|
parts.append(f"({year}). ")
|
||||||
|
parts.append(str(title))
|
||||||
|
if publisher:
|
||||||
|
parts.append(f". {publisher}")
|
||||||
|
if url:
|
||||||
|
parts.append(f". {url}")
|
||||||
|
return "".join(parts)
|
||||||
|
|
||||||
|
|
||||||
|
def convert_citations(text: str, mapping: dict[str, int]) -> str:
|
||||||
|
def repl(match: re.Match[str]) -> str:
|
||||||
|
ids = [item.strip() for item in match.group(1).split(",") if item.strip()]
|
||||||
|
nums: list[str] = []
|
||||||
|
seen: set[int] = set()
|
||||||
|
for sid in ids:
|
||||||
|
if sid not in mapping:
|
||||||
|
continue
|
||||||
|
number = mapping[sid]
|
||||||
|
if number in seen:
|
||||||
|
continue
|
||||||
|
seen.add(number)
|
||||||
|
nums.append(str(number))
|
||||||
|
return "<sup>[" + ", ".join(nums) + "]</sup>" if nums else match.group(0)
|
||||||
|
|
||||||
|
return SRC_CITE_RE.sub(repl, text)
|
||||||
|
|
||||||
|
|
||||||
|
def strip_existing_reference_section(text: str) -> str:
|
||||||
|
pattern = re.compile(r"\n##\s*(?:参考文献|参考来源清单|References)\s*\n.*\Z", re.S)
|
||||||
|
return pattern.sub("", text).rstrip() + "\n"
|
||||||
|
|
||||||
|
|
||||||
|
def number_citations(
|
||||||
|
*,
|
||||||
|
text: str,
|
||||||
|
sources: dict[str, dict[str, Any]],
|
||||||
|
) -> tuple[str, list[dict[str, Any]]]:
|
||||||
|
ordered_ids = extract_ordered_source_ids(text)
|
||||||
|
mapping, records = build_numeric_mapping(ordered_ids, sources)
|
||||||
|
body = convert_citations(strip_existing_reference_section(text), mapping).rstrip()
|
||||||
|
ref_lines = ["", "## 参考来源清单", ""]
|
||||||
|
for record in records:
|
||||||
|
ref_lines.append(format_reference(record["number"], record["source_id"], record["source"]))
|
||||||
|
return body + "\n" + "\n".join(ref_lines).rstrip() + "\n", records
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
parser = argparse.ArgumentParser(description="Convert [src_xxx] citations to numeric citations")
|
||||||
|
parser.add_argument("project", help="Project directory")
|
||||||
|
parser.add_argument("--input", default="phase4/final_zh.md")
|
||||||
|
parser.add_argument("--output", default="phase4/final_zh_numbered.md")
|
||||||
|
parser.add_argument("--sources", default="phase2/sources.jsonl")
|
||||||
|
parser.add_argument("--map", default="phase4/citation_map.json")
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
project = Path(args.project)
|
||||||
|
src_path = project / args.input
|
||||||
|
out_path = project / args.output
|
||||||
|
sources_path = project / args.sources
|
||||||
|
map_path = project / args.map
|
||||||
|
if not src_path.exists():
|
||||||
|
raise SystemExit(f"input not found: {src_path}")
|
||||||
|
sources = load_sources(sources_path)
|
||||||
|
numbered, records = number_citations(text=src_path.read_text(encoding="utf-8"), sources=sources)
|
||||||
|
out_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
out_path.write_text(numbered, encoding="utf-8")
|
||||||
|
map_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
map_path.write_text(json.dumps(records, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||||
|
print(f"Wrote: {out_path.relative_to(project)}")
|
||||||
|
print(f"Wrote: {map_path.relative_to(project)}")
|
||||||
|
print(f"Citations: {len(records)}")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
+12
-1
@@ -33,6 +33,7 @@ from scripts.lib.markdown_chunker import (
|
|||||||
split_by_headers,
|
split_by_headers,
|
||||||
)
|
)
|
||||||
from scripts.lib.zenmux_client import ZenMuxClient, ZenMuxError, load_secrets
|
from scripts.lib.zenmux_client import ZenMuxClient, ZenMuxError, load_secrets
|
||||||
|
from scripts.runtime.skills import SkillRegistry
|
||||||
|
|
||||||
DEFAULT_MODEL = "anthropic/claude-sonnet-4.6"
|
DEFAULT_MODEL = "anthropic/claude-sonnet-4.6"
|
||||||
MODEL_MAX_TOKENS = {
|
MODEL_MAX_TOKENS = {
|
||||||
@@ -43,6 +44,16 @@ MODEL_MAX_TOKENS = {
|
|||||||
"anthropic/claude-haiku-4.5": 16000,
|
"anthropic/claude-haiku-4.5": 16000,
|
||||||
}
|
}
|
||||||
PROMPT_FILE = Path(__file__).parent / "prompts" / "polish_system.txt"
|
PROMPT_FILE = Path(__file__).parent / "prompts" / "polish_system.txt"
|
||||||
|
POLISH_SKILLS = ("humanizer-cn", "output-hygiene")
|
||||||
|
|
||||||
|
|
||||||
|
def build_polish_system_prompt(skill_registry: SkillRegistry | None = None) -> str:
|
||||||
|
"""Build the Phase 4 polish prompt with canonical writing skills attached."""
|
||||||
|
registry = skill_registry or SkillRegistry()
|
||||||
|
parts = [PROMPT_FILE.read_text(encoding="utf-8").rstrip()]
|
||||||
|
for skill_name in POLISH_SKILLS:
|
||||||
|
parts.append(f"# Skill: {skill_name}\n\n{registry.read(skill_name).rstrip()}")
|
||||||
|
return "\n\n".join(parts) + "\n"
|
||||||
|
|
||||||
|
|
||||||
def resolve_project(arg: str) -> Path:
|
def resolve_project(arg: str) -> Path:
|
||||||
@@ -163,7 +174,7 @@ def main() -> int:
|
|||||||
log_file = logs_dir / "polish.jsonl"
|
log_file = logs_dir / "polish.jsonl"
|
||||||
notes_file = project_root / "phase4" / "polish_notes.jsonl"
|
notes_file = project_root / "phase4" / "polish_notes.jsonl"
|
||||||
|
|
||||||
system_prompt = PROMPT_FILE.read_text(encoding="utf-8")
|
system_prompt = build_polish_system_prompt()
|
||||||
text = src_path.read_text(encoding="utf-8")
|
text = src_path.read_text(encoding="utf-8")
|
||||||
blocks = split_by_headers(text, max_level=2)
|
blocks = split_by_headers(text, max_level=2)
|
||||||
|
|
||||||
|
|||||||
@@ -91,34 +91,82 @@ def _chapter_title_from_id(chapter_id: str) -> str:
|
|||||||
return chapter_id
|
return chapter_id
|
||||||
|
|
||||||
|
|
||||||
|
def _load_source_registry(sources_path: Path, source_ids: list[str]) -> list[dict]:
|
||||||
|
wanted = set(source_ids)
|
||||||
|
if not sources_path.exists() or not wanted:
|
||||||
|
return []
|
||||||
|
rows: list[dict] = []
|
||||||
|
for line in sources_path.read_text(encoding="utf-8").splitlines():
|
||||||
|
if not line.strip():
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
row = json.loads(line)
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
continue
|
||||||
|
if row.get("id") in wanted:
|
||||||
|
rows.append(row)
|
||||||
|
return rows
|
||||||
|
|
||||||
|
|
||||||
|
def _cached_source_excerpts(project_root: Path, cached_paths: list[str], *, max_sources: int = 5, max_chars: int = 1400) -> list[dict]:
|
||||||
|
excerpts: list[dict] = []
|
||||||
|
for rel in cached_paths[:max_sources]:
|
||||||
|
path = project_root / rel
|
||||||
|
if not path.exists():
|
||||||
|
continue
|
||||||
|
text = path.read_text(encoding="utf-8", errors="ignore").strip()
|
||||||
|
excerpts.append({"path": rel, "excerpt": text[:max_chars]})
|
||||||
|
return excerpts
|
||||||
|
|
||||||
|
|
||||||
def build_chapter_briefs(project_root: Path) -> list[dict]:
|
def build_chapter_briefs(project_root: Path) -> list[dict]:
|
||||||
cards = load_task_cards(project_root / "phase2" / "task_cards.json")
|
cards = load_task_cards(project_root / "phase2" / "task_cards.json")
|
||||||
grouped: dict[str, list[tuple[str, dict]]] = {}
|
grouped: dict[str, list[tuple[str, dict]]] = {}
|
||||||
|
skipped_packets: list[dict[str, str]] = []
|
||||||
for card in cards:
|
for card in cards:
|
||||||
packet_path = project_root / card.output_packet
|
packet_path = project_root / card.output_packet
|
||||||
if not packet_path.exists():
|
if not packet_path.exists():
|
||||||
|
skipped_packets.append({"task_id": card.task_id, "reason": "packet file missing"})
|
||||||
continue
|
continue
|
||||||
packet = json.loads(packet_path.read_text(encoding="utf-8"))
|
packet = json.loads(packet_path.read_text(encoding="utf-8"))
|
||||||
|
try:
|
||||||
validate_packet(packet)
|
validate_packet(packet)
|
||||||
|
except Exception as exc:
|
||||||
|
skipped_packets.append({"task_id": card.task_id, "reason": str(exc)})
|
||||||
|
continue
|
||||||
for chapter_id in card.chapter_ids:
|
for chapter_id in card.chapter_ids:
|
||||||
grouped.setdefault(chapter_id, []).append((card.task_id, packet))
|
grouped.setdefault(chapter_id, []).append((card.task_id, packet))
|
||||||
|
|
||||||
briefs: list[dict] = []
|
briefs: list[dict] = []
|
||||||
out_dir = project_root / "phase2" / "chapter_briefs"
|
out_dir = project_root / "phase2" / "chapter_briefs"
|
||||||
out_dir.mkdir(parents=True, exist_ok=True)
|
out_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
if skipped_packets:
|
||||||
|
(project_root / "phase2" / "brief_warnings.json").write_text(
|
||||||
|
json.dumps(skipped_packets, ensure_ascii=False, indent=2) + "\n",
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
for chapter_id in sorted(grouped):
|
for chapter_id in sorted(grouped):
|
||||||
packet_pairs = sorted(grouped[chapter_id], key=lambda item: item[0])
|
packet_pairs = grouped[chapter_id]
|
||||||
packet_ids = [item[0] for item in packet_pairs]
|
packet_ids = [item[0] for item in packet_pairs]
|
||||||
packets = [item[1] for item in packet_pairs]
|
packets = [item[1] for item in packet_pairs]
|
||||||
source_ids = sorted({sid for packet in packets for sid in packet.get("source_ids", [])})
|
source_ids = sorted({sid for packet in packets for sid in packet.get("source_ids", [])})
|
||||||
|
source_registry = _load_source_registry(project_root / "phase2" / "sources.jsonl", source_ids)
|
||||||
|
cached_paths = [
|
||||||
|
source["cached_text_path"]
|
||||||
|
for source in source_registry
|
||||||
|
if source.get("cached_text_path")
|
||||||
|
]
|
||||||
|
chapter_title = next((card.chapter_title for card in cards if chapter_id in card.chapter_ids and card.chapter_title), None)
|
||||||
brief = {
|
brief = {
|
||||||
"chapter_id": chapter_id,
|
"chapter_id": chapter_id,
|
||||||
"chapter_title": _chapter_title_from_id(chapter_id),
|
"chapter_title": chapter_title or _chapter_title_from_id(chapter_id),
|
||||||
"packet_ids": packet_ids,
|
"packet_ids": packet_ids,
|
||||||
"core_claims": [claim for packet in packets for claim in packet.get("claims", [])],
|
"core_claims": [claim for packet in packets for claim in packet.get("claims", [])],
|
||||||
"evidence_items": [item for packet in packets for item in packet.get("evidence_items", [])],
|
"evidence_items": [item for packet in packets for item in packet.get("evidence_items", [])],
|
||||||
"counter_evidence": [item for packet in packets for item in packet.get("counter_evidence", [])],
|
"counter_evidence": [item for packet in packets for item in packet.get("counter_evidence", [])],
|
||||||
"source_ids": source_ids,
|
"source_ids": source_ids,
|
||||||
|
"cached_source_paths": cached_paths,
|
||||||
|
"cached_source_excerpts": _cached_source_excerpts(project_root, cached_paths),
|
||||||
"open_questions": [q for packet in packets for q in packet.get("open_questions", [])],
|
"open_questions": [q for packet in packets for q in packet.get("open_questions", [])],
|
||||||
"assembly_notes": [
|
"assembly_notes": [
|
||||||
"用中文写正式章节,英文仅保留在必要的来源标题、原文摘录、DOI/URL 中。",
|
"用中文写正式章节,英文仅保留在必要的来源标题、原文摘录、DOI/URL 中。",
|
||||||
@@ -190,6 +238,8 @@ def build_compressed_findings(project_root: Path) -> list[dict]:
|
|||||||
],
|
],
|
||||||
"counter_evidence": brief["counter_evidence"],
|
"counter_evidence": brief["counter_evidence"],
|
||||||
"source_ids": brief["source_ids"],
|
"source_ids": brief["source_ids"],
|
||||||
|
"cached_source_paths": brief.get("cached_source_paths", []),
|
||||||
|
"cached_source_excerpts": brief.get("cached_source_excerpts", []),
|
||||||
"open_questions": brief["open_questions"],
|
"open_questions": brief["open_questions"],
|
||||||
"writing_plan": [
|
"writing_plan": [
|
||||||
"先写本章判断,不按 packet 顺序堆砌。",
|
"先写本章判断,不按 packet 顺序堆砌。",
|
||||||
@@ -211,6 +261,7 @@ def build_chapter_user_prompt(brief: dict) -> str:
|
|||||||
"请根据以下 compressed finding / chapter brief 写一章正式中文 Markdown 正文。\n"
|
"请根据以下 compressed finding / chapter brief 写一章正式中文 Markdown 正文。\n"
|
||||||
"目标是形成一个完整章节,而不是 packet 摘要。避免碎片化,按金字塔结构组织:章首先给结论,再用证据支撑。\n"
|
"目标是形成一个完整章节,而不是 packet 摘要。避免碎片化,按金字塔结构组织:章首先给结论,再用证据支撑。\n"
|
||||||
"要求:标题必须是观点型判断;每个数字和事实保留 [src_xxx];纳入反方证据;不要出现调度元数据。\n"
|
"要求:标题必须是观点型判断;每个数字和事实保留 [src_xxx];纳入反方证据;不要出现调度元数据。\n"
|
||||||
|
"如 brief 中包含 cached_source_paths,说明这些是已抓取到本地的核心一手/权威信源快照;优先使用 packet 已摘录的原文,并在证据不足时标记需要从本地快照补摘录,不要重新联网检索。\n"
|
||||||
"禁止写空泛咨询腔。每个二级小节都必须至少落下 2 个具体审计发现、法规要求、SOP/记录/参数/现场观察或整改证据;不要只写原则。\n"
|
"禁止写空泛咨询腔。每个二级小节都必须至少落下 2 个具体审计发现、法规要求、SOP/记录/参数/现场观察或整改证据;不要只写原则。\n"
|
||||||
"正文末尾必须增加“证据落点与待补证据”小节,用表格列出:关键判断、已使用证据 source_id、已落地整改动作、仍缺证据。若证据不足,直接标注需回炉 Phase 2,不要用泛泛表述补齐。\n"
|
"正文末尾必须增加“证据落点与待补证据”小节,用表格列出:关键判断、已使用证据 source_id、已落地整改动作、仍缺证据。若证据不足,直接标注需回炉 Phase 2,不要用泛泛表述补齐。\n"
|
||||||
"只输出 Markdown,不要输出解释。\n\n"
|
"只输出 Markdown,不要输出解释。\n\n"
|
||||||
@@ -238,6 +289,7 @@ class ChapterAssemblyWorker:
|
|||||||
except FileNotFoundError:
|
except FileNotFoundError:
|
||||||
skill_texts.append(f"# Skill: {name}\n\n[missing skill: {name}]")
|
skill_texts.append(f"# Skill: {name}\n\n[missing skill: {name}]")
|
||||||
return (
|
return (
|
||||||
|
f"{self.role.identity}\n\n"
|
||||||
"你是 Deep Research v0.20 的中文章节组装 worker。\n"
|
"你是 Deep Research v0.20 的中文章节组装 worker。\n"
|
||||||
"你的职责是把结构化证据包收束成连贯章节,解决并发研究造成的碎片化。\n"
|
"你的职责是把结构化证据包收束成连贯章节,解决并发研究造成的碎片化。\n"
|
||||||
"不得编造来源,不得删除关键反方证据。\n\n"
|
"不得编造来源,不得删除关键反方证据。\n\n"
|
||||||
|
|||||||
@@ -21,6 +21,7 @@ class ResearchMethod:
|
|||||||
structure_principle: str
|
structure_principle: str
|
||||||
task_axes: list[str]
|
task_axes: list[str]
|
||||||
framework_sections: list[str]
|
framework_sections: list[str]
|
||||||
|
integrated_lanes: list[str]
|
||||||
|
|
||||||
|
|
||||||
class ResearchMethodRegistry:
|
class ResearchMethodRegistry:
|
||||||
@@ -56,5 +57,5 @@ class ResearchMethodRegistry:
|
|||||||
structure_principle=item.get("structure_principle", ""),
|
structure_principle=item.get("structure_principle", ""),
|
||||||
task_axes=list(item.get("task_axes") or []),
|
task_axes=list(item.get("task_axes") or []),
|
||||||
framework_sections=list(item.get("framework_sections") or []),
|
framework_sections=list(item.get("framework_sections") or []),
|
||||||
|
integrated_lanes=list(item.get("integrated_lanes") or item.get("task_axes") or []),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|||||||
@@ -32,6 +32,15 @@ def create_phase2_task_cards(
|
|||||||
framework_text = framework.read_text(encoding="utf-8")
|
framework_text = framework.read_text(encoding="utf-8")
|
||||||
if research_brief_path.exists():
|
if research_brief_path.exists():
|
||||||
research_brief = json.loads(research_brief_path.read_text(encoding="utf-8"))
|
research_brief = json.loads(research_brief_path.read_text(encoding="utf-8"))
|
||||||
|
if not research_brief.get("materials"):
|
||||||
|
material_inventory = load_manifest(project_root).get("material_inventory") or []
|
||||||
|
materials = []
|
||||||
|
for item in material_inventory:
|
||||||
|
rel = item.get("ocr_extracted_to") or item.get("extracted_to") or item.get("copied_to")
|
||||||
|
if rel:
|
||||||
|
materials.append({"path": rel, "role": "input_material"})
|
||||||
|
if materials:
|
||||||
|
research_brief["materials"] = materials
|
||||||
cards = generate_task_cards_from_research_brief(
|
cards = generate_task_cards_from_research_brief(
|
||||||
project_root.name,
|
project_root.name,
|
||||||
framework_text,
|
framework_text,
|
||||||
|
|||||||
+455
-26
@@ -192,10 +192,10 @@ def write_material_brief(
|
|||||||
def _axis_prompt_brief(axis: str, method: ResearchMethod) -> str:
|
def _axis_prompt_brief(axis: str, method: ResearchMethod) -> str:
|
||||||
prompts = {
|
prompts = {
|
||||||
"input_material_findings": "从用户材料中提取现场事实、审计发现、复盘记录和内部答复,并标注原始材料位置。",
|
"input_material_findings": "从用户材料中提取现场事实、审计发现、复盘记录和内部答复,并标注原始材料位置。",
|
||||||
"nmpa_fda_ema_ich_who_baseline": "把 NMPA、FDA、EMA、ICH、WHO、药典或 Annex 1 等要求转化为可核验的法规基线。",
|
"nmpa_fda_ema_ich_who_baseline": "把 NMPA、FDA、EMA、ICH、WHO、药典或 Annex 1 等要求转化为可核验的法规基线,并纳入 FDA warning letters 与会议材料作为执法尺度参照。",
|
||||||
"quality_system_gap": "把现场发现映射到质量体系流程缺口,覆盖偏差、变更、CAPA、文件、培训和数据完整性。",
|
"quality_system_gap": "把现场发现映射到质量体系流程缺口,覆盖偏差、变更、CAPA、文件、培训和数据完整性;优先检索 FDA warning letters 中同类缺陷的执法表述。",
|
||||||
"manufacturing_process_risk": "围绕生产工艺、设施、公用系统、CPP/CQA、验证和无菌保障识别系统性风险。",
|
"manufacturing_process_risk": "围绕生产工艺、设施、公用系统、CPP/CQA、验证和无菌保障识别系统性风险,并用 FDA warning letters / inspection enforcement examples 校准严重度。",
|
||||||
"operations_management_gap": "诊断运营管理、跨部门协同、会议机制、指标体系和交付节奏的结构性问题。",
|
"operations_management_gap": "诊断运营管理、跨部门协同、会议机制、指标体系和交付节奏的结构性问题,并参考 FDA 会议纪要/meeting materials 中对质量治理的关注点。",
|
||||||
"team_capability": "识别人员能力、岗位职责、质量文化和管理梯队方面的缺口与建设路径。",
|
"team_capability": "识别人员能力、岗位职责、质量文化和管理梯队方面的缺口与建设路径。",
|
||||||
"capa_roadmap": "把差距转化为短中长期 CAPA 组合,要求绑定 owner、期限、优先级、关闭证据和复核机制。",
|
"capa_roadmap": "把差距转化为短中长期 CAPA 组合,要求绑定 owner、期限、优先级、关闭证据和复核机制。",
|
||||||
"verification_evidence": "定义整改完成后可被审计接受的验证证据,包括记录、报告、趋势和管理评审输入。",
|
"verification_evidence": "定义整改完成后可被审计接受的验证证据,包括记录、报告、趋势和管理评审输入。",
|
||||||
@@ -213,13 +213,366 @@ def _material_paths(manifest: dict[str, Any]) -> list[dict[str, str]]:
|
|||||||
return materials
|
return materials
|
||||||
|
|
||||||
|
|
||||||
|
def _keywords_from_title(title: str) -> list[str]:
|
||||||
|
english = re.findall(r"[A-Za-z][A-Za-z0-9/+-]{1,}", title)
|
||||||
|
chinese_parts = re.split(r"[,,、;;::\s]+|和|与|及|的|在|为|从|来自|集中|决定|需要|形成|成为|不是|而是", title)
|
||||||
|
domain_terms = [
|
||||||
|
"审计",
|
||||||
|
"商业化",
|
||||||
|
"阶段门",
|
||||||
|
"风险",
|
||||||
|
"法规",
|
||||||
|
"欧盟",
|
||||||
|
"NMPA",
|
||||||
|
"GMP",
|
||||||
|
"ICH",
|
||||||
|
"无菌",
|
||||||
|
"RABS",
|
||||||
|
"First Air",
|
||||||
|
"APS",
|
||||||
|
"灯检",
|
||||||
|
"隧道",
|
||||||
|
"原液",
|
||||||
|
"WFI",
|
||||||
|
"SCADA",
|
||||||
|
"EMS",
|
||||||
|
"CPP",
|
||||||
|
"CQA",
|
||||||
|
"PPQ",
|
||||||
|
"清洁验证",
|
||||||
|
"偏差",
|
||||||
|
"变更",
|
||||||
|
"CAPA",
|
||||||
|
"数据完整性",
|
||||||
|
"人员",
|
||||||
|
"培训",
|
||||||
|
"质量文化",
|
||||||
|
"运营",
|
||||||
|
"跨部门",
|
||||||
|
"指标",
|
||||||
|
"团队",
|
||||||
|
"CDMO",
|
||||||
|
"整改",
|
||||||
|
"owner",
|
||||||
|
]
|
||||||
|
title_terms = [term for term in domain_terms if term in title]
|
||||||
|
keywords = [item.strip() for item in [*english, *title_terms, *chinese_parts] if len(item.strip()) >= 2]
|
||||||
|
seen: set[str] = set()
|
||||||
|
unique: list[str] = []
|
||||||
|
for keyword in keywords:
|
||||||
|
if keyword not in seen:
|
||||||
|
seen.add(keyword)
|
||||||
|
unique.append(keyword)
|
||||||
|
return unique[:12]
|
||||||
|
|
||||||
|
|
||||||
|
def _material_lines_for_chapter(project_root: Path, manifest: dict[str, Any], title: str, *, limit: int = 4) -> list[str]:
|
||||||
|
keywords = _keywords_from_title(title)
|
||||||
|
candidates: list[tuple[int, int, str]] = []
|
||||||
|
order = 0
|
||||||
|
for item in manifest.get("material_inventory") or []:
|
||||||
|
rel = item.get("ocr_extracted_to") or item.get("extracted_to")
|
||||||
|
if not rel:
|
||||||
|
continue
|
||||||
|
path = project_root / rel
|
||||||
|
if not path.exists():
|
||||||
|
continue
|
||||||
|
for raw in path.read_text(encoding="utf-8").splitlines():
|
||||||
|
line = raw.strip()
|
||||||
|
if len(line) < 8 or len(line) > 220:
|
||||||
|
continue
|
||||||
|
if line.startswith("#") or line.startswith("- source_path:") or line.startswith("- extracted_at:"):
|
||||||
|
continue
|
||||||
|
if "OCR Material:" in line:
|
||||||
|
continue
|
||||||
|
if re.match(r"^(审计对象|审计执行方|审计执行人|审计时间)[::]", line):
|
||||||
|
continue
|
||||||
|
score = sum(1 for keyword in keywords if keyword and keyword in line)
|
||||||
|
if score:
|
||||||
|
order += 1
|
||||||
|
candidates.append((score, order, f"{rel}:{line}"))
|
||||||
|
candidates.sort(key=lambda item: (-item[0], item[1]))
|
||||||
|
return [line for _, _, line in candidates[:limit]]
|
||||||
|
|
||||||
|
|
||||||
|
def _minimum_evidence_for_method(method: ResearchMethod) -> dict[str, Any]:
|
||||||
|
if method.key == "gmp_quality_operations_diagnosis":
|
||||||
|
return {
|
||||||
|
"local_material_quotes": 2,
|
||||||
|
"official_regulatory_or_guideline_sources": 2,
|
||||||
|
"enforcement_or_best_practice_precedents": 1,
|
||||||
|
"counter_evidence_or_boundary_conditions": 1,
|
||||||
|
"actionable_remediation_items": 3,
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
"high_quality_sources": 4,
|
||||||
|
"tier_1_2_sources": 2,
|
||||||
|
"counter_evidence_or_boundary_conditions": 1,
|
||||||
|
"decision_relevant_implications": 2,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _central_thesis(manifest: dict[str, Any], method: ResearchMethod) -> str:
|
||||||
|
topic = manifest.get("topic") or manifest.get("report_title") or "本研究主题"
|
||||||
|
if method.key == "gmp_quality_operations_diagnosis":
|
||||||
|
return (
|
||||||
|
f"初始主判断:{topic} 不应只按审计风险项数量来评价,而应从商业化 readiness、"
|
||||||
|
"质量体系运行成熟度、生产工艺证据链和运营协同能力四条线同时诊断。Phase 2 必须用"
|
||||||
|
"现场材料原文、官方法规/指南、执法案例或标杆实践来证明、修正或推翻这一判断。"
|
||||||
|
)
|
||||||
|
return (
|
||||||
|
f"初始主判断:{topic} 需要先形成可被证据推翻的观点型框架,再由 Phase 2 按方法论证据线"
|
||||||
|
"逐项求证;不能把并发检索结果直接堆砌成报告。"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _strategy_for_chapter(title: str, method: ResearchMethod) -> dict[str, Any]:
|
||||||
|
"""Return non-tautological Phase 1 strategy text for a chapter title."""
|
||||||
|
if method.key != "gmp_quality_operations_diagnosis":
|
||||||
|
return {
|
||||||
|
"core_question": f"本章需要判断:在什么证据条件下“{title}”成立,它会怎样改变最终决策?",
|
||||||
|
"bold_hypothesis": f"初始假设不是复述标题,而是预判“{title}”背后存在一个可被验证的因果机制;Phase 2 需要找证据支持、修正或推翻这个机制。",
|
||||||
|
"writing_claim": f"本章要把“{title}”写成一个可被证据检验的判断,而不是资料综述。",
|
||||||
|
"counter_evidence": [
|
||||||
|
"是否存在更简单的替代解释,能削弱本章主判断?",
|
||||||
|
"关键证据是否只来自单一来源或利益相关来源?",
|
||||||
|
"是否有反例显示本章判断只适用于部分场景?",
|
||||||
|
],
|
||||||
|
}
|
||||||
|
|
||||||
|
strategies = [
|
||||||
|
(
|
||||||
|
("审计", "阶段门"),
|
||||||
|
{
|
||||||
|
"core_question": "审计报告的低/中风险项计数,是否低估了白帆从临床/受托生产走向商业化标准时需要跨过的阶段门?",
|
||||||
|
"bold_hypothesis": "初始假设:白帆的硬件和文件基础总体可用,但审计材料暴露的是商业化 readiness 缺口,而不是简单的若干孤立缺陷;Phase 2 应验证这些缺口是否集中在无菌保障、工艺验证、质量闭环和运营节奏。",
|
||||||
|
"writing_claim": "本章要先把“风险项清单”翻译成管理层可决策的阶段门地图,说明哪些问题影响商业化放行、客户审计和技术转移节奏。",
|
||||||
|
"counter_evidence": [
|
||||||
|
"是否已有整改证据证明这些问题只是审计时点的临时缺口?",
|
||||||
|
"低/中风险评级是否足以说明商业化阶段门影响有限?",
|
||||||
|
"审计范围有限是否导致本章不能外推到整体体系成熟度?",
|
||||||
|
],
|
||||||
|
},
|
||||||
|
),
|
||||||
|
(
|
||||||
|
("法规", "欧盟", "NMPA", "ICH"),
|
||||||
|
{
|
||||||
|
"core_question": "如果按 EU Annex 1、NMPA GMP、ICH Q9/Q10 以及 FDA 执法尺度校准,哪些现场发现的严重度和整改优先级会发生变化?",
|
||||||
|
"bold_hypothesis": "初始假设:白帆按国内 GMP 逻辑已具备基础合规框架,但若以欧盟无菌标准和质量风险管理要求衡量,部分“低风险/建议项”会转化为体系成熟度缺口。",
|
||||||
|
"writing_claim": "本章要建立后文共用的法规基线,避免整改优先级只跟随原审计评级,而忽略国际化和商业化标准。",
|
||||||
|
"counter_evidence": [
|
||||||
|
"相关国际标准是否并不适用于当前产品阶段或委托生产边界?",
|
||||||
|
"NMPA 与欧盟/美国要求之间是否存在可接受差异?",
|
||||||
|
"是否有企业内部标准已经覆盖但审计材料未呈现?",
|
||||||
|
],
|
||||||
|
},
|
||||||
|
),
|
||||||
|
(
|
||||||
|
("无菌", "RABS", "First Air", "APS", "灯检"),
|
||||||
|
{
|
||||||
|
"core_question": "制剂线的主要无菌风险,是硬件布局不足,还是人员干预、首次气流保护、APS 覆盖和灯检标准执行证据不足?",
|
||||||
|
"bold_hypothesis": "初始假设:白帆制剂车间硬件基础并非主要短板,真正风险在于关键无菌行为和模拟验证是否能持续证明受控;Phase 2 应重点查 First Air、RABS 干预、APS 场景设计和灯检阳性样品管理。",
|
||||||
|
"writing_claim": "本章要把无菌保障从“设施看起来合规”推进到“关键操作和验证证据可被审计接受”。",
|
||||||
|
"counter_evidence": [
|
||||||
|
"现场是否已有完整视频复核、APS 覆盖和再培训有效性证据?",
|
||||||
|
"观察到的无菌动作问题是否只是个别人员或单次拍摄偏差?",
|
||||||
|
"灯检和 RABS 风险是否已有 SOP、趋势和复核记录闭环?",
|
||||||
|
],
|
||||||
|
},
|
||||||
|
),
|
||||||
|
(
|
||||||
|
("原液", "WFI", "SCADA", "EMS"),
|
||||||
|
{
|
||||||
|
"core_question": "原液和公用系统的风险是否被一次性封闭工艺掩盖,真正缺口在 WFI、SCADA/EMS、离线记录和异常升级证据链?",
|
||||||
|
"bold_hypothesis": "初始假设:一次性反应器和封闭转移降低了暴露风险,但不能自动证明系统受控;Phase 2 应验证 WFI 冷却回流、环境/压差报警、SCADA 数据和离线检测记录是否形成完整证据链。",
|
||||||
|
"writing_claim": "本章要说明原液与公用系统不是“硬件先进即可”,而是要证明关键状态、报警、数据和异常处理持续受控。",
|
||||||
|
"counter_evidence": [
|
||||||
|
"WFI、SCADA/EMS 和离线记录是否已有验证报告与趋势复核?",
|
||||||
|
"一次性系统是否已经充分降低共线和交叉污染风险?",
|
||||||
|
"被指出的公用系统风险是否只是设计建议而非实际偏差?",
|
||||||
|
],
|
||||||
|
},
|
||||||
|
),
|
||||||
|
(
|
||||||
|
("工艺", "CPP", "CQA", "PPQ", "清洁验证"),
|
||||||
|
{
|
||||||
|
"core_question": "现有 IND 阶段工艺规程和批记录,距离商业化 PPQ、控制策略和清洁验证所需证据还差在哪里?",
|
||||||
|
"bold_hypothesis": "初始假设:白帆目前的工艺文件足以支撑临床阶段执行,但不足以支撑商业化批记录、CPP/CQA 控制、PPQ 和清洁验证闭环;Phase 2 应查明哪些字段、参数和验证证据必须前置补齐。",
|
||||||
|
"writing_claim": "本章要把技术转移风险具体化为文件、参数、验证和批记录的硬门槛。",
|
||||||
|
"counter_evidence": [
|
||||||
|
"是否已有商业化模板、控制策略或 PPQ 草案未体现在审计材料中?",
|
||||||
|
"当前项目阶段是否尚不需要完整商业化批记录要求?",
|
||||||
|
"清洁验证和工艺验证是否已有主计划覆盖?",
|
||||||
|
],
|
||||||
|
},
|
||||||
|
),
|
||||||
|
(
|
||||||
|
("偏差", "变更", "CAPA", "数据完整性"),
|
||||||
|
{
|
||||||
|
"core_question": "白帆的问题是没有质量流程,还是流程之间的事件分类、升级、CAPA 有效性和数据完整性尚未形成运行闭环?",
|
||||||
|
"bold_hypothesis": "初始假设:白帆已有偏差、变更和 CAPA 的流程框架,但事件何时启动偏差、何时作为变更、如何证明 CAPA 有效,以及电子/纸质数据如何贯通,仍存在运行机制缺口。",
|
||||||
|
"writing_claim": "本章要把质量体系从“有 SOP”推进到“事件能被正确分类、调查、纠正、验证并趋势复核”。",
|
||||||
|
"counter_evidence": [
|
||||||
|
"是否有趋势分析、管理评审和 CAPA effectiveness check 证明体系已经闭环?",
|
||||||
|
"个别事件分类问题是否不足以代表体系性缺口?",
|
||||||
|
"电子系统和纸质记录之间是否已有数据完整性控制?",
|
||||||
|
],
|
||||||
|
},
|
||||||
|
),
|
||||||
|
(
|
||||||
|
("人员", "培训", "质量文化"),
|
||||||
|
{
|
||||||
|
"core_question": "培训记录齐全是否真的转化为一线无菌行为、偏差判断和质量风险意识?哪些证据能证明培训有效?",
|
||||||
|
"bold_hypothesis": "初始假设:白帆不缺培训台账,缺的是把培训结果转化为现场行为的一致性证据;如果 First Air、干预动作、事件判断和灯检执行仍需反复提醒,问题就不是“再培训一次”,而是培训有效性确认和质量文化运行机制不足。",
|
||||||
|
"writing_claim": "本章要把人员问题从“有没有培训”改写为“培训是否改变行为、降低风险、形成可复核证据”。",
|
||||||
|
"counter_evidence": [
|
||||||
|
"现场抽问、资格确认和再培训记录是否已证明人员理解到位?",
|
||||||
|
"被观察到的行为问题是否只发生在少数岗位或单次演示?",
|
||||||
|
"是否有岗位胜任力矩阵、年度复评和行为观察数据支撑人员能力?",
|
||||||
|
],
|
||||||
|
},
|
||||||
|
),
|
||||||
|
(
|
||||||
|
("运营", "跨部门", "指标", "review"),
|
||||||
|
{
|
||||||
|
"core_question": "白帆当前整改和生产准备依赖个人推动,还是已经形成跨部门例会、问题升级、指标看板和管理层复核的运营系统?",
|
||||||
|
"bold_hypothesis": "初始假设:运营短板不在于团队不努力,而在于缺少固定节奏和可视化管理系统;如果 owner、关闭证据、升级阈值和管理层 review 不稳定,整改会停留在临时协调,难以支撑商业化节奏。",
|
||||||
|
"writing_claim": "本章要说明运营管理是 GMP 风险的放大器:没有节奏、看板和升级机制,技术和质量问题会反复跨部门漂移。",
|
||||||
|
"counter_evidence": [
|
||||||
|
"是否已经存在稳定 PMO/例会/看板,只是未进入审计材料?",
|
||||||
|
"短期临时协调是否足以覆盖当前项目阶段,不需要完整运营系统?",
|
||||||
|
"owner、期限和关闭证据是否已经在复盘文件中基本清楚?",
|
||||||
|
],
|
||||||
|
},
|
||||||
|
),
|
||||||
|
(
|
||||||
|
("团队", "CDMO", "能力矩阵"),
|
||||||
|
{
|
||||||
|
"core_question": "对标成熟 CDMO,白帆最需要补齐的是人数、岗位能力,还是 QA/MSAT/工程/项目管理之间的角色分工?",
|
||||||
|
"bold_hypothesis": "初始假设:白帆的能力缺口不是简单扩编,而是商业化 CDMO 所需的角色矩阵尚未完全成型;Phase 2 应验证 QA 独立性、MSAT 工艺支持、工程保障、生产班组和 PMO 协同能力。",
|
||||||
|
"writing_claim": "本章要给出面向商业化的团队能力地图,说明哪些能力必须自建,哪些可外部支持,哪些要通过机制补齐。",
|
||||||
|
"counter_evidence": [
|
||||||
|
"现有人员是否已具备商业化经验,只是材料未体现?",
|
||||||
|
"对标 CDMO 是否会高估当前阶段所需组织复杂度?",
|
||||||
|
"是否可通过顾问、外包或客户支持临时补足能力?",
|
||||||
|
],
|
||||||
|
},
|
||||||
|
),
|
||||||
|
(
|
||||||
|
("整改", "owner", "路线图"),
|
||||||
|
{
|
||||||
|
"core_question": "哪些整改必须立即完成,哪些属于体系补强,哪些是能力建设?每项如何绑定 owner、关闭证据和复核窗口?",
|
||||||
|
"bold_hypothesis": "初始假设:如果整改只按问题清单逐条关闭,会漏掉体系性根因;更有效的路线应分为立即纠偏、90 天体系补强和中长期能力建设三层,并为每层定义关闭证据。",
|
||||||
|
"writing_claim": "本章要把诊断转化为可执行 CAPA 组合,而不是泛泛的改进建议。",
|
||||||
|
"counter_evidence": [
|
||||||
|
"是否已有整改计划足以覆盖 owner、期限、关闭证据和 QA verification?",
|
||||||
|
"部分整改是否应前移或后移,避免资源过载?",
|
||||||
|
"哪些建议若缺少法规证据,不应被列为强制整改?",
|
||||||
|
],
|
||||||
|
},
|
||||||
|
),
|
||||||
|
(
|
||||||
|
("管理层", "CAPA", "总表"),
|
||||||
|
{
|
||||||
|
"core_question": "管理层应通过什么样的 CAPA 总表、法规映射表和复核节奏,持续判断整改是否真正降低风险?",
|
||||||
|
"bold_hypothesis": "初始假设:白帆需要的不只是一次性报告,而是一套管理层可追踪的整改仪表盘;否则 CAPA 关闭会变成文件动作,无法证明风险趋势下降和商业化 readiness 提升。",
|
||||||
|
"writing_claim": "本章要把报告成果固化成管理层治理工具:CAPA 总表、法规映射、证据包和复核节奏。",
|
||||||
|
"counter_evidence": [
|
||||||
|
"现有管理评审或质量例会是否已经能承担这个功能?",
|
||||||
|
"过度表格化是否会增加一线负担而不改善风险?",
|
||||||
|
"哪些指标真正能反映风险降低,而不是制造形式化 KPI?",
|
||||||
|
],
|
||||||
|
},
|
||||||
|
),
|
||||||
|
]
|
||||||
|
for needles, strategy in strategies:
|
||||||
|
if any(needle in title for needle in needles):
|
||||||
|
return strategy
|
||||||
|
return {
|
||||||
|
"core_question": f"本章需要判断“{title}”背后的真实风险、适用边界和整改优先级。",
|
||||||
|
"bold_hypothesis": f"初始假设:{title} 不是孤立问题,而是质量体系、工艺证据或运营机制中的一个可验证缺口;Phase 2 必须用材料原文和外部证据判断其严重度。",
|
||||||
|
"writing_claim": f"本章要把“{title}”转化为可执行的诊断结论和整改要求。",
|
||||||
|
"counter_evidence": [
|
||||||
|
"该问题是否已有充分整改或验证证据?",
|
||||||
|
"是否只是阶段性限制,而非系统性缺口?",
|
||||||
|
"外部标准是否适用于当前业务边界?",
|
||||||
|
],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def build_chapter_planning(
|
||||||
|
project_root: Path,
|
||||||
|
manifest: dict[str, Any],
|
||||||
|
method: ResearchMethod,
|
||||||
|
titles: list[str],
|
||||||
|
*,
|
||||||
|
quota: int,
|
||||||
|
) -> list[dict[str, Any]]:
|
||||||
|
"""Build hypothesis-driven chapter plans that become Phase 2 prompt context."""
|
||||||
|
lanes = list(method.integrated_lanes or method.task_axes)
|
||||||
|
minimum_evidence = _minimum_evidence_for_method(method)
|
||||||
|
plans: list[dict[str, Any]] = []
|
||||||
|
for idx, title in enumerate(titles, start=1):
|
||||||
|
chapter_id = f"ch{idx:02d}"
|
||||||
|
material_lines = _material_lines_for_chapter(project_root, manifest, title)
|
||||||
|
if not material_lines:
|
||||||
|
material_lines = ["未在材料中自动匹配到足够线索;Phase 2 必须先回读全部输入材料并补充原文摘录。"]
|
||||||
|
strategy = _strategy_for_chapter(title, method)
|
||||||
|
core_question = strategy["core_question"]
|
||||||
|
bold_hypothesis = strategy["bold_hypothesis"]
|
||||||
|
verification_plan = [
|
||||||
|
"先从允许的本地材料提取 2-4 条原文证据,保留出处和上下文。",
|
||||||
|
f"再按方法论 evidence lanes 求证:{';'.join(lanes)}。",
|
||||||
|
"每个核心判断至少匹配 2 个独立高质量来源;不足时降级为待验证判断。",
|
||||||
|
"主动搜索反方证据、低严重度解释、适用范围限制或替代原因。",
|
||||||
|
"输出时把证据、判断、整改/建议和待补证据分开,避免直接写成散文化正文。",
|
||||||
|
]
|
||||||
|
counter_evidence = strategy["counter_evidence"]
|
||||||
|
writing_claim = strategy["writing_claim"]
|
||||||
|
phase2_prompt_context = "\n".join(
|
||||||
|
[
|
||||||
|
f"章节:{chapter_id} {title}",
|
||||||
|
core_question,
|
||||||
|
bold_hypothesis,
|
||||||
|
"材料起点:",
|
||||||
|
*[f"- {line}" for line in material_lines],
|
||||||
|
"求证路线:",
|
||||||
|
*[f"- {item}" for item in verification_plan],
|
||||||
|
"必须寻找的反方/边界:",
|
||||||
|
*[f"- {item}" for item in counter_evidence],
|
||||||
|
f"写作主张:{writing_claim}",
|
||||||
|
f"最低证据要求:{json.dumps(minimum_evidence, ensure_ascii=False)}",
|
||||||
|
]
|
||||||
|
)
|
||||||
|
plans.append(
|
||||||
|
{
|
||||||
|
"chapter_id": chapter_id,
|
||||||
|
"title": title,
|
||||||
|
"suggested_words": quota,
|
||||||
|
"core_question": core_question,
|
||||||
|
"bold_hypothesis": bold_hypothesis,
|
||||||
|
"why_this_matters": "本章用于把 Phase1 的判断转化为 Phase2 可验证命题,并为最终报告保留清晰主线。",
|
||||||
|
"material_starting_points": material_lines,
|
||||||
|
"evidence_lanes": lanes,
|
||||||
|
"verification_plan": verification_plan,
|
||||||
|
"counter_evidence_to_seek": counter_evidence,
|
||||||
|
"writing_claim": writing_claim,
|
||||||
|
"minimum_evidence": minimum_evidence,
|
||||||
|
"phase2_prompt_context": phase2_prompt_context,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return plans
|
||||||
|
|
||||||
|
|
||||||
def build_research_brief_payload(
|
def build_research_brief_payload(
|
||||||
project_root: Path,
|
project_root: Path,
|
||||||
manifest: dict[str, Any],
|
manifest: dict[str, Any],
|
||||||
method: ResearchMethod,
|
method: ResearchMethod,
|
||||||
|
chapter_planning: list[dict[str, Any]] | None = None,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
"""Create the file-backed Phase 1 research brief used by task-card generation."""
|
"""Create the file-backed Phase 1 research brief used by task-card generation."""
|
||||||
axes = list(method.task_axes)
|
axes = list(method.task_axes)
|
||||||
|
chapter_planning = chapter_planning or []
|
||||||
return {
|
return {
|
||||||
"version": "0.21-alpha",
|
"version": "0.21-alpha",
|
||||||
"topic": manifest.get("topic", project_root.name),
|
"topic": manifest.get("topic", project_root.name),
|
||||||
@@ -228,6 +581,13 @@ def build_research_brief_payload(
|
|||||||
"work_language": "zh",
|
"work_language": "zh",
|
||||||
"tone": "事实型、整改导向、面向管理层和质量/生产负责人;避免空泛咨询腔。",
|
"tone": "事实型、整改导向、面向管理层和质量/生产负责人;避免空泛咨询腔。",
|
||||||
"central_question": f"如何基于已提供材料和权威法规/最佳实践,系统诊断“{manifest.get('topic', project_root.name)}”并形成可执行整改路线图?",
|
"central_question": f"如何基于已提供材料和权威法规/最佳实践,系统诊断“{manifest.get('topic', project_root.name)}”并形成可执行整改路线图?",
|
||||||
|
"central_thesis": _central_thesis(manifest, method),
|
||||||
|
"phase_logic": {
|
||||||
|
"phase1": "大胆假设:结合输入材料、访谈信息和初步搜索,定下主基调、章节命题和求证路线。",
|
||||||
|
"phase2": "小心求证:worker 只围绕 Phase1 命题收集、验证、证伪和补证,不自行重写研究方向。",
|
||||||
|
"phase3": "一致性审校:检查 Phase1 假设与 Phase2 证据是否自洽,指出需要回炉的章节或证据缺口。",
|
||||||
|
},
|
||||||
|
"phase2_mode": "chapter_integrated",
|
||||||
"success_criteria": [
|
"success_criteria": [
|
||||||
"每个核心判断都能回到用户材料、权威法规、最佳实践或反方证据。",
|
"每个核心判断都能回到用户材料、权威法规、最佳实践或反方证据。",
|
||||||
"短中长期整改建议必须绑定优先级、责任、关闭证据和复核机制。",
|
"短中长期整改建议必须绑定优先级、责任、关闭证据和复核机制。",
|
||||||
@@ -239,8 +599,10 @@ def build_research_brief_payload(
|
|||||||
"research_brief_path": "phase1/research_brief.json",
|
"research_brief_path": "phase1/research_brief.json",
|
||||||
},
|
},
|
||||||
"materials": _material_paths(manifest),
|
"materials": _material_paths(manifest),
|
||||||
|
"chapter_planning": chapter_planning,
|
||||||
"task_planning": {
|
"task_planning": {
|
||||||
"chapter_source": "phase1/framework.md",
|
"chapter_source": "phase1/framework.md",
|
||||||
|
"phase2_mode": "chapter_integrated",
|
||||||
"axes": axes,
|
"axes": axes,
|
||||||
"required_skills": [
|
"required_skills": [
|
||||||
"deep-research",
|
"deep-research",
|
||||||
@@ -275,29 +637,62 @@ def write_research_brief(
|
|||||||
project_root: Path,
|
project_root: Path,
|
||||||
manifest: dict[str, Any] | None = None,
|
manifest: dict[str, Any] | None = None,
|
||||||
method: ResearchMethod | None = None,
|
method: ResearchMethod | None = None,
|
||||||
|
chapter_planning: list[dict[str, Any]] | None = None,
|
||||||
) -> tuple[Path, Path]:
|
) -> tuple[Path, Path]:
|
||||||
manifest = manifest or load_manifest(project_root)
|
manifest = manifest or load_manifest(project_root)
|
||||||
method = method or ResearchMethodRegistry().get(manifest.get("research_method"))
|
method = method or ResearchMethodRegistry().get(manifest.get("research_method"))
|
||||||
payload = build_research_brief_payload(project_root, manifest, method)
|
payload = build_research_brief_payload(project_root, manifest, method, chapter_planning=chapter_planning)
|
||||||
json_path = project_root / "phase1" / "research_brief.json"
|
json_path = project_root / "phase1" / "research_brief.json"
|
||||||
md_path = project_root / "phase1" / "research_brief.md"
|
md_path = project_root / "phase1" / "research_brief.md"
|
||||||
json_path.parent.mkdir(parents=True, exist_ok=True)
|
json_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
hypothesis_path = project_root / "phase1" / "hypothesis_map.json"
|
||||||
json_path.write_text(json.dumps(payload, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
json_path.write_text(json.dumps(payload, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||||
|
hypothesis_path.write_text(json.dumps(payload.get("chapter_planning") or [], ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||||
lines = [
|
lines = [
|
||||||
f"# Phase 1 Research Brief:{payload['topic']}",
|
f"# Phase 1 Research Brief:{payload['topic']}",
|
||||||
"",
|
"",
|
||||||
f"- research_method: {payload['research_method']}",
|
f"- research_method: {payload['research_method']}",
|
||||||
f"- work_language: {payload['work_language']}",
|
f"- work_language: {payload['work_language']}",
|
||||||
f"- tone: {payload['tone']}",
|
f"- tone: {payload['tone']}",
|
||||||
|
f"- phase2_mode: {payload['phase2_mode']}",
|
||||||
"",
|
"",
|
||||||
"## 中心问题",
|
"## 中心问题",
|
||||||
"",
|
"",
|
||||||
payload["central_question"],
|
payload["central_question"],
|
||||||
"",
|
"",
|
||||||
"## 成功标准",
|
"## 主基调 / 大胆假设",
|
||||||
|
"",
|
||||||
|
payload["central_thesis"],
|
||||||
|
"",
|
||||||
|
"## Phase 逻辑",
|
||||||
"",
|
"",
|
||||||
]
|
]
|
||||||
|
for phase_name, phase_text in payload["phase_logic"].items():
|
||||||
|
lines.append(f"- `{phase_name}`:{phase_text}")
|
||||||
|
lines.extend([
|
||||||
|
"",
|
||||||
|
"## 成功标准",
|
||||||
|
"",
|
||||||
|
])
|
||||||
lines.extend(f"- {item}" for item in payload["success_criteria"])
|
lines.extend(f"- {item}" for item in payload["success_criteria"])
|
||||||
|
if payload.get("chapter_planning"):
|
||||||
|
lines.extend(["", "## 章节命题与求证计划", ""])
|
||||||
|
for item in payload["chapter_planning"]:
|
||||||
|
lines.extend(
|
||||||
|
[
|
||||||
|
f"### {item['chapter_id']} {item['title']}",
|
||||||
|
"",
|
||||||
|
f"- 核心问题:{item['core_question']}",
|
||||||
|
f"- 大胆假设:{item['bold_hypothesis']}",
|
||||||
|
f"- 写作主张:{item['writing_claim']}",
|
||||||
|
f"- 证据线:{';'.join(item['evidence_lanes'])}",
|
||||||
|
"- 材料起点:",
|
||||||
|
]
|
||||||
|
)
|
||||||
|
lines.extend(f" - {line}" for line in item["material_starting_points"])
|
||||||
|
lines.extend(["- 求证计划:"])
|
||||||
|
lines.extend(f" - {line}" for line in item["verification_plan"])
|
||||||
|
lines.extend([""])
|
||||||
lines.extend(["", "## 任务切分原则", ""])
|
lines.extend(["", "## 任务切分原则", ""])
|
||||||
planning = payload["task_planning"]
|
planning = payload["task_planning"]
|
||||||
lines.append(planning["fragmentation_guard"])
|
lines.append(planning["fragmentation_guard"])
|
||||||
@@ -377,30 +772,49 @@ CHAPTER_TEMPLATES: dict[str, list[str]] = {
|
|||||||
"落地机制决定咨询建议能否转化为成果",
|
"落地机制决定咨询建议能否转化为成果",
|
||||||
],
|
],
|
||||||
"gmp_quality_operations_diagnosis": [
|
"gmp_quality_operations_diagnosis": [
|
||||||
"现场审计发现需要先转化为可验证的系统性问题图谱",
|
"从审计清单到商业化阶段门",
|
||||||
"法规基线决定质量体系差距的严重度与整改边界",
|
"用法规基线重新校准整改优先级",
|
||||||
"生产工艺体系风险来自流程、设施、公用系统和验证证据的耦合缺口",
|
"制剂无菌保障:从硬件合规到行为受控",
|
||||||
"偏差、变更、CAPA 和数据完整性决定质量系统能否闭环",
|
"原液与公用系统:封闭工艺背后的证据缺口",
|
||||||
"人员能力与质量文化决定制度是否真正落地",
|
"工艺文件与验证:商业化转移的硬门槛",
|
||||||
"运营管理问题需要区分组织、流程、会议机制和指标体系缺口",
|
"质量系统闭环:偏差、变更、CAPA 与数据完整性",
|
||||||
"跨部门协同断点会放大 GMP 风险和交付风险",
|
"人员能力:培训有效性比培训记录更关键",
|
||||||
"标杆实践应转化为短中长期整改组合而非口号",
|
"运营节奏:从临时协调转向管理系统",
|
||||||
"整改路线图必须绑定责任、优先级、证据和复核机制",
|
"团队建设:按 CDMO 能力矩阵补齐角色",
|
||||||
"管理层治理机制决定白帆能否从一次整改转向持续改进",
|
"整改路线图:立即纠偏、体系补强、能力建设",
|
||||||
|
"管理层看板:用 CAPA 总表驱动复核",
|
||||||
],
|
],
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
def render_framework(project_root: Path, *, method_key: str | None = None, chapter_count: int = 10) -> Path:
|
def _existing_chapter_titles(project_root: Path) -> list[str]:
|
||||||
|
framework_path = project_root / "phase1" / "framework.md"
|
||||||
|
if not framework_path.exists():
|
||||||
|
return []
|
||||||
|
from scripts.runtime.tasks import parse_framework_chapters
|
||||||
|
|
||||||
|
chapters = parse_framework_chapters(framework_path.read_text(encoding="utf-8"))
|
||||||
|
return [chapter.title for chapter in chapters if chapter.title]
|
||||||
|
|
||||||
|
|
||||||
|
def render_framework(
|
||||||
|
project_root: Path,
|
||||||
|
*,
|
||||||
|
method_key: str | None = None,
|
||||||
|
chapter_count: int = 10,
|
||||||
|
preserve_existing_outline: bool = False,
|
||||||
|
) -> Path:
|
||||||
manifest = load_manifest(project_root)
|
manifest = load_manifest(project_root)
|
||||||
registry = ResearchMethodRegistry()
|
registry = ResearchMethodRegistry()
|
||||||
method = registry.get(method_key or manifest.get("research_method"))
|
method = registry.get(method_key or manifest.get("research_method"))
|
||||||
if method_key:
|
if method_key:
|
||||||
manifest["research_method"] = method.key
|
manifest["research_method"] = method.key
|
||||||
titles = CHAPTER_TEMPLATES.get(method.key) or CHAPTER_TEMPLATES["mckinsey_market"]
|
existing_titles = _existing_chapter_titles(project_root) if preserve_existing_outline else []
|
||||||
|
titles = existing_titles or CHAPTER_TEMPLATES.get(method.key) or CHAPTER_TEMPLATES["mckinsey_market"]
|
||||||
chapter_count = max(8, min(15, chapter_count))
|
chapter_count = max(8, min(15, chapter_count))
|
||||||
selected = titles[:chapter_count]
|
selected = titles[:chapter_count] if not existing_titles else titles
|
||||||
quota = max(800, int(manifest.get("target_words", 30000)) // len(selected))
|
quota = max(800, int(manifest.get("target_words", 30000)) // len(selected))
|
||||||
|
chapter_planning = build_chapter_planning(project_root, manifest, method, selected, quota=quota)
|
||||||
sections = "\n".join(f"- {item}" for item in method.framework_sections)
|
sections = "\n".join(f"- {item}" for item in method.framework_sections)
|
||||||
axes = "、".join(method.task_axes)
|
axes = "、".join(method.task_axes)
|
||||||
material_text = render_material_inventory(manifest.get("material_inventory") or [])
|
material_text = render_material_inventory(manifest.get("material_inventory") or [])
|
||||||
@@ -430,17 +844,32 @@ def render_framework(project_root: Path, *, method_key: str | None = None, chapt
|
|||||||
"",
|
"",
|
||||||
"## 中心假设",
|
"## 中心假设",
|
||||||
"",
|
"",
|
||||||
f"围绕“{manifest['topic']}”形成可被证据支持或证伪的中文主线;所有核心判断必须绑定来源 ID。",
|
_central_thesis(manifest, method),
|
||||||
|
"",
|
||||||
|
"Phase1 的职责是大胆假设:基于材料、访谈和初步搜索定下主基调、章节命题和求证路线。Phase2 的职责是小心求证:验证、证伪、补证,而不是重新发明报告方向。Phase3 则检查 Phase1 假设与 Phase2 证据是否自洽。",
|
||||||
"",
|
"",
|
||||||
]
|
]
|
||||||
for idx, title in enumerate(selected, start=1):
|
for item in chapter_planning:
|
||||||
lines.extend(
|
lines.extend(
|
||||||
[
|
[
|
||||||
f"## 第{idx}章 {title}",
|
f"## 第{int(item['chapter_id'][2:])}章 {item['title']}",
|
||||||
"",
|
"",
|
||||||
f"建议字数:约 {quota} 字。",
|
f"建议字数:约 {item['suggested_words']} 字。",
|
||||||
f"研究思路:围绕 `{method.key}` 的方法框架,从 {axes} 等任务轴并发收集 evidence packet,再由 chapter assembly 收束为完整中文章节。",
|
f"本章要解决的问题:{item['core_question']}",
|
||||||
"证据要求:至少 2 个独立 Tier 1-2 信源;不足时在正文标注待验证;必须包含反方证据。",
|
f"大胆假设:{item['bold_hypothesis']}",
|
||||||
|
f"写作主张:{item['writing_claim']}",
|
||||||
|
f"证据线:{';'.join(item['evidence_lanes'])}",
|
||||||
|
"",
|
||||||
|
"材料起点:",
|
||||||
|
*[f"- {line}" for line in item["material_starting_points"]],
|
||||||
|
"",
|
||||||
|
"求证计划:",
|
||||||
|
*[f"- {line}" for line in item["verification_plan"]],
|
||||||
|
"",
|
||||||
|
"必须寻找的反方/边界:",
|
||||||
|
*[f"- {line}" for line in item["counter_evidence_to_seek"]],
|
||||||
|
"",
|
||||||
|
f"最低证据要求:`{json.dumps(item['minimum_evidence'], ensure_ascii=False)}`",
|
||||||
"",
|
"",
|
||||||
]
|
]
|
||||||
)
|
)
|
||||||
@@ -457,7 +886,7 @@ def render_framework(project_root: Path, *, method_key: str | None = None, chapt
|
|||||||
out = project_root / "phase1" / "framework.md"
|
out = project_root / "phase1" / "framework.md"
|
||||||
out.parent.mkdir(parents=True, exist_ok=True)
|
out.parent.mkdir(parents=True, exist_ok=True)
|
||||||
out.write_text("\n".join(lines), encoding="utf-8")
|
out.write_text("\n".join(lines), encoding="utf-8")
|
||||||
research_brief_md, research_brief_json = write_research_brief(project_root, manifest, method)
|
research_brief_md, research_brief_json = write_research_brief(project_root, manifest, method, chapter_planning=chapter_planning)
|
||||||
manifest["phase1"] = {
|
manifest["phase1"] = {
|
||||||
"status": "completed",
|
"status": "completed",
|
||||||
"approved": False,
|
"approved": False,
|
||||||
|
|||||||
+172
-1
@@ -1,4 +1,4 @@
|
|||||||
"""Deterministic Phase 3 review checks for the Python core."""
|
"""Phase 3 review checks for the Python core."""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
@@ -52,6 +52,24 @@ def _ready_packet_stems(project_root: Path) -> set[str]:
|
|||||||
return ready
|
return ready
|
||||||
|
|
||||||
|
|
||||||
|
def _read_text_if_exists(path: Path, *, max_chars: int | None = None) -> str:
|
||||||
|
if not path.exists():
|
||||||
|
return ""
|
||||||
|
text = path.read_text(encoding="utf-8", errors="ignore")
|
||||||
|
return text[:max_chars] if max_chars is not None else text
|
||||||
|
|
||||||
|
|
||||||
|
def _json_if_exists(path: Path, *, max_chars: int | None = None) -> str:
|
||||||
|
if not path.exists():
|
||||||
|
return ""
|
||||||
|
try:
|
||||||
|
data = json.loads(path.read_text(encoding="utf-8"))
|
||||||
|
text = json.dumps(data, ensure_ascii=False, indent=2)
|
||||||
|
except Exception:
|
||||||
|
text = path.read_text(encoding="utf-8", errors="ignore")
|
||||||
|
return text[:max_chars] if max_chars is not None else text
|
||||||
|
|
||||||
|
|
||||||
def _draft_quality_findings(drafts: list[Path]) -> list[dict[str, Any]]:
|
def _draft_quality_findings(drafts: list[Path]) -> list[dict[str, Any]]:
|
||||||
findings: list[dict[str, Any]] = []
|
findings: list[dict[str, Any]] = []
|
||||||
generic_markers = [
|
generic_markers = [
|
||||||
@@ -77,6 +95,159 @@ def _draft_quality_findings(drafts: list[Path]) -> list[dict[str, Any]]:
|
|||||||
return findings
|
return findings
|
||||||
|
|
||||||
|
|
||||||
|
def build_phase3_model_review_context(project_root: Path, *, max_chars: int = 650_000) -> str:
|
||||||
|
"""Build a structured, bounded context packet for an independent model review."""
|
||||||
|
deterministic_path = build_phase3_critique(project_root)
|
||||||
|
deterministic_copy = project_root / "phase3" / "critique_deterministic.md"
|
||||||
|
deterministic_copy.write_text(deterministic_path.read_text(encoding="utf-8"), encoding="utf-8")
|
||||||
|
|
||||||
|
manifest = load_manifest(project_root)
|
||||||
|
parts: list[str] = [
|
||||||
|
f"# Phase 3 Model Review Context: {manifest.get('topic', project_root.name)}",
|
||||||
|
"",
|
||||||
|
"## Review Contract",
|
||||||
|
"",
|
||||||
|
"- 这是给非 Codex 模型的独立总编审校上下文,不要求重写正文。",
|
||||||
|
"- 请判断 Phase2 草稿能否进入 Phase4,或必须回炉补证据/重写。",
|
||||||
|
"- 重点关注:证据是否落纸面、并发 packet 是否造成碎片化、法规/最佳实践覆盖是否足够、整改建议是否具体可执行。",
|
||||||
|
"",
|
||||||
|
"## Manifest",
|
||||||
|
"",
|
||||||
|
"```json",
|
||||||
|
json.dumps(manifest, ensure_ascii=False, indent=2),
|
||||||
|
"```",
|
||||||
|
"",
|
||||||
|
"## Deterministic Review Baseline",
|
||||||
|
"",
|
||||||
|
_read_text_if_exists(deterministic_copy),
|
||||||
|
"",
|
||||||
|
"## Phase 1 Framework",
|
||||||
|
"",
|
||||||
|
_read_text_if_exists(project_root / "phase1" / "framework.md", max_chars=50_000),
|
||||||
|
"",
|
||||||
|
"## Phase 1 Research Brief",
|
||||||
|
"",
|
||||||
|
_read_text_if_exists(project_root / "phase1" / "research_brief.md", max_chars=30_000),
|
||||||
|
"",
|
||||||
|
"## Phase 2 Brief Warnings",
|
||||||
|
"",
|
||||||
|
_json_if_exists(project_root / "phase2" / "brief_warnings.json", max_chars=30_000) or "无",
|
||||||
|
"",
|
||||||
|
"## Phase 2 Packet Errors",
|
||||||
|
"",
|
||||||
|
]
|
||||||
|
errors = sorted((project_root / "phase2" / "packet_errors").glob("*.json"))
|
||||||
|
if errors:
|
||||||
|
for path in errors[:40]:
|
||||||
|
parts.extend([f"### {path.name}", "", _json_if_exists(path, max_chars=2_000), ""])
|
||||||
|
else:
|
||||||
|
parts.append("无")
|
||||||
|
|
||||||
|
parts.extend(["", "## Source Registry Summary", ""])
|
||||||
|
source_lines = []
|
||||||
|
sources_path = project_root / "phase2" / "sources.jsonl"
|
||||||
|
if sources_path.exists():
|
||||||
|
for line in sources_path.read_text(encoding="utf-8").splitlines()[:260]:
|
||||||
|
if not line.strip():
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
source = json.loads(line)
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
continue
|
||||||
|
source_lines.append(
|
||||||
|
"- {id} | {tier} | {title} | {url} | cached={cached}".format(
|
||||||
|
id=source.get("id", ""),
|
||||||
|
tier=source.get("tier", ""),
|
||||||
|
title=str(source.get("title", ""))[:120],
|
||||||
|
url=source.get("url", ""),
|
||||||
|
cached=source.get("cached_text_path", ""),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
parts.append("\n".join(source_lines) or "无")
|
||||||
|
|
||||||
|
parts.extend(["", "## Compressed Findings", ""])
|
||||||
|
for path in sorted((project_root / "phase2" / "compressed_findings").glob("ch*.json")):
|
||||||
|
parts.extend([f"### {path.name}", "", "```json", _json_if_exists(path, max_chars=35_000), "```", ""])
|
||||||
|
|
||||||
|
parts.extend(["", "## Chapter Drafts", ""])
|
||||||
|
for path in sorted((project_root / "phase2" / "drafts").glob("ch*.md")):
|
||||||
|
parts.extend([f"### {path.name}", "", _read_text_if_exists(path, max_chars=55_000), ""])
|
||||||
|
|
||||||
|
context = "\n".join(parts)
|
||||||
|
if len(context) > max_chars:
|
||||||
|
context = context[:max_chars] + "\n\n[Context truncated by max_chars; review should flag if truncation limits confidence.]\n"
|
||||||
|
|
||||||
|
out = project_root / "phase3" / "review_context_opus_4_7.md"
|
||||||
|
out.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
out.write_text(context, encoding="utf-8")
|
||||||
|
return context
|
||||||
|
|
||||||
|
|
||||||
|
def phase3_model_review_system_prompt() -> str:
|
||||||
|
return (
|
||||||
|
"你是 Deep Research Phase 3 的独立总编审校模型,本次由 ZenMux Claude Opus 4.7 执行,用于避免 Codex/OpenAI 模型偏见。\n"
|
||||||
|
"你的任务是审校,不是润色或重写。必须用中文输出,英文仅可保留 source title、URL、法规缩写和原文短摘录。\n"
|
||||||
|
"请严格检查:1) 研究目标与 Phase1 框架是否契合;2) Phase2 并发 evidence packets 是否被章节真正吸收,还是造成碎片化;"
|
||||||
|
"3) FDA/NMPA/EMA/ICH/WHO/EU GMP 等权威来源是否足以支撑关键判断;4) 用户材料是否被正确作为起点且被权威来源交叉验证;"
|
||||||
|
"5) 运营管理与团队能力章节是否具体,不得泛泛咨询腔;6) CAPA 建议是否包含 owner、期限、关闭证据、QA verification、复核窗口和升级阈值;"
|
||||||
|
"7) 引用链和 source_id 是否可追踪;8) 是否仍有明显 AI 味、中英文混杂或空泛表达。\n\n"
|
||||||
|
"输出必须使用以下 Markdown 结构:\n"
|
||||||
|
"# Phase 3 Opus 4.7 独立审校\n"
|
||||||
|
"## 总体判定\n"
|
||||||
|
"给出:通过 / 有条件通过 / 回炉 Phase2,并说明最核心理由。\n"
|
||||||
|
"## P0/P1 阻断问题\n"
|
||||||
|
"列出必须修复的问题;每条写明章节/文件、问题、为什么阻断、建议动作。\n"
|
||||||
|
"## 章节级审校表\n"
|
||||||
|
"用表格覆盖 ch01-ch11:主线质量、证据密度、法规覆盖、整改可执行性、是否需要回炉。\n"
|
||||||
|
"## 证据与信源质量\n"
|
||||||
|
"单独评价 FDA warning letters、ICH Q9/Q10、EU GMP Annex 1、本地缓存信源、第三方低质信源的使用情况。\n"
|
||||||
|
"## 碎片化与叙事连贯性\n"
|
||||||
|
"判断并发研究是否造成割裂,并给出具体整合建议。\n"
|
||||||
|
"## Phase2 回炉任务清单\n"
|
||||||
|
"如果需要回炉,列出可执行任务卡级别的补证据/重写要求。\n"
|
||||||
|
"## Phase4 准入条件\n"
|
||||||
|
"明确进入 final 前必须满足的条件。\n"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def build_phase3_model_critique(
|
||||||
|
project_root: Path,
|
||||||
|
*,
|
||||||
|
client: Any,
|
||||||
|
model: str = "zenmux-anthropic/claude-opus-4-7",
|
||||||
|
max_context_chars: int = 650_000,
|
||||||
|
) -> Path:
|
||||||
|
context = build_phase3_model_review_context(project_root, max_chars=max_context_chars)
|
||||||
|
content = client.chat_complete(
|
||||||
|
model=model,
|
||||||
|
system=phase3_model_review_system_prompt(),
|
||||||
|
user=context,
|
||||||
|
temperature=0.2,
|
||||||
|
max_tokens=20_000,
|
||||||
|
tag="phase3:opus-review",
|
||||||
|
)
|
||||||
|
out = project_root / "phase3" / "critique.md"
|
||||||
|
out.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
out.write_text(content.rstrip() + "\n", encoding="utf-8")
|
||||||
|
|
||||||
|
manifest = load_manifest(project_root)
|
||||||
|
phase3 = manifest.setdefault("phase3", {})
|
||||||
|
phase3.update(
|
||||||
|
{
|
||||||
|
"status": "completed",
|
||||||
|
"review_mode": "model",
|
||||||
|
"review_model": model,
|
||||||
|
"critique_path": "phase3/critique.md",
|
||||||
|
"context_path": "phase3/review_context_opus_4_7.md",
|
||||||
|
"deterministic_critique_path": "phase3/critique_deterministic.md",
|
||||||
|
"updated_at": utc_now_iso(),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
manifest["updated_at"] = utc_now_iso()
|
||||||
|
write_manifest(project_root, manifest)
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
def build_phase3_critique(project_root: Path) -> Path:
|
def build_phase3_critique(project_root: Path) -> Path:
|
||||||
manifest = load_manifest(project_root)
|
manifest = load_manifest(project_root)
|
||||||
drafts = sorted((project_root / "phase2" / "drafts").glob("ch*.md"))
|
drafts = sorted((project_root / "phase2" / "drafts").glob("ch*.md"))
|
||||||
|
|||||||
@@ -59,6 +59,42 @@ ROLE_DEFAULTS = {
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
ROLE_IDENTITIES = {
|
||||||
|
"dr_plan": (
|
||||||
|
"你是 Deep Research 的 Phase1 研究架构师。你的工作不是列目录,而是先消化材料、访谈和初步搜索,"
|
||||||
|
"形成可被证伪的主判断、章节命题和求证路线。你要大胆假设,但必须给 Phase2 留下清晰的验证和推翻条件。"
|
||||||
|
),
|
||||||
|
"dr_pm": (
|
||||||
|
"你是 Deep Research 的研究项目经理。你的职责是把研究意图转化为可并发执行、可回收校验的任务,"
|
||||||
|
"控制碎片化、重复检索和上下文污染。"
|
||||||
|
),
|
||||||
|
"dr_searcher": (
|
||||||
|
"你是 Deep Research 的信源发现员。你的职责是用短英文关键词和轴向词找到高质量入口,"
|
||||||
|
"优先官方、法规、学术和一手材料;你不写结论,只交付可追溯来源。"
|
||||||
|
),
|
||||||
|
"dr_analyst": (
|
||||||
|
"你是 Deep Research 的章节证据分析师。你的职责不是写一篇像样的空泛文章,而是围绕 Phase1 命题"
|
||||||
|
"小心求证:提取材料原文、检索权威证据、寻找反方边界,并把证据整理成可审计的结构化 packet。"
|
||||||
|
),
|
||||||
|
"dr_verifier": (
|
||||||
|
"你是 Deep Research 的独立反方审校员。你的默认姿态是质疑:找证据缺口、适用边界、反例和过度推断,"
|
||||||
|
"并指出哪些结论必须降级或回炉。"
|
||||||
|
),
|
||||||
|
"dr_chief_editor": (
|
||||||
|
"你是 Deep Research 的 Phase3 总编审校。你的职责是通读 Phase1 假设与 Phase2 证据,判断二者是否自洽,"
|
||||||
|
"优先指出结构性失败、证据不足和需要回炉的章节。"
|
||||||
|
),
|
||||||
|
"dr_editor_in_chief": (
|
||||||
|
"你是 Deep Research 的终稿主编。你的职责是把已验证证据组织成客户可读的中文报告,"
|
||||||
|
"保持观点清晰、证据密实、表达克制,避免翻译腔和 AI 味。"
|
||||||
|
),
|
||||||
|
"dr_reporter": (
|
||||||
|
"你是 Deep Research 的报告制作负责人。你的职责是把已定稿内容可靠渲染为 PDF/DOCX,"
|
||||||
|
"确保引用、排版、中文字体、表格和输出卫生可交付。"
|
||||||
|
),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class RoleDefinition:
|
class RoleDefinition:
|
||||||
name: str
|
name: str
|
||||||
@@ -67,6 +103,7 @@ class RoleDefinition:
|
|||||||
temperature: float
|
temperature: float
|
||||||
max_tokens: int
|
max_tokens: int
|
||||||
max_concurrency: int
|
max_concurrency: int
|
||||||
|
identity: str = ""
|
||||||
|
|
||||||
|
|
||||||
class RuntimeProfile:
|
class RuntimeProfile:
|
||||||
@@ -103,6 +140,7 @@ def resolve_runtime_profile(
|
|||||||
temperature=float(defaults["temperature"]),
|
temperature=float(defaults["temperature"]),
|
||||||
max_tokens=int(defaults["max_tokens"]),
|
max_tokens=int(defaults["max_tokens"]),
|
||||||
max_concurrency=int(defaults["max_concurrency"]),
|
max_concurrency=int(defaults["max_concurrency"]),
|
||||||
|
identity=ROLE_IDENTITIES.get(name, ""),
|
||||||
)
|
)
|
||||||
return RuntimeProfile(
|
return RuntimeProfile(
|
||||||
profile=resolved["profile"],
|
profile=resolved["profile"],
|
||||||
|
|||||||
@@ -34,9 +34,10 @@ class SkillRegistry:
|
|||||||
self.canonical_dir = canonical_dir or CANONICAL_SKILLS_DIR
|
self.canonical_dir = canonical_dir or CANONICAL_SKILLS_DIR
|
||||||
|
|
||||||
def roots(self) -> list[Path]:
|
def roots(self) -> list[Path]:
|
||||||
roots = [self.canonical_dir]
|
roots = []
|
||||||
if self.canonical_dir == CANONICAL_SKILLS_DIR and PROJECT_SKILLS_DIR.exists():
|
if self.canonical_dir == CANONICAL_SKILLS_DIR and PROJECT_SKILLS_DIR.exists():
|
||||||
roots.append(PROJECT_SKILLS_DIR)
|
roots.append(PROJECT_SKILLS_DIR)
|
||||||
|
roots.append(self.canonical_dir)
|
||||||
return roots
|
return roots
|
||||||
|
|
||||||
def list(self) -> list[SkillInfo]:
|
def list(self) -> list[SkillInfo]:
|
||||||
|
|||||||
@@ -0,0 +1,230 @@
|
|||||||
|
"""Cache important external sources as local Markdown snapshots."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from pathlib import Path
|
||||||
|
from urllib.parse import urlparse
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
from lxml import html
|
||||||
|
|
||||||
|
|
||||||
|
IMPORTANT_DOMAINS = (
|
||||||
|
"fda.gov",
|
||||||
|
"ema.europa.eu",
|
||||||
|
"nmpa.gov.cn",
|
||||||
|
"cde.org.cn",
|
||||||
|
"ich.org",
|
||||||
|
"who.int",
|
||||||
|
"edqm.eu",
|
||||||
|
"pmda.go.jp",
|
||||||
|
"ec.europa.eu",
|
||||||
|
"health.ec.europa.eu",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class CacheResult:
|
||||||
|
source_id: str
|
||||||
|
url: str
|
||||||
|
cached_text_path: str
|
||||||
|
raw_path: str
|
||||||
|
status: str
|
||||||
|
chars: int
|
||||||
|
|
||||||
|
|
||||||
|
def _safe_stem(source: dict) -> str:
|
||||||
|
source_id = str(source.get("id") or "source")
|
||||||
|
digest = hashlib.sha1(str(source.get("url") or source_id).encode("utf-8")).hexdigest()[:10]
|
||||||
|
safe_id = re.sub(r"[^A-Za-z0-9_-]+", "_", source_id).strip("_") or "source"
|
||||||
|
return f"{safe_id}-{digest}"
|
||||||
|
|
||||||
|
|
||||||
|
def _domain(url: str) -> str:
|
||||||
|
return urlparse(url).netloc.lower()
|
||||||
|
|
||||||
|
|
||||||
|
def is_important_source(source: dict) -> bool:
|
||||||
|
url = str(source.get("url") or "")
|
||||||
|
if not url.startswith(("http://", "https://")):
|
||||||
|
return False
|
||||||
|
domain = _domain(url)
|
||||||
|
if any(domain.endswith(item) for item in IMPORTANT_DOMAINS):
|
||||||
|
return True
|
||||||
|
tier = str(source.get("tier") or "").lower()
|
||||||
|
if "tier 1" in tier or tier in {"1", "1.0"}:
|
||||||
|
return True
|
||||||
|
title = str(source.get("title") or "").lower()
|
||||||
|
return any(term in title for term in ("ich q9", "ich q10", "annex 1", "fda guidance", "who guideline"))
|
||||||
|
|
||||||
|
|
||||||
|
def load_sources(path: Path) -> list[dict]:
|
||||||
|
if not path.exists():
|
||||||
|
return []
|
||||||
|
rows: list[dict] = []
|
||||||
|
for line in path.read_text(encoding="utf-8").splitlines():
|
||||||
|
if not line.strip():
|
||||||
|
continue
|
||||||
|
rows.append(json.loads(line))
|
||||||
|
return rows
|
||||||
|
|
||||||
|
|
||||||
|
def write_sources(path: Path, rows: list[dict]) -> None:
|
||||||
|
path.write_text("".join(json.dumps(row, ensure_ascii=False) + "\n" for row in rows), encoding="utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
def _response_ext(url: str, content_type: str) -> str:
|
||||||
|
lowered = url.lower()
|
||||||
|
if "pdf" in content_type or lowered.endswith(".pdf"):
|
||||||
|
return ".pdf"
|
||||||
|
if "html" in content_type or lowered.endswith((".html", ".htm", "/")):
|
||||||
|
return ".html"
|
||||||
|
return ".bin"
|
||||||
|
|
||||||
|
|
||||||
|
def _html_to_text(content: bytes) -> str:
|
||||||
|
doc = html.fromstring(content)
|
||||||
|
for bad in doc.xpath("//script|//style|//noscript"):
|
||||||
|
bad.drop_tree()
|
||||||
|
return "\n".join(line.strip() for line in doc.text_content().splitlines() if line.strip())
|
||||||
|
|
||||||
|
|
||||||
|
def _pdf_to_text(path: Path) -> str:
|
||||||
|
try:
|
||||||
|
import fitz
|
||||||
|
except Exception:
|
||||||
|
return ""
|
||||||
|
doc = fitz.open(path)
|
||||||
|
parts: list[str] = []
|
||||||
|
for index, page in enumerate(doc, start=1):
|
||||||
|
text = page.get_text("text").strip()
|
||||||
|
if text:
|
||||||
|
parts.append(f"## Page {index}\n\n{text}")
|
||||||
|
return "\n\n".join(parts)
|
||||||
|
|
||||||
|
|
||||||
|
def _bytes_to_text(*, raw_path: Path, content: bytes, content_type: str, url: str) -> str:
|
||||||
|
if raw_path.suffix == ".pdf" or "pdf" in content_type or url.lower().endswith(".pdf"):
|
||||||
|
return _pdf_to_text(raw_path)
|
||||||
|
if raw_path.suffix in {".html", ".htm"} or "html" in content_type:
|
||||||
|
return _html_to_text(content)
|
||||||
|
try:
|
||||||
|
return content.decode("utf-8")
|
||||||
|
except UnicodeDecodeError:
|
||||||
|
return content.decode("utf-8", errors="ignore")
|
||||||
|
|
||||||
|
|
||||||
|
def cache_source(
|
||||||
|
project_root: Path,
|
||||||
|
source: dict,
|
||||||
|
*,
|
||||||
|
client: httpx.Client | None = None,
|
||||||
|
force: bool = False,
|
||||||
|
timeout: float = 45.0,
|
||||||
|
) -> CacheResult:
|
||||||
|
url = str(source.get("url") or "")
|
||||||
|
if not url.startswith(("http://", "https://")):
|
||||||
|
raise ValueError(f"source URL is not remote: {url}")
|
||||||
|
cache_dir = project_root / "phase2" / "source_cache"
|
||||||
|
raw_dir = cache_dir / "raw"
|
||||||
|
text_dir = cache_dir / "md"
|
||||||
|
raw_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
text_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
stem = _safe_stem(source)
|
||||||
|
md_path = text_dir / f"{stem}.md"
|
||||||
|
if md_path.exists() and not force:
|
||||||
|
return CacheResult(
|
||||||
|
source_id=str(source.get("id") or ""),
|
||||||
|
url=url,
|
||||||
|
cached_text_path=str(md_path.relative_to(project_root)),
|
||||||
|
raw_path=str(source.get("cached_raw_path") or ""),
|
||||||
|
status="cached",
|
||||||
|
chars=len(md_path.read_text(encoding="utf-8")),
|
||||||
|
)
|
||||||
|
|
||||||
|
owns_client = client is None
|
||||||
|
http = client or httpx.Client(trust_env=False, follow_redirects=True, timeout=timeout)
|
||||||
|
try:
|
||||||
|
response = http.get(url)
|
||||||
|
response.raise_for_status()
|
||||||
|
content_type = response.headers.get("content-type", "").lower()
|
||||||
|
ext = _response_ext(str(response.url), content_type)
|
||||||
|
raw_path = raw_dir / f"{stem}{ext}"
|
||||||
|
raw_path.write_bytes(response.content)
|
||||||
|
text = _bytes_to_text(raw_path=raw_path, content=response.content, content_type=content_type, url=str(response.url))
|
||||||
|
lines = [
|
||||||
|
f"# Source Snapshot: {source.get('title') or source.get('id') or url}",
|
||||||
|
"",
|
||||||
|
f"- source_id: {source.get('id', '')}",
|
||||||
|
f"- original_url: {url}",
|
||||||
|
f"- fetched_url: {response.url}",
|
||||||
|
f"- content_type: {content_type}",
|
||||||
|
f"- raw_path: {raw_path.relative_to(project_root)}",
|
||||||
|
"",
|
||||||
|
"## Extracted Text",
|
||||||
|
"",
|
||||||
|
text.strip() or "[No extractable text. Keep raw file for manual review.]",
|
||||||
|
"",
|
||||||
|
]
|
||||||
|
md_path.write_text("\n".join(lines), encoding="utf-8")
|
||||||
|
return CacheResult(
|
||||||
|
source_id=str(source.get("id") or ""),
|
||||||
|
url=url,
|
||||||
|
cached_text_path=str(md_path.relative_to(project_root)),
|
||||||
|
raw_path=str(raw_path.relative_to(project_root)),
|
||||||
|
status="fetched",
|
||||||
|
chars=len(text),
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
if owns_client:
|
||||||
|
http.close()
|
||||||
|
|
||||||
|
|
||||||
|
def cache_sources(
|
||||||
|
project_root: Path,
|
||||||
|
*,
|
||||||
|
sources_rel: str = "phase2/sources.jsonl",
|
||||||
|
important_only: bool = True,
|
||||||
|
limit: int | None = None,
|
||||||
|
force: bool = False,
|
||||||
|
) -> list[CacheResult]:
|
||||||
|
sources_path = project_root / sources_rel
|
||||||
|
rows = load_sources(sources_path)
|
||||||
|
results: list[CacheResult] = []
|
||||||
|
selected_indexes = [
|
||||||
|
index
|
||||||
|
for index, row in enumerate(rows)
|
||||||
|
if row.get("url")
|
||||||
|
and (not row.get("cached_text_path") or force)
|
||||||
|
and (not important_only or is_important_source(row))
|
||||||
|
]
|
||||||
|
if limit is not None:
|
||||||
|
selected_indexes = selected_indexes[:limit]
|
||||||
|
|
||||||
|
with httpx.Client(trust_env=False, follow_redirects=True, timeout=45.0) as client:
|
||||||
|
for index in selected_indexes:
|
||||||
|
row = rows[index]
|
||||||
|
try:
|
||||||
|
result = cache_source(project_root, row, client=client, force=force)
|
||||||
|
except Exception as exc:
|
||||||
|
row["cache_status"] = "failed"
|
||||||
|
row["cache_error"] = str(exc)[:300]
|
||||||
|
continue
|
||||||
|
row["cached_text_path"] = result.cached_text_path
|
||||||
|
row["cached_raw_path"] = result.raw_path
|
||||||
|
row["cache_status"] = result.status
|
||||||
|
row["cached_text_chars"] = result.chars
|
||||||
|
results.append(result)
|
||||||
|
write_sources(sources_path, rows)
|
||||||
|
manifest = project_root / "phase2" / "source_cache" / "manifest.json"
|
||||||
|
manifest.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
manifest.write_text(
|
||||||
|
json.dumps([result.__dict__ for result in results], ensure_ascii=False, indent=2) + "\n",
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
return results
|
||||||
@@ -8,11 +8,11 @@ from typing import Any
|
|||||||
|
|
||||||
|
|
||||||
def _source_key(source: dict[str, Any]) -> str:
|
def _source_key(source: dict[str, Any]) -> str:
|
||||||
return (source.get("url") or source.get("doi") or source.get("id") or "").strip()
|
return (source.get("id") or source.get("source_id") or source.get("doi") or source.get("url") or "").strip()
|
||||||
|
|
||||||
|
|
||||||
def append_packet_sources(sources_path: Path, packet: dict[str, Any]) -> int:
|
def append_packet_sources(sources_path: Path, packet: dict[str, Any]) -> int:
|
||||||
"""Append packet sources to sources.jsonl, deduping by URL/DOI/id."""
|
"""Append packet sources to sources.jsonl, preserving every citeable source_id."""
|
||||||
sources_path.parent.mkdir(parents=True, exist_ok=True)
|
sources_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
existing: set[str] = set()
|
existing: set[str] = set()
|
||||||
if sources_path.exists():
|
if sources_path.exists():
|
||||||
@@ -37,10 +37,27 @@ def append_packet_sources(sources_path: Path, packet: dict[str, Any]) -> int:
|
|||||||
|
|
||||||
|
|
||||||
def rebuild_sources_from_packets(project_root: Path) -> int:
|
def rebuild_sources_from_packets(project_root: Path) -> int:
|
||||||
"""Rebuild phase2/sources.jsonl from packet-level source metadata."""
|
"""Rebuild phase2/sources.jsonl from packet-level source metadata.
|
||||||
|
|
||||||
|
The registry is keyed by source_id, not URL. Two packet sources may point to
|
||||||
|
the same URL but have different source_ids already cited in drafts; dropping
|
||||||
|
either row would break citation traceability.
|
||||||
|
"""
|
||||||
packets_dir = project_root / "phase2" / "packets"
|
packets_dir = project_root / "phase2" / "packets"
|
||||||
sources_path = project_root / "phase2" / "sources.jsonl"
|
sources_path = project_root / "phase2" / "sources.jsonl"
|
||||||
sources_path.parent.mkdir(parents=True, exist_ok=True)
|
sources_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
existing_by_key: dict[str, dict[str, Any]] = {}
|
||||||
|
if sources_path.exists():
|
||||||
|
for line in sources_path.read_text(encoding="utf-8").splitlines():
|
||||||
|
if not line.strip():
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
row = json.loads(line)
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
continue
|
||||||
|
key = _source_key(row)
|
||||||
|
if key:
|
||||||
|
existing_by_key[key] = row
|
||||||
seen: set[str] = set()
|
seen: set[str] = set()
|
||||||
rows: list[dict[str, Any]] = []
|
rows: list[dict[str, Any]] = []
|
||||||
|
|
||||||
@@ -56,7 +73,8 @@ def rebuild_sources_from_packets(project_root: Path) -> int:
|
|||||||
if not key or key in seen:
|
if not key or key in seen:
|
||||||
continue
|
continue
|
||||||
seen.add(key)
|
seen.add(key)
|
||||||
rows.append(source)
|
previous = existing_by_key.get(key, {})
|
||||||
|
rows.append({**source, **{k: v for k, v in previous.items() if k.startswith("cache") or k.startswith("cached_")}})
|
||||||
|
|
||||||
sources_path.write_text(
|
sources_path.write_text(
|
||||||
"".join(json.dumps(row, ensure_ascii=False) + "\n" for row in rows),
|
"".join(json.dumps(row, ensure_ascii=False) + "\n" for row in rows),
|
||||||
|
|||||||
+154
-33
@@ -11,36 +11,45 @@ from typing import Any
|
|||||||
from scripts.runtime.methods import ResearchMethod
|
from scripts.runtime.methods import ResearchMethod
|
||||||
|
|
||||||
|
|
||||||
VALID_ROUTES = {"general", "scholar", "patents", "news"}
|
VALID_ROUTES = {"general", "evidence", "scholar", "patents", "news", "fda"}
|
||||||
DEFAULT_AXES = ["literature", "regulatory", "patents", "market", "counter"]
|
DEFAULT_AXES = ["literature", "regulatory", "patents", "market", "counter"]
|
||||||
AXIS_ROUTES = {
|
AXIS_ROUTES = {
|
||||||
"literature": ["scholar", "general"],
|
"literature": ["scholar", "evidence", "general"],
|
||||||
"clinical": ["scholar", "general"],
|
"clinical": ["scholar", "evidence", "general"],
|
||||||
"regulatory": ["general", "news"],
|
"regulatory": ["fda", "evidence", "general", "news"],
|
||||||
"patents": ["patents", "general"],
|
"patents": ["patents", "evidence", "general"],
|
||||||
"market": ["news", "general"],
|
"market": ["news", "general"],
|
||||||
"china": ["news", "general"],
|
"china": ["news", "general"],
|
||||||
"counter": ["scholar", "general"],
|
"counter": ["fda", "scholar", "evidence", "general"],
|
||||||
"regulatory_gap": ["general", "news"],
|
"regulatory_gap": ["fda", "evidence", "general", "news"],
|
||||||
"risk_classification": ["general", "scholar"],
|
"risk_classification": ["evidence", "general", "scholar"],
|
||||||
"capa_design": ["general", "news"],
|
"capa_design": ["evidence", "general", "news"],
|
||||||
"ownership_timeline": ["general"],
|
"ownership_timeline": ["general"],
|
||||||
"verification_evidence": ["general", "scholar"],
|
"verification_evidence": ["fda", "evidence", "general", "scholar"],
|
||||||
"process_flow": ["scholar", "general"],
|
"process_flow": ["scholar", "evidence", "general"],
|
||||||
"cqa_cpp": ["scholar", "general"],
|
"cqa_cpp": ["scholar", "evidence", "general"],
|
||||||
"scale_up_risk": ["scholar", "general"],
|
"scale_up_risk": ["scholar", "evidence", "general"],
|
||||||
"control_strategy": ["scholar", "general"],
|
"control_strategy": ["scholar", "evidence", "general"],
|
||||||
"supply_chain": ["news", "general"],
|
"supply_chain": ["news", "general"],
|
||||||
"scientific_rationale": ["scholar", "general"],
|
"scientific_rationale": ["scholar", "evidence", "general"],
|
||||||
"poc_evidence": ["scholar", "general"],
|
"poc_evidence": ["scholar", "evidence", "general"],
|
||||||
"ip_fto": ["patents", "general"],
|
"ip_fto": ["patents", "evidence", "general"],
|
||||||
"development_path": ["scholar", "general"],
|
"development_path": ["scholar", "evidence", "general"],
|
||||||
"commercial_window": ["news", "general"],
|
"commercial_window": ["news", "general"],
|
||||||
"current_state": ["general"],
|
"current_state": ["evidence", "general"],
|
||||||
"capability_gap": ["general"],
|
"capability_gap": ["evidence", "general"],
|
||||||
"operating_model": ["general"],
|
"operating_model": ["evidence", "general"],
|
||||||
"governance": ["general"],
|
"governance": ["evidence", "general"],
|
||||||
"implementation_roadmap": ["general"],
|
"implementation_roadmap": ["evidence", "general"],
|
||||||
|
"nmpa_fda_ema_ich_who_baseline": ["fda", "evidence", "general", "news"],
|
||||||
|
"quality_system_gap": ["fda", "evidence", "general"],
|
||||||
|
"manufacturing_process_risk": ["fda", "scholar", "evidence", "general"],
|
||||||
|
"operations_management_gap": ["fda", "evidence", "general"],
|
||||||
|
"team_capability": ["evidence", "general", "news"],
|
||||||
|
"capa_roadmap": ["fda", "evidence", "general"],
|
||||||
|
"input_material_findings": ["evidence", "general"],
|
||||||
|
"fda_enforcement_precedents": ["fda"],
|
||||||
|
"chapter_integrated": ["fda", "scholar", "evidence", "general"],
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
@@ -60,6 +69,7 @@ class TaskCard:
|
|||||||
questions: list[str]
|
questions: list[str]
|
||||||
search_routes: list[str]
|
search_routes: list[str]
|
||||||
output_packet: str
|
output_packet: str
|
||||||
|
chapter_title: str = ""
|
||||||
preferred_model_role: str = "dr_analyst"
|
preferred_model_role: str = "dr_analyst"
|
||||||
status: str = "pending"
|
status: str = "pending"
|
||||||
dependencies: list[str] = field(default_factory=list)
|
dependencies: list[str] = field(default_factory=list)
|
||||||
@@ -105,12 +115,34 @@ def parse_framework_chapters(framework_text: str) -> list[Chapter]:
|
|||||||
return chapters
|
return chapters
|
||||||
|
|
||||||
|
|
||||||
def _questions_for_axis(chapter: Chapter, axis: str) -> list[str]:
|
def _questions_for_axis(chapter: Chapter, axis: str, method: ResearchMethod | None = None) -> list[str]:
|
||||||
|
if axis == "chapter_integrated":
|
||||||
|
lanes = ";".join(method.integrated_lanes if method else [])
|
||||||
return [
|
return [
|
||||||
|
f"围绕《{chapter.title}》形成章节级综合证据包,不再拆成孤立小轴。",
|
||||||
|
f"必须按当前 research_method 的 evidence lanes 组织证据:{lanes or '本地材料、权威来源、反方证据、可执行建议'}。",
|
||||||
|
"若项目有用户材料,必须先读取本地材料证据并提取原文;再用本方法适用的权威来源交叉验证。",
|
||||||
|
"必须形成:材料/事实基线、外部权威证据、差距或机会判断、反方/限制条件、可执行建议和待补证据。",
|
||||||
|
]
|
||||||
|
questions = [
|
||||||
f"围绕《{chapter.title}》从 {axis} 角度提炼可证伪的核心结论。",
|
f"围绕《{chapter.title}》从 {axis} 角度提炼可证伪的核心结论。",
|
||||||
"至少寻找两个 Tier 1-2 来源支撑主要结论;不足时标注待验证。",
|
"至少寻找两个 Tier 1-2 来源支撑主要结论;不足时标注待验证。",
|
||||||
"主动检索反方证据、限制条件或失败案例。",
|
"主动检索反方证据、限制条件或失败案例。",
|
||||||
]
|
]
|
||||||
|
if axis in {
|
||||||
|
"nmpa_fda_ema_ich_who_baseline",
|
||||||
|
"quality_system_gap",
|
||||||
|
"manufacturing_process_risk",
|
||||||
|
"operations_management_gap",
|
||||||
|
"capa_roadmap",
|
||||||
|
"verification_evidence",
|
||||||
|
"counter",
|
||||||
|
"fda_enforcement_precedents",
|
||||||
|
}:
|
||||||
|
questions.append(
|
||||||
|
"必须检索并优先评估 FDA Warning Letters、inspection/enforcement 页面、会议纪要或 meeting materials,作为 GMP 缺陷严重度和整改优先级的佐证。"
|
||||||
|
)
|
||||||
|
return questions
|
||||||
|
|
||||||
|
|
||||||
def _default_required_skills(axis: str) -> list[str]:
|
def _default_required_skills(axis: str) -> list[str]:
|
||||||
@@ -121,18 +153,37 @@ def _default_required_skills(axis: str) -> list[str]:
|
|||||||
|
|
||||||
|
|
||||||
def _default_expected_evidence(axis: str) -> dict[str, Any]:
|
def _default_expected_evidence(axis: str) -> dict[str, Any]:
|
||||||
return {
|
expected = {
|
||||||
"min_tier_1_2_sources": 2,
|
"min_tier_1_2_sources": 2,
|
||||||
"must_include_counter_evidence": True,
|
"must_include_counter_evidence": True,
|
||||||
"must_include_source_metadata": True,
|
"must_include_source_metadata": True,
|
||||||
"preferred_evidence_types": [
|
"preferred_evidence_types": [
|
||||||
"regulatory_or_best_practice_requirement",
|
"regulatory_or_best_practice_requirement",
|
||||||
|
"fda_warning_letter_or_meeting_record",
|
||||||
"site_or_material_finding",
|
"site_or_material_finding",
|
||||||
"quantitative_fact_or_record",
|
"quantitative_fact_or_record",
|
||||||
"implementation_or_verification_evidence",
|
"implementation_or_verification_evidence",
|
||||||
],
|
],
|
||||||
"axis": axis,
|
"axis": axis,
|
||||||
}
|
}
|
||||||
|
if axis == "chapter_integrated":
|
||||||
|
expected.update(
|
||||||
|
{
|
||||||
|
"min_local_material_evidence": 2,
|
||||||
|
"min_official_sources": 2,
|
||||||
|
"min_fda_or_regulatory_precedents": 1,
|
||||||
|
"min_capa_actions": 3,
|
||||||
|
"preferred_evidence_types": [
|
||||||
|
"local_audit_or_recap_quote",
|
||||||
|
"official_regulatory_requirement",
|
||||||
|
"fda_warning_letter_or_meeting_record",
|
||||||
|
"gap_analysis",
|
||||||
|
"capa_action_with_owner_and_verification",
|
||||||
|
"counter_evidence_or_boundary_condition",
|
||||||
|
],
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return expected
|
||||||
|
|
||||||
|
|
||||||
def _default_stop_conditions() -> list[str]:
|
def _default_stop_conditions() -> list[str]:
|
||||||
@@ -143,6 +194,15 @@ def _default_stop_conditions() -> list[str]:
|
|||||||
]
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def _integrated_prompt_brief(chapter: Chapter, method: ResearchMethod | None) -> str:
|
||||||
|
lanes = ";".join(method.integrated_lanes if method else [])
|
||||||
|
return (
|
||||||
|
f"本任务是《{chapter.title}》的章节级综合证据包。不要把多条窄轴 packet 机械拼贴;"
|
||||||
|
f"必须围绕当前研究方法的 lanes 一次性收束主线:{lanes or '事实材料、权威证据、反方证据、行动建议'}。"
|
||||||
|
"输出必须让章节作者能直接写出判断、证据落点和可执行建议。"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def _task_card_for_chapter_axis(
|
def _task_card_for_chapter_axis(
|
||||||
*,
|
*,
|
||||||
chapter: Chapter,
|
chapter: Chapter,
|
||||||
@@ -152,22 +212,27 @@ def _task_card_for_chapter_axis(
|
|||||||
required_skills: list[str] | None = None,
|
required_skills: list[str] | None = None,
|
||||||
allowed_materials: list[str] | None = None,
|
allowed_materials: list[str] | None = None,
|
||||||
prompt_brief: str | None = None,
|
prompt_brief: str | None = None,
|
||||||
|
questions: list[str] | None = None,
|
||||||
|
research_goal: str | None = None,
|
||||||
|
expected_evidence: dict[str, Any] | None = None,
|
||||||
stop_conditions: list[str] | None = None,
|
stop_conditions: list[str] | None = None,
|
||||||
|
method: ResearchMethod | None = None,
|
||||||
) -> TaskCard:
|
) -> TaskCard:
|
||||||
return TaskCard(
|
return TaskCard(
|
||||||
task_id=f"{chapter.chapter_id}-{axis}",
|
task_id=f"{chapter.chapter_id}-{axis}",
|
||||||
chapter_ids=[chapter.chapter_id],
|
chapter_ids=[chapter.chapter_id],
|
||||||
topic_axis=axis,
|
topic_axis=axis,
|
||||||
questions=_questions_for_axis(chapter, axis),
|
questions=questions or _questions_for_axis(chapter, axis, method),
|
||||||
search_routes=routes,
|
search_routes=routes,
|
||||||
output_packet=f"phase2/packets/{chapter.chapter_id}-{axis}.json",
|
output_packet=f"phase2/packets/{chapter.chapter_id}-{axis}.json",
|
||||||
|
chapter_title=chapter.title,
|
||||||
preferred_model_role="dr_verifier" if axis == "counter" else "dr_analyst",
|
preferred_model_role="dr_verifier" if axis == "counter" else "dr_analyst",
|
||||||
research_goal=f"为《{chapter.title}》收集并验证 {axis} 轴证据,形成可写入章节的具体判断与证据落点。",
|
research_goal=research_goal or f"为《{chapter.title}》收集并验证 {axis} 轴证据,形成可写入章节的具体判断与证据落点。",
|
||||||
research_method=method_key,
|
research_method=method_key,
|
||||||
prompt_brief=prompt_brief or f"围绕《{chapter.title}》的 {axis} 轴,优先形成可证伪、可引用、可落地的证据包。",
|
prompt_brief=prompt_brief or (_integrated_prompt_brief(chapter, method) if axis == "chapter_integrated" else f"围绕《{chapter.title}》的 {axis} 轴,优先形成可证伪、可引用、可落地的证据包。"),
|
||||||
required_skills=required_skills or _default_required_skills(axis),
|
required_skills=required_skills or _default_required_skills(axis),
|
||||||
allowed_materials=allowed_materials or [],
|
allowed_materials=allowed_materials or [],
|
||||||
expected_evidence=_default_expected_evidence(axis),
|
expected_evidence=expected_evidence or _default_expected_evidence(axis),
|
||||||
stop_conditions=stop_conditions or _default_stop_conditions(),
|
stop_conditions=stop_conditions or _default_stop_conditions(),
|
||||||
model_hint="use_cross_model_verifier" if axis == "counter" else "use_cost_effective_research_worker",
|
model_hint="use_cross_model_verifier" if axis == "counter" else "use_cost_effective_research_worker",
|
||||||
)
|
)
|
||||||
@@ -193,6 +258,7 @@ def generate_task_cards(
|
|||||||
axis=axis,
|
axis=axis,
|
||||||
routes=routes,
|
routes=routes,
|
||||||
method_key=method.key if method else "",
|
method_key=method.key if method else "",
|
||||||
|
method=method,
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
validate_task_cards(cards)
|
validate_task_cards(cards)
|
||||||
@@ -211,7 +277,17 @@ def generate_task_cards_from_research_brief(
|
|||||||
chapters = parse_framework_chapters(framework_text)
|
chapters = parse_framework_chapters(framework_text)
|
||||||
planning = research_brief.get("task_planning") or {}
|
planning = research_brief.get("task_planning") or {}
|
||||||
method_key = research_brief.get("research_method") or (method.key if method else "")
|
method_key = research_brief.get("research_method") or (method.key if method else "")
|
||||||
selected_axes = axes or (method.task_axes if method else None) or list(planning.get("search_routes_by_axis") or []) or DEFAULT_AXES
|
if method is None and method_key:
|
||||||
|
from scripts.runtime.methods import ResearchMethodRegistry
|
||||||
|
|
||||||
|
method = ResearchMethodRegistry().get(method_key)
|
||||||
|
phase2_mode = planning.get("phase2_mode") or research_brief.get("phase2_mode")
|
||||||
|
if axes:
|
||||||
|
selected_axes = axes
|
||||||
|
elif phase2_mode == "chapter_integrated":
|
||||||
|
selected_axes = ["chapter_integrated"]
|
||||||
|
else:
|
||||||
|
selected_axes = (method.task_axes if method else None) or list(planning.get("search_routes_by_axis") or []) or DEFAULT_AXES
|
||||||
routes_by_axis = planning.get("search_routes_by_axis") or {}
|
routes_by_axis = planning.get("search_routes_by_axis") or {}
|
||||||
prompt_by_axis = planning.get("axis_prompt_briefs") or {}
|
prompt_by_axis = planning.get("axis_prompt_briefs") or {}
|
||||||
base_skills = list(planning.get("required_skills") or [])
|
base_skills = list(planning.get("required_skills") or [])
|
||||||
@@ -221,6 +297,15 @@ def generate_task_cards_from_research_brief(
|
|||||||
for item in research_brief.get("materials", [])
|
for item in research_brief.get("materials", [])
|
||||||
if item.get("path")
|
if item.get("path")
|
||||||
]
|
]
|
||||||
|
if not allowed_materials:
|
||||||
|
material_digest = (research_brief.get("phase1_inputs") or {}).get("material_digest")
|
||||||
|
if material_digest:
|
||||||
|
allowed_materials.append(str(material_digest))
|
||||||
|
chapter_plan_by_id = {
|
||||||
|
str(item.get("chapter_id")): item
|
||||||
|
for item in research_brief.get("chapter_planning", [])
|
||||||
|
if item.get("chapter_id")
|
||||||
|
}
|
||||||
cards: list[TaskCard] = []
|
cards: list[TaskCard] = []
|
||||||
for chapter in chapters:
|
for chapter in chapters:
|
||||||
for axis in selected_axes:
|
for axis in selected_axes:
|
||||||
@@ -228,6 +313,35 @@ def generate_task_cards_from_research_brief(
|
|||||||
skills = base_skills or _default_required_skills(axis)
|
skills = base_skills or _default_required_skills(axis)
|
||||||
if "search-gateway" not in skills:
|
if "search-gateway" not in skills:
|
||||||
skills = ["search-gateway", *skills]
|
skills = ["search-gateway", *skills]
|
||||||
|
chapter_plan = chapter_plan_by_id.get(chapter.chapter_id) if axis == "chapter_integrated" else None
|
||||||
|
prompt_brief = prompt_by_axis.get(axis)
|
||||||
|
questions = None
|
||||||
|
research_goal = None
|
||||||
|
expected_evidence = None
|
||||||
|
card_stop_conditions = stop_conditions or None
|
||||||
|
if chapter_plan:
|
||||||
|
prompt_brief = chapter_plan.get("phase2_prompt_context") or prompt_brief
|
||||||
|
research_goal = chapter_plan.get("core_question")
|
||||||
|
questions = [
|
||||||
|
chapter_plan.get("core_question", ""),
|
||||||
|
chapter_plan.get("bold_hypothesis", ""),
|
||||||
|
"按 Phase1 求证计划逐条收集支持证据、反方证据和待补证据。",
|
||||||
|
"不得绕开 Phase1 主基调另起炉灶;若证据推翻假设,必须明确写出修正建议。",
|
||||||
|
]
|
||||||
|
questions.extend(str(item) for item in chapter_plan.get("verification_plan", []))
|
||||||
|
expected_evidence = _default_expected_evidence(axis)
|
||||||
|
expected_evidence.update(
|
||||||
|
{
|
||||||
|
"phase1_minimum_evidence": chapter_plan.get("minimum_evidence") or {},
|
||||||
|
"evidence_lanes": chapter_plan.get("evidence_lanes") or [],
|
||||||
|
"must_address_phase1_hypothesis": True,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
card_stop_conditions = [
|
||||||
|
*(stop_conditions or _default_stop_conditions()),
|
||||||
|
"已经逐条回应 Phase1 的大胆假设:支持、修正或推翻,并说明依据。",
|
||||||
|
"已经把本地材料原文、外部证据、反方边界和行动建议分开记录。",
|
||||||
|
]
|
||||||
cards.append(
|
cards.append(
|
||||||
_task_card_for_chapter_axis(
|
_task_card_for_chapter_axis(
|
||||||
chapter=chapter,
|
chapter=chapter,
|
||||||
@@ -236,8 +350,12 @@ def generate_task_cards_from_research_brief(
|
|||||||
method_key=method_key,
|
method_key=method_key,
|
||||||
required_skills=skills,
|
required_skills=skills,
|
||||||
allowed_materials=allowed_materials,
|
allowed_materials=allowed_materials,
|
||||||
prompt_brief=prompt_by_axis.get(axis),
|
prompt_brief=prompt_brief,
|
||||||
stop_conditions=stop_conditions or None,
|
questions=questions,
|
||||||
|
research_goal=research_goal,
|
||||||
|
expected_evidence=expected_evidence,
|
||||||
|
stop_conditions=card_stop_conditions,
|
||||||
|
method=method,
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
validate_task_cards(cards)
|
validate_task_cards(cards)
|
||||||
@@ -284,6 +402,8 @@ def validate_task_cards(cards: list[TaskCard]) -> None:
|
|||||||
seen.add(card.task_id)
|
seen.add(card.task_id)
|
||||||
if not card.chapter_ids:
|
if not card.chapter_ids:
|
||||||
raise ValueError(f"{card.task_id}: chapter_ids required")
|
raise ValueError(f"{card.task_id}: chapter_ids required")
|
||||||
|
if not card.chapter_title:
|
||||||
|
card.chapter_title = card.chapter_ids[0]
|
||||||
if not card.questions:
|
if not card.questions:
|
||||||
raise ValueError(f"{card.task_id}: questions required")
|
raise ValueError(f"{card.task_id}: questions required")
|
||||||
if not card.output_packet.endswith(".json"):
|
if not card.output_packet.endswith(".json"):
|
||||||
@@ -344,7 +464,8 @@ def validate_packet(packet: dict[str, Any]) -> None:
|
|||||||
if undeclared:
|
if undeclared:
|
||||||
raise ValueError(f"packet source_ids referenced but not declared: {undeclared}")
|
raise ValueError(f"packet source_ids referenced but not declared: {undeclared}")
|
||||||
packet_sources = packet.get("sources") or []
|
packet_sources = packet.get("sources") or []
|
||||||
if packet_sources:
|
if not packet_sources:
|
||||||
|
raise ValueError("packet sources must not be empty")
|
||||||
known_source_ids = {source.get("id") for source in packet_sources}
|
known_source_ids = {source.get("id") for source in packet_sources}
|
||||||
missing_sources = sorted(declared - known_source_ids)
|
missing_sources = sorted(declared - known_source_ids)
|
||||||
if missing_sources:
|
if missing_sources:
|
||||||
|
|||||||
+245
-9
@@ -39,6 +39,10 @@ class ProjectSearchProvider:
|
|||||||
hits = self.client.patents(query, num_results=num_results)
|
hits = self.client.patents(query, num_results=num_results)
|
||||||
elif route == "news":
|
elif route == "news":
|
||||||
hits = self.client.news(query, num_results=num_results, time_range="y")
|
hits = self.client.news(query, num_results=num_results, time_range="y")
|
||||||
|
elif route == "fda":
|
||||||
|
hits = self.client.fda(query, num_results=num_results)
|
||||||
|
elif route == "evidence":
|
||||||
|
hits = self.client.evidence(query, num_results=num_results)
|
||||||
else:
|
else:
|
||||||
hits = self.client.search(query, num_results=num_results)
|
hits = self.client.search(query, num_results=num_results)
|
||||||
return [
|
return [
|
||||||
@@ -72,6 +76,213 @@ def _safe_source_stem(task_id: str) -> str:
|
|||||||
return re.sub(r"[^a-zA-Z0-9]+", "_", task_id).strip("_").lower()
|
return re.sub(r"[^a-zA-Z0-9]+", "_", task_id).strip("_").lower()
|
||||||
|
|
||||||
|
|
||||||
|
def contains_cjk(text: str) -> bool:
|
||||||
|
return any("\u4e00" <= char <= "\u9fff" for char in text)
|
||||||
|
|
||||||
|
|
||||||
|
def strip_cjk(text: str) -> str:
|
||||||
|
return re.sub(r"[\u3400-\u9fff]+", " ", text)
|
||||||
|
|
||||||
|
|
||||||
|
def validate_packet_against_allowed_context(
|
||||||
|
packet: dict,
|
||||||
|
search_context: dict[str, Any] | None,
|
||||||
|
material_context: dict[str, Any] | None,
|
||||||
|
) -> None:
|
||||||
|
"""Ensure the model did not invent source IDs or URLs beyond candidates."""
|
||||||
|
if not search_context and not material_context:
|
||||||
|
return
|
||||||
|
candidates = (search_context or {}).get("candidate_sources") or []
|
||||||
|
materials = (material_context or {}).get("materials") or []
|
||||||
|
if not candidates and not materials:
|
||||||
|
return
|
||||||
|
candidate_ids = {source.get("id") for source in candidates}
|
||||||
|
candidate_ids.update(item.get("source_id") for item in materials)
|
||||||
|
candidate_urls = {source.get("url") for source in candidates if source.get("url")}
|
||||||
|
candidate_urls.update(item.get("path") for item in materials if item.get("path"))
|
||||||
|
packet_sources = packet.get("sources") or []
|
||||||
|
unknown_ids = sorted(
|
||||||
|
source.get("id")
|
||||||
|
for source in packet_sources
|
||||||
|
if source.get("id") and source.get("id") not in candidate_ids
|
||||||
|
)
|
||||||
|
unknown_urls = sorted(
|
||||||
|
source.get("url")
|
||||||
|
for source in packet_sources
|
||||||
|
if source.get("url") and source.get("url") not in candidate_urls
|
||||||
|
)
|
||||||
|
if (candidates or materials) and not packet_sources:
|
||||||
|
raise ValueError("packet must include source metadata from candidate_sources or local materials")
|
||||||
|
if unknown_ids:
|
||||||
|
raise ValueError(f"packet sources include non-candidate source IDs: {unknown_ids}")
|
||||||
|
if unknown_urls:
|
||||||
|
raise ValueError(f"packet sources include non-candidate URLs: {unknown_urls}")
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_packet_against_context(
|
||||||
|
packet: dict[str, Any],
|
||||||
|
search_context: dict[str, Any] | None,
|
||||||
|
material_context: dict[str, Any] | None,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""Deterministically fill schema metadata the model often omits."""
|
||||||
|
packet = dict(packet)
|
||||||
|
referenced: set[str] = set(packet.get("source_ids") or [])
|
||||||
|
for section in ("claims", "counter_evidence"):
|
||||||
|
for item in packet.get(section) or []:
|
||||||
|
referenced.update(item.get("source_ids") or [])
|
||||||
|
for item in packet.get("evidence_items") or []:
|
||||||
|
if item.get("source_id"):
|
||||||
|
referenced.add(item["source_id"])
|
||||||
|
if "source_ids" not in packet or not packet.get("source_ids"):
|
||||||
|
packet["source_ids"] = sorted(referenced)
|
||||||
|
|
||||||
|
available_sources: dict[str, dict[str, Any]] = {}
|
||||||
|
for source in (search_context or {}).get("candidate_sources") or []:
|
||||||
|
if source.get("id"):
|
||||||
|
available_sources[source["id"]] = source
|
||||||
|
for material in (material_context or {}).get("materials") or []:
|
||||||
|
source_id = material.get("source_id")
|
||||||
|
if source_id:
|
||||||
|
available_sources[source_id] = {
|
||||||
|
"id": source_id,
|
||||||
|
"title": material.get("title") or Path(material.get("path", "")).name,
|
||||||
|
"url": material.get("path") or "",
|
||||||
|
"tier": "local_material",
|
||||||
|
"score": 8,
|
||||||
|
}
|
||||||
|
|
||||||
|
existing_sources = {
|
||||||
|
source.get("id"): source
|
||||||
|
for source in packet.get("sources") or []
|
||||||
|
if source.get("id")
|
||||||
|
}
|
||||||
|
for source_id in packet.get("source_ids") or []:
|
||||||
|
if source_id not in existing_sources and source_id in available_sources:
|
||||||
|
existing_sources[source_id] = available_sources[source_id]
|
||||||
|
if existing_sources:
|
||||||
|
packet["sources"] = [existing_sources[source_id] for source_id in packet.get("source_ids", []) if source_id in existing_sources]
|
||||||
|
return packet
|
||||||
|
|
||||||
|
|
||||||
|
FDA_AXIS_TERMS = {
|
||||||
|
"nmpa_fda_ema_ich_who_baseline": "CGMP pharmaceutical quality system process validation aseptic processing data integrity",
|
||||||
|
"quality_system_gap": "CGMP CAPA deviation change control data integrity quality unit pharmaceutical",
|
||||||
|
"manufacturing_process_risk": "aseptic processing sterile drug manufacturing process validation PPQ cleaning validation water system",
|
||||||
|
"operations_management_gap": "pharmaceutical quality system quality metrics management review senior management FDA",
|
||||||
|
"capa_roadmap": "CGMP CAPA effectiveness remediation warning letter close-out pharmaceutical",
|
||||||
|
"verification_evidence": "FDA 483 response CAPA effectiveness verification EIR pharmaceutical quality",
|
||||||
|
"counter": "FDA warning letter CGMP pharmaceutical quality data integrity remediation limitations",
|
||||||
|
"fda_enforcement_precedents": "FDA warning letter CGMP pharmaceutical aseptic processing data integrity CAPA process validation",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
FDA_CHAPTER_TERMS = {
|
||||||
|
"ch01": "commercial readiness phase gate remediation governance",
|
||||||
|
"ch02": "regulatory baseline CGMP EU GMP Annex 1 ICH Q9 ICH Q10",
|
||||||
|
"ch03": "aseptic processing RABS first air media fill visual inspection depyrogenation tunnel",
|
||||||
|
"ch04": "biologics drug substance WFI clean utilities SCADA EMS single-use system",
|
||||||
|
"ch05": "process validation master batch record CPP CQA PPQ cleaning validation technology transfer",
|
||||||
|
"ch06": "deviation change control CAPA document control training data integrity quality unit",
|
||||||
|
"ch07": "training effectiveness quality culture operator qualification human factors",
|
||||||
|
"ch08": "quality metrics management review escalation cross-functional governance operations",
|
||||||
|
"ch09": "CDMO quality organization technology transfer project governance capability matrix",
|
||||||
|
"ch10": "CAPA remediation plan effectiveness check owner due date verification evidence",
|
||||||
|
"ch11": "regulatory mapping CAPA tracker closure evidence quality assurance verification",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
ROUTE_CHAPTER_TERMS = {
|
||||||
|
**FDA_CHAPTER_TERMS,
|
||||||
|
}
|
||||||
|
|
||||||
|
ROUTE_SUFFIX_TERMS = {
|
||||||
|
"scholar": "pharmaceutical GMP review validation risk management quality system",
|
||||||
|
"patents": "biologics manufacturing patent process formulation device",
|
||||||
|
"news": "pharmaceutical quality operations CDMO quality governance",
|
||||||
|
"evidence": "pharmaceutical GMP evidence guidance enforcement best practice quality operations",
|
||||||
|
"general": "pharmaceutical GMP best practice guidance quality operations remediation",
|
||||||
|
}
|
||||||
|
|
||||||
|
INTERNAL_QUERY_TOKENS = {
|
||||||
|
"chapter_integrated",
|
||||||
|
"input_material_findings",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _compact_english_query(*parts: str, max_terms: int = 16) -> str:
|
||||||
|
text = strip_cjk(" ".join(part for part in parts if part))
|
||||||
|
text = re.sub(r"[^A-Za-z0-9./+-]+", " ", text)
|
||||||
|
terms: list[str] = []
|
||||||
|
seen: set[str] = set()
|
||||||
|
for raw in text.split():
|
||||||
|
term = raw.strip(" ./+-").lower()
|
||||||
|
if not term or term in INTERNAL_QUERY_TOKENS:
|
||||||
|
continue
|
||||||
|
key = term.casefold()
|
||||||
|
if key in seen:
|
||||||
|
continue
|
||||||
|
seen.add(key)
|
||||||
|
terms.append(term)
|
||||||
|
if len(terms) >= max_terms:
|
||||||
|
break
|
||||||
|
return " ".join(terms)
|
||||||
|
|
||||||
|
|
||||||
|
def _chapter_terms(card: TaskCard) -> str:
|
||||||
|
mapped = " ".join(ROUTE_CHAPTER_TERMS.get(chapter_id, "") for chapter_id in card.chapter_ids)
|
||||||
|
if mapped.strip():
|
||||||
|
return mapped
|
||||||
|
return strip_cjk(card.chapter_title)
|
||||||
|
|
||||||
|
|
||||||
|
def build_route_query(card: TaskCard, route: str) -> str:
|
||||||
|
"""Build short, route-aware queries instead of sending whole task cards."""
|
||||||
|
if route == "fda":
|
||||||
|
terms = FDA_AXIS_TERMS.get(card.topic_axis, "FDA warning letter CGMP pharmaceutical quality")
|
||||||
|
chapter_terms = " ".join(FDA_CHAPTER_TERMS.get(chapter_id, "") for chapter_id in card.chapter_ids)
|
||||||
|
query = f"{terms} {chapter_terms}".strip()
|
||||||
|
if contains_cjk(query):
|
||||||
|
raise ValueError(f"FDA route query must not contain Chinese text: {query}")
|
||||||
|
return query
|
||||||
|
if route == "scholar":
|
||||||
|
return _compact_english_query(_chapter_terms(card), ROUTE_SUFFIX_TERMS["scholar"])
|
||||||
|
if route == "patents":
|
||||||
|
return _compact_english_query(_chapter_terms(card), ROUTE_SUFFIX_TERMS["patents"])
|
||||||
|
if route == "news":
|
||||||
|
return _compact_english_query(_chapter_terms(card), ROUTE_SUFFIX_TERMS["news"])
|
||||||
|
if route == "evidence":
|
||||||
|
return _compact_english_query(_chapter_terms(card), ROUTE_SUFFIX_TERMS["evidence"])
|
||||||
|
return _compact_english_query(_chapter_terms(card), ROUTE_SUFFIX_TERMS["general"])
|
||||||
|
|
||||||
|
|
||||||
|
def _material_excerpt(project_root: Path | None, rel_path: str, *, max_chars: int = 6000) -> dict[str, str] | None:
|
||||||
|
if project_root is None:
|
||||||
|
return None
|
||||||
|
path = project_root / rel_path
|
||||||
|
if not path.exists() or not path.is_file():
|
||||||
|
return None
|
||||||
|
text = path.read_text(encoding="utf-8", errors="ignore")
|
||||||
|
return {
|
||||||
|
"path": rel_path,
|
||||||
|
"source_id": f"src_local_{_safe_source_stem(Path(rel_path).stem)}",
|
||||||
|
"title": Path(rel_path).name,
|
||||||
|
"excerpt": text[:max_chars],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def build_material_context(card: TaskCard, project_root: Path | None, *, max_chars_per_material: int = 6000) -> dict[str, Any]:
|
||||||
|
materials = []
|
||||||
|
seen: set[str] = set()
|
||||||
|
for rel in card.allowed_materials:
|
||||||
|
if rel in seen:
|
||||||
|
continue
|
||||||
|
seen.add(rel)
|
||||||
|
item = _material_excerpt(project_root, rel, max_chars=max_chars_per_material)
|
||||||
|
if item:
|
||||||
|
materials.append(item)
|
||||||
|
return {"materials": materials}
|
||||||
|
|
||||||
|
|
||||||
def build_search_context(
|
def build_search_context(
|
||||||
card: TaskCard,
|
card: TaskCard,
|
||||||
search_provider: SearchProvider,
|
search_provider: SearchProvider,
|
||||||
@@ -82,9 +293,9 @@ def build_search_context(
|
|||||||
routes_used: list[str] = []
|
routes_used: list[str] = []
|
||||||
source_stem = _safe_source_stem(card.task_id)
|
source_stem = _safe_source_stem(card.task_id)
|
||||||
idx = 1
|
idx = 1
|
||||||
query = " ".join(card.questions)
|
|
||||||
for route in card.search_routes:
|
for route in card.search_routes:
|
||||||
routes_used.append(route)
|
routes_used.append(route)
|
||||||
|
query = build_route_query(card, route)
|
||||||
hits = search_provider.search(query=query, route=route, num_results=num_results_per_route)
|
hits = search_provider.search(query=query, route=route, num_results=num_results_per_route)
|
||||||
for hit in hits:
|
for hit in hits:
|
||||||
candidate_sources.append(
|
candidate_sources.append(
|
||||||
@@ -102,15 +313,22 @@ def build_search_context(
|
|||||||
return {"routes_used": routes_used, "candidate_sources": candidate_sources}
|
return {"routes_used": routes_used, "candidate_sources": candidate_sources}
|
||||||
|
|
||||||
|
|
||||||
def build_packet_user_prompt(card: TaskCard, search_context: dict[str, Any] | None = None) -> str:
|
def build_packet_user_prompt(
|
||||||
|
card: TaskCard,
|
||||||
|
search_context: dict[str, Any] | None = None,
|
||||||
|
material_context: dict[str, Any] | None = None,
|
||||||
|
) -> str:
|
||||||
context = search_context or {"routes_used": [], "candidate_sources": []}
|
context = search_context or {"routes_used": [], "candidate_sources": []}
|
||||||
|
materials = material_context or {"materials": []}
|
||||||
return (
|
return (
|
||||||
"请根据以下 task card 产出一个证据包 JSON。\n"
|
"请根据以下 task card 产出一个证据包 JSON。\n"
|
||||||
"正式结论、summary、open_questions 用中文;英文原文摘录、source title、DOI/URL 可以保留英文。\n"
|
"正式结论、summary、open_questions 用中文;英文原文摘录、source title、DOI/URL 可以保留英文。\n"
|
||||||
"必须主动包含 counter_evidence,且所有引用的 source_id 必须出现在 source_ids 中。\n\n"
|
"必须主动包含 counter_evidence,且所有引用的 source_id 必须出现在 source_ids 中。\n\n"
|
||||||
"只能使用 candidate_sources 中的来源,不得编造 URL、DOI、trial ID 或 source_id。\n"
|
"只能使用 candidate_sources 或 Local material context 中的来源,不得编造 URL、DOI、trial ID 或 source_id。\n"
|
||||||
"输出 JSON 必须包含 sources 字段,且 sources 只能来自 candidate_sources。\n\n"
|
"输出 JSON 必须包含 sources 字段;sources 只能来自 candidate_sources 或 Local material context。\n"
|
||||||
|
"如 Local material context 非空,必须至少提取 1 条本地材料原文证据;如果与本章无关,必须在 open_questions 说明为什么无关。\n\n"
|
||||||
f"{json.dumps(card.to_dict(), ensure_ascii=False, indent=2)}\n\n"
|
f"{json.dumps(card.to_dict(), ensure_ascii=False, indent=2)}\n\n"
|
||||||
|
f"Local material context:\n{json.dumps(materials, ensure_ascii=False, indent=2)}\n\n"
|
||||||
f"Search context:\n{json.dumps(context, ensure_ascii=False, indent=2)}\n\n"
|
f"Search context:\n{json.dumps(context, ensure_ascii=False, indent=2)}\n\n"
|
||||||
"只输出 JSON,不要输出 Markdown 解释。"
|
"只输出 JSON,不要输出 Markdown 解释。"
|
||||||
)
|
)
|
||||||
@@ -122,16 +340,19 @@ def build_packet_repair_prompt(
|
|||||||
raw_response: str,
|
raw_response: str,
|
||||||
error: Exception,
|
error: Exception,
|
||||||
search_context: dict[str, Any] | None = None,
|
search_context: dict[str, Any] | None = None,
|
||||||
|
material_context: dict[str, Any] | None = None,
|
||||||
) -> str:
|
) -> str:
|
||||||
context = search_context or {"routes_used": [], "candidate_sources": []}
|
context = search_context or {"routes_used": [], "candidate_sources": []}
|
||||||
|
materials = material_context or {"materials": []}
|
||||||
return (
|
return (
|
||||||
"请修复上一次 evidence packet 输出,使其成为合法且通过 schema 校验的 JSON。\n"
|
"请修复上一次 evidence packet 输出,使其成为合法且通过 schema 校验的 JSON。\n"
|
||||||
"只输出 JSON 对象,不要输出 Markdown、解释或代码块。\n"
|
"只输出 JSON 对象,不要输出 Markdown、解释或代码块。\n"
|
||||||
"保留中文主写作;英文只允许出现在来源标题、URL、DOI、原文摘录或检索笔记中。\n"
|
"保留中文主写作;英文只允许出现在来源标题、URL、DOI、原文摘录或检索笔记中。\n"
|
||||||
"不得编造 candidate_sources 以外的来源、URL、DOI、trial ID 或 source_id。\n\n"
|
"不得编造 candidate_sources 或 Local material context 以外的来源、URL、DOI、trial ID 或 source_id。\n\n"
|
||||||
f"Schema error:\n{error}\n\n"
|
f"Schema error:\n{error}\n\n"
|
||||||
f"Task card:\n{json.dumps(card.to_dict(), ensure_ascii=False, indent=2)}\n\n"
|
f"Task card:\n{json.dumps(card.to_dict(), ensure_ascii=False, indent=2)}\n\n"
|
||||||
f"Search context:\n{json.dumps(context, ensure_ascii=False, indent=2)}\n\n"
|
f"Search context:\n{json.dumps(context, ensure_ascii=False, indent=2)}\n\n"
|
||||||
|
f"Local material context:\n{json.dumps(materials, ensure_ascii=False, indent=2)}\n\n"
|
||||||
f"Previous raw response:\n{raw_response[:12000]}"
|
f"Previous raw response:\n{raw_response[:12000]}"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -142,12 +363,14 @@ class PacketWorker:
|
|||||||
*,
|
*,
|
||||||
role: RoleDefinition,
|
role: RoleDefinition,
|
||||||
client: ChatClient,
|
client: ChatClient,
|
||||||
|
project_root: Path | None = None,
|
||||||
search_provider: SearchProvider | None = None,
|
search_provider: SearchProvider | None = None,
|
||||||
skill_registry: SkillRegistry | None = None,
|
skill_registry: SkillRegistry | None = None,
|
||||||
num_results_per_route: int = 5,
|
num_results_per_route: int = 5,
|
||||||
) -> None:
|
) -> None:
|
||||||
self.role = role
|
self.role = role
|
||||||
self.client = client
|
self.client = client
|
||||||
|
self.project_root = project_root
|
||||||
self.search_provider = search_provider
|
self.search_provider = search_provider
|
||||||
self.skill_registry = skill_registry or SkillRegistry()
|
self.skill_registry = skill_registry or SkillRegistry()
|
||||||
self.num_results_per_route = num_results_per_route
|
self.num_results_per_route = num_results_per_route
|
||||||
@@ -160,6 +383,7 @@ class PacketWorker:
|
|||||||
except FileNotFoundError:
|
except FileNotFoundError:
|
||||||
skill_texts.append(f"# Skill: {name}\n\n[missing skill: {name}]")
|
skill_texts.append(f"# Skill: {name}\n\n[missing skill: {name}]")
|
||||||
return (
|
return (
|
||||||
|
f"{self.role.identity}\n\n"
|
||||||
"你是 Deep Research v0.20 Python runtime 的证据包 worker。\n"
|
"你是 Deep Research v0.20 Python runtime 的证据包 worker。\n"
|
||||||
"你的唯一任务是把一个 task card 转换为结构化 evidence packet。\n"
|
"你的唯一任务是把一个 task card 转换为结构化 evidence packet。\n"
|
||||||
"遵循中文主写作原则;不要写章节正文;不要编造 URL、DOI、trial ID 或 source_id。\n\n"
|
"遵循中文主写作原则;不要写章节正文;不要编造 URL、DOI、trial ID 或 source_id。\n\n"
|
||||||
@@ -175,17 +399,23 @@ class PacketWorker:
|
|||||||
self.search_provider,
|
self.search_provider,
|
||||||
num_results_per_route=self.num_results_per_route,
|
num_results_per_route=self.num_results_per_route,
|
||||||
)
|
)
|
||||||
|
material_context = build_material_context(card, self.project_root)
|
||||||
raw = self.client.chat_complete(
|
raw = self.client.chat_complete(
|
||||||
model=self.role.model,
|
model=self.role.model,
|
||||||
system=self._system_prompt(),
|
system=self._system_prompt(),
|
||||||
user=build_packet_user_prompt(card, search_context),
|
user=build_packet_user_prompt(card, search_context, material_context),
|
||||||
temperature=self.role.temperature,
|
temperature=self.role.temperature,
|
||||||
max_tokens=self.role.max_tokens,
|
max_tokens=self.role.max_tokens,
|
||||||
tag=f"packet:{card.task_id}",
|
tag=f"packet:{card.task_id}",
|
||||||
)
|
)
|
||||||
try:
|
try:
|
||||||
packet = _extract_json_object(raw)
|
packet = normalize_packet_against_context(
|
||||||
|
_extract_json_object(raw),
|
||||||
|
search_context,
|
||||||
|
material_context,
|
||||||
|
)
|
||||||
validate_packet(packet)
|
validate_packet(packet)
|
||||||
|
validate_packet_against_allowed_context(packet, search_context, material_context)
|
||||||
return packet
|
return packet
|
||||||
except Exception as error:
|
except Exception as error:
|
||||||
repaired = self.client.chat_complete(
|
repaired = self.client.chat_complete(
|
||||||
@@ -196,13 +426,19 @@ class PacketWorker:
|
|||||||
raw_response=raw,
|
raw_response=raw,
|
||||||
error=error,
|
error=error,
|
||||||
search_context=search_context,
|
search_context=search_context,
|
||||||
|
material_context=material_context,
|
||||||
),
|
),
|
||||||
temperature=0,
|
temperature=0,
|
||||||
max_tokens=self.role.max_tokens,
|
max_tokens=self.role.max_tokens,
|
||||||
tag=f"packet-repair:{card.task_id}",
|
tag=f"packet-repair:{card.task_id}",
|
||||||
)
|
)
|
||||||
packet = _extract_json_object(repaired)
|
packet = normalize_packet_against_context(
|
||||||
|
_extract_json_object(repaired),
|
||||||
|
search_context,
|
||||||
|
material_context,
|
||||||
|
)
|
||||||
validate_packet(packet)
|
validate_packet(packet)
|
||||||
|
validate_packet_against_allowed_context(packet, search_context, material_context)
|
||||||
return packet
|
return packet
|
||||||
|
|
||||||
|
|
||||||
@@ -233,7 +469,7 @@ def run_packet_workers(
|
|||||||
def run_one(card: TaskCard) -> tuple[TaskCard, dict | None, Exception | None]:
|
def run_one(card: TaskCard) -> tuple[TaskCard, dict | None, Exception | None]:
|
||||||
search_provider = search_provider_factory() if search_provider_factory else None
|
search_provider = search_provider_factory() if search_provider_factory else None
|
||||||
try:
|
try:
|
||||||
worker = PacketWorker(role=role, client=client_factory(role), search_provider=search_provider)
|
worker = PacketWorker(role=role, client=client_factory(role), project_root=project_root, search_provider=search_provider)
|
||||||
return card, worker.run(card), None
|
return card, worker.run(card), None
|
||||||
except Exception as error:
|
except Exception as error:
|
||||||
return card, None, error
|
return card, None, error
|
||||||
|
|||||||
+16
-5
@@ -25,17 +25,19 @@ from scripts.lib.zenmux_client import load_secrets
|
|||||||
|
|
||||||
|
|
||||||
ROUTE_HELP = {
|
ROUTE_HELP = {
|
||||||
"general": "Exa -> Tavily generic web discovery",
|
"general": "Tavily -> Exa -> Brave generic web discovery",
|
||||||
|
"evidence": "Exa highlights -> Tavily -> Brave controlled evidence discovery",
|
||||||
"scholar": "Serper Scholar -> generic fallback",
|
"scholar": "Serper Scholar -> generic fallback",
|
||||||
"patents": "Serper Google Patents -> site:patents.google.com fallback",
|
"patents": "Serper Google Patents -> site:patents.google.com fallback",
|
||||||
"news": "Serper News -> generic fallback",
|
"news": "Serper News -> generic fallback",
|
||||||
|
"fda": "FDA-focused discovery for warning letters, enforcement pages, and meeting materials",
|
||||||
}
|
}
|
||||||
|
|
||||||
PROFILE_ROUTES = {
|
PROFILE_ROUTES = {
|
||||||
"biomed_literature": ["scholar", "general"],
|
"biomed_literature": ["scholar", "evidence", "general"],
|
||||||
"patent_heavy": ["patents", "general"],
|
"patent_heavy": ["patents", "evidence", "general"],
|
||||||
"china_market": ["news", "general"],
|
"china_market": ["news", "evidence", "general"],
|
||||||
"investment": ["news", "general"],
|
"investment": ["news", "evidence", "general"],
|
||||||
}
|
}
|
||||||
|
|
||||||
PROFILE_QUERY_PREFIX = {
|
PROFILE_QUERY_PREFIX = {
|
||||||
@@ -46,12 +48,16 @@ PROFILE_QUERY_PREFIX = {
|
|||||||
def search_route(client: SearchClient, route: str, query: str, args: argparse.Namespace) -> list[SearchHit]:
|
def search_route(client: SearchClient, route: str, query: str, args: argparse.Namespace) -> list[SearchHit]:
|
||||||
if route == "general":
|
if route == "general":
|
||||||
return client.search(query, num_results=args.num_results)
|
return client.search(query, num_results=args.num_results)
|
||||||
|
if route == "evidence":
|
||||||
|
return client.evidence(query, num_results=args.num_results, category=args.exa_category)
|
||||||
if route == "scholar":
|
if route == "scholar":
|
||||||
return client.scholar(query, num_results=args.num_results, year_low=args.year_low)
|
return client.scholar(query, num_results=args.num_results, year_low=args.year_low)
|
||||||
if route == "patents":
|
if route == "patents":
|
||||||
return client.patents(query, num_results=args.num_results)
|
return client.patents(query, num_results=args.num_results)
|
||||||
if route == "news":
|
if route == "news":
|
||||||
return client.news(query, num_results=args.num_results, time_range=args.time_range)
|
return client.news(query, num_results=args.num_results, time_range=args.time_range)
|
||||||
|
if route == "fda":
|
||||||
|
return client.fda(query, num_results=args.num_results)
|
||||||
raise SystemExit(f"unknown route: {route}")
|
raise SystemExit(f"unknown route: {route}")
|
||||||
|
|
||||||
|
|
||||||
@@ -107,6 +113,11 @@ def build_parser() -> argparse.ArgumentParser:
|
|||||||
help="Run a strategy profile instead of a single route",
|
help="Run a strategy profile instead of a single route",
|
||||||
)
|
)
|
||||||
parser.add_argument("--num-results", type=int, default=10)
|
parser.add_argument("--num-results", type=int, default=10)
|
||||||
|
parser.add_argument(
|
||||||
|
"--exa-category",
|
||||||
|
choices=["research paper", "news", "company", "financial report", "github", "tweet", "personal site", "pdf"],
|
||||||
|
help="Optional Exa category for the evidence route",
|
||||||
|
)
|
||||||
parser.add_argument("--year-low", type=int, help="Lower year bound for scholar searches")
|
parser.add_argument("--year-low", type=int, help="Lower year bound for scholar searches")
|
||||||
parser.add_argument("--time-range", choices=["d", "w", "m", "y"], help="Serper news time range")
|
parser.add_argument("--time-range", choices=["d", "w", "m", "y"], help="Serper news time range")
|
||||||
parser.add_argument("--json", action="store_true", help="Emit JSON instead of Markdown")
|
parser.add_argument("--json", action="store_true", help="Emit JSON instead of Markdown")
|
||||||
|
|||||||
@@ -25,6 +25,7 @@ Deep Research is driven by the repository Python core, not by chat context. Trea
|
|||||||
6. Run Phase 2 with file-backed task cards and packets:
|
6. Run Phase 2 with file-backed task cards and packets:
|
||||||
`uv run python scripts/dr.py research <slug> --workers 6 --execute-packets --allow-search-fallback`
|
`uv run python scripts/dr.py research <slug> --workers 6 --execute-packets --allow-search-fallback`
|
||||||
7. Build briefs and chapters only from persisted packets:
|
7. Build briefs and chapters only from persisted packets:
|
||||||
|
`uv run python scripts/dr.py sources cache <slug> --limit 50`
|
||||||
`uv run python scripts/dr.py research <slug> --build-briefs`
|
`uv run python scripts/dr.py research <slug> --build-briefs`
|
||||||
`uv run python scripts/dr.py research <slug> --assemble-chapters --workers 4`
|
`uv run python scripts/dr.py research <slug> --assemble-chapters --workers 4`
|
||||||
8. Review and finalize through Python:
|
8. Review and finalize through Python:
|
||||||
@@ -37,6 +38,7 @@ Deep Research is driven by the repository Python core, not by chat context. Trea
|
|||||||
- Do not invent evidence when model/API access fails. Stop at the last durable artifact and report the exact blocker.
|
- Do not invent evidence when model/API access fails. Stop at the last durable artifact and report the exact blocker.
|
||||||
- Phase 2 concurrency must use task cards and packet files, not platform subagents as the default mechanism.
|
- Phase 2 concurrency must use task cards and packet files, not platform subagents as the default mechanism.
|
||||||
- Search must use the project Python gateway (`scripts/search.py` / `scripts.lib.search_client`) by default. Do not use Tavily MCP, browser MCP, or platform-native web search in subagents unless the user explicitly requests that escape hatch.
|
- Search must use the project Python gateway (`scripts/search.py` / `scripts.lib.search_client`) by default. Do not use Tavily MCP, browser MCP, or platform-native web search in subagents unless the user explicitly requests that escape hatch.
|
||||||
|
- Key Tier 1-2 sources such as ICH Q9/Q10, EU GMP Annex 1, FDA guidance/warning letters, EMA/NMPA/WHO pages, and pharmacopeia materials should be cached as local Markdown snapshots under `phase2/source_cache/` before chapter assembly.
|
||||||
- User materials are starting evidence, not final truth. Cross-check against authoritative sources such as NMPA, FDA, EMA, ICH, WHO, pharmacopeias, and recognized best-practice references.
|
- User materials are starting evidence, not final truth. Cross-check against authoritative sources such as NMPA, FDA, EMA, ICH, WHO, pharmacopeias, and recognized best-practice references.
|
||||||
- For GMP/quality/operations diagnosis, prefer `--method gmp_quality_operations_diagnosis`.
|
- For GMP/quality/operations diagnosis, prefer `--method gmp_quality_operations_diagnosis`.
|
||||||
- Chapter drafts are not acceptable if they merely summarize principles. Each section must turn evidence into concrete findings, risk implications, and整改动作;otherwise return to Phase 2 enrichment.
|
- Chapter drafts are not acceptable if they merely summarize principles. Each section must turn evidence into concrete findings, risk implications, and整改动作;otherwise return to Phase 2 enrichment.
|
||||||
|
|||||||
@@ -15,6 +15,7 @@ Run searches from the repository root:
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
uv run python scripts/search.py "<query>" --route general --json --trace
|
uv run python scripts/search.py "<query>" --route general --json --trace
|
||||||
|
uv run python scripts/search.py "<query>" --route evidence --json --trace
|
||||||
uv run python scripts/search.py "<query>" --route scholar --year-low 2020 --json --trace
|
uv run python scripts/search.py "<query>" --route scholar --year-low 2020 --json --trace
|
||||||
uv run python scripts/search.py "<query>" --route news --time-range y --json --trace
|
uv run python scripts/search.py "<query>" --route news --time-range y --json --trace
|
||||||
uv run python scripts/search.py "<query>" --route patents --json --trace
|
uv run python scripts/search.py "<query>" --route patents --json --trace
|
||||||
@@ -29,12 +30,19 @@ UV_CACHE_DIR=/private/tmp/deep_research_uv_cache uv run python scripts/search.py
|
|||||||
|
|
||||||
## Routing
|
## Routing
|
||||||
|
|
||||||
- `general`: Exa first, Tavily fallback.
|
- `general`: Tavily first, Exa fallback, Brave fallback; use for broad discovery and gap filling.
|
||||||
- `scholar`: Serper Scholar first; use for papers, guidelines, and technical literature.
|
- `evidence`: Exa highlights first, Tavily fallback, Brave fallback; use when a task card needs concise, source-level candidate evidence for an evidence packet.
|
||||||
|
- `scholar`: Serper Scholar first; use for papers, reviews, technical literature, and academic validation only.
|
||||||
- `news`: Serper News first; use for recent industry/current information.
|
- `news`: Serper News first; use for recent industry/current information.
|
||||||
- `patents`: Serper Google Patents first.
|
- `patents`: Serper Google Patents first.
|
||||||
- `biomed_literature`: scholar plus general discovery.
|
- `biomed_literature`: scholar plus general discovery.
|
||||||
|
|
||||||
|
Serper is not the default general web search source. Keep it mainly for Scholar, Google Patents, News, and targeted `site:` searches where Google coverage matters.
|
||||||
|
|
||||||
|
Tavily Research is a phase-level scan tool, not a packet-writing shortcut. Use it for Phase 1 initial landscape scans, Phase 2 gap-fill after a chapter is thin, or Phase 3回炉补证据;its output must be saved, source-scored, deduplicated, and converted into candidate evidence before citation.
|
||||||
|
|
||||||
|
Exa is the preferred controlled evidence discovery route for agents because it can return short highlights/text per URL. Treat Exa hits as candidate sources unless the URL itself is an original Tier 1-2 source.
|
||||||
|
|
||||||
API keys are loaded from `secrets.env` by `scripts/search.py`; do not ask the user to authorize MCP calls when the env keys are available.
|
API keys are loaded from `secrets.env` by `scripts/search.py`; do not ask the user to authorize MCP calls when the env keys are available.
|
||||||
|
|
||||||
## Subagent Protocol
|
## Subagent Protocol
|
||||||
@@ -45,6 +53,7 @@ For evidence packets:
|
|||||||
2. Use search hits only as candidate sources; whenever possible, cite the original regulator, guideline, paper, or official document.
|
2. Use search hits only as candidate sources; whenever possible, cite the original regulator, guideline, paper, or official document.
|
||||||
3. Put every used source in `sources` with `id`, `title`, `url`, `tier`, and `score`.
|
3. Put every used source in `sources` with `id`, `title`, `url`, `tier`, and `score`.
|
||||||
4. Do not write a final chapter during search; produce structured evidence only.
|
4. Do not write a final chapter during search; produce structured evidence only.
|
||||||
|
5. For repeatedly used Tier 1-2 sources, run `uv run python scripts/dr.py sources cache <project>` so later phases can cite a local Markdown snapshot rather than only a URL.
|
||||||
|
|
||||||
For chapter assembly:
|
For chapter assembly:
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,76 @@
|
|||||||
|
---
|
||||||
|
name: search-strategy
|
||||||
|
description: 生物医药深度研究的统一检索策略。规定信源优先级、检索轮次、关键词构造、API 路由,以及何时切换到专业信源。所有做信息收集的 worker/agent 必须加载此技能。
|
||||||
|
---
|
||||||
|
|
||||||
|
# Search Strategy
|
||||||
|
|
||||||
|
## Core Rule
|
||||||
|
|
||||||
|
Do not send Chinese chapter titles, interview paragraphs, or full task-card text directly to search APIs. For formal search, first convert the task into short English query terms plus axis terms, then add source/domain constraints when useful.
|
||||||
|
|
||||||
|
## Query Construction
|
||||||
|
|
||||||
|
Build every query from three parts:
|
||||||
|
|
||||||
|
- `entity/domain`: the object or field, such as `pharmaceutical`, `biologics`, `sterile drug`, `CDMO`, `quality system`.
|
||||||
|
- `axis`: the research axis, such as `CAPA deviation change control`, `aseptic processing process validation PPQ`, `quality metrics management review`.
|
||||||
|
- `evidence type`: the evidence to retrieve, such as `Warning Letter`, `meeting materials`, `guidance`, `systematic review`, `patent`, `best practices`.
|
||||||
|
|
||||||
|
Default English query length is 5-12 keywords. Chinese terms are useful for NMPA, local industry sources, and internal-material matching, but Chinese long sentences must not be the default query form.
|
||||||
|
|
||||||
|
For route-specific searches, do not append the original Chinese chapter title after the English query. If chapter context is needed, map the chapter to short English concept terms first, such as `commercial readiness phase gate`, `aseptic processing`, `quality metrics management review`, or `CAPA effectiveness check`.
|
||||||
|
|
||||||
|
## Route Patterns
|
||||||
|
|
||||||
|
- `fda`: use `site:fda.gov` plus `Warning Letter`, `inspection`, `enforcement`, `meeting materials`, or `meeting minutes`, then add the axis terms.
|
||||||
|
- `scholar`: use technical/scientific terms plus `review`, `validation`, `risk management`, `quality system`, or disease/mechanism terms.
|
||||||
|
- `evidence`: use Exa highlights for controlled evidence discovery when a packet needs concise source-level excerpts; still trace important hits back to original Tier 1-2 sources.
|
||||||
|
- `patents`: use technology route plus material, target, process, formulation, device, or manufacturing terms.
|
||||||
|
- `news`: use company/industry plus event type and recency terms.
|
||||||
|
- `general`: use Tavily/Exa/Brave for discovery and gap filling; trace useful hits back to Tier 1-2 original sources before citing. Do not route generic web discovery through Serper by default.
|
||||||
|
- `tavily_research` conceptually means a phase-level scan, not a normal packet route. Save the research result, score/deduplicate sources, then convert it into candidate evidence before writing claims.
|
||||||
|
|
||||||
|
## GMP/FDA Examples
|
||||||
|
|
||||||
|
Bad query:
|
||||||
|
|
||||||
|
```text
|
||||||
|
围绕《审计发现应先转化为商业化阶段门缺口,而不是停留在风险项计数》从质量体系角度提炼可证伪的核心结论
|
||||||
|
```
|
||||||
|
|
||||||
|
Good queries:
|
||||||
|
|
||||||
|
```text
|
||||||
|
site:fda.gov "Warning Letter" CGMP CAPA deviation change control data integrity pharmaceutical
|
||||||
|
site:fda.gov "meeting materials" "pharmaceutical quality" "quality metrics"
|
||||||
|
site:fda.gov/inspections-compliance-enforcement-and-criminal-investigations "Warning Letter" aseptic processing process validation
|
||||||
|
```
|
||||||
|
|
||||||
|
## Source Priority
|
||||||
|
|
||||||
|
- Tier 1: regulator, guideline, pharmacopeia, primary literature, trial registry, patent original, company filing.
|
||||||
|
- Tier 2: systematic review, recognized consulting or industry association report, professional database/media.
|
||||||
|
- Tier 3: conference abstract, broker report, preprint, vendor white paper.
|
||||||
|
- Tier 4: generic web search result; discovery only, not conclusion support.
|
||||||
|
|
||||||
|
## Four-Round Search Discipline
|
||||||
|
|
||||||
|
1. Tier 1 direct hit: regulator, PubMed/Scholar, trial registry, patent original, or official filing.
|
||||||
|
2. Tier 2 synthesis: recognized review, guideline interpretation, consulting/association report.
|
||||||
|
3. Counter-evidence: limitations, failures, enforcement actions, contradictory interpretations.
|
||||||
|
4. Gap fill: Exa evidence discovery or Tavily/Brave general discovery, then trace back to original sources. Use Serper here only for Google-specific needs such as `site:` targeting, Scholar, Patents, or News.
|
||||||
|
|
||||||
|
## Tavily Research vs Exa Evidence
|
||||||
|
|
||||||
|
- Tavily Research is best for Phase 1 initial landscape scans, thin-chapter补证据, and Phase 3回炉. Prompt in English, specify source priority, counter-evidence, and structured output. Do not cite its synthesized prose directly.
|
||||||
|
- Exa evidence discovery is best for Phase 2 packet work because highlights/text are compact enough for source-quality scoring and evidence-table mapping.
|
||||||
|
- Serper remains preferred for Scholar, Google Patents, News, and Google-specific `site:` targeting.
|
||||||
|
- Brave remains a cross-check and mixed-language fallback, not the first evidence route.
|
||||||
|
|
||||||
|
## Required Packet Behavior
|
||||||
|
|
||||||
|
- Put search keywords or route notes in `raw_quotes_or_notes` when evidence is weak or no suitable source was found.
|
||||||
|
- FDA/GMP tasks must explicitly check Warning Letters, inspection/enforcement pages, and meeting materials/minutes.
|
||||||
|
- Do not cite search snippets as final evidence when an original regulator, guideline, paper, or official PDF can be reached.
|
||||||
|
- If the candidate sources are not sufficient, stop and record the gap in `open_questions` instead of writing generic prose.
|
||||||
@@ -62,6 +62,10 @@ def write_packet(path: Path, task_id: str, claim: str, source_id: str) -> None:
|
|||||||
"evidence_items": [{"source_id": source_id, "summary": f"{claim} 的证据"}],
|
"evidence_items": [{"source_id": source_id, "summary": f"{claim} 的证据"}],
|
||||||
"counter_evidence": [{"claim": "仍需关注样本量和外推限制", "source_ids": ["src_counter"]}],
|
"counter_evidence": [{"claim": "仍需关注样本量和外推限制", "source_ids": ["src_counter"]}],
|
||||||
"source_ids": [source_id, "src_counter"],
|
"source_ids": [source_id, "src_counter"],
|
||||||
|
"sources": [
|
||||||
|
{"id": source_id, "title": "来源", "url": f"https://example.com/{source_id}"},
|
||||||
|
{"id": "src_counter", "title": "反方来源", "url": "https://example.com/counter"},
|
||||||
|
],
|
||||||
"source_quality_notes": [f"{source_id} Tier 1"],
|
"source_quality_notes": [f"{source_id} Tier 1"],
|
||||||
"open_questions": ["还需要补充中国市场数据"],
|
"open_questions": ["还需要补充中国市场数据"],
|
||||||
"raw_quotes_or_notes": ["English note can remain as source material."],
|
"raw_quotes_or_notes": ["English note can remain as source material."],
|
||||||
@@ -76,6 +80,7 @@ def test_build_chapter_briefs_aggregates_packets_by_chapter(tmp_path: Path) -> N
|
|||||||
{
|
{
|
||||||
"task_id": "ch01-clinical",
|
"task_id": "ch01-clinical",
|
||||||
"chapter_ids": ["ch01"],
|
"chapter_ids": ["ch01"],
|
||||||
|
"chapter_title": "临床证据正在重塑需求判断",
|
||||||
"topic_axis": "clinical",
|
"topic_axis": "clinical",
|
||||||
"questions": ["q"],
|
"questions": ["q"],
|
||||||
"search_routes": ["scholar"],
|
"search_routes": ["scholar"],
|
||||||
@@ -84,6 +89,7 @@ def test_build_chapter_briefs_aggregates_packets_by_chapter(tmp_path: Path) -> N
|
|||||||
{
|
{
|
||||||
"task_id": "ch01-market",
|
"task_id": "ch01-market",
|
||||||
"chapter_ids": ["ch01"],
|
"chapter_ids": ["ch01"],
|
||||||
|
"chapter_title": "临床证据正在重塑需求判断",
|
||||||
"topic_axis": "market",
|
"topic_axis": "market",
|
||||||
"questions": ["q"],
|
"questions": ["q"],
|
||||||
"search_routes": ["news"],
|
"search_routes": ["news"],
|
||||||
@@ -101,6 +107,7 @@ def test_build_chapter_briefs_aggregates_packets_by_chapter(tmp_path: Path) -> N
|
|||||||
brief = briefs[0]
|
brief = briefs[0]
|
||||||
validate_chapter_brief(brief)
|
validate_chapter_brief(brief)
|
||||||
assert brief["chapter_id"] == "ch01"
|
assert brief["chapter_id"] == "ch01"
|
||||||
|
assert brief["chapter_title"] == "临床证据正在重塑需求判断"
|
||||||
assert brief["packet_ids"] == ["ch01-clinical", "ch01-market"]
|
assert brief["packet_ids"] == ["ch01-clinical", "ch01-market"]
|
||||||
assert "src_001" in brief["source_ids"]
|
assert "src_001" in brief["source_ids"]
|
||||||
assert "src_002" in brief["source_ids"]
|
assert "src_002" in brief["source_ids"]
|
||||||
@@ -118,6 +125,50 @@ def test_build_chapter_briefs_aggregates_packets_by_chapter(tmp_path: Path) -> N
|
|||||||
assert (project / "phase2/compressed_findings/ch01.json").exists()
|
assert (project / "phase2/compressed_findings/ch01.json").exists()
|
||||||
|
|
||||||
|
|
||||||
|
def test_build_chapter_briefs_skips_placeholder_packets(tmp_path: Path) -> None:
|
||||||
|
project = tmp_path / "project"
|
||||||
|
cards = [
|
||||||
|
{
|
||||||
|
"task_id": "ch01-good",
|
||||||
|
"chapter_ids": ["ch01"],
|
||||||
|
"chapter_title": "临床证据正在重塑需求判断",
|
||||||
|
"topic_axis": "clinical",
|
||||||
|
"questions": ["q"],
|
||||||
|
"search_routes": ["scholar"],
|
||||||
|
"output_packet": "phase2/packets/ch01-good.json",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"task_id": "ch01-empty",
|
||||||
|
"chapter_ids": ["ch01"],
|
||||||
|
"chapter_title": "临床证据正在重塑需求判断",
|
||||||
|
"topic_axis": "clinical",
|
||||||
|
"questions": ["q"],
|
||||||
|
"search_routes": ["scholar"],
|
||||||
|
"output_packet": "phase2/packets/ch01-empty.json",
|
||||||
|
},
|
||||||
|
]
|
||||||
|
(project / "phase2").mkdir(parents=True)
|
||||||
|
(project / "phase2/task_cards.json").write_text(json.dumps(cards, ensure_ascii=False), encoding="utf-8")
|
||||||
|
write_packet(project / "phase2/packets/ch01-good.json", "ch01-good", "临床证据支持核心判断", "src_001")
|
||||||
|
empty = {
|
||||||
|
"task_id": "ch01-empty",
|
||||||
|
"claims": [],
|
||||||
|
"evidence_items": [],
|
||||||
|
"counter_evidence": [],
|
||||||
|
"source_ids": [],
|
||||||
|
"source_quality_notes": [],
|
||||||
|
"open_questions": [],
|
||||||
|
"raw_quotes_or_notes": [],
|
||||||
|
}
|
||||||
|
(project / "phase2/packets/ch01-empty.json").write_text(json.dumps(empty, ensure_ascii=False), encoding="utf-8")
|
||||||
|
|
||||||
|
briefs = build_chapter_briefs(project)
|
||||||
|
|
||||||
|
assert len(briefs) == 1
|
||||||
|
warnings = json.loads((project / "phase2/brief_warnings.json").read_text(encoding="utf-8"))
|
||||||
|
assert warnings[0]["task_id"] == "ch01-empty"
|
||||||
|
|
||||||
|
|
||||||
def test_chapter_prompt_contains_brief_and_fragmentation_guard() -> None:
|
def test_chapter_prompt_contains_brief_and_fragmentation_guard() -> None:
|
||||||
brief = {
|
brief = {
|
||||||
"chapter_id": "ch01",
|
"chapter_id": "ch01",
|
||||||
@@ -161,6 +212,8 @@ def test_chapter_assembly_worker_writes_markdown(tmp_path: Path) -> None:
|
|||||||
assert output == tmp_path / "phase2/drafts/ch01.md"
|
assert output == tmp_path / "phase2/drafts/ch01.md"
|
||||||
assert "结论先行" in output.read_text(encoding="utf-8")
|
assert "结论先行" in output.read_text(encoding="utf-8")
|
||||||
assert fake.calls[0]["model"] == role.model
|
assert fake.calls[0]["model"] == role.model
|
||||||
|
assert "章节证据分析师" in fake.calls[0]["system"]
|
||||||
|
assert "中文章节组装 worker" in fake.calls[0]["system"]
|
||||||
|
|
||||||
|
|
||||||
def test_validate_chapter_markdown_rejects_unknown_source_ids() -> None:
|
def test_validate_chapter_markdown_rejects_unknown_source_ids() -> None:
|
||||||
|
|||||||
@@ -68,13 +68,22 @@ def test_framework_mentions_ingested_materials(tmp_path: Path) -> None:
|
|||||||
|
|
||||||
assert "phase0/extracted/audit.md" in framework
|
assert "phase0/extracted/audit.md" in framework
|
||||||
assert "NMPA、FDA、EMA、ICH、WHO" in framework
|
assert "NMPA、FDA、EMA、ICH、WHO" in framework
|
||||||
|
assert "Phase1 的职责是大胆假设" in framework
|
||||||
|
assert "本章要解决的问题" in framework
|
||||||
assert "请先确认 `phase1/material_brief.md`" in framework
|
assert "请先确认 `phase1/material_brief.md`" in framework
|
||||||
assert research_brief_md.exists()
|
assert research_brief_md.exists()
|
||||||
assert "任务切分原则" in research_brief_md.read_text(encoding="utf-8")
|
assert "任务切分原则" in research_brief_md.read_text(encoding="utf-8")
|
||||||
|
assert "章节命题与求证计划" in research_brief_md.read_text(encoding="utf-8")
|
||||||
|
assert (project / "phase1" / "hypothesis_map.json").exists()
|
||||||
assert brief["research_method"] == "gmp_quality_operations_diagnosis"
|
assert brief["research_method"] == "gmp_quality_operations_diagnosis"
|
||||||
assert brief["work_language"] == "zh"
|
assert brief["work_language"] == "zh"
|
||||||
|
assert brief["phase2_mode"] == "chapter_integrated"
|
||||||
|
assert brief["central_thesis"]
|
||||||
|
assert brief["chapter_planning"][0]["phase2_prompt_context"]
|
||||||
assert brief["task_planning"]["required_skills"]
|
assert brief["task_planning"]["required_skills"]
|
||||||
assert brief["task_planning"]["search_routes_by_axis"]["counter"] == ["scholar", "general"]
|
assert brief["task_planning"]["phase2_mode"] == "chapter_integrated"
|
||||||
|
assert brief["task_planning"]["search_routes_by_axis"]["counter"] == ["fda", "scholar", "evidence", "general"]
|
||||||
|
assert brief["task_planning"]["search_routes_by_axis"]["quality_system_gap"] == ["fda", "evidence", "general"]
|
||||||
assert brief["phase2_inputs"]["framework_path"] == "phase1/framework.md"
|
assert brief["phase2_inputs"]["framework_path"] == "phase1/framework.md"
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,80 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
REPO_ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
if str(REPO_ROOT) not in sys.path:
|
||||||
|
sys.path.insert(0, str(REPO_ROOT))
|
||||||
|
|
||||||
|
from scripts.runtime.review import (
|
||||||
|
build_phase3_model_review_context,
|
||||||
|
build_phase3_model_critique,
|
||||||
|
phase3_model_review_system_prompt,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class FakeClient:
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self.calls: list[dict[str, object]] = []
|
||||||
|
|
||||||
|
def chat_complete(self, **kwargs) -> str:
|
||||||
|
self.calls.append(kwargs)
|
||||||
|
return "# Phase 3 Opus 4.7 独立审校\n\n## 总体判定\n\n回炉 Phase2。\n"
|
||||||
|
|
||||||
|
|
||||||
|
def make_project(tmp_path: Path) -> Path:
|
||||||
|
project = tmp_path / "project"
|
||||||
|
(project / "phase1").mkdir(parents=True)
|
||||||
|
(project / "phase2/drafts").mkdir(parents=True)
|
||||||
|
(project / "phase2/compressed_findings").mkdir(parents=True)
|
||||||
|
(project / "manifest.json").write_text(
|
||||||
|
json.dumps({"topic": "白帆测试项目", "phase3": {}}, ensure_ascii=False),
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
(project / "phase1/framework.md").write_text("## 第1章 质量体系判断\n", encoding="utf-8")
|
||||||
|
(project / "phase2/drafts/ch01.md").write_text("## 质量体系判断\n\n正文。[src_001]\n", encoding="utf-8")
|
||||||
|
(project / "phase2/sources.jsonl").write_text('{"id":"src_001","title":"来源","url":"https://www.fda.gov/example"}\n', encoding="utf-8")
|
||||||
|
(project / "phase2/compressed_findings/ch01.json").write_text(
|
||||||
|
json.dumps(
|
||||||
|
{
|
||||||
|
"chapter_id": "ch01",
|
||||||
|
"chapter_title": "质量体系判断",
|
||||||
|
"packet_ids": ["ch01-a"],
|
||||||
|
"chapter_thesis": "质量体系需要补证据",
|
||||||
|
"key_findings": [],
|
||||||
|
"evidence_landings": [],
|
||||||
|
"counter_evidence": [],
|
||||||
|
"source_ids": ["src_001"],
|
||||||
|
"open_questions": [],
|
||||||
|
"writing_plan": [],
|
||||||
|
},
|
||||||
|
ensure_ascii=False,
|
||||||
|
),
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
return project
|
||||||
|
|
||||||
|
|
||||||
|
def test_phase3_model_context_contains_structured_inputs(tmp_path: Path) -> None:
|
||||||
|
project = make_project(tmp_path)
|
||||||
|
|
||||||
|
context = build_phase3_model_review_context(project)
|
||||||
|
|
||||||
|
assert "Deterministic Review Baseline" in context
|
||||||
|
assert "Compressed Findings" in context
|
||||||
|
assert "Chapter Drafts" in context
|
||||||
|
assert "src_001" in context
|
||||||
|
|
||||||
|
|
||||||
|
def test_phase3_model_review_calls_requested_model_and_writes_critique(tmp_path: Path) -> None:
|
||||||
|
project = make_project(tmp_path)
|
||||||
|
fake = FakeClient()
|
||||||
|
|
||||||
|
out = build_phase3_model_critique(project, client=fake, model="zenmux-anthropic/claude-opus-4-7")
|
||||||
|
|
||||||
|
assert out.exists()
|
||||||
|
assert fake.calls[0]["model"] == "zenmux-anthropic/claude-opus-4-7"
|
||||||
|
assert "独立总编审校" in fake.calls[0]["system"]
|
||||||
|
assert "FDA/NMPA/EMA/ICH/WHO" in phase3_model_review_system_prompt()
|
||||||
@@ -0,0 +1,19 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
REPO_ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
if str(REPO_ROOT) not in sys.path:
|
||||||
|
sys.path.insert(0, str(REPO_ROOT))
|
||||||
|
|
||||||
|
from scripts.polish import build_polish_system_prompt
|
||||||
|
|
||||||
|
|
||||||
|
def test_polish_system_prompt_loads_humanizer_and_output_hygiene() -> None:
|
||||||
|
prompt = build_polish_system_prompt()
|
||||||
|
|
||||||
|
assert "# Skill: humanizer-cn" in prompt
|
||||||
|
assert "CN-1" in prompt
|
||||||
|
assert "# Skill: output-hygiene" in prompt
|
||||||
|
assert "禁止词" in prompt
|
||||||
@@ -10,6 +10,7 @@ if str(REPO_ROOT) not in sys.path:
|
|||||||
|
|
||||||
from scripts.reporting.fonts import resolve_quarto_fonts
|
from scripts.reporting.fonts import resolve_quarto_fonts
|
||||||
from scripts.reporting.references import build_references_block
|
from scripts.reporting.references import build_references_block
|
||||||
|
from scripts.number_citations import number_citations
|
||||||
|
|
||||||
|
|
||||||
def test_build_references_block_uses_only_cited_sources(tmp_path: Path) -> None:
|
def test_build_references_block_uses_only_cited_sources(tmp_path: Path) -> None:
|
||||||
@@ -32,3 +33,38 @@ def test_resolve_quarto_fonts_returns_stable_defaults_for_missing_dir(tmp_path:
|
|||||||
assert fonts.main_font == "Source Han Serif CN"
|
assert fonts.main_font == "Source Han Serif CN"
|
||||||
assert fonts.sans_font == "Source Han Sans CN"
|
assert fonts.sans_font == "Source Han Sans CN"
|
||||||
assert fonts.requires_system_fonts is True
|
assert fonts.requires_system_fonts is True
|
||||||
|
|
||||||
|
|
||||||
|
def test_number_citations_replaces_source_ids_and_keeps_url() -> None:
|
||||||
|
text = "# 报告\n\n关键判断。[src_a, src_b]\n\n## 参考文献\n\n旧列表\n"
|
||||||
|
sources = {
|
||||||
|
"src_a": {"title": "法规 A", "url": "https://example.com/a"},
|
||||||
|
"src_b": {"title": "指南 B", "url": "https://example.com/b"},
|
||||||
|
}
|
||||||
|
|
||||||
|
numbered, records = number_citations(text=text, sources=sources)
|
||||||
|
|
||||||
|
assert "关键判断。<sup>[1, 2]</sup>" in numbered
|
||||||
|
assert "[src_a" not in numbered
|
||||||
|
assert "## 参考来源清单" in numbered
|
||||||
|
assert "1. 法规 A. https://example.com/a" in numbered
|
||||||
|
assert "旧列表" not in numbered
|
||||||
|
assert [record["source_id"] for record in records] == ["src_a", "src_b"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_number_citations_deduplicates_same_underlying_source() -> None:
|
||||||
|
text = "甲。[src_a]\n\n乙。[src_b, src_c]\n"
|
||||||
|
sources = {
|
||||||
|
"src_a": {"title": "同一报告 OCR", "path": "phase0/report.md"},
|
||||||
|
"src_b": {"title": "同一报告", "path": "phase0/report.md"},
|
||||||
|
"src_c": {"title": "法规 C", "url": "https://example.com/c"},
|
||||||
|
}
|
||||||
|
|
||||||
|
numbered, records = number_citations(text=text, sources=sources)
|
||||||
|
|
||||||
|
assert "甲。<sup>[1]</sup>" in numbered
|
||||||
|
assert "乙。<sup>[1, 2]</sup>" in numbered
|
||||||
|
assert numbered.count("同一报告") == 1
|
||||||
|
assert "同一报告 OCR" not in numbered
|
||||||
|
assert len(records) == 2
|
||||||
|
assert records[0]["source_ids"] == ["src_a", "src_b"]
|
||||||
|
|||||||
@@ -11,7 +11,7 @@ if str(REPO_ROOT) not in sys.path:
|
|||||||
from scripts.runtime.roles import resolve_runtime_profile
|
from scripts.runtime.roles import resolve_runtime_profile
|
||||||
from scripts.runtime.sources import append_packet_sources, rebuild_sources_from_packets
|
from scripts.runtime.sources import append_packet_sources, rebuild_sources_from_packets
|
||||||
from scripts.runtime.tasks import TaskCard
|
from scripts.runtime.tasks import TaskCard
|
||||||
from scripts.runtime.workers import PacketWorker, build_search_context
|
from scripts.runtime.workers import PacketWorker, build_material_context, build_route_query, build_search_context, normalize_packet_against_context
|
||||||
|
|
||||||
|
|
||||||
class FakeSearchProvider:
|
class FakeSearchProvider:
|
||||||
@@ -57,6 +57,66 @@ def test_build_search_context_assigns_stable_source_ids() -> None:
|
|||||||
assert context["routes_used"] == ["scholar", "general"]
|
assert context["routes_used"] == ["scholar", "general"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_fda_route_query_uses_english_axis_terms_not_chinese_title() -> None:
|
||||||
|
card = TaskCard(
|
||||||
|
task_id="ch10-fda_enforcement_precedents",
|
||||||
|
chapter_ids=["ch10"],
|
||||||
|
topic_axis="fda_enforcement_precedents",
|
||||||
|
questions=["立即纠偏、体系补强、能力建设三层整改路线图必须绑定 owner、关闭证据和复核机制"],
|
||||||
|
search_routes=["fda"],
|
||||||
|
output_packet="phase2/packets/ch10-fda_enforcement_precedents.json",
|
||||||
|
chapter_title="立即纠偏、体系补强、能力建设三层整改路线图必须绑定 owner、关闭证据和复核机制",
|
||||||
|
)
|
||||||
|
|
||||||
|
query = build_route_query(card, "fda")
|
||||||
|
|
||||||
|
assert "立即纠偏" not in query
|
||||||
|
assert "CAPA" in query
|
||||||
|
assert "remediation" in query
|
||||||
|
assert "verification evidence" in query
|
||||||
|
|
||||||
|
|
||||||
|
def test_integrated_scholar_query_does_not_leak_internal_axis_or_cjk_punctuation() -> None:
|
||||||
|
card = TaskCard(
|
||||||
|
task_id="ch07-chapter_integrated",
|
||||||
|
chapter_ids=["ch07"],
|
||||||
|
topic_axis="chapter_integrated",
|
||||||
|
questions=["人员能力:培训有效性比培训记录更关键"],
|
||||||
|
search_routes=["scholar"],
|
||||||
|
output_packet="phase2/packets/ch07-chapter_integrated.json",
|
||||||
|
chapter_title="人员能力:培训有效性比培训记录更关键",
|
||||||
|
)
|
||||||
|
|
||||||
|
query = build_route_query(card, "scholar")
|
||||||
|
|
||||||
|
assert "chapter_integrated" not in query
|
||||||
|
assert "、" not in query
|
||||||
|
assert " " not in query
|
||||||
|
assert "training" in query
|
||||||
|
assert "quality" in query
|
||||||
|
assert not any("\u4e00" <= char <= "\u9fff" for char in query)
|
||||||
|
|
||||||
|
|
||||||
|
def test_evidence_route_query_is_short_english_candidate_evidence_query() -> None:
|
||||||
|
card = TaskCard(
|
||||||
|
task_id="ch08-chapter_integrated",
|
||||||
|
chapter_ids=["ch08"],
|
||||||
|
topic_axis="chapter_integrated",
|
||||||
|
questions=["运营管理需要建立跨部门节奏、问题升级、指标看板和管理层 review"],
|
||||||
|
search_routes=["evidence"],
|
||||||
|
output_packet="phase2/packets/ch08-chapter_integrated.json",
|
||||||
|
chapter_title="运营管理需要建立跨部门节奏、问题升级、指标看板和管理层 review",
|
||||||
|
)
|
||||||
|
|
||||||
|
query = build_route_query(card, "evidence")
|
||||||
|
|
||||||
|
assert "evidence" in query
|
||||||
|
assert "quality" in query
|
||||||
|
assert "operations" in query
|
||||||
|
assert "运营管理" not in query
|
||||||
|
assert not any("\u4e00" <= char <= "\u9fff" for char in query)
|
||||||
|
|
||||||
|
|
||||||
def test_packet_worker_includes_search_context_in_prompt() -> None:
|
def test_packet_worker_includes_search_context_in_prompt() -> None:
|
||||||
context = build_search_context(sample_card(), FakeSearchProvider(), num_results_per_route=1)
|
context = build_search_context(sample_card(), FakeSearchProvider(), num_results_per_route=1)
|
||||||
response = {
|
response = {
|
||||||
@@ -79,7 +139,71 @@ def test_packet_worker_includes_search_context_in_prompt() -> None:
|
|||||||
assert "candidate_sources" in fake.calls[0]["user"]
|
assert "candidate_sources" in fake.calls[0]["user"]
|
||||||
|
|
||||||
|
|
||||||
def test_append_packet_sources_dedupes_by_url(tmp_path: Path) -> None:
|
def test_normalize_packet_fills_source_ids_and_sources_from_context() -> None:
|
||||||
|
context = {
|
||||||
|
"candidate_sources": [
|
||||||
|
{"id": "src_a", "title": "A", "url": "https://example.com/a", "tier": "Tier 2", "score": 7}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
packet = {
|
||||||
|
"task_id": "ch01",
|
||||||
|
"claims": [{"claim": "判断", "source_ids": ["src_a"]}],
|
||||||
|
"evidence_items": [{"source_id": "src_a", "summary": "证据"}],
|
||||||
|
"counter_evidence": [{"claim": "反方", "source_ids": ["src_a"]}],
|
||||||
|
"source_quality_notes": [],
|
||||||
|
"open_questions": [],
|
||||||
|
"raw_quotes_or_notes": [],
|
||||||
|
}
|
||||||
|
|
||||||
|
normalized = normalize_packet_against_context(packet, context, None)
|
||||||
|
|
||||||
|
assert normalized["source_ids"] == ["src_a"]
|
||||||
|
assert normalized["sources"] == context["candidate_sources"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_material_context_is_loaded_and_allowed_as_source(tmp_path: Path) -> None:
|
||||||
|
project = tmp_path / "project"
|
||||||
|
material = project / "phase0/extracted/audit.md"
|
||||||
|
material.parent.mkdir(parents=True)
|
||||||
|
material.write_text("白帆现场发现:偏差调查未闭环。", encoding="utf-8")
|
||||||
|
card = TaskCard(
|
||||||
|
task_id="ch01-chapter_integrated",
|
||||||
|
chapter_ids=["ch01"],
|
||||||
|
topic_axis="chapter_integrated",
|
||||||
|
questions=["q"],
|
||||||
|
search_routes=[],
|
||||||
|
output_packet="phase2/packets/ch01-chapter_integrated.json",
|
||||||
|
allowed_materials=["phase0/extracted/audit.md"],
|
||||||
|
)
|
||||||
|
context = build_material_context(card, project)
|
||||||
|
response = {
|
||||||
|
"task_id": "ch01-chapter_integrated",
|
||||||
|
"claims": [{"claim": "现场材料显示偏差调查需要补强", "source_ids": [context["materials"][0]["source_id"]]}],
|
||||||
|
"evidence_items": [{"source_id": context["materials"][0]["source_id"], "summary": "偏差调查未闭环。"}],
|
||||||
|
"counter_evidence": [{"claim": "需与完整审计报告交叉确认", "source_ids": [context["materials"][0]["source_id"]]}],
|
||||||
|
"source_ids": [context["materials"][0]["source_id"]],
|
||||||
|
"sources": [
|
||||||
|
{
|
||||||
|
"id": context["materials"][0]["source_id"],
|
||||||
|
"title": "audit.md",
|
||||||
|
"url": "phase0/extracted/audit.md",
|
||||||
|
"tier": "local_material",
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"source_quality_notes": ["本地材料作为起点证据"],
|
||||||
|
"open_questions": [],
|
||||||
|
"raw_quotes_or_notes": ["白帆现场发现:偏差调查未闭环。"],
|
||||||
|
}
|
||||||
|
fake = FakeClient(response)
|
||||||
|
role = resolve_runtime_profile(profile="medium").role_for_task("evidence_packet")
|
||||||
|
|
||||||
|
packet = PacketWorker(role=role, client=fake, project_root=project).run(card)
|
||||||
|
|
||||||
|
assert packet["source_ids"] == [context["materials"][0]["source_id"]]
|
||||||
|
assert "白帆现场发现" in fake.calls[0]["user"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_append_packet_sources_preserves_distinct_source_ids_for_same_url(tmp_path: Path) -> None:
|
||||||
packet = {
|
packet = {
|
||||||
"sources": [
|
"sources": [
|
||||||
{"id": "src_a", "title": "A", "url": "https://example.com/a", "tier": "Tier 2", "score": 7},
|
{"id": "src_a", "title": "A", "url": "https://example.com/a", "tier": "Tier 2", "score": 7},
|
||||||
@@ -89,11 +213,11 @@ def test_append_packet_sources_dedupes_by_url(tmp_path: Path) -> None:
|
|||||||
|
|
||||||
written = append_packet_sources(tmp_path / "sources.jsonl", packet)
|
written = append_packet_sources(tmp_path / "sources.jsonl", packet)
|
||||||
|
|
||||||
assert written == 1
|
assert written == 2
|
||||||
assert len((tmp_path / "sources.jsonl").read_text(encoding="utf-8").splitlines()) == 1
|
assert len((tmp_path / "sources.jsonl").read_text(encoding="utf-8").splitlines()) == 2
|
||||||
|
|
||||||
|
|
||||||
def test_rebuild_sources_from_packets_dedupes_manual_packets(tmp_path: Path) -> None:
|
def test_rebuild_sources_from_packets_preserves_distinct_source_ids(tmp_path: Path) -> None:
|
||||||
project = tmp_path / "project"
|
project = tmp_path / "project"
|
||||||
packets = project / "phase2" / "packets"
|
packets = project / "phase2" / "packets"
|
||||||
packets.mkdir(parents=True)
|
packets.mkdir(parents=True)
|
||||||
@@ -109,7 +233,34 @@ def test_rebuild_sources_from_packets_dedupes_manual_packets(tmp_path: Path) ->
|
|||||||
count = rebuild_sources_from_packets(project)
|
count = rebuild_sources_from_packets(project)
|
||||||
|
|
||||||
lines = (project / "phase2" / "sources.jsonl").read_text(encoding="utf-8").splitlines()
|
lines = (project / "phase2" / "sources.jsonl").read_text(encoding="utf-8").splitlines()
|
||||||
assert count == 2
|
assert count == 3
|
||||||
assert len(lines) == 2
|
assert len(lines) == 3
|
||||||
assert "src_001" in lines[0]
|
assert "src_001" in lines[0]
|
||||||
assert "src_003" in lines[1]
|
assert "src_002" in lines[1]
|
||||||
|
assert "src_003" in lines[2]
|
||||||
|
|
||||||
|
|
||||||
|
def test_rebuild_sources_preserves_cache_metadata(tmp_path: Path) -> None:
|
||||||
|
project = tmp_path / "project"
|
||||||
|
packets = project / "phase2" / "packets"
|
||||||
|
packets.mkdir(parents=True)
|
||||||
|
source = {"id": "src_001", "title": "A", "url": "https://example.com/a"}
|
||||||
|
(packets / "ch01-a.json").write_text(json.dumps({"sources": [source]}, ensure_ascii=False), encoding="utf-8")
|
||||||
|
(project / "phase2/sources.jsonl").write_text(
|
||||||
|
json.dumps(
|
||||||
|
{
|
||||||
|
**source,
|
||||||
|
"cached_text_path": "phase2/source_cache/md/src_001.md",
|
||||||
|
"cache_status": "fetched",
|
||||||
|
},
|
||||||
|
ensure_ascii=False,
|
||||||
|
)
|
||||||
|
+ "\n",
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
|
||||||
|
rebuild_sources_from_packets(project)
|
||||||
|
|
||||||
|
row = json.loads((project / "phase2/sources.jsonl").read_text(encoding="utf-8"))
|
||||||
|
assert row["cached_text_path"] == "phase2/source_cache/md/src_001.md"
|
||||||
|
assert row["cache_status"] == "fetched"
|
||||||
|
|||||||
@@ -0,0 +1,70 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
REPO_ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
if str(REPO_ROOT) not in sys.path:
|
||||||
|
sys.path.insert(0, str(REPO_ROOT))
|
||||||
|
|
||||||
|
from scripts.runtime.source_cache import cache_sources, is_important_source
|
||||||
|
|
||||||
|
|
||||||
|
class FakeResponse:
|
||||||
|
headers = {"content-type": "text/html; charset=utf-8"}
|
||||||
|
url = "https://www.fda.gov/example"
|
||||||
|
content = b"<html><body><h1>FDA Guidance</h1><p>Important CGMP text.</p></body></html>"
|
||||||
|
|
||||||
|
def raise_for_status(self) -> None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
class FakeClient:
|
||||||
|
def __enter__(self) -> "FakeClient":
|
||||||
|
return self
|
||||||
|
|
||||||
|
def __exit__(self, *_args) -> None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
def get(self, url: str) -> FakeResponse:
|
||||||
|
assert url == "https://www.fda.gov/example"
|
||||||
|
return FakeResponse()
|
||||||
|
|
||||||
|
def close(self) -> None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def test_is_important_source_detects_official_regulator() -> None:
|
||||||
|
assert is_important_source({"url": "https://www.fda.gov/example", "title": "FDA"})
|
||||||
|
assert not is_important_source({"url": "https://example.com/blog", "title": "Blog"})
|
||||||
|
|
||||||
|
|
||||||
|
def test_cache_sources_writes_markdown_and_updates_registry(tmp_path: Path, monkeypatch) -> None:
|
||||||
|
project = tmp_path / "project"
|
||||||
|
sources = project / "phase2" / "sources.jsonl"
|
||||||
|
sources.parent.mkdir(parents=True)
|
||||||
|
sources.write_text(
|
||||||
|
json.dumps(
|
||||||
|
{
|
||||||
|
"id": "src_fda_001",
|
||||||
|
"title": "FDA Guidance",
|
||||||
|
"url": "https://www.fda.gov/example",
|
||||||
|
"tier": "Tier 1",
|
||||||
|
},
|
||||||
|
ensure_ascii=False,
|
||||||
|
)
|
||||||
|
+ "\n",
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
|
||||||
|
monkeypatch.setattr("scripts.runtime.source_cache.httpx.Client", lambda **_kwargs: FakeClient())
|
||||||
|
|
||||||
|
results = cache_sources(project)
|
||||||
|
|
||||||
|
rows = [json.loads(line) for line in sources.read_text(encoding="utf-8").splitlines()]
|
||||||
|
assert len(results) == 1
|
||||||
|
assert rows[0]["cached_text_path"].startswith("phase2/source_cache/md/")
|
||||||
|
cached = project / rows[0]["cached_text_path"]
|
||||||
|
assert cached.exists()
|
||||||
|
assert "Important CGMP text." in cached.read_text(encoding="utf-8")
|
||||||
@@ -68,6 +68,10 @@ def test_research_build_briefs_does_not_overwrite_existing_packets(tmp_path: Pat
|
|||||||
"evidence_items": [{"source_id": "src_001", "summary": "证据"}],
|
"evidence_items": [{"source_id": "src_001", "summary": "证据"}],
|
||||||
"counter_evidence": [{"claim": "限制", "source_ids": ["src_002"]}],
|
"counter_evidence": [{"claim": "限制", "source_ids": ["src_002"]}],
|
||||||
"source_ids": ["src_001", "src_002"],
|
"source_ids": ["src_001", "src_002"],
|
||||||
|
"sources": [
|
||||||
|
{"id": "src_001", "title": "来源1", "url": "https://example.com/1"},
|
||||||
|
{"id": "src_002", "title": "来源2", "url": "https://example.com/2"},
|
||||||
|
],
|
||||||
"source_quality_notes": ["src_001 Tier 1"],
|
"source_quality_notes": ["src_001 Tier 1"],
|
||||||
"open_questions": [],
|
"open_questions": [],
|
||||||
"raw_quotes_or_notes": [],
|
"raw_quotes_or_notes": [],
|
||||||
@@ -80,6 +84,28 @@ def test_research_build_briefs_does_not_overwrite_existing_packets(tmp_path: Pat
|
|||||||
assert "真实证据不能被 skeleton 覆盖" in packet_path.read_text(encoding="utf-8")
|
assert "真实证据不能被 skeleton 覆盖" in packet_path.read_text(encoding="utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
def test_codex_native_profile_does_not_claim_python_core_model_execution(tmp_path: Path) -> None:
|
||||||
|
project = tmp_path / "project"
|
||||||
|
(project / "phase1").mkdir(parents=True)
|
||||||
|
(project / "manifest.json").write_text(
|
||||||
|
'{"research_method": "mckinsey_market", "phase1": {"approved": true}, "phase2": {}}\n',
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
(project / "phase1/framework.md").write_text(
|
||||||
|
"## 第1章 临床证据正在重塑需求判断\n\n研究思路。",
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
|
||||||
|
args = dr.build_parser().parse_args(["research", str(project), "--profile", "codex_native", "--execute-packets"])
|
||||||
|
|
||||||
|
try:
|
||||||
|
dr.cmd_research(args)
|
||||||
|
except SystemExit as exc:
|
||||||
|
assert "not Codex App built-in models" in str(exc)
|
||||||
|
else:
|
||||||
|
raise AssertionError("codex_native must not execute through Python external clients")
|
||||||
|
|
||||||
|
|
||||||
def test_packet_state_counts_ignores_stale_errors_for_ready_packets(tmp_path: Path) -> None:
|
def test_packet_state_counts_ignores_stale_errors_for_ready_packets(tmp_path: Path) -> None:
|
||||||
project = tmp_path / "project"
|
project = tmp_path / "project"
|
||||||
(project / "phase2/packets").mkdir(parents=True)
|
(project / "phase2/packets").mkdir(parents=True)
|
||||||
@@ -184,6 +210,30 @@ def test_init_and_frame_create_executable_python_core_project(tmp_path: Path) ->
|
|||||||
assert "中文" in framework
|
assert "中文" in framework
|
||||||
|
|
||||||
|
|
||||||
|
def test_frame_can_preserve_existing_outline(tmp_path: Path) -> None:
|
||||||
|
project = tmp_path / "custom-outline"
|
||||||
|
(project / "phase1").mkdir(parents=True)
|
||||||
|
(project / "manifest.json").write_text(
|
||||||
|
'{"topic": "自定义研究", "research_method": "mckinsey_market", "target_words": 12000, "phase1": {}}\n',
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
(project / "phase1/framework.md").write_text(
|
||||||
|
"## 第1章 第一条自定义主线\n\n## 第2章 第二条自定义主线\n\n## 第3章 第三条自定义主线\n\n"
|
||||||
|
"## 第4章 第四条自定义主线\n\n## 第5章 第五条自定义主线\n\n## 第6章 第六条自定义主线\n\n"
|
||||||
|
"## 第7章 第七条自定义主线\n\n## 第8章 第八条自定义主线\n",
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
|
||||||
|
args = dr.build_parser().parse_args(["frame", str(project), "--preserve-existing-outline"])
|
||||||
|
|
||||||
|
assert dr.cmd_frame(args) == 0
|
||||||
|
framework = (project / "phase1/framework.md").read_text(encoding="utf-8")
|
||||||
|
brief = json.loads((project / "phase1/research_brief.json").read_text(encoding="utf-8"))
|
||||||
|
assert "第一条自定义主线" in framework
|
||||||
|
assert "本章要解决的问题" in framework
|
||||||
|
assert brief["chapter_planning"][0]["title"] == "第一条自定义主线"
|
||||||
|
|
||||||
|
|
||||||
def test_review_writes_phase3_critique(tmp_path: Path) -> None:
|
def test_review_writes_phase3_critique(tmp_path: Path) -> None:
|
||||||
project = tmp_path / "project"
|
project = tmp_path / "project"
|
||||||
(project / "phase1").mkdir(parents=True)
|
(project / "phase1").mkdir(parents=True)
|
||||||
@@ -205,6 +255,54 @@ def test_review_writes_phase3_critique(tmp_path: Path) -> None:
|
|||||||
assert "src_001" in text
|
assert "src_001" in text
|
||||||
|
|
||||||
|
|
||||||
|
def test_review_model_dry_run_exposes_opus_context_plan(tmp_path: Path, capsys) -> None:
|
||||||
|
project = tmp_path / "project"
|
||||||
|
project.mkdir()
|
||||||
|
(project / "manifest.json").write_text('{"topic": "测试项目"}\n', encoding="utf-8")
|
||||||
|
|
||||||
|
args = dr.build_parser().parse_args(["review", str(project), "--model-review", "--dry-run"])
|
||||||
|
|
||||||
|
assert dr.cmd_review(args) == 0
|
||||||
|
out = capsys.readouterr().out
|
||||||
|
assert "zenmux-anthropic/claude-opus-4-7" in out
|
||||||
|
assert "review_context_opus_4_7.md" in out
|
||||||
|
|
||||||
|
|
||||||
|
def test_finalize_polish_dry_run_uses_polish_source_argument(tmp_path: Path, capsys) -> None:
|
||||||
|
project = tmp_path / "project"
|
||||||
|
(project / "phase4").mkdir(parents=True)
|
||||||
|
(project / "manifest.json").write_text(
|
||||||
|
'{"model_profile": "medium", "phase4": {}}\n',
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
(project / "phase4/final_zh.md").write_text("# 中文终稿\n", encoding="utf-8")
|
||||||
|
|
||||||
|
args = dr.build_parser().parse_args(["finalize", str(project), "--polish", "--dry-run"])
|
||||||
|
|
||||||
|
assert dr.cmd_finalize(args) == 0
|
||||||
|
out = capsys.readouterr().out
|
||||||
|
assert "scripts/polish.py" in out
|
||||||
|
assert "--source phase4/final_zh.md" in out
|
||||||
|
assert "--input phase4/final_zh.md" not in out.split("scripts/polish.py", 1)[1]
|
||||||
|
|
||||||
|
|
||||||
|
def test_finalize_number_citations_dry_run_builds_numbered_markdown(tmp_path: Path, capsys) -> None:
|
||||||
|
project = tmp_path / "project"
|
||||||
|
(project / "phase4").mkdir(parents=True)
|
||||||
|
(project / "manifest.json").write_text(
|
||||||
|
'{"model_profile": "medium", "phase4": {}}\n',
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
(project / "phase4/final_zh.md").write_text("# 中文终稿\n\n正文。[src_001]\n", encoding="utf-8")
|
||||||
|
|
||||||
|
args = dr.build_parser().parse_args(["finalize", str(project), "--number-citations", "--dry-run"])
|
||||||
|
|
||||||
|
assert dr.cmd_finalize(args) == 0
|
||||||
|
out = capsys.readouterr().out
|
||||||
|
assert "scripts/number_citations.py" in out
|
||||||
|
assert "--input phase4/final_zh_numbered.md" in out
|
||||||
|
|
||||||
|
|
||||||
def test_run_new_topic_initializes_and_frames_project(tmp_path: Path) -> None:
|
def test_run_new_topic_initializes_and_frames_project(tmp_path: Path) -> None:
|
||||||
args = dr.build_parser().parse_args(
|
args = dr.build_parser().parse_args(
|
||||||
[
|
[
|
||||||
|
|||||||
@@ -12,6 +12,8 @@ if str(REPO_ROOT) not in sys.path:
|
|||||||
|
|
||||||
from scripts.lib.model_config import resolve_model_profile
|
from scripts.lib.model_config import resolve_model_profile
|
||||||
from scripts.runtime.roles import resolve_runtime_profile
|
from scripts.runtime.roles import resolve_runtime_profile
|
||||||
|
from scripts.runtime.methods import ResearchMethodRegistry
|
||||||
|
from scripts.runtime.phase1 import build_chapter_planning
|
||||||
from scripts.runtime.skills import SkillRegistry
|
from scripts.runtime.skills import SkillRegistry
|
||||||
from scripts.runtime.tasks import (
|
from scripts.runtime.tasks import (
|
||||||
TaskCard,
|
TaskCard,
|
||||||
@@ -78,6 +80,7 @@ def test_generate_task_cards_from_chinese_framework() -> None:
|
|||||||
"ch02-regulatory",
|
"ch02-regulatory",
|
||||||
]
|
]
|
||||||
assert cards[0].output_packet == "phase2/packets/ch01-clinical.json"
|
assert cards[0].output_packet == "phase2/packets/ch01-clinical.json"
|
||||||
|
assert cards[0].chapter_title == "GLP-1 产业链的增量来自适应症扩张"
|
||||||
assert cards[0].research_goal
|
assert cards[0].research_goal
|
||||||
assert "search-gateway" in cards[0].required_skills
|
assert "search-gateway" in cards[0].required_skills
|
||||||
assert cards[0].expected_evidence["min_tier_1_2_sources"] == 2
|
assert cards[0].expected_evidence["min_tier_1_2_sources"] == 2
|
||||||
@@ -121,6 +124,122 @@ def test_generate_task_cards_from_research_brief_carries_prompt_and_skills() ->
|
|||||||
assert "search-gateway" in cards[0].required_skills
|
assert "search-gateway" in cards[0].required_skills
|
||||||
|
|
||||||
|
|
||||||
|
def test_gmp_task_cards_include_fda_enforcement_route() -> None:
|
||||||
|
brief = {
|
||||||
|
"research_method": "gmp_quality_operations_diagnosis",
|
||||||
|
"task_planning": {
|
||||||
|
"search_routes_by_axis": {
|
||||||
|
"quality_system_gap": ["fda", "general"],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
framework = "## 第1章 偏差和 CAPA 闭环能力决定质量体系可信度\n\n研究思路。"
|
||||||
|
|
||||||
|
cards = generate_task_cards_from_research_brief(
|
||||||
|
"baifan-test",
|
||||||
|
framework,
|
||||||
|
brief,
|
||||||
|
axes=["quality_system_gap"],
|
||||||
|
)
|
||||||
|
|
||||||
|
assert cards[0].search_routes == ["fda", "general"]
|
||||||
|
assert "FDA Warning Letters" in " ".join(cards[0].questions)
|
||||||
|
assert "fda_warning_letter_or_meeting_record" in cards[0].expected_evidence["preferred_evidence_types"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_integrated_chapter_mode_is_method_driven_not_gmp_hardcoded() -> None:
|
||||||
|
brief = {
|
||||||
|
"research_method": "mckinsey_market",
|
||||||
|
"phase2_mode": "chapter_integrated",
|
||||||
|
"task_planning": {},
|
||||||
|
}
|
||||||
|
framework = "## 第1章 市场需求正在被支付政策重塑\n\n研究思路。"
|
||||||
|
|
||||||
|
cards = generate_task_cards_from_research_brief(
|
||||||
|
"market-test",
|
||||||
|
framework,
|
||||||
|
brief,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert [card.task_id for card in cards] == ["ch01-chapter_integrated"]
|
||||||
|
assert "literature evidence" in " ".join(cards[0].questions)
|
||||||
|
assert "FDA Warning Letters" not in " ".join(cards[0].questions)
|
||||||
|
|
||||||
|
|
||||||
|
def test_integrated_task_card_uses_phase1_chapter_planning() -> None:
|
||||||
|
brief = {
|
||||||
|
"research_method": "gmp_quality_operations_diagnosis",
|
||||||
|
"phase2_mode": "chapter_integrated",
|
||||||
|
"chapter_planning": [
|
||||||
|
{
|
||||||
|
"chapter_id": "ch01",
|
||||||
|
"title": "审计发现应先转化为商业化阶段门缺口",
|
||||||
|
"core_question": "本章要判断审计发现是否反映阶段门缺口。",
|
||||||
|
"bold_hypothesis": "大胆假设:风险项计数低估了商业化 readiness 缺口。",
|
||||||
|
"verification_plan": ["提取现场材料原文", "检索官方法规和执法案例"],
|
||||||
|
"evidence_lanes": ["site audit findings", "official baseline"],
|
||||||
|
"minimum_evidence": {"local_material_quotes": 2},
|
||||||
|
"phase2_prompt_context": "章节:ch01\n必须围绕阶段门缺口求证。",
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"task_planning": {"phase2_mode": "chapter_integrated"},
|
||||||
|
}
|
||||||
|
framework = "## 第1章 审计发现应先转化为商业化阶段门缺口\n\n研究思路。"
|
||||||
|
|
||||||
|
cards = generate_task_cards_from_research_brief("baifan-test", framework, brief)
|
||||||
|
|
||||||
|
assert cards[0].prompt_brief == "章节:ch01\n必须围绕阶段门缺口求证。"
|
||||||
|
assert cards[0].research_goal == "本章要判断审计发现是否反映阶段门缺口。"
|
||||||
|
assert "大胆假设:风险项计数低估了商业化 readiness 缺口。" in cards[0].questions
|
||||||
|
assert cards[0].expected_evidence["phase1_minimum_evidence"] == {"local_material_quotes": 2}
|
||||||
|
assert cards[0].expected_evidence["must_address_phase1_hypothesis"] is True
|
||||||
|
assert any("Phase1 的大胆假设" in item for item in cards[0].stop_conditions)
|
||||||
|
|
||||||
|
|
||||||
|
def test_phase1_gmp_hypotheses_are_not_title_restatements(tmp_path: Path) -> None:
|
||||||
|
project = tmp_path / "baifan"
|
||||||
|
project.mkdir()
|
||||||
|
(project / "phase0/extracted").mkdir(parents=True)
|
||||||
|
material = project / "phase0/extracted/audit.md"
|
||||||
|
material.write_text(
|
||||||
|
"人员培训记录齐全,但无菌操作动作违反 First Air 原则,需要进一步培训。\n"
|
||||||
|
"复盘显示 owner、关闭证据和问题升级机制仍需补齐。\n",
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
manifest = {
|
||||||
|
"topic": "白帆生物 GMP 与运营诊断",
|
||||||
|
"material_inventory": [{"extracted_to": "phase0/extracted/audit.md"}],
|
||||||
|
}
|
||||||
|
method = ResearchMethodRegistry().get("gmp_quality_operations_diagnosis")
|
||||||
|
|
||||||
|
plans = build_chapter_planning(
|
||||||
|
project,
|
||||||
|
manifest,
|
||||||
|
method,
|
||||||
|
["人员能力:培训有效性比培训记录更关键", "运营节奏:从临时协调转向管理系统"],
|
||||||
|
quota=2000,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert "关键解释变量" not in plans[0]["bold_hypothesis"]
|
||||||
|
assert "不缺培训台账" in plans[0]["bold_hypothesis"]
|
||||||
|
assert "固定节奏和可视化管理系统" in plans[1]["bold_hypothesis"]
|
||||||
|
assert plans[0]["core_question"] != plans[0]["title"]
|
||||||
|
assert "章节成稿应围绕这一观点展开" not in plans[1]["writing_claim"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_research_brief_without_materials_falls_back_to_material_digest() -> None:
|
||||||
|
brief = {
|
||||||
|
"research_method": "mckinsey_market",
|
||||||
|
"phase2_mode": "chapter_integrated",
|
||||||
|
"phase1_inputs": {"material_digest": "phase1/material_digest.md"},
|
||||||
|
}
|
||||||
|
framework = "## 第1章 市场需求正在被支付政策重塑\n\n研究思路。"
|
||||||
|
|
||||||
|
cards = generate_task_cards_from_research_brief("market-test", framework, brief)
|
||||||
|
|
||||||
|
assert cards[0].allowed_materials == ["phase1/material_digest.md"]
|
||||||
|
|
||||||
|
|
||||||
def test_task_card_validation_rejects_duplicates_and_cycles() -> None:
|
def test_task_card_validation_rejects_duplicates_and_cycles() -> None:
|
||||||
cards = [
|
cards = [
|
||||||
TaskCard(task_id="a", chapter_ids=["ch01"], topic_axis="clinical", questions=["q"], search_routes=["scholar"], output_packet="phase2/packets/a.json", dependencies=["b"]),
|
TaskCard(task_id="a", chapter_ids=["ch01"], topic_axis="clinical", questions=["q"], search_routes=["scholar"], output_packet="phase2/packets/a.json", dependencies=["b"]),
|
||||||
@@ -154,6 +273,10 @@ def test_packet_validation_requires_sources_and_counter_evidence() -> None:
|
|||||||
validate_packet(packet)
|
validate_packet(packet)
|
||||||
|
|
||||||
packet["source_ids"].append("src_002")
|
packet["source_ids"].append("src_002")
|
||||||
|
packet["sources"] = [
|
||||||
|
{"id": "src_001", "title": "来源1", "url": "https://example.com/1"},
|
||||||
|
{"id": "src_002", "title": "来源2", "url": "https://example.com/2"},
|
||||||
|
]
|
||||||
validate_packet(packet)
|
validate_packet(packet)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -74,6 +74,10 @@ def valid_response() -> dict:
|
|||||||
"evidence_items": [{"source_id": "src_001", "summary": "III 期结果支持主要终点。"}],
|
"evidence_items": [{"source_id": "src_001", "summary": "III 期结果支持主要终点。"}],
|
||||||
"counter_evidence": [{"claim": "长期安全性仍需随访", "source_ids": ["src_002"]}],
|
"counter_evidence": [{"claim": "长期安全性仍需随访", "source_ids": ["src_002"]}],
|
||||||
"source_ids": ["src_001", "src_002"],
|
"source_ids": ["src_001", "src_002"],
|
||||||
|
"sources": [
|
||||||
|
{"id": "src_001", "title": "来源1", "url": "https://example.com/1"},
|
||||||
|
{"id": "src_002", "title": "来源2", "url": "https://example.com/2"},
|
||||||
|
],
|
||||||
"source_quality_notes": ["src_001 Tier 1; src_002 Tier 2"],
|
"source_quality_notes": ["src_001 Tier 1; src_002 Tier 2"],
|
||||||
"open_questions": [],
|
"open_questions": [],
|
||||||
"raw_quotes_or_notes": ["Original English evidence note is allowed."],
|
"raw_quotes_or_notes": ["Original English evidence note is allowed."],
|
||||||
@@ -98,6 +102,7 @@ def test_packet_worker_generates_valid_packet_with_fake_client(tmp_path: Path) -
|
|||||||
|
|
||||||
validate_packet(packet)
|
validate_packet(packet)
|
||||||
assert fake.calls[0]["model"] == role.model
|
assert fake.calls[0]["model"] == role.model
|
||||||
|
assert "章节证据分析师" in fake.calls[0]["system"]
|
||||||
assert "search-strategy" in fake.calls[0]["system"]
|
assert "search-strategy" in fake.calls[0]["system"]
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
Binary file not shown.
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user