213 lines
7.8 KiB
Markdown
213 lines
7.8 KiB
Markdown
---
|
|
name: research-quality-gates
|
|
description: Applies stage gates for Antigravity-native Deep Research: search receipts, source access, claims ledger, triangulation, contradiction search, continuation state, and final assembly checks.
|
|
---
|
|
|
|
# Research Quality Gates
|
|
|
|
Use this skill after every phase boundary and before any user-facing "done" claim. It is designed for Antigravity-native runs where model work happens inside Antigravity, while this repository provides the evidence discipline.
|
|
|
|
## Core Principle
|
|
|
|
Evidence must persist outside the model context. A claim is not verified because the model remembers it, summarized it, or saw it in a search snippet. It is verified only when the artifact trail contains:
|
|
|
|
1. a search receipt,
|
|
2. an opened original source or explicit access failure,
|
|
3. a registered `source_id`,
|
|
4. an evidence span or data locator,
|
|
5. a claim-ledger record linking the claim to supporting and counter evidence.
|
|
|
|
## Required Artifacts
|
|
|
|
Create these files when the relevant phase begins. Empty files are acceptable at creation time, but they must be populated before the phase gate passes.
|
|
|
|
| Phase | Artifact | Purpose |
|
|
|---|---|---|
|
|
| Phase 1 | `phase1/method_decision.md` | Selected method, rejected methods, evidence types, and search routes. |
|
|
| Phase 1 | `phase1/assumptions.md` | Material assumptions and scope assumptions that need validation. |
|
|
| Phase 2 | `phase2/search_log.jsonl` | One row per search/open/extract action. |
|
|
| Phase 2 | `phase2/sources.jsonl` | Stable source registry with scores and tiers. |
|
|
| Phase 2 | `phase2/rejected_sources.jsonl` | Rejected or low-quality sources with reasons. |
|
|
| Phase 2 | `phase2/claims_ledger.jsonl` | Atomic factual and analytical claims with verification status. |
|
|
| Phase 2 | `phase2/coverage_matrix.md` | Coverage of questions, chapters, methods, and evidence gaps. |
|
|
| Phase 2 | `phase2/unsupported_claims.md` | Claims that could not be verified or need user-visible caveats. |
|
|
| All phases | `continuation_state.json` | Current phase, completed sections, artifact paths, open gaps, and next actions. |
|
|
| Phase 4 | `phase4/final_fact_check.md` | Final audit showing each core fact exists in the verified claim ledger. |
|
|
|
|
## Search Receipt Gate
|
|
|
|
Every search or retrieval action must write a row to `search_log.jsonl`.
|
|
|
|
Required fields:
|
|
|
|
```json
|
|
{
|
|
"receipt_id": "srch_0001",
|
|
"timestamp": "YYYY-MM-DDTHH:MM:SSZ",
|
|
"tool": "search-cli|browser|project-search|other",
|
|
"mode": "academic|scholar|patents|news|extract|general|deep",
|
|
"query_or_url": "...",
|
|
"purpose": "which task/card/chapter this supports",
|
|
"result_count": 10,
|
|
"opened_urls": ["https://..."],
|
|
"status": "ok|partial|failed",
|
|
"failure_reason": null
|
|
}
|
|
```
|
|
|
|
Gate fails if a model says it searched, confirmed, checked, extracted, opened, or verified a fact without a corresponding receipt.
|
|
|
|
## Source Access Gate
|
|
|
|
For any source used as evidence, `sources.jsonl` must record access status.
|
|
|
|
Required additional fields:
|
|
|
|
```json
|
|
{
|
|
"id": "src_042",
|
|
"search_receipt_ids": ["srch_0001"],
|
|
"raw_url": "https://...",
|
|
"canonical_url": "https://...",
|
|
"title": "...",
|
|
"publication_date": "YYYY-MM-DD",
|
|
"source_type": "paper|regulator|trial_registry|patent|filing|database|consulting|media|other",
|
|
"tier": 1,
|
|
"score": 8.6,
|
|
"access_status": "opened_original|opened_abstract|paywalled|failed|snippet_only",
|
|
"evidence_locator": "page/table/section/trial id/patent claim",
|
|
"independence_cluster": "cluster_...",
|
|
"notes": "..."
|
|
}
|
|
```
|
|
|
|
`snippet_only` sources cannot support final claims. Paywalled or abstract-only sources can be used only with a visible caveat or a second opened source that carries the same fact.
|
|
|
|
## Claim Ledger Gate
|
|
|
|
Each chapter draft must be backed by `claims_ledger.jsonl`. Store atomic claims, not paragraphs.
|
|
|
|
Required fields:
|
|
|
|
```json
|
|
{
|
|
"claim_id": "ch03_C012",
|
|
"chapter": "ch03",
|
|
"claim_type": "fact|trend|comparison|causal|forecast|recommendation",
|
|
"claim": "...",
|
|
"supporting_source_ids": ["src_042", "src_087"],
|
|
"counter_source_ids": ["src_103"],
|
|
"evidence_spans": [
|
|
{
|
|
"source_id": "src_042",
|
|
"locator": "Table 2",
|
|
"summary": "..."
|
|
}
|
|
],
|
|
"independence_clusters": ["cluster_a", "cluster_b"],
|
|
"confidence": "high|medium|low|unsupported",
|
|
"verification_status": "verified|partially_verified|conflicted|unsupported",
|
|
"needs_delta_retrieve": false,
|
|
"visible_caveat_required": false
|
|
}
|
|
```
|
|
|
|
Gate fails if:
|
|
|
|
- a core claim has fewer than 2 independent Tier 1-2 supporting sources;
|
|
- a claim has source IDs but no evidence locator;
|
|
- supporting sources all come from the same independence cluster;
|
|
- a conflicted or unsupported claim is written as a settled conclusion.
|
|
|
|
## Triangulation Gate
|
|
|
|
Counting URLs is not enough. Sources are independent only when they do not trace back to the same press release, same trial registry entry, same company deck, same sell-side note, or same syndicated article.
|
|
|
|
For core conclusions, require at least two of these source families where available:
|
|
|
|
- regulator / official registry,
|
|
- peer-reviewed paper or systematic review,
|
|
- company filing / annual report / exchange disclosure,
|
|
- patent or legal record,
|
|
- independent database,
|
|
- reputable industry or consulting report,
|
|
- independent expert or professional media analysis.
|
|
|
|
If a field has only one source family available, mark the limitation explicitly in `unsupported_claims.md` and in the report limitations section.
|
|
|
|
## Contradiction Gate
|
|
|
|
Every chapter requires at least one deliberate counter-search or falsification pass. Record it in `search_log.jsonl` and summarize it in the chapter brief.
|
|
|
|
Minimum counter-evidence record:
|
|
|
|
```json
|
|
{
|
|
"claim_id": "ch03_C012",
|
|
"counter_query": "...",
|
|
"counter_source_ids": ["src_103"],
|
|
"result": "none_found|weak_counter|material_counter",
|
|
"impact": "keep|qualify|rewrite|delete"
|
|
}
|
|
```
|
|
|
|
Gate fails if the chapter contains no counter-evidence section, no counter-search receipt, or no decision about how contradictions affected the draft.
|
|
|
|
## Coverage Gate
|
|
|
|
Before drafting, create `phase2/coverage_matrix.md` with one row per chapter/task axis:
|
|
|
|
| Chapter | Method | Core Question | Required Evidence | Found Evidence | Gaps | Decision |
|
|
|---|---|---|---|---|---|---|
|
|
|
|
Gate fails if a chapter is drafted while its method, required evidence, or gaps are blank.
|
|
|
|
## Delta-Retrieve Protocol
|
|
|
|
When a gap appears, do not patch it from memory. Create a targeted delta task:
|
|
|
|
1. write the gap in `coverage_matrix.md` or `claims_ledger.jsonl`;
|
|
2. run 1-3 targeted searches with explicit query purpose;
|
|
3. register any usable sources;
|
|
4. update the relevant claim record;
|
|
5. if still unsupported, keep the caveat visible.
|
|
|
|
Delta retrieval is mandatory when Phase 3 finds a critical evidence gap.
|
|
|
|
## Continuation Protocol
|
|
|
|
For long reports or interrupted runs, keep `projects/<slug>/continuation_state.json` current.
|
|
|
|
Minimum fields:
|
|
|
|
```json
|
|
{
|
|
"version": "0.20",
|
|
"slug": "...",
|
|
"current_phase": "phase2",
|
|
"active_model": "Gemini 3.1 Pro (Low)",
|
|
"completed_artifacts": ["phase2/task_cards.json"],
|
|
"open_gates": ["claim_ledger_gate"],
|
|
"artifact_paths": {
|
|
"sources": "phase2/sources.jsonl",
|
|
"claims": "phase2/claims_ledger.jsonl",
|
|
"search_log": "phase2/search_log.jsonl"
|
|
},
|
|
"open_questions": [],
|
|
"next_actions": []
|
|
}
|
|
```
|
|
|
|
Before resuming a project, read this state and the latest artifacts. Do not rely on chat history alone.
|
|
|
|
## Final Assembly Gate
|
|
|
|
Phase 4 must prove that final facts are a subset of verified claims:
|
|
|
|
- Sample at least 20 high-impact facts or all core claims, whichever is smaller.
|
|
- For each sampled fact, record `claim_id`, `source_ids`, verification status, and final wording decision in `phase4/final_fact_check.md`.
|
|
- New facts introduced during editing require new search receipts and claim-ledger records.
|
|
- Delete or caveat any unsupported factual claim before rendering PDF/DOCX.
|
|
|
|
Do not render final deliverables until the final assembly gate passes.
|