131 lines
6.2 KiB
Markdown
131 lines
6.2 KiB
Markdown
# ASR Pipeline v2
|
|
|
|
## Goal
|
|
|
|
The pipeline recovers useful meeting facts and traces rather than pretending that
|
|
speech can always be reconstructed word for word. The original transcript remains
|
|
the audit source. A draft may omit low-value repetition, but must not invent a
|
|
subject, number, decision, or relationship.
|
|
|
|
## Stages
|
|
|
|
1. S0 parses speaker turns and collapses decoder loops.
|
|
2. S1 generates candidates from the untouched transcript. Phonetics, observed ASR
|
|
errors, aliases, entity relationships, and recency are independent signals.
|
|
3. S2 only removes fillers and repairs punctuation. It receives no Hindsight context
|
|
and makes no entity decisions.
|
|
4. S3 applies settled mappings deterministically. Flash Lite sees only unresolved
|
|
candidates and Chinese-number spans.
|
|
5. S4 extracts source-labelled evidence cards per chunk.
|
|
6. S5 Sonnet groups those cards by topic and writes a rough draft.
|
|
|
|
## Personal fact store
|
|
|
|
`state/lexicon.db` is intentionally reusable outside ASR:
|
|
|
|
- `entity`, `alias`, `correction`: identity and observed surface forms.
|
|
- `relation`: person/company/project relations with source, confidence, and dates.
|
|
- `interaction`: contact recency and frequency.
|
|
- `observation`: what a transcript emitted and how it was resolved.
|
|
- `phonetic_confusion`: confirmed personal accent/ASR confusion pairs.
|
|
- `review_question`: transport-independent human review queue.
|
|
- `source_document`: incremental indexing state for historical minutes.
|
|
|
|
Obsidian remains a curated source. Hindsight supplies broad recall. SQLite holds
|
|
small deterministic facts that applications can safely query.
|
|
|
|
## Human review loop
|
|
|
|
The pipeline never blocks waiting for an answer. Ambiguous high-impact entity
|
|
candidates are queued and the draft keeps an uncertainty marker.
|
|
|
|
```bash
|
|
bin/asr-refine review pending
|
|
bin/asr-refine review answer 12 万亚坤
|
|
```
|
|
|
|
A confirmed candidate is learned as an alias, correction, and observation. The
|
|
candidate-pool cache includes the database timestamp, so the next recording uses it
|
|
immediately.
|
|
|
|
Pi, Telegram, and the web UI are thin transports over this queue:
|
|
|
|
1. Read pending rows and display the question, source sentence, and choices.
|
|
2. Submit `question_id + answer`.
|
|
3. Call the same `review.answer()` function or CLI command.
|
|
4. Optionally request a replay of S3-S5 for the affected recording.
|
|
|
|
The production web transport is a persistent mobile control plane:
|
|
|
|
```bash
|
|
bin/asr-refine review serve # normally started by systemd
|
|
bin/asr-refine review state # print the active one-time URL
|
|
```
|
|
|
|
- Every generated draft is a Web delivery, even when entity candidate detection
|
|
finds no ambiguity. Review order is: inferred call background (with a corrected
|
|
background field that requests a redraft), unnamed voiceprints, the draft's
|
|
`## 待你确认` list, then one overall review item.
|
|
- It listens on `192.168.50.145:8765`. `/` accepts streamed audio uploads,
|
|
shows recent job state, and atomically hands completed files to the existing
|
|
recording watcher. `/hook/intake` exposes the same raw-upload contract for a
|
|
future reverse proxy or external uploader.
|
|
- Telegram receives a 256-bit capability URL. The token exists only in Telegram and
|
|
the mode-0600 runtime state file; it is redacted from service/request logs.
|
|
- The page exposes only one queued question at a time: candidate buttons, an
|
|
explicit "无法确认", a short custom answer, and an expandable full-draft view.
|
|
- Rough drafts live under `state/drafts/`, outside the Vault and Git. After all
|
|
questions are answered, an auditable feedback block is saved privately.
|
|
`asr-review-revise@.service` applies those answers with Sonnet, validates the
|
|
formal schema/Summary/provenance, then writes and pushes exactly one formal note
|
|
under `minutes/`. The private reviewed source moves to `state/published/`.
|
|
- A confirmed unnamed `speaker_N` is written to `voiceprint_speakers`, marked for
|
|
retry by the daily enrollment job, and immediately submitted to the persistent
|
|
`:10099/voiceprints/enroll` API.
|
|
- SQLite records one `entity_mention` per entity per formal minute. When the current
|
|
minute would bring an uncarded person/company to two distinct minutes, Web offers
|
|
a conservative card scaffold. No card or judgment field is created without an
|
|
explicit human choice.
|
|
- Upload metadata (title and optional user-supplied background) is kept separate
|
|
from the acoustic transcript. The title survives into the task slug; the
|
|
background is explicitly labelled as user-supplied context for drafting.
|
|
- `asr-review-web.service` is enabled at boot and restarts on failure. A successful
|
|
private draft ensures it is running and sends its stable review URL through
|
|
Telegram. `dry-run` never queues questions.
|
|
|
|
The service unit lives at `deploy/asr-review-web.service`. Telegram may later gain
|
|
inline choice buttons, but neither transport owns facts or writes SQLite directly.
|
|
|
|
## Confidence policy
|
|
|
|
- Confirmed: voiceprint, explicit human mapping, or multiple independent strong
|
|
signals.
|
|
- Probable: context and relation graph agree, but identity is inferred.
|
|
- Pending: weak or conflicting evidence, including numbers whose type/unit is not
|
|
clear.
|
|
|
|
Only confirmed facts become deterministic aliases automatically.
|
|
|
|
## Graph and recency
|
|
|
|
```bash
|
|
bin/asr-refine graph sync
|
|
bin/asr-refine graph stats
|
|
```
|
|
|
|
The graph incrementally scans finalized minutes, imports person/company/project
|
|
frontmatter and known wikilinks, records meeting interactions, creates low-confidence
|
|
co-occurrence edges, and promotes explicit linked relations such as 投资/负责/合作 at a
|
|
higher confidence. Deleted or changed notes remove only facts owned by that source.
|
|
|
|
Candidate context weight combines relation confidence, repeated support, last contact,
|
|
and a 45-day half-life. It influences ranking but cannot activate the personal
|
|
phonetic-confusion channel outside entities already in the current graph context.
|
|
|
|
## Numbers
|
|
|
|
The deterministic number pass classifies percentage, date, money, quantity, and
|
|
project/code speech. Every decision records raw value, normalized value, type,
|
|
confidence, source span, and nearby context. Ambiguous forms such as `两万五九份`
|
|
remain unchanged and are passed to the uncertainty stage instead of being guessed.
|