Compare commits

..
52 Commits
Author SHA1 Message Date
Codex 3c70861463 feat(asr-pipeline): merge PR #1 webauth/CSRF/locking hardening and web_password fallback
Build image / build (push) Canceled after 0s
Build ASR Pipeline / image (push) Canceled after 0s
2026-11-02 06:37:33 -08:00
Codex 9debd1c47d fix(tools-pve/asr-pipeline): support /state/vault and /state/ssh inside container
Build image / build (push) Canceled after 0s
Build ASR Pipeline / image (push) Canceled after 0s
2026-11-02 03:07:41 -08:00
Codex bf5ea57d99 fix(tools-pve/asr-pipeline): store Obsidian git clone on local ext4 /opt/asr-pipeline/vault
Build image / build (push) Canceled after 0s
Build ASR Pipeline / image (push) Canceled after 0s
2026-11-02 02:58:34 -08:00
Codex 56e0305188 fix(tools-pve/asr-pipeline): mount gitea ssh key for Obsidian vault sync and fix provider source_detail
Build image / build (push) Canceled after 0s
Build ASR Pipeline / image (push) Canceled after 0s
2026-11-02 02:57:42 -08:00
Codex c42456be22 Retire ZenMux subscription and keep BYOK as terminal fallback 2026-11-01 23:25:00 -08:00
Codex be589ff5e0 feat(tools-pve/asr-pipeline): SQLite model config, task delete/rerefine, multi-recording review, lexicon CRUD
Build image / build (push) Canceled after 0s
Build ASR Pipeline / image (push) Canceled after 0s
2026-11-01 23:14:31 -08:00
Codex 2367c5eb4e feat(tools-pve/asr-pipeline): multi-backend ASR web app with LiteLLM routing
Build image / build (push) Canceled after 0s
Build ASR Pipeline / image (push) Canceled after 0s
2026-11-01 22:07:57 -08:00
Codex d1790a05b0 Upgrade BookOrbit to 3.6.0 2026-10-29 20:57:56 -07:00
Codex 3d44b74d8c Route asr-gpt to official OpenAI with ZenMux backup 2026-10-28 19:23:58 -07:00
Codex df391cd41f Deploy public LiteLLM on US DC1 with preserved HK routes 2026-10-28 19:12:03 -07:00
Codex 81c475464e 补充 OpenAI 主备验收与可复用双协议测试 2026-10-28 18:23:21 -07:00
Codex 205bd3bbbc 接入 OpenAI 官方 API 并为 Codex 订阅配置同型号回退 2026-10-28 18:19:04 -07:00
Codex ed5f6514bb 升级 Hindsight 到 1.1.0 固定镜像 2026-10-27 19:00:54 -07:00
Codex d583490bad feat(litellm): add native Beijing Bailian ASR and personal vocabulary 2026-10-26 17:52:46 -07:00
Codex daa907d029 feat(litellm): add provider namespaces with legacy compatibility 2026-10-26 17:38:55 -07:00
Codex 6a88160e0f Upgrade BookOrbit to 3.5.0
Pin the app image to the current GHCR OCI index. Leave PostgreSQL unchanged.
2026-10-26 06:46:39 -07:00
Codex 91daf426f2 在香港 LiteLLM 接入四款 ZenMux ASR 对比路由 2026-10-25 05:22:50 -07:00
Codex 45293b1b53 将个人听写词库收敛为高频人名与公司名 2026-10-25 01:57:25 -07:00
Codex 05e0800581 记录 tools-pve SMART 真实测试与响应格式限制 2026-10-25 01:12:24 -07:00
Codex f9ccf7af2b 为 tools-pve 个人语音输入启用 SMART 2026-10-25 01:09:53 -07:00
Codex 443ed46bf9 记录香港个人语音输入验收及 JSON 响应限制 2026-10-25 01:01:32 -07:00
Codex 71ccac6d72 接入香港 Vertex SMART 个人语音输入与热词 2026-10-25 00:59:39 -07:00
Codex ba0c198c0c 在香港禁用区域不可用的 AI Studio 路由 2026-10-25 00:46:03 -07:00
Codex 641991043f 同步香港 LiteLLM 镜像与多提供商路由 2026-10-25 00:42:57 -07:00
Codex 7831d5f319 Document isolated personal vocabulary transcription route 2026-10-25 00:22:07 -07:00
Codex 035daa177d Add isolated Gemini dictation route with native personal vocabulary 2026-10-25 00:19:22 -07:00
Codex 8746be24ce Upgrade BookOrbit to 3.4.0 2026-10-19 05:34:58 -07:00
Codex 94ffeca06e Allow Camoufox execution from Byparr cache 2026-10-12 22:57:22 -07:00
Codex d5625cc610 Re-enable Byparr debug for cache diagnosis 2026-10-12 22:55:46 -07:00
Codex 7672ff4f6c Fix Byparr browser cache capacity 2026-10-12 22:53:53 -07:00
Codex c8a5fe3d03 Enable temporary Byparr debug logging 2026-10-12 22:52:01 -07:00
Codex a0e86bb47f Deploy Byparr on tools-pve 2026-10-12 22:45:03 -07:00
Codex c4a2182b5e Document Hermes latest Grok Luna Sol policy 2026-10-11 23:10:51 -07:00
Codex 8db1483345 Add Hermes auxiliary and subscription model routes 2026-10-11 22:44:10 -07:00
Codex cd50972093 Route open models through ZenMux with OpenRouter fallback 2026-10-11 18:40:45 -07:00
Codex 9ebc213d71 Keep Google rerank synthetic-only per user decision 2026-10-09 09:29:31 -07:00
Codex a8c2a8d88f 升级 Hindsight 至 0.10.3 并配置 Google rerank 回退链 2026-10-09 09:22:04 -07:00
Codex 3f4b329018 开放 Google Discovery Engine 两条 rerank 路由 2026-10-09 09:16:00 -07:00
Codex 6791addafd 记录 Google 赠金接入与检索模型兼容性验收 2026-10-09 09:05:36 -07:00
Codex d3a487f86a 接入 Google Agent Platform 并开放已验证的 AI Studio embedding 2026-10-09 09:02:00 -07:00
Codex db09881b2d docs(litellm): record verified provider routes and client access 2026-10-08 21:56:31 -07:00
Codex 9af7b98f17 feat(litellm): enable OAuth models and AI Studio aliases 2026-10-08 21:53:19 -07:00
Codex 6473297555 fix(litellm): gate OAuth routes on real credentials 2026-10-08 21:43:18 -07:00
Codex 9a4020b7ec fix(litellm): restrict UI login to database users 2026-10-08 20:56:30 -07:00
Codex aaff6db668 docs(litellm): record live gateway and verified prompt cache 2026-10-08 20:41:28 -07:00
Codex 8b5c1b9927 feat(tools-pve): prepare LiteLLM gateway with persistent OAuth routes 2026-10-08 19:41:22 -07:00
Codex ac7508fdd9 Update BookOrbit to 3.3.0 2026-10-07 20:01:47 -07:00
Codex 15383c2dc9 Mount clean Grimmory library alongside legacy catalog 2026-10-07 07:50:38 -07:00
Codex 3e4caaef91 Point Grimmory at isolated clean library 2026-10-07 07:46:50 -07:00
Codex f691817d64 Add Grimmory evaluation stack for media-core 2026-10-07 06:53:43 -07:00
Codex fbc3067d0c ci: grant GHCR write permission to callers
Build image / build (push) Canceled after 0s
Build ASR Pipeline / image (push) Canceled after 0s
Build image / build (push) Canceled after 0s
Build Snell DC9 / image (push) Canceled after 0s
Build image / build (push) Canceled after 0s
Build Snell HK / image (push) Canceled after 0s
Build image / build (push) Canceled after 0s
Build WeChat Fulltext / image (push) Canceled after 0s
2026-10-05 04:36:12 -07:00
Codex 4157ee2ddf ci: build Arcane service images on GitHub
Build image / build (push) Canceled after 0s
Build ASR Pipeline / image (push) Canceled after 0s
Build image / build (push) Canceled after 0s
Build Snell DC9 / image (push) Canceled after 0s
Build image / build (push) Canceled after 0s
Build Snell HK / image (push) Canceled after 0s
Build image / build (push) Canceled after 0s
Build WeChat Fulltext / image (push) Canceled after 0s
2026-10-05 04:30:48 -07:00
110 changed files with 11720 additions and 556 deletions
+56
View File
@@ -0,0 +1,56 @@
name: Build image
on:
workflow_call:
inputs:
context:
required: true
type: string
image_name:
required: true
type: string
jobs:
build:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
concurrency:
group: image-${{ inputs.image_name }}-${{ github.ref }}
cancel-in-progress: true
steps:
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Set up Buildx
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
- name: Log in to GHCR
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Generate image metadata
id: meta
uses: docker/metadata-action@dc802804100637a589fabce1cb79ff13a1411302 # v6.2.0
with:
images: ghcr.io/${{ github.repository_owner }}/${{ inputs.image_name }}
tags: |
type=raw,value=latest,enable={{is_default_branch}}
type=sha,format=long
- name: Build and push image
uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0
with:
context: ${{ inputs.context }}
platforms: linux/amd64
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha,scope=${{ inputs.image_name }}
cache-to: type=gha,mode=max,scope=${{ inputs.image_name }}
provenance: mode=max
sbom: true
+21
View File
@@ -0,0 +1,21 @@
name: Build ASR Pipeline
on:
push:
branches: [main]
paths:
- servers/tools-pve/asr-pipeline/**
- .github/workflows/_build-image.yml
- .github/workflows/build-asr-pipeline.yml
workflow_dispatch:
permissions:
contents: read
packages: write
jobs:
image:
uses: ./.github/workflows/_build-image.yml
with:
context: servers/tools-pve/asr-pipeline
image_name: arcane-asr-pipeline
+21
View File
@@ -0,0 +1,21 @@
name: Build Snell DC9
on:
push:
branches: [main]
paths:
- servers/bwh-us-dc9/snell/**
- .github/workflows/_build-image.yml
- .github/workflows/build-snell-dc9.yml
workflow_dispatch:
permissions:
contents: read
packages: write
jobs:
image:
uses: ./.github/workflows/_build-image.yml
with:
context: servers/bwh-us-dc9/snell
image_name: arcane-snell-dc9
+21
View File
@@ -0,0 +1,21 @@
name: Build Snell HK
on:
push:
branches: [main]
paths:
- servers/bwh-hk-cmi/snell/**
- .github/workflows/_build-image.yml
- .github/workflows/build-snell-hk.yml
workflow_dispatch:
permissions:
contents: read
packages: write
jobs:
image:
uses: ./.github/workflows/_build-image.yml
with:
context: servers/bwh-hk-cmi/snell
image_name: arcane-snell-hk
@@ -0,0 +1,21 @@
name: Build WeChat Fulltext
on:
push:
branches: [main]
paths:
- servers/tools-pve/wechat-fulltext/**
- .github/workflows/_build-image.yml
- .github/workflows/build-wechat-fulltext.yml
workflow_dispatch:
permissions:
contents: read
packages: write
jobs:
image:
uses: ./.github/workflows/_build-image.yml
with:
context: servers/tools-pve/wechat-fulltext
image_name: arcane-wechat-fulltext
+1
View File
@@ -0,0 +1 @@
__pycache__/
+40 -2
View File
@@ -1,4 +1,4 @@
# Hong Kong CMI LiteLLM Vertex AI Relay # 香港 CMI LiteLLM 多提供商代理
- 公网 OpenAI-compatible 入口为 `https://api.opfo.xyz/v1`,由 Caddy 终止 TLS 后通过外部 Docker 网络 `hk-web` 反向代理到 `litellm-api:1049`。 - 公网 OpenAI-compatible 入口为 `https://api.opfo.xyz/v1`,由 Caddy 终止 TLS 后通过外部 Docker 网络 `hk-web` 反向代理到 `litellm-api:1049`。
- LiteLLM 不使用 host network,也不向宿主机发布端口;容器内监听 `0.0.0.0:1049` 仅供 `hk-web` 上的 Caddy 访问。 - LiteLLM 不使用 host network,也不向宿主机发布端口;容器内监听 `0.0.0.0:1049` 仅供 `hk-web` 上的 Caddy 访问。
@@ -8,4 +8,42 @@
- 对外模型别名包括 `gemini-3.6-flash`、`gemini-3.5-flash-lite`、 - 对外模型别名包括 `gemini-3.6-flash`、`gemini-3.5-flash-lite`、
`google/gemini-embedding-2` 和 `google/gemini-3.8-flash`;后两者仍通过 `google/gemini-embedding-2` 和 `google/gemini-3.8-flash`;后两者仍通过
Vertex AI ADC 路由。 Vertex AI ADC 路由。
- 后续 Vertex Anthropic 模型在 `config.yaml` 增加独立模型项。 - 2026-10-25 同步 tools-pve 已验证的 LiteLLM 1.104.2 固定镜像,以及 `models.yaml` 中的提供商映射。`latest` 是维护的别名,不是自动发现所有厂商的新模型。
- `provider_routes.py` 是同日 tools-pve 的路由生成器快照;更新提供商逻辑时应同步比对两端。HK 的 `render_config.py` 不接入数据库管理或 OAuth,保留现有主密钥认证,个人语音输入走独立的 Vertex 路线。
- Arcane Project 本地变量增加 `ZENMUX_SUBSCRIPTION_API_KEY`、`ZENMUX_PAYG_API_KEY`、`OPENROUTER_API_KEY`。只有相应凭据存在时才生成该组路由;真实值不进入 Git。
- Claude 使用 ZenMux 原生 Anthropic Messages 路径,订阅与 `-payg` 显式分开;开源模型使用 ZenMux 订阅优先、同模型 OpenRouter 后备。不自动回退至 ZenMux PAYG。
- HK 合成请求实测 AI Studio 返回 `User location is not supported for the API use`,因此 Compose 不传入 AI Studio 凭据,也不发布其路线。tools-pve 上原有 AI Studio 功能不受影响。OAuth 会话和会议处理程序没有复制至 HK。
- Vertex 继续使用 HK 原有 ADC。旧 `gemini-3.6-flash` 别名保留,但指向当前映射的 `gemini-3.8-flash`;其余旧别名保留。
- 当前实际配置由启动脚本生成至 `/tmp/litellm.yaml`;旧 `config.yaml` 仅作为历史参考,不再挂载。
- 升级只重建 LiteLLM,不变更 Caddy、HAProxy、其他项目及 ADC 卷。失败时回退本次 Git 提交并同步,再按需要 Redeploy,禁止删除卷。
## Spokenly 个人语音输入
- 基础地址:`https://api.opfo.xyz/v1`。
- 转写完整地址:`https://api.opfo.xyz/v1/audio/transcriptions`。
- 模型:`gemini-transcribe-personal`,真实上游为 `vertex_ai/gemini-3.5-transcribe`,区域 `global`,复用 HK ADC。
- API Key:HK 现有 LiteLLM 主密钥,不是 tools-pve 客户端 key,也不是 ZenMux key。
- 响应使用 `json`;不请求时间戳、说话人分离或 `verbose_json`。当前 LiteLLM 请求 `text` 时实测仍返回 JSON 包装,因此不要选 text。额外 prompt 留空;去口语化由 SMART 负责,不靠提示词。
- 仅显式带 `personal_dictation: true` 的这条路由经 `vertex_dictation.py` 添加 `generationConfig.audioTranscriptionConfig.mode=SMART` 和 `customVocabulary`。其他转写路线、会议录音以及 tools-pve 配置不改。
- 当前个人词库只有 10 个人名、10 个公司名,与 tools-pve 相同;从 Obsidian People/Companies 卡片取规范名,以正式工作笔记文档频次与 Hindsight 实体 mention_count 交叉筛选。产品、项目、文档标题和错误别名暂不纳入。旧 SQLite 自动抽取的 100 词快照已因过度引导被替代,SMART 保留。
- `PERSONAL_VOCABULARY_B64` 只存 Arcane Project 环境(Base64 不是加密);init 将其解码至现有私有凭据卷 `/credentials/personal-vocabulary.json`,umask 077,LiteLLM 只读挂载,不进入 Git。
- 更新词库时使用 server-management 的 `scripts/export-entity-dictation-vocabulary.py`,先审查纯实体名称与频次,替换 Arcane 中上述变量并重建 init;单改环境变量不会更新已生成文件。保留旧私有词库以便回滚。升级 LiteLLM 前重跑 `scripts/test-vertex-dictation-wire.py`,确认原生 SMART/热词和普通路线隔离。
- 验收:OpenAI multipart 音频端点接受一秒静音及 Google 公开 Brooklyn Bridge 语音示例,json 均返回 200,公开样本正确输出 `How old is the Brooklyn Bridge?`。测试证明接口和原生参数有效,不代表中文口语及个人名词已进行识别质量对比。
## 四款 ZenMux ASR 对比入口
同一 Base URL `https://api.opfo.xyz/v1`、HK 现有密钥,Spokenly 选择自定义 OpenAI-compatible transcription,response_format=json,Prompt 留空,额外 AI Processing 关闭。只切换模型名:
| 客户端模型名 | ZenMux 实际型号 |
| --- | --- |
| `asr-gpt` | `openai/gpt-transcribe` |
| `asr-qwen` | `qwen/qwen-audio-3.1-asr-flash` |
| `asr-doubao` | `bytedance/doubao-seed-asr-2.0` |
| `asr-mimo` | `xiaomi/mimo-v2.5-asr` |
- 四条路线只使用 ZenMux 订阅 key,不注入个人热词或 SMART、不加额外 LLM 改写,不自动切模型或 PAYG。供应商自身默认的标点和规范化未强制覆盖。
- `zenmux_transcription.py` 将 multipart 文件转换成 ZenMux 要求的 JSON/base64,固定上游 `/api/v1/audio/transcriptions`;不新增服务、端口或音频存储。
- 适配器只识别四个上游型号,校验固定 ZenMux API 地址,其他 OpenAI/Vertex 路线仍用原适配器。原 `gemini-transcribe-personal` 的 SMART 与个人词库不变,会议管线不动。
- 本轮只验收非流式文件转写和 JSON 输出。不要请求 speaker labels、时间戳或 verbose_json,也不使用提示词来冒充润色能力。
- ZenMux 没有返回细分 audio/text token 计量,适配器只转发其音频秒数,不编造 token 拆分;费用以 ZenMux 的实际用量记录为准。
- 每次升级 LiteLLM,重跑 server-management 的 `scripts/test-zenmux-asr-wire.py`(同步/异步、原音频字节、JSON 转换、其他 provider 不变)和现有 Vertex dictation 回归测试。
@@ -0,0 +1,119 @@
"""Bridge OpenAI file uploads to Beijing Bailian's native ASR API."""
import base64
import io
import json
from pathlib import Path
from urllib.parse import urlsplit
import wave
from litellm.llms.base_llm.audio_transcription.transformation import AudioTranscriptionRequestData
from litellm.llms.openai.transcriptions.whisper_transformation import OpenAIWhisperAudioTranscriptionConfig
from litellm.litellm_core_utils.audio_utils.utils import process_audio_file
from litellm.types.utils import TranscriptionResponse, TranscriptionUsageDurationObject
MODEL = 'qwen-audio-3.1-asr-flash'
FORMATS = {'audio/wav':'wav','audio/x-wav':'wav','audio/wave':'wav',
'audio/mpeg':'mp3','audio/mp3':'mp3','audio/flac':'flac','audio/x-flac':'flac',
'audio/mp4':'m4a','audio/m4a':'m4a','audio/x-m4a':'m4a',
'audio/ogg':'ogg','audio/webm':'webm','video/webm':'webm','audio/aac':'aac'}
VOCABULARY_PATHS = frozenset({'/etc/litellm/personal-vocabulary.json',
'/credentials/personal-vocabulary.json'})
def vocabulary(path):
if path not in VOCABULARY_PATHS:
raise ValueError('Unexpected personal vocabulary path')
words = json.loads(Path(path).read_text())
if not isinstance(words,list) or not 1 <= len(words) <= 100:
raise ValueError('Personal vocabulary must contain 1-100 terms')
if any(not isinstance(w,str) or not w.strip() or len(w)>100 for w in words):
raise ValueError('Invalid personal vocabulary term')
return {w.strip():3 for w in words}
class BailianTranscriptionConfig(OpenAIWhisperAudioTranscriptionConfig):
@property
def has_native_transcription_endpoint(self):
return True
def get_supported_openai_params(self,model):
return ['language','response_format','prompt']
def map_openai_params(self,non_default_params,optional_params,model,drop_params):
result=super().map_openai_params(non_default_params,optional_params,model,drop_params)
if result.get('response_format') not in (None,'json','text'):
raise ValueError('Bailian dictation supports json/text without timestamps')
return result
def validate_environment(self,headers,model,messages,optional_params,litellm_params,
api_key=None,api_base=None):
if not api_key:raise ValueError('Beijing Bailian API key required')
return {**headers,'Authorization':'Bearer '+api_key,'Content-Type':'application/json',
'X-DashScope-SSE':'disable'}
def get_complete_url(self,api_base,api_key,model,optional_params,litellm_params,stream=None):
parsed=urlsplit(api_base or '')
if (model != MODEL or parsed.scheme != 'https' or
not (parsed.hostname or '').endswith('.cn-beijing.maas.aliyuncs.com') or
parsed.path.rstrip('/') != '/api/v1' or parsed.port not in (None,443) or
parsed.username or parsed.password or parsed.query or parsed.fragment):
raise ValueError('Bailian ASR restricted to Beijing workspace native endpoints')
if stream:raise ValueError('Streaming not enabled on this dictation route')
return api_base.rstrip('/')+'/services/aigc/multimodal-generation/generation'
def transform_audio_transcription_request(self,model,audio_file,optional_params,litellm_params):
if model != MODEL or optional_params.get('timestamp_granularities'):
raise ValueError('Unsupported Bailian dictation model or timestamps')
audio=process_audio_file(audio_file)
mime=audio.content_type.split(';',1)[0].lower()
if mime not in FORMATS:raise ValueError('Unsupported Bailian audio MIME type')
if len(audio.file_content)>10_000_000:raise ValueError('Bailian inline audio exceeds 10 MB')
parameters={'format':FORMATS[mime]}
if parameters['format']=='wav':
with wave.open(io.BytesIO(audio.file_content),'rb') as wav:
parameters['sample_rate']=str(wav.getframerate())
language=optional_params.get('language')
if language:parameters['language_hints']=[language]
# LiteLLM classifies DashScope as OpenAI-compatible and nests custom kwargs.
extra=optional_params.get('extra_body') or {}
path=optional_params.get('bailian_vocabulary_path') or extra.get('bailian_vocabulary_path')
if path:parameters['vocabulary']=vocabulary(path)
prompt=optional_params.get('prompt')
messages=[]
if prompt:
if not isinstance(prompt,str) or len(prompt)>400:
raise ValueError('Bailian context prompt must be at most 400 characters')
messages.append({'role':'user','content':[{'type':'input_text','text':prompt}]})
# Always preserve the original bytes; only WAV metadata supplies sample rate.
uri_mime='audio/mp3' if parameters['format']=='mp3' else mime
data_uri='data:'+uri_mime+';base64,'+base64.b64encode(audio.file_content).decode()
if len(data_uri)>10_000_000:
raise ValueError('Bailian inline audio exceeds 10 MB after Base64 encoding')
messages.append({'role':'user','content':[{'type':'input_audio','input_audio':{
'data':data_uri}}]})
return AudioTranscriptionRequestData(data={'model':MODEL,'input':{'messages':messages},
'parameters':parameters})
def transform_audio_transcription_response(self,raw_response):
body=raw_response.json()
text=body.get('output',{}).get('text',body.get('text'))
if not isinstance(text,str):raise ValueError('Bailian returned no transcript')
result=TranscriptionResponse(text=text)
duration=(body.get('usage') or {}).get('duration')
if duration is not None:
result.usage=TranscriptionUsageDurationObject(type='duration',seconds=duration)
result['duration']=duration
return result
def install():
import litellm
from litellm.utils import ProviderConfigManager
original=ProviderConfigManager.get_provider_audio_transcription_config
if getattr(original,'_bailian_asr_adapter',False):return
def select(model,provider):
if provider==litellm.LlmProviders.DASHSCOPE and model==MODEL:
return BailianTranscriptionConfig()
return original(model,provider)
select._bailian_asr_adapter=True
ProviderConfigManager.get_provider_audio_transcription_config=staticmethod(select)
+19 -3
View File
@@ -4,17 +4,20 @@ services:
restart: "no" restart: "no"
environment: environment:
GOOGLE_ADC_B64: ${GOOGLE_ADC_B64} GOOGLE_ADC_B64: ${GOOGLE_ADC_B64}
PERSONAL_VOCABULARY_B64: ${PERSONAL_VOCABULARY_B64:?Personal vocabulary is required}
volumes: volumes:
- google-adc:/credentials - google-adc:/credentials
command: command:
- /bin/sh - /bin/sh
- -c - -c
- | - |
set -eu
umask 077 umask 077
printf '%s' "$${GOOGLE_ADC_B64}" | base64 -d > /credentials/application_default_credentials.json printf '%s' "$${GOOGLE_ADC_B64}" | base64 -d > /credentials/application_default_credentials.json
printf '%s' "$${PERSONAL_VOCABULARY_B64}" | base64 -d > /credentials/personal-vocabulary.json
litellm: litellm:
image: ghcr.io/berriai/litellm:main-stable@sha256:625981c83410a3ea68eb0697590a57ec1d764d634514d54fa5db0591077ee839 image: ghcr.io/berriai/litellm:main-stable@sha256:3d88bd757134a00e8c00237b85a021ab1f6d54abb9c1df750f411dbe7256a500
restart: unless-stopped restart: unless-stopped
depends_on: depends_on:
adc-init: adc-init:
@@ -28,10 +31,23 @@ services:
VERTEXAI_PROJECT: ${VERTEXAI_PROJECT} VERTEXAI_PROJECT: ${VERTEXAI_PROJECT}
VERTEXAI_LOCATION: ${VERTEXAI_LOCATION:-global} VERTEXAI_LOCATION: ${VERTEXAI_LOCATION:-global}
LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY} LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY}
VERTEXAI_ENABLED: "true"
ZENMUX_SUBSCRIPTION_API_KEY: ${ZENMUX_SUBSCRIPTION_API_KEY:-}
ZENMUX_PAYG_API_KEY: ${ZENMUX_PAYG_API_KEY:-}
OPENROUTER_API_KEY: ${OPENROUTER_API_KEY:-}
BAILIAN_CN_API_KEY: ${BAILIAN_CN_API_KEY:-}
BAILIAN_CN_API_BASE: ${BAILIAN_CN_API_BASE:-https://ws-v68kvjnmhkiejaxo.cn-beijing.maas.aliyuncs.com/api/v1}
volumes: volumes:
- ./config.yaml:/app/config.yaml:ro - ./models.yaml:/etc/litellm/models.yaml:ro
- ./provider_routes.py:/etc/litellm/provider_routes.py:ro
- ./render_config.py:/etc/litellm/render_config.py:ro
- ./vertex_dictation.py:/etc/litellm/vertex_dictation.py:ro
- ./zenmux_transcription.py:/etc/litellm/zenmux_transcription.py:ro
- ./bailian_transcription.py:/etc/litellm/bailian_transcription.py:ro
- google-adc:/credentials:ro - google-adc:/credentials:ro
command: ["--config", "/app/config.yaml", "--host", "0.0.0.0", "--port", "${LITELLM_PORT:-1049}"] entrypoint: ["/bin/sh", "-c"]
command:
- python /etc/litellm/render_config.py && exec python /etc/litellm/vertex_dictation.py --config /tmp/litellm.yaml --host 0.0.0.0 --port ${LITELLM_PORT:-1049} --num_workers 1
logging: logging:
driver: json-file driver: json-file
options: options:
+43
View File
@@ -0,0 +1,43 @@
# 除 Google 自带 latest 别名外,本项目映射不会自行追踪上游发布。
zenmux:
claude-sonnet-latest: anthropic/claude-sonnet-5.5
claude-opus-latest: anthropic/claude-opus-5.5
claude-haiku-latest: anthropic/claude-haiku-5.5
openrouter:
deepseek-flash-latest: deepseek/deepseek-v4.1-flash
deepseek-pro-latest: deepseek/deepseek-v4-pro
glm-latest: z-ai/glm-5.3-prime
glm-flash-latest: z-ai/glm-5.3-flash
kimi-latest: moonshotai/kimi-k3
mimo-flash-latest: xiaomi/mimo-v2.6-flash
# These existing public aliases are subscription-first, not load-balanced.
zenmux_open_models:
deepseek-flash-latest: deepseek/deepseek-v4.1-flash
deepseek-pro-latest: deepseek/deepseek-v4-pro
glm-latest: z-ai/glm-5.3-prime
glm-flash-latest: z-ai/glm-5.3-flash
kimi-latest: moonshotai/kimi-k3
mimo-flash-latest: xiaomi/mimo-v2.6-flash
zenmux_chat:
gemini-flash-zenmux: google/gemini-3.8-flash
google_ai_studio:
gemini-flash-latest: gemini-flash-latest
gemini-flash-lite-latest: gemini-flash-lite-latest
gemini-pro-latest: gemini-pro-latest
gemini-embedding-2-studio: gemini-embedding-2
google_vertex:
gemini-flash-vertex: gemini-3.8-flash
gemini-flash-lite-vertex: gemini-3.5-flash-lite
gemini-pro-vertex: gemini-3.1-pro-preview
google-rerank-default: semantic-ranker-default@latest
google-rerank-fast: semantic-ranker-fast@latest
# 已有 Hindsight 部署用过的候选模型;本代理仍需独立授权与请求验收。
# renderer 在实际 token 出现前排除这些路线。
chatgpt:
gpt-luna-latest: gpt-6-luna
gpt-sol-latest: gpt-6.1-sol
gpt-work-sol: gpt-6-sol
gpt-review-5.5: gpt-5.5
gpt-astra-latest: gpt-6-astra
xai:
grok-latest: grok-4.7
@@ -0,0 +1,175 @@
"""Render only credential-ready routes; credentials remain environment references."""
import os
import json
from copy import deepcopy
from pathlib import Path
import yaml
def oauth_ready(path):
try:
record = json.loads(path.read_text())
return isinstance(record, dict) and bool(record.get('access_token'))
except (OSError, ValueError):
return False
def namespace_config(config):
"""Add explicit provider names without removing existing client aliases."""
routes = config['model_list']
existing = {r['model_name']: r for r in routes}
fallbacks = config['router_settings']['fallbacks']
automatic = {name for group in fallbacks for name in group}
renamed = {}
def clone(route, alias):
if alias in existing:
if existing[alias]['litellm_params'] != route['litellm_params']:
raise ValueError('Conflicting namespace alias: ' + alias)
return
new = deepcopy(route)
new['model_name'] = alias
existing[alias] = new
routes.append(new)
for route in list(routes):
alias = route['model_name']
model = route['litellm_params']['model']
if alias in {'gemini-3.6-flash', 'gemini-3.5-flash-lite',
'google/gemini-embedding-2', 'google/gemini-3.8-flash'}:
continue
if alias.endswith('-openrouter'):
new = 'openrouter/' + alias.removesuffix('-openrouter')
elif alias in automatic:
new = 'auto/' + alias
clone(route, 'zenmux/' + alias)
elif model.startswith('anthropic/'):
new = ('zenmux-payg/' + alias.removesuffix('-payg')
if alias.endswith('-payg') else 'zenmux/' + alias)
elif model.startswith('openai/'):
name = model.removeprefix('openai/').split('/', 1)[-1]
new = 'zenmux/' + (name if alias.startswith('asr-') else
alias.replace('-zenmux', '-latest'))
elif model.startswith('openrouter/'):
new = 'openrouter/' + alias
elif model.startswith('gemini/'):
new = 'aistudio/' + alias.removesuffix('-studio')
elif model.startswith('vertex_ai/'):
name = {'google-rerank-default': 'rerank-default-latest',
'google-rerank-fast': 'rerank-fast-latest'}.get(alias,
alias.replace('-vertex', '-latest'))
new = 'vertex/' + name
elif model.startswith('chatgpt/'):
name = model.split('/', 1)[1] if alias in {'gpt-work-sol', 'gpt-review-5.5'} else alias
new = 'codex/' + name
elif model.startswith('xai/'):
new = 'xai/' + alias
else:
raise ValueError('Unrecognized namespace provider: ' + alias)
clone(route, new)
renamed[alias] = new
for group in list(fallbacks):
for source, targets in group.items():
fallbacks.append({renamed[source]: [renamed[target] for target in targets]})
return config
def add_bailian_routes(config, env, vocabulary_path):
if env.get('BAILIAN_CN_API_KEY'):
base = env.get('BAILIAN_CN_API_BASE')
if not base:
raise ValueError('BAILIAN_CN_API_BASE required for Beijing Bailian')
for personal in [False, True]:
params = {'model': 'dashscope/qwen-audio-3.1-asr-flash',
'api_base': base, 'api_key': 'os.environ/BAILIAN_CN_API_KEY',
'max_retries': 0}
if personal:
params['bailian_vocabulary_path'] = vocabulary_path
config['model_list'].append({'model_name': 'bailian-cn/qwen-audio-3.1-asr-flash' +
('-personal' if personal else ''), 'litellm_params': params})
return config
def build_config(catalog, env, exists, namespaced=True):
routes = []
fallbacks = []
def add(alias, model, key=None, **params):
if key:
params['api_key'] = 'os.environ/' + key
routes.append({'model_name': alias, 'litellm_params': {'model': model, **params}})
for key, suffix in [('ZENMUX_SUBSCRIPTION_API_KEY', ''), ('ZENMUX_PAYG_API_KEY', '-payg')]:
if env.get(key):
for alias, model in catalog['zenmux'].items():
# Strip only the LiteLLM provider prefix, retaining ZenMux's vendor prefix.
add(alias + suffix, 'anthropic/' + model, key,
api_base='https://zenmux.ai/api/anthropic')
if env.get('ZENMUX_SUBSCRIPTION_API_KEY'):
for alias, model in catalog.get('zenmux_chat', {}).items():
add(alias, 'openai/' + model, 'ZENMUX_SUBSCRIPTION_API_KEY',
api_base='https://zenmux.ai/api/v1')
if env.get('OPENROUTER_API_KEY'):
for alias, model in catalog['openrouter'].items():
primary = catalog.get('zenmux_open_models', {}).get(alias)
if primary and env.get('ZENMUX_SUBSCRIPTION_API_KEY'):
if primary != model:
raise ValueError('Cross-provider model mismatch: ' + alias)
add(alias, 'openai/' + primary, 'ZENMUX_SUBSCRIPTION_API_KEY',
api_base='https://zenmux.ai/api/v1', max_retries=0)
routes[-1]['model_info'] = {'cooldown_time': 60,
'allowed_fails_policy': {'RateLimitErrorAllowedFails': 0,
'AuthenticationErrorAllowedFails': 0,
'InternalServerErrorAllowedFails': 0,
'ServiceUnavailableErrorAllowedFails': 0,
'TimeoutErrorAllowedFails': 0}}
add(alias + '-openrouter', 'openrouter/' + model, 'OPENROUTER_API_KEY')
fallbacks.append({alias: [alias + '-openrouter']})
else:
add(alias, 'openrouter/' + model, 'OPENROUTER_API_KEY')
elif env.get('ZENMUX_SUBSCRIPTION_API_KEY'):
for alias, model in catalog.get('zenmux_open_models', {}).items():
add(alias, 'openai/' + model, 'ZENMUX_SUBSCRIPTION_API_KEY',
api_base='https://zenmux.ai/api/v1', max_retries=0)
if env.get('GOOGLE_AI_STUDIO_API_KEY'):
add('gemini-transcribe-personal', 'gemini/gemini-3.5-transcribe',
'GOOGLE_AI_STUDIO_API_KEY', personal_vocabulary=True)
for alias, model in catalog['google_ai_studio'].items():
add(alias, 'gemini/' + model, 'GOOGLE_AI_STUDIO_API_KEY')
if env.get('VERTEXAI_ENABLED', '').lower() == 'true' and env.get('VERTEXAI_PROJECT'):
for alias, model in catalog.get('google_vertex', {}).items():
add(alias, 'vertex_ai/' + model,
vertex_project='os.environ/VERTEXAI_PROJECT',
vertex_location='os.environ/VERTEXAI_LOCATION')
for provider, token_var, default_dir in [
('chatgpt', 'CHATGPT_TOKEN_DIR', '/tokens/chatgpt'),
('xai', 'XAI_OAUTH_TOKEN_DIR', '/tokens/xai'),
]:
if exists(Path(env.get(token_var, default_dir)) / 'auth.json'):
for alias, model in catalog[provider].items():
extra = {'use_xai_oauth': True} if provider == 'xai' else {}
add(alias, provider + '/' + model, **extra)
config = {
'model_list': routes,
'general_settings': {
'master_key': 'os.environ/LITELLM_MASTER_KEY',
'database_url': 'os.environ/DATABASE_URL',
'store_model_in_db': False,
'disable_env_credential_login': True,
},
'litellm_settings': {'drop_params': False, 'set_verbose': False},
# Only explicitly approved open-model routes have cross-provider fallbacks.
'router_settings': {'num_retries': 1, 'timeout': 180, 'fallbacks': fallbacks},
}
return add_bailian_routes(namespace_config(config), env,
'/etc/litellm/personal-vocabulary.json') if namespaced else config
if __name__ == '__main__':
catalog = yaml.safe_load(Path('/etc/litellm/models.yaml').read_text())
config = build_config(catalog, os.environ, oauth_ready)
output = Path('/tmp/litellm.yaml')
output.write_text(yaml.safe_dump(config, allow_unicode=True, sort_keys=False))
output.chmod(0o600)
print('Configured routes:', ', '.join(route['model_name'] for route in config['model_list']))
@@ -0,0 +1,59 @@
"""HK provider routes and separately opted-in personal Vertex dictation."""
import os
from pathlib import Path
import yaml
from provider_routes import build_config, namespace_config, add_bailian_routes
def build_hk_config(catalog, env):
config = build_config(catalog, env, lambda path: False, namespaced=False)
config['general_settings'] = {'master_key': 'os.environ/LITELLM_MASTER_KEY'}
routes = config['model_list']
routes[:] = [route for route in routes if route['model_name'] != 'gemini-transcribe-personal']
if env.get('ZENMUX_SUBSCRIPTION_API_KEY'):
asr_models = {
'asr-gpt': 'openai/gpt-transcribe',
'asr-qwen': 'qwen/qwen-audio-3.1-asr-flash',
'asr-doubao': 'bytedance/doubao-seed-asr-2.0',
'asr-mimo': 'xiaomi/mimo-v2.5-asr',
}
for alias, model in asr_models.items():
routes.append({'model_name': alias, 'litellm_params': {
'model': 'openai/' + model,
'api_key': 'os.environ/ZENMUX_SUBSCRIPTION_API_KEY',
'api_base': 'https://zenmux.ai/api/v1',
'max_retries': 0}})
if env.get('GOOGLE_AI_STUDIO_API_KEY'):
routes.append({'model_name': 'gemini-transcribe', 'litellm_params': {
'model': 'gemini/gemini-3.5-transcribe',
'api_key': 'os.environ/GOOGLE_AI_STUDIO_API_KEY'}})
if env.get('VERTEXAI_ENABLED', '').lower() == 'true' and env.get('VERTEXAI_PROJECT'):
routes.append({'model_name': 'gemini-transcribe-personal', 'litellm_params': {
'model': 'vertex_ai/gemini-3.5-transcribe',
'vertex_project': 'os.environ/VERTEXAI_PROJECT',
'vertex_location': 'global',
'personal_dictation': True}})
# Keep old client aliases; the old 3.6 alias now follows the verified Flash map.
legacy = {
'gemini-3.6-flash': catalog['google_vertex']['gemini-flash-vertex'],
'gemini-3.5-flash-lite': catalog['google_vertex']['gemini-flash-lite-vertex'],
'google/gemini-embedding-2': 'gemini-embedding-2',
'google/gemini-3.8-flash': catalog['google_vertex']['gemini-flash-vertex'],
}
for alias, model in legacy.items():
routes.append({'model_name': alias, 'litellm_params': {
'model': 'vertex_ai/' + model,
'vertex_project': 'os.environ/VERTEXAI_PROJECT',
'vertex_location': 'os.environ/VERTEXAI_LOCATION'}})
return add_bailian_routes(namespace_config(config), env, '/credentials/personal-vocabulary.json')
if __name__ == '__main__':
catalog = yaml.safe_load(Path('/etc/litellm/models.yaml').read_text())
config = build_hk_config(catalog, os.environ)
output = Path('/tmp/litellm.yaml')
output.write_text(yaml.safe_dump(config, allow_unicode=True, sort_keys=False))
output.chmod(0o600)
print('Configured routes:', ', '.join(route['model_name'] for route in config['model_list']))
+56
View File
@@ -0,0 +1,56 @@
import unittest
from pathlib import Path
import yaml
from render_config import build_hk_config
class RoutingTests(unittest.TestCase):
def test_bailian_personal_uses_hk_private_vocabulary(self):
catalog = yaml.safe_load(Path(__file__).with_name('models.yaml').read_text())
routes = build_hk_config(catalog, {'BAILIAN_CN_API_KEY':'synthetic',
'BAILIAN_CN_API_BASE':'https://ws-test.cn-beijing.maas.aliyuncs.com/api/v1'})['model_list']
self.assertEqual(len(routes),2)
self.assertEqual(routes[1]['litellm_params']['bailian_vocabulary_path'],
'/credentials/personal-vocabulary.json')
def test_all_providers(self):
catalog = yaml.safe_load(Path(__file__).with_name('models.yaml').read_text())
env = {key: 'synthetic' for key in ['ZENMUX_SUBSCRIPTION_API_KEY',
'ZENMUX_PAYG_API_KEY', 'OPENROUTER_API_KEY', 'GOOGLE_AI_STUDIO_API_KEY',
'VERTEXAI_PROJECT']}
env['VERTEXAI_ENABLED'] = 'true'
result = build_hk_config(catalog, env)
routes = {r['model_name']: r['litellm_params'] for r in result['model_list']}
self.assertEqual(len(routes), len(result['model_list']))
self.assertEqual(routes['gemini-transcribe-personal']['model'], 'vertex_ai/gemini-3.5-transcribe')
self.assertIs(routes['gemini-transcribe-personal']['personal_dictation'], True)
self.assertEqual(routes['gemini-transcribe-personal']['vertex_location'], 'global')
self.assertNotIn('grok-latest', routes)
self.assertNotIn('gpt-sol-latest', routes)
self.assertEqual(routes['asr-gpt']['model'], 'openai/openai/gpt-transcribe')
self.assertEqual(routes['asr-qwen']['model'], 'openai/qwen/qwen-audio-3.1-asr-flash')
self.assertEqual(routes['asr-doubao']['model'], 'openai/bytedance/doubao-seed-asr-2.0')
self.assertEqual(routes['asr-mimo']['model'], 'openai/xiaomi/mimo-v2.5-asr')
for alias in ['asr-gpt', 'asr-qwen', 'asr-doubao', 'asr-mimo']:
self.assertNotIn('personal_dictation', routes[alias])
self.assertEqual(routes[alias]['api_key'], 'os.environ/ZENMUX_SUBSCRIPTION_API_KEY')
self.assertEqual(routes['gemini-transcribe']['model'], 'gemini/gemini-3.5-transcribe')
self.assertEqual(routes['gemini-3.6-flash']['model'], 'vertex_ai/gemini-3.8-flash')
self.assertEqual(len(result['router_settings']['fallbacks']), 12)
self.assertEqual(routes['vertex/gemini-transcribe-personal'], routes['gemini-transcribe-personal'])
self.assertNotIn('aistudio/gemini-transcribe-personal', routes)
self.assertEqual(routes['zenmux/gpt-transcribe'], routes['asr-gpt'])
self.assertEqual(result['general_settings'], {'master_key': 'os.environ/LITELLM_MASTER_KEY'})
self.assertNotIn('synthetic', yaml.safe_dump(result))
def test_missing_credentials(self):
catalog = yaml.safe_load(Path(__file__).with_name('models.yaml').read_text())
result = build_hk_config(catalog, {})
self.assertEqual(result['model_list'], [])
self.assertEqual(result['router_settings']['fallbacks'], [])
if __name__ == '__main__':
unittest.main()
@@ -0,0 +1,54 @@
"""Scoped Vertex dictation settings; ordinary transcription calls stay unchanged."""
import json
from pathlib import Path
VOCABULARY = Path('/credentials/personal-vocabulary.json')
def vocabulary(path=VOCABULARY):
words = json.loads(path.read_text())
if not isinstance(words, list) or not 1 <= len(words) <= 100:
raise ValueError('Personal vocabulary must contain 1-100 terms')
if any(not isinstance(word, str) or not word.strip() or len(word) > 100 for word in words):
raise ValueError('Invalid personal vocabulary term')
return list(dict.fromkeys(word.strip() for word in words))
def install():
from litellm.llms.vertex_ai.audio_transcription.gemini_transcribe_transformation import (
VertexGeminiAudioTranscriptionConfig,
)
original = VertexGeminiAudioTranscriptionConfig.transform_audio_transcription_request
if getattr(original, '_personal_dictation_patch', False):
return
def transform(self, model, audio_file, optional_params, litellm_params):
if optional_params.get('personal_dictation') is not True:
return original(self, model, audio_file, optional_params, litellm_params)
if model.removeprefix('vertex_ai/') != 'gemini-3.5-transcribe':
raise ValueError('Personal dictation is restricted to Vertex Gemini 3.5 Transcribe')
if optional_params.get('timestamp_granularities') or optional_params.get('response_format') not in (None, 'json', 'text'):
raise ValueError('Personal dictation supports json/text without timestamps')
clean = {key: val for key, val in optional_params.items() if key != 'personal_dictation'}
result = original(self, model, audio_file, clean, litellm_params)
config = result.data.setdefault('generationConfig', {}).setdefault('audioTranscriptionConfig', {})
if set(config) - {'mode', 'languageCodes'} or config.get('mode') not in (None, 'VERBATIM'):
raise ValueError('Unexpected Vertex transcription defaults; review adapter compatibility')
config['mode'] = 'SMART'
config['customVocabulary'] = vocabulary()
return result
transform._personal_dictation_patch = True
VertexGeminiAudioTranscriptionConfig.transform_audio_transcription_request = transform
if __name__ == '__main__':
import sys
from litellm import run_server
from zenmux_transcription import install as install_zenmux_asr
from bailian_transcription import install as install_bailian_asr
install_zenmux_asr()
install_bailian_asr()
install()
sys.argv[0] = 'litellm'
sys.exit(run_server())
@@ -0,0 +1,91 @@
"""Adapt four allowlisted ZenMux JSON ASR models to LiteLLM's audio endpoint."""
import base64
from litellm.llms.base_llm.audio_transcription.transformation import AudioTranscriptionRequestData
from litellm.llms.openai.transcriptions.whisper_transformation import OpenAIWhisperAudioTranscriptionConfig
from litellm.litellm_core_utils.audio_utils.utils import process_audio_file
from litellm.types.utils import TranscriptionResponse, TranscriptionUsageDurationObject
MODELS = frozenset({'openai/gpt-transcribe', 'qwen/qwen-audio-3.1-asr-flash',
'bytedance/doubao-seed-asr-2.0', 'xiaomi/mimo-v2.5-asr'})
API_BASE = 'https://zenmux.ai/api/v1'
class ZenMuxTranscriptionConfig(OpenAIWhisperAudioTranscriptionConfig):
@property
def has_native_transcription_endpoint(self):
return True
def get_supported_openai_params(self, model):
return ['language', 'response_format', 'prompt']
def map_openai_params(self, non_default_params, optional_params, model, drop_params):
params = super().map_openai_params(non_default_params, optional_params, model, drop_params)
if params.get('prompt'):
raise ValueError('ZenMux comparison routes do not accept prompts; leave prompt empty')
if params.get('response_format') not in (None, 'json', 'text'):
raise ValueError('ZenMux comparison routes support json/text only')
return params
def validate_environment(self, headers, model, messages, optional_params,
litellm_params, api_key=None, api_base=None):
if not api_key:
raise ValueError('ZenMux subscription API key is required')
return {**headers, 'Authorization': 'Bearer ' + api_key, 'Content-Type': 'application/json'}
def get_complete_url(self, api_base, api_key, model, optional_params, litellm_params, stream=None):
if (api_base or '').rstrip('/') != API_BASE or model not in MODELS:
raise ValueError('ZenMux ASR adapter is restricted to the configured endpoint and model allowlist')
if stream:
raise ValueError('Streaming is not enabled for comparison dictation routes')
return API_BASE + '/audio/transcriptions'
def transform_audio_transcription_request(self, model, audio_file, optional_params, litellm_params):
if model not in MODELS:
raise ValueError('Unknown ZenMux ASR model')
if optional_params.get('timestamp_granularities') or optional_params.get('prompt'):
raise ValueError('Timestamps and prompt injection are disabled on comparison routes')
audio = process_audio_file(audio_file)
formats = {'audio/wav': 'wav', 'audio/x-wav': 'wav', 'audio/wave': 'wav',
'audio/mpeg': 'mp3', 'audio/mp3': 'mp3', 'audio/flac': 'flac',
'audio/x-flac': 'flac', 'audio/m4a': 'm4a', 'audio/mp4': 'm4a',
'audio/x-m4a': 'm4a', 'audio/ogg': 'ogg', 'audio/webm': 'webm',
'video/webm': 'webm', 'audio/aac': 'aac'}
mime = audio.content_type.split(';', 1)[0].lower()
if mime not in formats:
raise ValueError('Unsupported audio MIME type for ZenMux ASR')
body = {'model': model, 'input_audio': {
'data': base64.b64encode(audio.file_content).decode(), 'format': formats[mime]},
'stream': False}
language = optional_params.get('language')
if language:
body['language'] = language
return AudioTranscriptionRequestData(data=body)
def transform_audio_transcription_response(self, raw_response):
body = raw_response.json()
if not isinstance(body.get('text'), str):
raise ValueError('ZenMux ASR returned no transcript text')
response = TranscriptionResponse(text=body['text'])
usage = body.get('usage') or {}
if usage.get('seconds') is not None:
# ZenMux has no audio/text token breakdown; report duration without inventing one.
response.usage = TranscriptionUsageDurationObject(type='duration', seconds=usage['seconds'])
response['duration'] = usage['seconds']
return response
def install():
import litellm
from litellm.utils import ProviderConfigManager
original = ProviderConfigManager.get_provider_audio_transcription_config
if getattr(original, '_zenmux_asr_adapter', False):
return
def select(model, provider):
if provider == litellm.LlmProviders.OPENAI and model in MODELS:
return ZenMuxTranscriptionConfig()
return original(model, provider)
select._zenmux_asr_adapter = True
ProviderConfigManager.get_provider_audio_transcription_config = staticmethod(select)
@@ -0,0 +1,24 @@
# 模型命名空间
2026-10-27 起新增客户端命名空间,保留旧名兼容。实际型号、协议、凭据、个人热词和 SMART 均不改变;latest 不代表自动跟踪发布。
| 前缀 | 含义 |
| --- | --- |
| zenmux/ | ZenMux 订阅直连,不跨 provider 回退 |
| zenmux-payg/ | ZenMux PAYG,仅显式请求 |
| openrouter/ | OpenRouter 直连 |
| auto/ | 开源模型 ZenMux 订阅优先,故障回退 OpenRouter |
| aistudio/ | Google AI Studio |
| vertex/ | Google Vertex |
| codex/ | Codex OAuth 订阅 |
| xai/ | xAI OAuth 订阅 |
常规旧别名加对应前缀。特殊转换:`gemini-flash-zenmux` → `zenmux/gemini-flash-latest`;`gemini-*-vertex` → `vertex/gemini-*-latest`;`gemini-embedding-2-studio` → `aistudio/gemini-embedding-2`;`google-rerank-default/fast` → `vertex/rerank-default/fast-latest`;`gpt-work-sol` → `codex/gpt-6-sol`;`gpt-review-5.5` → `codex/gpt-5.5`。PAYG 去掉旧 `-payg` 后缀,改加 `zenmux-payg/` 前缀。
六组开源模型 `deepseek-flash-latest`、`deepseek-pro-latest`、`glm-latest`、`glm-flash-latest`、`kimi-latest`、`mimo-flash-latest` 各提供 auto/、zenmux/、openrouter/ 三种选择。旧自动回退名和旧 -openrouter 名保留。
tools-pve 的个人语音名为 `aistudio/gemini-transcribe-personal`;HK 使用 `vertex/gemini-transcribe-personal`,不能互换两端 provider。HK 四款 ZenMux ASR 别名为 `zenmux/gpt-transcribe`、`zenmux/qwen-audio-3.1-asr-flash`、`zenmux/doubao-seed-asr-2.0`、`zenmux/mimo-v2.5-asr`。HK 的四个历史 Google 兼容名保留。
只有凭据已就绪的路线才生成。新旧别名的 litellm_params 必须完全相同,不能把客户端前缀当作上游 provider 参数。受模型白名单限制的 virtual key 需要另行授权新名;不扩大既有 key 权限。旧名暂不删除,待所有客户端完成迁移后单独处理。
修改 renderer 后,通过 Arcane Git Sync 同步,并重启对应 LiteLLM 进程使 /tmp/litellm.yaml 重建;仅同步代码不代表进程已加载。逐节点确认 /v1/models、新旧路由参数等价、fallback 配置及实际请求。
@@ -0,0 +1,24 @@
# 百炼北京原厂 ASR
2026-10-27 上线,仅北京,不配置新加坡或跨地域回退。客户端沿用本机 LiteLLM 的 /v1/audio/transcriptions、现有客户端 key、multipart 文件上传,推荐 response_format=json。
| 模型名 | 行为 |
| --- | --- |
| bailian-cn/qwen-audio-3.1-asr-flash | 原厂识别,无词表注入 |
| bailian-cn/qwen-audio-3.1-asr-flash-personal | 相同模型,原生 vocabulary 注入既有 20 个人名/公司名,每词权重 3 |
上游为业务空间专属域名 `https://ws-v68kvjnmhkiejaxo.cn-beijing.maas.aliyuncs.com/api/v1`,使用原生 `/services/aigc/multimodal-generation/generation`,不是 compatible-mode。已用 .145 现有 BAILIAN_API_KEY 实测该空间可用;部署后凭据权威是各 Arcane Project 的 BAILIAN_CN_API_KEY,原始 key 不进入 Git。
当前 LiteLLM 1.104.2 没有 DashScope 原生转写配置,启动适配器仅接管 dashscope provider 的 qwen-audio-3.1-asr-flash。普通 Gemini、Vertex、ZenMux 和 OpenAI 不变。不增加服务、端口或镜像,不重新部署会议管线。
文件原字节转为 Base64 Data URI。WAV 从文件头读取采样率,非 WAV 不伪造采样率;上游可选 prompt 作为 input_text 上下文,不覆盖、不做二次 LLM 改写,最多 400 字。生产个人词表从当前节点已有私有文件读取,不把词表放到 Git。DashScope 自定义参数在 SDK 中进入 extra_body,适配器需取出私有词表路径,再生成 parameters.vocabulary;私有路径不能泄漏上游。
不启用 streaming、时间戳或说话人分离,不承诺自动去口语化;原厂默认识别行为不额外覆盖。上传 Base64 Data URI 保守限制 10 MB,超限明确拒绝,不自动把文件上传公网。json 为已验证的推荐响应格式。
配置验证:先运行每目录 test_config.py,再在固定镜像中执行 scripts/test-bailian-asr-wire.py,检查同步/异步真实 SDK 的 HTTP 请求体、热词、原字节保持及其他 provider 不受影响。
原厂 A/B:2026-07-21 电话前 90 秒,无热词两处“洛奇”,指定临时热词“洛启、普华”(权重 3)后两处“洛启”,耗时分别 3.432/3.649 秒。该结果只证明该片段短词表有改善,不代表已有 20 词表一定同等改善或整体 CER/WER。
上线后必须验证:两节点普通/个人路由公开音频、现有客户端 key 权限、词表开关和 native vocabulary 请求体。新模型不因部署便自动获得 virtual key 权限;仅为用户批准的应用增加这两条路线,不扩大到全部模型。
官方文档:https://help.aliyun.com/zh/model-studio/fun-asr-flash-recorded-speech-recognition-http-api
@@ -20,3 +20,10 @@ dc1.opfo.xyz {
</html> </html>
HTML HTML
} }
api.opfo.xyz {
encode zstd gzip
reverse_proxy litellm-api:1049 {
flush_interval -1
}
}
@@ -28,6 +28,9 @@ services:
- ./caddy/Caddyfile:/etc/caddy/Caddyfile:ro - ./caddy/Caddyfile:/etc/caddy/Caddyfile:ro
- caddy-data:/data - caddy-data:/data
- caddy-config:/config - caddy-config:/config
networks:
- default
- edge-api
logging: logging:
driver: json-file driver: json-file
options: options:
@@ -39,6 +42,9 @@ volumes:
caddy-config: caddy-config:
networks: networks:
edge-api:
external: true
name: dc1-api
proxy: proxy:
external: true external: true
name: dc1-proxy name: dc1-proxy
+3
View File
@@ -0,0 +1,3 @@
__pycache__/
*.pyc
.env
+37
View File
@@ -0,0 +1,37 @@
# DC1 公网模型代理
## 2026-11-02:取消 ZenMux 订阅
已禁用订阅 Key。Claude/开源模型采用 OpenRouter → ZenMux BYOK,后者只作最后备用;
显式 `zenmux-payg/...` 仍直连 BYOK。旧 `zenmux/...` 名称保留作兼容入口,
不代表订阅后端仍存在;新配置优先用 `auto/...`。
`asr-gpt`:OpenAI 官方 → ZenMux BYOK;`asr-qwen`:已配置的百炼北京 → ZenMux BYOK。
豆包/MiMo 没有已配置的同模型替代后端,因此只能经 BYOK,不跨型号替换。
SMART、个人词库及会议处理配置不变。BYOK 是按量付费,不与 OpenRouter 共享缓存。
OpenRouter Claude 合成直连在两台生产主机均返回条款限制类 403,首选上游未通过验收。
2026-10-29从HK迁移,保留`https://api.opfo.xyz/v1`、原客户端认证、全部旧别名、
Vertex ADC与SMART个人听写、四款ZenMux ASR、百炼及开源模型主备。
无数据库,不迁移`.49`的OAuth或数据库,不从其他Agent复制refresh token。
代码与非敏感配置在本目录,真实环境变量在DC1 Arcane `litellm-managed`私有.env。
ADC和个人词库从原HK项目环境快照初始化到私有named volume,只读提供给应用;
Base64不是加密,不能提交env或凭据卷。旧HK保留作回滚,不自动删除任何数据。
单worker,应用CPU上限1核、内存896MiB;不在VPS构建镜像。固定原LiteLLM镜像,
本轮不升级镜像或SDK。Docker网络`dc1-api`由此项目创建,现有frontdoor Caddy加入它,
HAProxy的既有SNI分流与代理端口不变,公网不直接发布LiteLLM或管理端口。
新增`openai/...`为官方API独立路由,只有明确批准的OPENAI_API_KEY出现才启用;
不把ZenMux GPT转写别名偷换为官方API。这里没有订阅OAuth,因此不伪造Codex备用链。
`.49`的订阅到API回退是另一个独立实例。本次不自动启用AI Studio。
迁移顺序:源元数据/私有环境备份;DC1新Git Sync/Project;部署后以原key验证;
再更新frontdoor并确认HTTPS签发条件;最后只切api.opfo.xyz的CNAME,保留旧值回滚。
TLS证书由DC1 Caddy自动签发,正式DNS切换后须检查证书链和SSE/音频请求,失败回滚。
其他域名、HK网站、Gitea、代理协议、防火墙和Tailscale不改。
```bash
python3 -m unittest discover -s . -p test_config.py
```
@@ -0,0 +1,119 @@
"""Bridge OpenAI file uploads to Beijing Bailian's native ASR API."""
import base64
import io
import json
from pathlib import Path
from urllib.parse import urlsplit
import wave
from litellm.llms.base_llm.audio_transcription.transformation import AudioTranscriptionRequestData
from litellm.llms.openai.transcriptions.whisper_transformation import OpenAIWhisperAudioTranscriptionConfig
from litellm.litellm_core_utils.audio_utils.utils import process_audio_file
from litellm.types.utils import TranscriptionResponse, TranscriptionUsageDurationObject
MODEL = 'qwen-audio-3.1-asr-flash'
FORMATS = {'audio/wav':'wav','audio/x-wav':'wav','audio/wave':'wav',
'audio/mpeg':'mp3','audio/mp3':'mp3','audio/flac':'flac','audio/x-flac':'flac',
'audio/mp4':'m4a','audio/m4a':'m4a','audio/x-m4a':'m4a',
'audio/ogg':'ogg','audio/webm':'webm','video/webm':'webm','audio/aac':'aac'}
VOCABULARY_PATHS = frozenset({'/etc/litellm/personal-vocabulary.json',
'/credentials/personal-vocabulary.json'})
def vocabulary(path):
if path not in VOCABULARY_PATHS:
raise ValueError('Unexpected personal vocabulary path')
words = json.loads(Path(path).read_text())
if not isinstance(words,list) or not 1 <= len(words) <= 100:
raise ValueError('Personal vocabulary must contain 1-100 terms')
if any(not isinstance(w,str) or not w.strip() or len(w)>100 for w in words):
raise ValueError('Invalid personal vocabulary term')
return {w.strip():3 for w in words}
class BailianTranscriptionConfig(OpenAIWhisperAudioTranscriptionConfig):
@property
def has_native_transcription_endpoint(self):
return True
def get_supported_openai_params(self,model):
return ['language','response_format','prompt']
def map_openai_params(self,non_default_params,optional_params,model,drop_params):
result=super().map_openai_params(non_default_params,optional_params,model,drop_params)
if result.get('response_format') not in (None,'json','text'):
raise ValueError('Bailian dictation supports json/text without timestamps')
return result
def validate_environment(self,headers,model,messages,optional_params,litellm_params,
api_key=None,api_base=None):
if not api_key:raise ValueError('Beijing Bailian API key required')
return {**headers,'Authorization':'Bearer '+api_key,'Content-Type':'application/json',
'X-DashScope-SSE':'disable'}
def get_complete_url(self,api_base,api_key,model,optional_params,litellm_params,stream=None):
parsed=urlsplit(api_base or '')
if (model != MODEL or parsed.scheme != 'https' or
not (parsed.hostname or '').endswith('.cn-beijing.maas.aliyuncs.com') or
parsed.path.rstrip('/') != '/api/v1' or parsed.port not in (None,443) or
parsed.username or parsed.password or parsed.query or parsed.fragment):
raise ValueError('Bailian ASR restricted to Beijing workspace native endpoints')
if stream:raise ValueError('Streaming not enabled on this dictation route')
return api_base.rstrip('/')+'/services/aigc/multimodal-generation/generation'
def transform_audio_transcription_request(self,model,audio_file,optional_params,litellm_params):
if model != MODEL or optional_params.get('timestamp_granularities'):
raise ValueError('Unsupported Bailian dictation model or timestamps')
audio=process_audio_file(audio_file)
mime=audio.content_type.split(';',1)[0].lower()
if mime not in FORMATS:raise ValueError('Unsupported Bailian audio MIME type')
if len(audio.file_content)>10_000_000:raise ValueError('Bailian inline audio exceeds 10 MB')
parameters={'format':FORMATS[mime]}
if parameters['format']=='wav':
with wave.open(io.BytesIO(audio.file_content),'rb') as wav:
parameters['sample_rate']=str(wav.getframerate())
language=optional_params.get('language')
if language:parameters['language_hints']=[language]
# LiteLLM classifies DashScope as OpenAI-compatible and nests custom kwargs.
extra=optional_params.get('extra_body') or {}
path=optional_params.get('bailian_vocabulary_path') or extra.get('bailian_vocabulary_path')
if path:parameters['vocabulary']=vocabulary(path)
prompt=optional_params.get('prompt')
messages=[]
if prompt:
if not isinstance(prompt,str) or len(prompt)>400:
raise ValueError('Bailian context prompt must be at most 400 characters')
messages.append({'role':'user','content':[{'type':'input_text','text':prompt}]})
# Always preserve the original bytes; only WAV metadata supplies sample rate.
uri_mime='audio/mp3' if parameters['format']=='mp3' else mime
data_uri='data:'+uri_mime+';base64,'+base64.b64encode(audio.file_content).decode()
if len(data_uri)>10_000_000:
raise ValueError('Bailian inline audio exceeds 10 MB after Base64 encoding')
messages.append({'role':'user','content':[{'type':'input_audio','input_audio':{
'data':data_uri}}]})
return AudioTranscriptionRequestData(data={'model':MODEL,'input':{'messages':messages},
'parameters':parameters})
def transform_audio_transcription_response(self,raw_response):
body=raw_response.json()
text=body.get('output',{}).get('text',body.get('text'))
if not isinstance(text,str):raise ValueError('Bailian returned no transcript')
result=TranscriptionResponse(text=text)
duration=(body.get('usage') or {}).get('duration')
if duration is not None:
result.usage=TranscriptionUsageDurationObject(type='duration',seconds=duration)
result['duration']=duration
return result
def install():
import litellm
from litellm.utils import ProviderConfigManager
original=ProviderConfigManager.get_provider_audio_transcription_config
if getattr(original,'_bailian_asr_adapter',False):return
def select(model,provider):
if provider==litellm.LlmProviders.DASHSCOPE and model==MODEL:
return BailianTranscriptionConfig()
return original(model,provider)
select._bailian_asr_adapter=True
ProviderConfigManager.get_provider_audio_transcription_config=staticmethod(select)
+71
View File
@@ -0,0 +1,71 @@
services:
adc-init:
image: alpine:3.24@sha256:294b683cb724975bec92580e1e685676bd4b50bda910ddb8c51d4cabeaec77e6
restart: "no"
environment:
GOOGLE_ADC_B64: ${GOOGLE_ADC_B64}
PERSONAL_VOCABULARY_B64: ${PERSONAL_VOCABULARY_B64:?Personal vocabulary is required}
volumes:
- google-adc:/credentials
command:
- /bin/sh
- -c
- |
set -eu
umask 077
printf '%s' "$${GOOGLE_ADC_B64}" | base64 -d > /credentials/application_default_credentials.json
printf '%s' "$${PERSONAL_VOCABULARY_B64}" | base64 -d > /credentials/personal-vocabulary.json
litellm:
image: ghcr.io/berriai/litellm:main-stable@sha256:3d88bd757134a00e8c00237b85a021ab1f6d54abb9c1df750f411dbe7256a500
restart: unless-stopped
cpus: "1.0"
mem_limit: 896m
depends_on:
adc-init:
condition: service_completed_successfully
networks:
edge-api:
aliases:
- litellm-api
environment:
GOOGLE_APPLICATION_CREDENTIALS: /credentials/application_default_credentials.json
VERTEXAI_PROJECT: ${VERTEXAI_PROJECT}
VERTEXAI_LOCATION: ${VERTEXAI_LOCATION:-global}
LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY}
VERTEXAI_ENABLED: "true"
ZENMUX_PAYG_API_KEY: ${ZENMUX_PAYG_API_KEY:-}
OPENROUTER_API_KEY: ${OPENROUTER_API_KEY:-}
OPENAI_API_KEY: ${OPENAI_API_KEY:-}
BAILIAN_CN_API_KEY: ${BAILIAN_CN_API_KEY:-}
BAILIAN_CN_API_BASE: ${BAILIAN_CN_API_BASE:-https://ws-v68kvjnmhkiejaxo.cn-beijing.maas.aliyuncs.com/api/v1}
volumes:
- ./models.yaml:/etc/litellm/models.yaml:ro
- ./provider_routes.py:/etc/litellm/provider_routes.py:ro
- ./provider_policy.py:/etc/litellm/provider_policy.py:ro
- ./render_config.py:/etc/litellm/render_config.py:ro
- ./vertex_dictation.py:/etc/litellm/vertex_dictation.py:ro
- ./zenmux_transcription.py:/etc/litellm/zenmux_transcription.py:ro
- ./bailian_transcription.py:/etc/litellm/bailian_transcription.py:ro
- google-adc:/credentials:ro
entrypoint: ["/bin/sh", "-c"]
command:
- python /etc/litellm/render_config.py && exec python /etc/litellm/vertex_dictation.py --config /tmp/litellm.yaml --host 0.0.0.0 --port ${LITELLM_PORT:-1049} --num_workers 1
logging:
driver: json-file
options:
max-size: 1m
max-file: "2"
healthcheck:
test: [CMD, python, -c, 'import urllib.request; urllib.request.urlopen("http://127.0.0.1:1049/health/liveliness", timeout=5)']
interval: 30s
timeout: 10s
retries: 5
start_period: 120s
volumes:
google-adc:
networks:
edge-api:
name: dc1-api
+55
View File
@@ -0,0 +1,55 @@
# 除 Google 自带 latest 别名外,本项目映射不会自行追踪上游发布。
zenmux_subscription_enabled: false
zenmux:
claude-sonnet-latest: anthropic/claude-sonnet-5.5
claude-opus-latest: anthropic/claude-opus-5.5
claude-haiku-latest: anthropic/claude-haiku-5.5
openrouter:
deepseek-flash-latest: deepseek/deepseek-v4.1-flash
deepseek-pro-latest: deepseek/deepseek-v4-pro
glm-latest: z-ai/glm-5.3-prime
glm-flash-latest: z-ai/glm-5.3-flash
kimi-latest: moonshotai/kimi-k3
mimo-flash-latest: xiaomi/mimo-v2.6-flash
# OpenRouter first; ZenMux BYOK is the final same-model fallback.
zenmux_open_models:
deepseek-flash-latest: deepseek/deepseek-v4.1-flash
deepseek-pro-latest: deepseek/deepseek-v4-pro
glm-latest: z-ai/glm-5.3-prime
glm-flash-latest: z-ai/glm-5.3-flash
kimi-latest: moonshotai/kimi-k3
mimo-flash-latest: xiaomi/mimo-v2.6-flash
zenmux_chat:
gemini-flash-zenmux: google/gemini-3.8-flash
google_ai_studio:
gemini-flash-latest: gemini-flash-latest
gemini-flash-lite-latest: gemini-flash-lite-latest
gemini-pro-latest: gemini-pro-latest
gemini-embedding-2-studio: gemini-embedding-2
google_vertex:
gemini-flash-vertex: gemini-3.8-flash
gemini-flash-lite-vertex: gemini-3.5-flash-lite
gemini-pro-vertex: gemini-3.1-pro-preview
google-rerank-default: semantic-ranker-default@latest
google-rerank-fast: semantic-ranker-fast@latest
# 已有 Hindsight 部署用过的候选模型;本代理仍需独立授权与请求验收。
# renderer 在实际 token 出现前排除这些路线。
chatgpt:
gpt-luna-latest: gpt-6-luna
gpt-sol-latest: gpt-6.1-sol
gpt-work-sol: gpt-6-sol
gpt-review-5.5: gpt-5.5
gpt-astra-latest: gpt-6-astra
xai:
grok-latest: grok-4.7
openai:
gpt-terra-latest: gpt-6.1-terra
gpt-transcribe: gpt-transcribe
gpt-4o-transcribe: gpt-4o-transcribe
gpt-4o-mini-transcribe: gpt-4o-mini-transcribe
gpt-4o-transcribe-diarize: gpt-4o-transcribe-diarize
text-embedding-3-small: text-embedding-3-small
text-embedding-3-large: text-embedding-3-large
gpt-image-2.5-flare: gpt-image-2.5-flare
gpt-image-2.5-sunburst: gpt-image-2.5-sunburst
gpt-4o-mini-tts: gpt-4o-mini-tts
@@ -0,0 +1,118 @@
"""Subscription retirement: existing aliases stay usable, BYOK is terminal only."""
from copy import deepcopy
def subscription_disabled(catalog):
return catalog.get('zenmux_subscription_enabled') is False
def without_subscription(catalog, env):
if not subscription_disabled(catalog):
return env
return {key: value for key, value in env.items()
if key != 'ZENMUX_SUBSCRIPTION_API_KEY'}
def apply_provider_policy(config, catalog, env):
if not subscription_disabled(catalog):
return config
routes = config['model_list']
if any(r['litellm_params'].get('api_key') ==
'os.environ/ZENMUX_SUBSCRIPTION_API_KEY' for r in routes):
raise ValueError('Retired subscription route remains')
indexed = {r['model_name']: r for r in routes}
groups = {k: list(v) for row in config['router_settings']['fallbacks']
for k, v in row.items()}
def add(name, params):
if name in indexed:
return indexed[name]
route = {'model_name': name, 'litellm_params': deepcopy(params)}
indexed[name] = route
routes.append(route)
return route
def fallback(source, target):
if source == target or source not in indexed or target not in indexed:
return
targets = groups.setdefault(source, [])
if target not in targets:
targets.append(target)
def payg_params(model, anthropic=False):
return {'model': ('anthropic/' if anthropic else 'openai/') + model,
'api_key': 'os.environ/ZENMUX_PAYG_API_KEY',
'api_base': 'https://zenmux.ai/api/anthropic' if anthropic
else 'https://zenmux.ai/api/v1',
'max_retries': 0}
# Explicit BYOK aliases remain direct; automatic/legacy aliases prefer OpenRouter.
for family in ('zenmux', 'zenmux_open_models'):
for alias, model in catalog.get(family, {}).items():
byok = 'zenmux-payg/' + alias
if env.get('ZENMUX_PAYG_API_KEY'):
add(byok, payg_params(model, anthropic=family == 'zenmux'))
primary = None
if env.get('OPENROUTER_API_KEY'):
primary = {'model': 'openrouter/' + model,
'api_key': 'os.environ/OPENROUTER_API_KEY', 'max_retries': 0}
elif byok in indexed:
primary = indexed[byok]['litellm_params']
if primary is None:
continue
names = [alias, 'auto/' + alias, 'zenmux/' + alias]
if env.get('OPENROUTER_API_KEY'):
names += ['openrouter/' + alias]
if family == 'zenmux_open_models':
names += [alias + '-openrouter']
for name in names:
route = add(name, primary)
if route['litellm_params'].get('api_key') != 'os.environ/ZENMUX_PAYG_API_KEY':
fallback(name, byok)
# Former Gemini subscription aliases follow the existing Google primary.
for alias, model in catalog.get('zenmux_chat', {}).items():
byok = 'zenmux-payg/' + alias.replace('-zenmux', '-latest')
if env.get('ZENMUX_PAYG_API_KEY'):
add(byok, payg_params(model))
google = indexed.get('aistudio/gemini-flash-latest') or indexed.get('vertex/gemini-flash-latest')
primary = google or indexed.get(byok)
if primary:
for name in [alias, 'zenmux/' + alias.replace('-zenmux', '-latest')]:
route = add(name, primary['litellm_params'])
if route['litellm_params'].get('api_key') != 'os.environ/ZENMUX_PAYG_API_KEY':
fallback(name, byok)
# Public ASR compatibility aliases use their same-model direct provider first.
asr = {'asr-gpt': ('openai/gpt-transcribe', 'openai/gpt-transcribe'),
'asr-qwen': ('bailian-cn/qwen-audio-3.1-asr-flash', 'qwen/qwen-audio-3.1-asr-flash'),
'asr-doubao': (None, 'bytedance/doubao-seed-asr-2.0'),
'asr-mimo': (None, 'xiaomi/mimo-v2.5-asr')}
for alias, (direct, model) in asr.items():
# Do not introduce new ASR models to the LAN instance.
if not config.get('_public_gateway'):
break
byok = 'zenmux-payg/' + model.split('/', 1)[1]
if env.get('ZENMUX_PAYG_API_KEY'):
add(byok, payg_params(model))
primary = indexed.get(direct) or indexed.get(byok)
if not primary:
continue
if alias in indexed:
indexed[alias]['litellm_params'] = deepcopy(primary['litellm_params'])
else:
add(alias, primary['litellm_params'])
# Old explicit ZenMux names are compatibility aliases, no subscription calls.
add('zenmux/' + model.split('/', 1)[1], primary['litellm_params'])
if primary['litellm_params'].get('api_key') != 'os.environ/ZENMUX_PAYG_API_KEY':
fallback(alias, byok)
fallback('zenmux/' + model.split('/', 1)[1], byok)
config.pop('_public_gateway', None)
config['router_settings']['fallbacks'] = [{k: v} for k, v in groups.items() if v]
for source, targets in groups.items():
if source not in indexed or any(t not in indexed for t in targets):
raise ValueError('Dangling fallback: ' + source)
if indexed[source]['litellm_params'].get('api_key') == 'os.environ/ZENMUX_PAYG_API_KEY':
raise ValueError('BYOK must be terminal: ' + source)
return config
@@ -0,0 +1,177 @@
"""Render only credential-ready routes; credentials remain environment references."""
import os
import json
from copy import deepcopy
from pathlib import Path
import yaml
from provider_policy import without_subscription
def oauth_ready(path):
try:
record = json.loads(path.read_text())
return isinstance(record, dict) and bool(record.get('access_token'))
except (OSError, ValueError):
return False
def namespace_config(config):
"""Add explicit provider names without removing existing client aliases."""
routes = config['model_list']
existing = {r['model_name']: r for r in routes}
fallbacks = config['router_settings']['fallbacks']
automatic = {name for group in fallbacks for name in group}
renamed = {}
def clone(route, alias):
if alias in existing:
if existing[alias]['litellm_params'] != route['litellm_params']:
raise ValueError('Conflicting namespace alias: ' + alias)
return
new = deepcopy(route)
new['model_name'] = alias
existing[alias] = new
routes.append(new)
for route in list(routes):
alias = route['model_name']
model = route['litellm_params']['model']
if alias in {'gemini-3.6-flash', 'gemini-3.5-flash-lite',
'google/gemini-embedding-2', 'google/gemini-3.8-flash'}:
continue
if alias.endswith('-openrouter'):
new = 'openrouter/' + alias.removesuffix('-openrouter')
elif alias in automatic:
new = 'auto/' + alias
clone(route, 'zenmux/' + alias)
elif model.startswith('anthropic/'):
new = ('zenmux-payg/' + alias.removesuffix('-payg')
if alias.endswith('-payg') else 'zenmux/' + alias)
elif model.startswith('openai/'):
name = model.removeprefix('openai/').split('/', 1)[-1]
new = 'zenmux/' + (name if alias.startswith('asr-') else
alias.replace('-zenmux', '-latest'))
elif model.startswith('openrouter/'):
new = 'openrouter/' + alias
elif model.startswith('gemini/'):
new = 'aistudio/' + alias.removesuffix('-studio')
elif model.startswith('vertex_ai/'):
name = {'google-rerank-default': 'rerank-default-latest',
'google-rerank-fast': 'rerank-fast-latest'}.get(alias,
alias.replace('-vertex', '-latest'))
new = 'vertex/' + name
elif model.startswith('chatgpt/'):
name = model.split('/', 1)[1] if alias in {'gpt-work-sol', 'gpt-review-5.5'} else alias
new = 'codex/' + name
elif model.startswith('xai/'):
new = 'xai/' + alias
else:
raise ValueError('Unrecognized namespace provider: ' + alias)
clone(route, new)
renamed[alias] = new
for group in list(fallbacks):
for source, targets in group.items():
fallbacks.append({renamed[source]: [renamed[target] for target in targets]})
return config
def add_bailian_routes(config, env, vocabulary_path):
if env.get('BAILIAN_CN_API_KEY'):
base = env.get('BAILIAN_CN_API_BASE')
if not base:
raise ValueError('BAILIAN_CN_API_BASE required for Beijing Bailian')
for personal in [False, True]:
params = {'model': 'dashscope/qwen-audio-3.1-asr-flash',
'api_base': base, 'api_key': 'os.environ/BAILIAN_CN_API_KEY',
'max_retries': 0}
if personal:
params['bailian_vocabulary_path'] = vocabulary_path
config['model_list'].append({'model_name': 'bailian-cn/qwen-audio-3.1-asr-flash' +
('-personal' if personal else ''), 'litellm_params': params})
return config
def build_config(catalog, env, exists, namespaced=True):
env = without_subscription(catalog, env)
routes = []
fallbacks = []
def add(alias, model, key=None, **params):
if key:
params['api_key'] = 'os.environ/' + key
routes.append({'model_name': alias, 'litellm_params': {'model': model, **params}})
for key, suffix in [('ZENMUX_SUBSCRIPTION_API_KEY', ''), ('ZENMUX_PAYG_API_KEY', '-payg')]:
if env.get(key):
for alias, model in catalog['zenmux'].items():
# Strip only the LiteLLM provider prefix, retaining ZenMux's vendor prefix.
add(alias + suffix, 'anthropic/' + model, key,
api_base='https://zenmux.ai/api/anthropic')
if env.get('ZENMUX_SUBSCRIPTION_API_KEY'):
for alias, model in catalog.get('zenmux_chat', {}).items():
add(alias, 'openai/' + model, 'ZENMUX_SUBSCRIPTION_API_KEY',
api_base='https://zenmux.ai/api/v1')
if env.get('OPENROUTER_API_KEY'):
for alias, model in catalog['openrouter'].items():
primary = catalog.get('zenmux_open_models', {}).get(alias)
if primary and env.get('ZENMUX_SUBSCRIPTION_API_KEY'):
if primary != model:
raise ValueError('Cross-provider model mismatch: ' + alias)
add(alias, 'openai/' + primary, 'ZENMUX_SUBSCRIPTION_API_KEY',
api_base='https://zenmux.ai/api/v1', max_retries=0)
routes[-1]['model_info'] = {'cooldown_time': 60,
'allowed_fails_policy': {'RateLimitErrorAllowedFails': 0,
'AuthenticationErrorAllowedFails': 0,
'InternalServerErrorAllowedFails': 0,
'ServiceUnavailableErrorAllowedFails': 0,
'TimeoutErrorAllowedFails': 0}}
add(alias + '-openrouter', 'openrouter/' + model, 'OPENROUTER_API_KEY')
fallbacks.append({alias: [alias + '-openrouter']})
else:
add(alias, 'openrouter/' + model, 'OPENROUTER_API_KEY')
elif env.get('ZENMUX_SUBSCRIPTION_API_KEY'):
for alias, model in catalog.get('zenmux_open_models', {}).items():
add(alias, 'openai/' + model, 'ZENMUX_SUBSCRIPTION_API_KEY',
api_base='https://zenmux.ai/api/v1', max_retries=0)
if env.get('GOOGLE_AI_STUDIO_API_KEY'):
add('gemini-transcribe-personal', 'gemini/gemini-3.5-transcribe',
'GOOGLE_AI_STUDIO_API_KEY', personal_vocabulary=True)
for alias, model in catalog['google_ai_studio'].items():
add(alias, 'gemini/' + model, 'GOOGLE_AI_STUDIO_API_KEY')
if env.get('VERTEXAI_ENABLED', '').lower() == 'true' and env.get('VERTEXAI_PROJECT'):
for alias, model in catalog.get('google_vertex', {}).items():
add(alias, 'vertex_ai/' + model,
vertex_project='os.environ/VERTEXAI_PROJECT',
vertex_location='os.environ/VERTEXAI_LOCATION')
for provider, token_var, default_dir in [
('chatgpt', 'CHATGPT_TOKEN_DIR', '/tokens/chatgpt'),
('xai', 'XAI_OAUTH_TOKEN_DIR', '/tokens/xai'),
]:
if exists(Path(env.get(token_var, default_dir)) / 'auth.json'):
for alias, model in catalog[provider].items():
extra = {'use_xai_oauth': True} if provider == 'xai' else {}
add(alias, provider + '/' + model, **extra)
config = {
'model_list': routes,
'general_settings': {
'master_key': 'os.environ/LITELLM_MASTER_KEY',
'database_url': 'os.environ/DATABASE_URL',
'store_model_in_db': False,
'disable_env_credential_login': True,
},
'litellm_settings': {'drop_params': False, 'set_verbose': False},
# Only explicitly approved open-model routes have cross-provider fallbacks.
'router_settings': {'num_retries': 1, 'timeout': 180, 'fallbacks': fallbacks},
}
return add_bailian_routes(namespace_config(config), env,
'/etc/litellm/personal-vocabulary.json') if namespaced else config
if __name__ == '__main__':
catalog = yaml.safe_load(Path('/etc/litellm/models.yaml').read_text())
config = build_config(catalog, os.environ, oauth_ready)
output = Path('/tmp/litellm.yaml')
output.write_text(yaml.safe_dump(config, allow_unicode=True, sort_keys=False))
output.chmod(0o600)
print('Configured routes:', ', '.join(route['model_name'] for route in config['model_list']))
@@ -0,0 +1,77 @@
"""HK-compatible public routes on DC1, with separately named official OpenAI API."""
import os
from pathlib import Path
import yaml
from provider_routes import build_config, namespace_config, add_bailian_routes
from provider_policy import without_subscription, apply_provider_policy
def build_hk_config(catalog, env):
env = without_subscription(catalog, env)
config = build_config(catalog, env, lambda path: False, namespaced=False)
config['general_settings'] = {'master_key': 'os.environ/LITELLM_MASTER_KEY'}
routes = config['model_list']
routes[:] = [route for route in routes if route['model_name'] != 'gemini-transcribe-personal']
if env.get('ZENMUX_SUBSCRIPTION_API_KEY'):
asr_models = {
'asr-gpt': 'openai/gpt-transcribe',
'asr-qwen': 'qwen/qwen-audio-3.1-asr-flash',
'asr-doubao': 'bytedance/doubao-seed-asr-2.0',
'asr-mimo': 'xiaomi/mimo-v2.5-asr',
}
for alias, model in asr_models.items():
routes.append({'model_name': alias, 'litellm_params': {
'model': 'openai/' + model,
'api_key': 'os.environ/ZENMUX_SUBSCRIPTION_API_KEY',
'api_base': 'https://zenmux.ai/api/v1',
'max_retries': 0}})
if env.get('GOOGLE_AI_STUDIO_API_KEY'):
routes.append({'model_name': 'gemini-transcribe', 'litellm_params': {
'model': 'gemini/gemini-3.5-transcribe',
'api_key': 'os.environ/GOOGLE_AI_STUDIO_API_KEY'}})
if env.get('VERTEXAI_ENABLED', '').lower() == 'true' and env.get('VERTEXAI_PROJECT'):
routes.append({'model_name': 'gemini-transcribe-personal', 'litellm_params': {
'model': 'vertex_ai/gemini-3.5-transcribe',
'vertex_project': 'os.environ/VERTEXAI_PROJECT',
'vertex_location': 'global',
'personal_dictation': True}})
# Keep old client aliases; the old 3.6 alias now follows the verified Flash map.
legacy = {
'gemini-3.6-flash': catalog['google_vertex']['gemini-flash-vertex'],
'gemini-3.5-flash-lite': catalog['google_vertex']['gemini-flash-lite-vertex'],
'google/gemini-embedding-2': 'gemini-embedding-2',
'google/gemini-3.8-flash': catalog['google_vertex']['gemini-flash-vertex'],
}
for alias, model in legacy.items():
routes.append({'model_name': alias, 'litellm_params': {
'model': 'vertex_ai/' + model,
'vertex_project': 'os.environ/VERTEXAI_PROJECT',
'vertex_location': 'os.environ/VERTEXAI_LOCATION'}})
config = add_bailian_routes(namespace_config(config), env, '/credentials/personal-vocabulary.json')
if env.get('OPENAI_API_KEY'):
models = dict(catalog.get('openai', {}))
for alias, model in catalog.get('chatgpt', {}).items():
models[alias if alias.endswith('-latest') else model] = model
for alias, model in models.items():
config['model_list'].append({'model_name': 'openai/' + alias, 'litellm_params': {
'model': 'openai/' + model, 'api_base': 'https://api.openai.com/v1',
'api_key': 'os.environ/OPENAI_API_KEY', 'max_retries': 0}})
config['model_list'] = [r for r in config['model_list'] if r['model_name'] != 'asr-gpt']
config['model_list'].append({'model_name': 'asr-gpt', 'litellm_params': {
'model': 'openai/gpt-transcribe', 'api_base': 'https://api.openai.com/v1',
'api_key': 'os.environ/OPENAI_API_KEY', 'max_retries': 0}})
if env.get('ZENMUX_SUBSCRIPTION_API_KEY'):
config['router_settings']['fallbacks'].append({'asr-gpt': ['zenmux/gpt-transcribe']})
config['_public_gateway'] = True
return apply_provider_policy(config, catalog, env)
if __name__ == '__main__':
catalog = yaml.safe_load(Path('/etc/litellm/models.yaml').read_text())
config = build_hk_config(catalog, os.environ)
output = Path('/tmp/litellm.yaml')
output.write_text(yaml.safe_dump(config, allow_unicode=True, sort_keys=False))
output.chmod(0o600)
print('Configured routes:', ', '.join(route['model_name'] for route in config['model_list']))
+79
View File
@@ -0,0 +1,79 @@
import unittest
from pathlib import Path
import yaml
from render_config import build_hk_config
class RoutingTests(unittest.TestCase):
def test_official_api_names_do_not_replace_hk_routes(self):
catalog = yaml.safe_load(Path(__file__).with_name('models.yaml').read_text())
env = {'ZENMUX_SUBSCRIPTION_API_KEY': 'synthetic', 'ZENMUX_PAYG_API_KEY': 'synthetic', 'OPENAI_API_KEY': 'synthetic'}
config = build_hk_config(catalog, env)
routes = {r['model_name']: r['litellm_params'] for r in config['model_list']}
self.assertEqual(routes['openai/gpt-transcribe']['api_base'], 'https://api.openai.com/v1')
self.assertEqual(routes['zenmux/gpt-transcribe']['api_base'], 'https://api.openai.com/v1')
self.assertEqual(routes['asr-gpt'], routes['openai/gpt-transcribe'])
self.assertIn({'asr-gpt': ['zenmux-payg/gpt-transcribe']}, config['router_settings']['fallbacks'])
self.assertEqual(routes['openai/gpt-sol-latest']['model'], 'openai/gpt-6.1-sol')
self.assertNotIn('codex/gpt-sol-latest', routes)
self.assertNotIn('synthetic', yaml.safe_dump(config))
def test_official_asr_without_zenmux_has_no_dangling_fallback(self):
catalog = yaml.safe_load(Path(__file__).with_name('models.yaml').read_text())
config = build_hk_config(catalog, {'OPENAI_API_KEY': 'synthetic'})
routes = {r['model_name']: r['litellm_params'] for r in config['model_list']}
self.assertEqual(routes['asr-gpt']['model'], 'openai/gpt-transcribe')
self.assertEqual(routes['zenmux/gpt-transcribe'], routes['asr-gpt'])
self.assertNotIn({'asr-gpt': ['zenmux/gpt-transcribe']}, config['router_settings']['fallbacks'])
def test_bailian_personal_uses_hk_private_vocabulary(self):
catalog = yaml.safe_load(Path(__file__).with_name('models.yaml').read_text())
routes = build_hk_config(catalog, {'BAILIAN_CN_API_KEY':'synthetic',
'BAILIAN_CN_API_BASE':'https://ws-test.cn-beijing.maas.aliyuncs.com/api/v1'})['model_list']
self.assertEqual(len(routes),4)
self.assertEqual(routes[1]['litellm_params']['bailian_vocabulary_path'],
'/credentials/personal-vocabulary.json')
def test_all_providers(self):
catalog = yaml.safe_load(Path(__file__).with_name('models.yaml').read_text())
env = {key: 'synthetic' for key in ['ZENMUX_SUBSCRIPTION_API_KEY',
'ZENMUX_PAYG_API_KEY', 'OPENROUTER_API_KEY', 'GOOGLE_AI_STUDIO_API_KEY',
'VERTEXAI_PROJECT']}
env['VERTEXAI_ENABLED'] = 'true'
result = build_hk_config(catalog, env)
routes = {r['model_name']: r['litellm_params'] for r in result['model_list']}
self.assertEqual(len(routes), len(result['model_list']))
self.assertEqual(routes['gemini-transcribe-personal']['model'], 'vertex_ai/gemini-3.5-transcribe')
self.assertIs(routes['gemini-transcribe-personal']['personal_dictation'], True)
self.assertEqual(routes['gemini-transcribe-personal']['vertex_location'], 'global')
self.assertNotIn('grok-latest', routes)
self.assertNotIn('gpt-sol-latest', routes)
self.assertEqual(routes['asr-gpt']['model'], 'openai/openai/gpt-transcribe')
self.assertEqual(routes['asr-qwen']['model'], 'openai/qwen/qwen-audio-3.1-asr-flash')
self.assertEqual(routes['asr-doubao']['model'], 'openai/bytedance/doubao-seed-asr-2.0')
self.assertEqual(routes['asr-mimo']['model'], 'openai/xiaomi/mimo-v2.5-asr')
for alias in ['asr-gpt', 'asr-qwen', 'asr-doubao', 'asr-mimo']:
self.assertNotIn('personal_dictation', routes[alias])
self.assertEqual(routes[alias]['api_key'], 'os.environ/ZENMUX_PAYG_API_KEY')
self.assertEqual(routes['gemini-transcribe']['model'], 'gemini/gemini-3.5-transcribe')
self.assertEqual(routes['gemini-3.6-flash']['model'], 'vertex_ai/gemini-3.8-flash')
self.assertTrue(result['router_settings']['fallbacks'])
self.assertNotIn('os.environ/ZENMUX_SUBSCRIPTION_API_KEY', yaml.safe_dump(result))
self.assertEqual(routes['auto/glm-latest']['api_key'], 'os.environ/OPENROUTER_API_KEY')
self.assertEqual(routes['vertex/gemini-transcribe-personal'], routes['gemini-transcribe-personal'])
self.assertNotIn('aistudio/gemini-transcribe-personal', routes)
self.assertEqual(routes['zenmux/gpt-transcribe'], routes['asr-gpt'])
self.assertEqual(result['general_settings'], {'master_key': 'os.environ/LITELLM_MASTER_KEY'})
self.assertNotIn('synthetic', yaml.safe_dump(result))
def test_missing_credentials(self):
catalog = yaml.safe_load(Path(__file__).with_name('models.yaml').read_text())
result = build_hk_config(catalog, {})
self.assertEqual(result['model_list'], [])
self.assertEqual(result['router_settings']['fallbacks'], [])
if __name__ == '__main__':
unittest.main()
@@ -0,0 +1,54 @@
"""Scoped Vertex dictation settings; ordinary transcription calls stay unchanged."""
import json
from pathlib import Path
VOCABULARY = Path('/credentials/personal-vocabulary.json')
def vocabulary(path=VOCABULARY):
words = json.loads(path.read_text())
if not isinstance(words, list) or not 1 <= len(words) <= 100:
raise ValueError('Personal vocabulary must contain 1-100 terms')
if any(not isinstance(word, str) or not word.strip() or len(word) > 100 for word in words):
raise ValueError('Invalid personal vocabulary term')
return list(dict.fromkeys(word.strip() for word in words))
def install():
from litellm.llms.vertex_ai.audio_transcription.gemini_transcribe_transformation import (
VertexGeminiAudioTranscriptionConfig,
)
original = VertexGeminiAudioTranscriptionConfig.transform_audio_transcription_request
if getattr(original, '_personal_dictation_patch', False):
return
def transform(self, model, audio_file, optional_params, litellm_params):
if optional_params.get('personal_dictation') is not True:
return original(self, model, audio_file, optional_params, litellm_params)
if model.removeprefix('vertex_ai/') != 'gemini-3.5-transcribe':
raise ValueError('Personal dictation is restricted to Vertex Gemini 3.5 Transcribe')
if optional_params.get('timestamp_granularities') or optional_params.get('response_format') not in (None, 'json', 'text'):
raise ValueError('Personal dictation supports json/text without timestamps')
clean = {key: val for key, val in optional_params.items() if key != 'personal_dictation'}
result = original(self, model, audio_file, clean, litellm_params)
config = result.data.setdefault('generationConfig', {}).setdefault('audioTranscriptionConfig', {})
if set(config) - {'mode', 'languageCodes'} or config.get('mode') not in (None, 'VERBATIM'):
raise ValueError('Unexpected Vertex transcription defaults; review adapter compatibility')
config['mode'] = 'SMART'
config['customVocabulary'] = vocabulary()
return result
transform._personal_dictation_patch = True
VertexGeminiAudioTranscriptionConfig.transform_audio_transcription_request = transform
if __name__ == '__main__':
import sys
from litellm import run_server
from zenmux_transcription import install as install_zenmux_asr
from bailian_transcription import install as install_bailian_asr
install_zenmux_asr()
install_bailian_asr()
install()
sys.argv[0] = 'litellm'
sys.exit(run_server())
@@ -0,0 +1,91 @@
"""Adapt four allowlisted ZenMux JSON ASR models to LiteLLM's audio endpoint."""
import base64
from litellm.llms.base_llm.audio_transcription.transformation import AudioTranscriptionRequestData
from litellm.llms.openai.transcriptions.whisper_transformation import OpenAIWhisperAudioTranscriptionConfig
from litellm.litellm_core_utils.audio_utils.utils import process_audio_file
from litellm.types.utils import TranscriptionResponse, TranscriptionUsageDurationObject
MODELS = frozenset({'openai/gpt-transcribe', 'qwen/qwen-audio-3.1-asr-flash',
'bytedance/doubao-seed-asr-2.0', 'xiaomi/mimo-v2.5-asr'})
API_BASE = 'https://zenmux.ai/api/v1'
class ZenMuxTranscriptionConfig(OpenAIWhisperAudioTranscriptionConfig):
@property
def has_native_transcription_endpoint(self):
return True
def get_supported_openai_params(self, model):
return ['language', 'response_format', 'prompt']
def map_openai_params(self, non_default_params, optional_params, model, drop_params):
params = super().map_openai_params(non_default_params, optional_params, model, drop_params)
if params.get('prompt'):
raise ValueError('ZenMux comparison routes do not accept prompts; leave prompt empty')
if params.get('response_format') not in (None, 'json', 'text'):
raise ValueError('ZenMux comparison routes support json/text only')
return params
def validate_environment(self, headers, model, messages, optional_params,
litellm_params, api_key=None, api_base=None):
if not api_key:
raise ValueError('ZenMux subscription API key is required')
return {**headers, 'Authorization': 'Bearer ' + api_key, 'Content-Type': 'application/json'}
def get_complete_url(self, api_base, api_key, model, optional_params, litellm_params, stream=None):
if (api_base or '').rstrip('/') != API_BASE or model not in MODELS:
raise ValueError('ZenMux ASR adapter is restricted to the configured endpoint and model allowlist')
if stream:
raise ValueError('Streaming is not enabled for comparison dictation routes')
return API_BASE + '/audio/transcriptions'
def transform_audio_transcription_request(self, model, audio_file, optional_params, litellm_params):
if model not in MODELS:
raise ValueError('Unknown ZenMux ASR model')
if optional_params.get('timestamp_granularities') or optional_params.get('prompt'):
raise ValueError('Timestamps and prompt injection are disabled on comparison routes')
audio = process_audio_file(audio_file)
formats = {'audio/wav': 'wav', 'audio/x-wav': 'wav', 'audio/wave': 'wav',
'audio/mpeg': 'mp3', 'audio/mp3': 'mp3', 'audio/flac': 'flac',
'audio/x-flac': 'flac', 'audio/m4a': 'm4a', 'audio/mp4': 'm4a',
'audio/x-m4a': 'm4a', 'audio/ogg': 'ogg', 'audio/webm': 'webm',
'video/webm': 'webm', 'audio/aac': 'aac'}
mime = audio.content_type.split(';', 1)[0].lower()
if mime not in formats:
raise ValueError('Unsupported audio MIME type for ZenMux ASR')
body = {'model': model, 'input_audio': {
'data': base64.b64encode(audio.file_content).decode(), 'format': formats[mime]},
'stream': False}
language = optional_params.get('language')
if language:
body['language'] = language
return AudioTranscriptionRequestData(data=body)
def transform_audio_transcription_response(self, raw_response):
body = raw_response.json()
if not isinstance(body.get('text'), str):
raise ValueError('ZenMux ASR returned no transcript text')
response = TranscriptionResponse(text=body['text'])
usage = body.get('usage') or {}
if usage.get('seconds') is not None:
# ZenMux has no audio/text token breakdown; report duration without inventing one.
response.usage = TranscriptionUsageDurationObject(type='duration', seconds=usage['seconds'])
response['duration'] = usage['seconds']
return response
def install():
import litellm
from litellm.utils import ProviderConfigManager
original = ProviderConfigManager.get_provider_audio_transcription_config
if getattr(original, '_zenmux_asr_adapter', False):
return
def select(model, provider):
if provider == litellm.LlmProviders.OPENAI and model in MODELS:
return ZenMuxTranscriptionConfig()
return original(model, provider)
select._zenmux_asr_adapter = True
ProviderConfigManager.get_provider_audio_transcription_config = staticmethod(select)
+5 -1
View File
@@ -70,8 +70,12 @@ HINDSIGHT_API_EMBEDDINGS_OPENROUTER_MODEL=baai/bge-m3
HINDSIGHT_API_EMBEDDINGS_OPENAI_BATCH_SIZE=32 HINDSIGHT_API_EMBEDDINGS_OPENAI_BATCH_SIZE=32
HINDSIGHT_API_EMBEDDINGS_MAX_INPUT_TOKENS=8192 HINDSIGHT_API_EMBEDDINGS_MAX_INPUT_TOKENS=8192
# Local TEI reranker, then OpenRouter, then fail-open RRF. # Google tested with synthetic text only; keep production reranking local.
HINDSIGHT_API_RERANKER_PROVIDER=tei HINDSIGHT_API_RERANKER_PROVIDER=tei
HINDSIGHT_API_RERANKER_LITELLM_API_BASE=http://192.168.50.49:4000
HINDSIGHT_API_RERANKER_LITELLM_API_KEY=replace-with-litellm-client-key
HINDSIGHT_API_RERANKER_LITELLM_MODEL=google-rerank-default
HINDSIGHT_API_RERANKER_LITELLM_TIMEOUT=10
HINDSIGHT_API_RERANKER_TEI_URL=http://192.168.50.100:18084 HINDSIGHT_API_RERANKER_TEI_URL=http://192.168.50.100:18084
HINDSIGHT_API_RERANKER_TEI_BATCH_SIZE=32 HINDSIGHT_API_RERANKER_TEI_BATCH_SIZE=32
HINDSIGHT_API_RERANKER_TEI_MAX_CONCURRENT=4 HINDSIGHT_API_RERANKER_TEI_MAX_CONCURRENT=4
+17 -6
View File
@@ -1,17 +1,28 @@
# Hindsight 独立 VM # Hindsight 独立 VM
本项目部署到 PVE VM `hindsight`(`192.168.50.47`)的 Arcane Environment。 本项目部署到 PVE VM `hindsight`(`192.168.50.47`)的 Arcane Environment。
它是新一代空白实例,不直接挂载或修改 unRaid 上的旧数据库。 它是已迁移生产数据的独立实例,不直接挂载 unRaid 上的旧数据库。
## 组件 ## 组件
- Hindsight `0.10.1-slim`; - Hindsight `0.10.3-slim`;
- PostgreSQL 18 + pgvector; - PostgreSQL 18 + pgvector;
- unRaid GPU0 上的本地 TEI `BAAI/bge-m3` embedding;OpenRouter 同模型保留为手动回滚; - unRaid GPU0 上的本地 TEI `BAAI/bge-m3` embedding;OpenRouter 同模型保留为手动回滚;
- unRaid GPU0 上的本地 TEI `Alibaba-NLP/gte-multilingual-reranker-base` reranker,失败时依次回退 OpenRouter `cohere/rerank-4-fast` 与 RRF; - 本地 TEI 用于生产 rerank,保留原有 OpenRouter `cohere/rerank-4-fast` 与 RRF 回退;Google 仅用于合成测试;
- ZenMux GPT-6 Luna 用于 Retain; - Codex OAuth GPT-6 Luna 用于 Retain;
- ZenMux Grok 4.7 用于 Consolidation; - xAI OAuth Grok 4.7 用于 Consolidation;
- ZenMux Sonnet 5 用于 Reflect。 - Codex OAuth GPT-6 Sol 用于 Reflect 与知识页刷新;原有 ZenMux 回退保持不变。
## 本次升级验证
升级前已生成 `/srv/hindsight/backups/pre-google-0.10.3-20261010.dump`,并通过
`pg_restore --list` 校验。应用和数据库均健康,数据库容器没有重建。
embedding 仍为本地 bge-m3,未重新生成向量。
六组合成排序样本在 Google default、fast 与本地 TEI 上均通过,不代表已证明
Google 质量优于 TEI。Google 单次请求约 1 秒,本地 TEI 为毫秒级。
用户选择暂不向 Google 发送私人记忆,因此生产排序已恢复本地 TEI,
Google 不在生产排序链中。embedding 迁移另行安排。
运行数据固定在 `/srv/hindsight/postgres`。Compose 和非敏感模板进入 Git,真实 运行数据固定在 `/srv/hindsight/postgres`。Compose 和非敏感模板进入 Git,真实
数据库密码、ZenMux key 和 OpenRouter key 仅保存在该 Arcane Environment 的 数据库密码、ZenMux key 和 OpenRouter key 仅保存在该 Arcane Environment 的
+1 -1
View File
@@ -26,7 +26,7 @@ services:
max-file: "3" max-file: "3"
hindsight: hindsight:
image: ghcr.io/vectorize-io/hindsight:0.10.1-slim@sha256:4d14ce6643ecb2dacb9446e0b967a35f811ce8d0172a39131281520b4c425da9 image: ghcr.io/vectorize-io/hindsight:1.1.0-slim@sha256:fbc37d194bce007b6cce2fd9f66cd7f8c050bd18b80e662090a92835d9cb191d
container_name: hindsight container_name: hindsight
restart: unless-stopped restart: unless-stopped
env_file: env_file:
+1 -1
View File
@@ -5,7 +5,7 @@ x-arcane:
services: services:
app: app:
image: ghcr.io/bookorbit/bookorbit:3.1.0@sha256:78b6eac18306eb7eba0dbf6592ecdee01b15f24de9a941cb853db4a0232d7999 image: ghcr.io/bookorbit/bookorbit:3.6.0@sha256:bbb5da52e96dbbff1dd36f55454f9845e6cde264b4b3428a3ea783124a4ca62e
container_name: bookorbit-app container_name: bookorbit-app
restart: unless-stopped restart: unless-stopped
init: true init: true
+2
View File
@@ -0,0 +1,2 @@
GRIMMORY_DB_PASSWORD=replace-with-a-random-password
GRIMMORY_DB_ROOT_PASSWORD=replace-with-a-different-random-password
+30
View File
@@ -0,0 +1,30 @@
# Grimmory on media-core
- LAN URL: `http://192.168.50.46:6060`
- Legacy evaluation library: `/mnt/truenas/multimedia/books` -> `/books` (read-only)
- Clean library projection: `/mnt/truenas/multimedia/grimmory/library` -> `/books-clean` (read-only)
- BookDrop: `/mnt/truenas/multimedia/grimmory/staging` -> `/bookdrop`
- Catalog snapshots and review reports: `/mnt/truenas/multimedia/grimmory/manifests`
- Rebuild state: `/opt/media-core/appdata/grimmory-rebuild/catalog.sqlite3`
- Application data: `/opt/media-core/appdata/grimmory/data`
- MariaDB data: `/opt/media-core/appdata/grimmory/mariadb`
- Storage mode: `NETWORK`
This is a parallel evaluation deployment. BookOrbit remains on the legacy
`/mnt/truenas/multimedia/books` tree, while Grimmory indexes a separately
materialized Work -> Edition -> Asset projection. Only Grimmory's dedicated
BookDrop is writable. Do not enable file rename, move, delete, or metadata
write-back against `/books` during the evaluation.
Keep the existing `Main Library` at `/books` until the clean projection has
been accepted. Create a second library at `/books-clean` with organization
mode `Book Per Folder`, metadata source `Prefer Sidecar`, and folder watching
disabled. Organization mode is immutable after library creation.
Set
`GRIMMORY_DB_PASSWORD` and `GRIMMORY_DB_ROOT_PASSWORD` only in the Arcane
project environment; do not commit their values.
Before an upgrade, dump the `grimmory` MariaDB database and back up
`/opt/media-core/appdata/grimmory/data`. Image tags describe the release and
the pinned OCI index digests are authoritative.
+94
View File
@@ -0,0 +1,94 @@
x-arcane:
icon: book-open
urls:
- http://192.168.50.46:6060
services:
grimmory:
image: ghcr.io/grimmory-tools/grimmory:v3.5.0@sha256:bf6fe21c6e247597f7869664a440b5a34242e2d2940575cae39c17bfaa66dc0b
container_name: grimmory
restart: unless-stopped
init: true
depends_on:
mariadb:
condition: service_healthy
ports:
- "6060:6060"
environment:
USER_ID: "1000"
GROUP_ID: "1000"
TZ: Asia/Shanghai
DATABASE_URL: jdbc:mariadb://mariadb:3306/grimmory
DATABASE_USERNAME: grimmory
DATABASE_PASSWORD: ${GRIMMORY_DB_PASSWORD:?set GRIMMORY_DB_PASSWORD in the Arcane project environment}
SWAGGER_ENABLED: "false"
FORCE_DISABLE_OIDC: "false"
DISK_TYPE: NETWORK
volumes:
- type: bind
source: /opt/media-core/appdata/grimmory/data
target: /app/data
bind:
create_host_path: false
- type: bind
source: /mnt/truenas/multimedia/books
target: /books
read_only: true
bind:
create_host_path: false
- type: bind
source: /mnt/truenas/multimedia/grimmory/library
target: /books-clean
read_only: true
bind:
create_host_path: false
- type: bind
source: /mnt/truenas/multimedia/grimmory/staging
target: /bookdrop
bind:
create_host_path: false
healthcheck:
test: ["CMD-SHELL", "wget -q --spider http://127.0.0.1:6060/api/v1/healthcheck"]
interval: 30s
timeout: 10s
retries: 5
start_period: 90s
labels:
com.getarcaneapp.arcane.icon: book-open
logging:
driver: json-file
options:
max-size: 10m
max-file: "3"
mariadb:
image: lscr.io/linuxserver/mariadb:11.4.8@sha256:91de7f701bc7fc3a424b81beafca7a7c6c4c5b7c8be6afd2ae148698695c0b0c
container_name: grimmory-db
restart: unless-stopped
environment:
PUID: "1000"
PGID: "1000"
TZ: Asia/Shanghai
MYSQL_ROOT_PASSWORD: ${GRIMMORY_DB_ROOT_PASSWORD:?set GRIMMORY_DB_ROOT_PASSWORD in the Arcane project environment}
MYSQL_DATABASE: grimmory
MYSQL_USER: grimmory
MYSQL_PASSWORD: ${GRIMMORY_DB_PASSWORD:?set GRIMMORY_DB_PASSWORD in the Arcane project environment}
volumes:
- type: bind
source: /opt/media-core/appdata/grimmory/mariadb
target: /config
bind:
create_host_path: false
healthcheck:
test: ["CMD", "mariadb-admin", "ping", "-h", "localhost"]
interval: 5s
timeout: 5s
retries: 12
start_period: 30s
labels:
com.getarcaneapp.arcane.icon: mariadb
logging:
driver: json-file
options:
max-size: 10m
max-file: "3"
+6 -4
View File
@@ -1,7 +1,9 @@
.git .git
.env .venv
tests
README.md
__pycache__ __pycache__
*.pyc *.pyc
seed state/
cache/
dry-run/
logs/
.env
+43 -7
View File
@@ -1,11 +1,47 @@
ZENMUX_API_KEY= # Deployment example only. Do not commit real secrets.
ASR_AGENT_TOKEN= ASR_WORKSPACE=/home/claw/codex-workspace/meeting-asr
ASR_PUBLIC_URL=http://192.168.50.49:8788 ASR_VAULT=/home/claw/obsidian-vault
ASR_STATE_DIR=/home/claw/codex-workspace/meeting-asr/state
ASR_CACHE_DIR=/home/claw/codex-workspace/meeting-asr/cache
ASR_LOG=/home/claw/codex-workspace/meeting-asr/logs/asr-pipeline.log
ASR_ENV_FILE=/home/claw/codex-workspace/meeting-asr/.env
ASR_RECORDINGS_ROOT=/home/claw/codex-workspace/meeting-asr/recordings
MEETING_INBOX_BASE=/home/claw/codex-workspace/meeting-asr/recordings
ASR_CONTROL_DB=/home/claw/codex-workspace/meeting-asr/state/intake.db
# ASR Provider: qwen3cam | soniox | gemini | openai | grok
ASR_PROVIDER=qwen3cam
ASR_DIARIZE_URL=http://192.168.50.100:10099/asr ASR_DIARIZE_URL=http://192.168.50.100:10099/asr
ASR_VOICEPRINT_ENROLL_URL=http://192.168.50.100:10099/voiceprints/enroll
# Unified LiteLLM gateway (replaces deprecated ZenMux)
LITELLM_BASE_URL=http://192.168.50.49:4000/v1
LITELLM_API_KEY=
ASR_GEMINI_MODEL=aistudio/gemini-3.5-transcribe
ASR_OPENAI_MODEL=openai/gpt-4o-transcribe-diarize
ASR_CLEAN_MODEL=aistudio/gemini-flash-lite-latest
ASR_KEYWORD_MODEL=aistudio/gemini-flash-lite-latest
ASR_DRAFT_MODEL=codex/gpt-sol-latest
# Direct cloud ASR keys (when not routed through LiteLLM)
SONIOX_API_KEY=
SONIOX_MODEL=stt-async-v5
XAI_API_KEY=
ASR_GROK_MODEL=grok-voice-transcribe-2.0
# Knowledge & Obsidian integration
HINDSIGHT_API=http://192.168.50.47:8888 HINDSIGHT_API=http://192.168.50.47:8888
HINDSIGHT_BANK=personal-context HINDSIGHT_BANK=personal-context
ASR_GIT_SYNC=0
# Web & Notifications
# ASR_WEB_PASSWORD is required. An empty value closes the browser UI.
# ASR_WEB_ALLOW_ANON=1 opens it with no password; use only on a process that is not reachable.
# ASR_WEB_COOKIE_SECURE=1 when the site is served over HTTPS.
ASR_PUBLIC_URL=http://192.168.50.49:8788
ASR_REVIEW_PUBLIC_HOST=192.168.50.145
ASR_REVIEW_PORT=8788
ASR_AGENT_TOKEN=
ASR_WEB_PASSWORD=
ASR_NOTIFY=0
ASR_NOTIFY_BACKEND=none ASR_NOTIFY_BACKEND=none
HERMES_WEBHOOK_URL=
HERMES_WEBHOOK_SECRET=
TELEGRAM_BOT_TOKEN=
TELEGRAM_CHAT_ID=
+17 -1
View File
@@ -1,4 +1,20 @@
.venv/
.env .env
seed/*.db .env.*
!.env.example
__pycache__/ __pycache__/
*.pyc *.pyc
*.egg-info/
build/
dist/
state/
cache/
logs/
dry-run/
*.db
*.db-*
*.sqlite*
*.bak*
recordings/*
!recordings/bin/
recordings/bin/__pycache__/
+22
View File
@@ -0,0 +1,22 @@
# Meeting ASR
这是独立的会议、电话录音转录与纪要项目,不是 Agent 插件。先读 README.md 和
docs/current-state.md。历史 HANDOFF.md 只作设计背景,不能当成现态。
## 边界
- ASR GPU 推理由 unRaid 外部服务提供;本项目负责上传、编排、纠错、审核与发布。
- 不依赖 Claude/Codex/Hermes CLI。Hermes 仅是现有通知适配器,可独立替换。
- 原音频、转写、SQLite、审核答案和凭据属于私有运行数据,不得提交到 Git。
- 人工确认的别名和声纹身份不可猜测覆盖;更换 SQLite 前必须做一致性备份。
- 原文是证据,Hindsight 是背景;不能把记忆中的事实伪装成录音中说过的话。
- 纪要以关键内容、结论、线索和行动项为目标,不追求逐字还原,不补造金额或身份。
- 候选词从原始转写提取,去口语化与纠错分开;含糊项保留,交人工确认。
- 只有审核通过的正式纪要可进入 Obsidian。发布遵循 vault 自己的 AGENTS.md/SOP。
- 当前写入边界尚未重新审计;不得宣称已实现不可绕过的目录白名单。
- 拆分后的项目尚未替换生产服务。禁止启动第二个监听器消费相同录音目录。
## 验证
使用独立临时目录运行 `python -m unittest discover -s tests -v`;不得拿生产词库做测试。
测试期间关闭通知、Git 写入和外部推理。模型更新、真实录音测试和服务切换分别验收。
+7 -1
View File
@@ -2,7 +2,7 @@ FROM python:3.13-slim-bookworm@sha256:ed86c82274b3c69b52fb5820f358f0bd7df0b60333
RUN apt-get update \ RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \ && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
ca-certificates curl tini \ ca-certificates curl ffmpeg git openssh-client tini \
&& rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/apt/lists/*
WORKDIR /app WORKDIR /app
@@ -15,6 +15,7 @@ COPY termbank/ ./termbank/
ENV PYTHONUNBUFFERED=1 \ ENV PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1 \ PYTHONDONTWRITEBYTECODE=1 \
PYTHONPATH=/app \ PYTHONPATH=/app \
ASR_DATA_DIR=/data \
ASR_STATE_DIR=/state \ ASR_STATE_DIR=/state \
ASR_VAULT=/data/vault \ ASR_VAULT=/data/vault \
ASR_TERMBANK_DIR=/app/termbank \ ASR_TERMBANK_DIR=/app/termbank \
@@ -24,6 +25,11 @@ ENV PYTHONUNBUFFERED=1 \
ASR_DRAFTS_DIR=/data/artifacts/drafts \ ASR_DRAFTS_DIR=/data/artifacts/drafts \
ASR_RESULTS_DIR=/data/artifacts/results \ ASR_RESULTS_DIR=/data/artifacts/results \
ASR_REVISION_QUEUE_DIR=/state/revision-queue \ ASR_REVISION_QUEUE_DIR=/state/revision-queue \
ASR_PROVIDER=qwen3cam \
LITELLM_BASE_URL=http://192.168.50.49:4000/v1 \
ASR_CLEAN_MODEL=aistudio/gemini-flash-lite-latest \
ASR_KEYWORD_MODEL=aistudio/gemini-flash-lite-latest \
ASR_DRAFT_MODEL=codex/gpt-sol-latest \
ASR_GIT_SYNC=0 \ ASR_GIT_SYNC=0 \
ASR_NOTIFY=0 \ ASR_NOTIFY=0 \
ASR_REVIEW_HOST=0.0.0.0 \ ASR_REVIEW_HOST=0.0.0.0 \
+300
View File
@@ -0,0 +1,300 @@
# ASR 后处理管线 — 项目方案与交接状态
最后更新:2026-07-26(会话结束时移交)
---
## 0. 一句话现状
独立 Python 程序已上线并接管 systemd,**召回问题已定位并大幅修复(1/13 → 12/13)**,
**结构归拢已生效**;但 Kai 指出的更根本的 6 项设计缺陷(S2 职责过载、背景重复、
数字处理、音通道过窄、缺关系图谱、纪要目标定位)**只完成方案设计,未实现**。
下一个接手的模型应从「第 4 节 待实施方案」开始。
---
## 1. 项目路径
| 用途 | 路径 |
|---|---|
| **生产程序(主)** | `/home/claw/claude-workspace/OBSIDIAN/asr-pipeline/` |
| 入口 | `asr-pipeline/bin/asr-refine`(自带 `.venv`,依赖仅 pypinyin / rapidfuzz / numpy) |
| 代码包 | `asr-pipeline/asrpipe/*.py` |
| 词库(SQLite) | `asr-pipeline/state/lexicon.db` |
| 术语库 | `asr-pipeline/termbank/terms.json`(27.7k) + `allowlist.txt`(9.5k) |
| 缓存(可删) | `asr-pipeline/cache/` |
| dry-run 产物 | `asr-pipeline/dry-run/<slug>/{cleaned.txt,draft.md,report.json}` |
| **评测实验室** | `/home/claw/claude-workspace/OBSIDIAN/asr-lab/`(12 篇 fixture + 11 篇人工定稿 + `REPORT.md`) |
| fixture | `asr-lab/fixtures/<name>/{raw.txt,final.md,draft_baseline.md}` |
| 转写落地(触发点) | `~/obsidian-vault/.asr-inbox/<slug>.txt`(gitignored) |
| 转写持久归档 | `OBSIDIAN/recordings/transcripts/<fid>-<datetime>.txt` |
| 私有粗稿 | `asr-pipeline/state/drafts/`(不进 Vault/Git) |
| 正式输出 | Web 全部确认后直接写 `~/obsidian-vault/minutes/` |
| 术语表(**只读**) | `~/obsidian-vault/minutes/_glossary.md` |
| 人物/公司卡(**只读**) | `~/obsidian-vault/People/*.md`、`Companies/*.md` |
| 日志 | `OBSIDIAN/logs/asr-pipeline.log` |
| 凭证(600) | `OBSIDIAN/bin/.asr-refine.env`(`ANTHROPIC_AUTH_TOKEN=` ZenMux key) |
| systemd 触发 | `~/.config/systemd/user/obsidian-asr-refine.path` → `obsidian-asr-refine-native.service` |
| 旧链路(保留可回退) | `OBSIDIAN/bin/obsidian-asr-refine.sh` + `obsidian-asr-refine.service` |
| skill 文档 | `~/.claude/skills/memo-inbox/references/asr-refine-pipeline.md` |
### 常用命令
```bash
P=~/claude-workspace/OBSIDIAN/asr-pipeline/bin/asr-refine
$P doctor # 路径/凭证/词库/Hindsight/两个模型 全检
$P dry-run <file.txt> # 全流程,不写 vault、不 commit
$P run-all # 处理 .asr-inbox 中所有未处理转写
$P lexicon stats|sync|show|learn <别名> <规范名> --note "…"
$P build-termbank # 重建术语库(联网)
```
回退到旧 `claude -p` 链路:
```bash
sed -i 's/^Unit=.*/Unit=obsidian-asr-refine.service/' ~/.config/systemd/user/obsidian-asr-refine.path
systemctl --user daemon-reload && systemctl --user restart obsidian-asr-refine.path
```
---
## 2. 当前架构(已实现)
```
① 音频 → recordings/inbox/ obsidian-recording-watch.service
② meeting-watch.py → asr-transcribe.sh MEATING_ASR_BACKEND=qwen3cam
= :10099 Qwen3-ASR-1.7B(vLLM) + FSMN-VAD + ForcedAligner-0.6B + CAM++ + 声纹(11人)
输出 【谭凯】…(声纹已命名,身份可信)/【speaker_N】…(未命名)
※ FunASR 不是独立后端,只是该服务内部 VAD/hotword 组件
③ 桥接为 ~/obsidian-vault/.asr-inbox/<date>-<HHMMSS>录音.txt(原子 mv)
④ path unit → obsidian-asr-refine-native.service → asr-refine run-all
⑤ 五阶段(见下)→ state/drafts/<slug>.md
⑥ git commit+push → 更新 _待确认.md → Telegram 通知(hermes-gateway :8644 签名 webhook)
⑦ Kai 在一次性 Web 页复核 → Sonnet 应用确认 → 直接发布正式 minutes/
```
### 五阶段
| 阶段 | 模块 | 内容 | 成本 |
|---|---|---|---|
| S0 | `segment.py` | 退化噪声折叠 + 说话人感知分片(1800/200 overlap) | 免费 |
| S1c | `refine.keyword_recall` | LLM 抽专有名词 → Hindsight recall | 1 次便宜调用 |
| S1 | `flagger.py` | 候选生成:音近 + 别名 + 变体 + 关系支持 | 免费(4~13s) |
| S2 | `refine.clean_chunks` | 去口语化 **+ 候选裁决**(并发 6) | N 次便宜调用 |
| S3 | `fidelity.py` | 确定性保真闸门 → 失败则严格重试 → 再失败保留原文 | 免费 |
| S4 | `refine.write_draft` | 一次强模型调用,结构化 + 出处标注 | 1 次调用 |
模型(实测选型,勿随意改):
- 清洗 `google/gemini-3.5-flash-lite`。对比 `claude-haiku-4.5`:闸门失败 19/87 vs 78/87,
haiku 有 69 次是**过度压缩**(当摘要器,删掉 AUTOIMMUNE/ADCS/CD),且慢 2×、贵 2.9×。
- 成稿 `anthropic/claude-sonnet-5`。
- 端点 `https://zenmux.ai/api/v1/chat/completions`;并发 12 触发 429、6 稳定;
`temperature` 被 sonnet-5/opus-5 拒(客户端已自愈并记忆);HTTP 402 是瞬时的(按 max_tokens 预留额度)。
---
## 3. 本轮已完成(可直接使用)
### 3.1 诊断结论(重要,别重复走弯路)
Kai 反馈"纠错太次",实测复现 `2026-07-21-160435录音`:
```
8 个规范实体全部 in_play=True,池子里也都有,13 个口语面形只有 1 个产出候选
```
**不是召回/推断能力问题,是匹配器把答案扔了。** 三条硬规则造成:
1. `if span == cand.form: return` — 精确别名被当"已正确"丢弃。`万亚坤` 卡片写着 `aliases:["老万"]`,`迈威生物` 写着 `["迈威"]`,全被吃掉。
2. `abs(len(span)-len(cand.form)) > 1` — 简称永远匹配不上全名(麦威↔迈威生物、洛奇↔洛启生物、保山↔宝山基金)。
3. 只做等长整串 `seq_score`,无子串/称谓形态(光辉↔李光辉、张泰海↔张成海)。
另有第 4 条:`in_play` 里的名字若池子里没有 Cand(宝山基金无公司卡,仅靠纪要**文件名**进 in_play)则永不可匹配;且 in_play 被文件名/日期污染 25/70。
### 3.2 已实现的修复
**`lexicon.py`(新)** — SQLite 词库。设计铁律:**Obsidian 只读**,不回写 `_glossary.md`、不改卡片;
学到的别名/纠错单独沉淀(类比声纹库)。选 SQLite 而非 Hindsight 的理由:别名解析要确定性精确查表,
Hindsight 是 LLM 抽取 + 语义召回(1.4s/次,概率性),两者互补 —— lexicon 定身份,Hindsight 供语境。
```
entity(canonical, kind, company, role, related, card) 251 条(95 有归属,70 有关系)
alias(alias, canonical, kind, origin, confirmed, hits, note) 210 条(82 卡片 + 128 学习/术语表)
correction(surface, canonical, n, confirmed, note) 158 条
meta(vault_sig, synced_at)
```
- 从卡片 frontmatter 拉 `aliases/company/role/related_people`
- 导入 `_glossary.md` 153 条,**保留一对多语境映射**(`许博→许俊彦/许恒皓`、`徐总→徐劲前/邹洵`)
- 已录入 Kai 本轮 12 条确认(老尚→单永强、麦威→迈威生物、张泰海→张成海 等)
- 坑已修:`_fm` 用 `[ \t]*` 而非 `\s*`(`\s` 吃换行,空 `role:` 会把下一行 `status: active` 当值)
**`flagger.py` 重构** — 变体展开 + 多通道匹配:
- `variants()`:机构简称(org_head)、去姓称名、称谓(老X/小X/X总/X博/X哥/X姐/X老师/X董,`博=博士` 有 glossary 三例佐证)
- 每条候选带 `rule`(推导方式)+ `owner`(任职)+ `support`(`X(本次在谈)` / `X(近期相关)`)
- `RULE_MIN` 分规则阈值;`anchor` 结构锁定:称谓字必须**字面相同**,只有名字音节可漂
(否则 0.82 阈值等于"一个音节对上就行",产生 36 条垃圾:想谈→谭凯、理解→李光辉、就动→刘冀)
- `one-off`:等长、n-1 字相同、**首字必须相同**、且只对 canonical(不对别名 —— 否则 `林老板` 会让 我老板/跟老板/那老板 全指向邹洵,18 条垃圾)
- `consistency`:文档内一致性传播,目标已确立后其他写法放宽到 0.80,但 `<0.90 时首字必须相同`
- in_play 实体化(补 Cand)+ 去文件名/日期污染 + 去重(`迈威` 既是别名又被材料化 → 两个竞争候选)
- 一对多别名**全部**加入(原来只取 `targets[0]`,把 `徐总→邹洵` 丢了)
- 平局时优先保留 alias 出处(`万金→万津` 两条都 1.0,丢了 alias 就降级成猜测)
- `settled` 判定 + `render_for_prompt` A/B 分组:A=已确认唯一映射(必须替换),B=待判断
- 性能:rapidfuzz 预筛一致性通道,最慢 24.5s → 12.9s
**`prompts.py`**:JUDGE_RULES 说明各 rule 含义与证据强度;DRAFT_PROMPT 加**按议题归拢**要求
(结论→支撑事实→待办,单节 ≤7 bullet,禁止平铺)+ 反过度回避条款(能确定就写全名,别写"身份不确定")。
**`refine.py`**:S1c 移到 S1 **之前**,keyword_recall 的实体名回灌 `extra_in_play`
(原来只进背景文本,从不进候选空间 —— 无说话人标签的转写因此几乎没候选)。
**`cli.py`**:新增 `lexicon sync|stats|learn|show`;`doctor` 增加 lexicon 检查。
### 3.3 效果实测
| 指标 | 修复前 | 修复后 |
|---|---|---|
| 07-21 目标面形召回 | 1/13 | **12/13**(仅 `趟鬼` 未召回,Kai 本人也未给答案) |
| flags 总数 / 密度 | 10 | 34(12 篇 fixture 合计 400,10.5/万字) |
| S1 耗时(最慢) | 4.7s | 12.9s |
| 草稿结构 | 平铺 bullet | 5 议题节,结论→事实→待办 |
| 保真 | — | num/latin/term = 1.00,gate_fail 0 |
`老万` 现在带交叉证据:`万亚坤(人名·已确认别名,洛启生物(本次在谈)、实控人)`。
---
## 4. 待实施方案(Kai 2026-07-26 给出的 8 点,**核心工作在这里**)
### 4.1 已量化的三条症状
| Kai 判断 | 实测证据 |
|---|---|
| 去口语化 input 太复杂,flash-lite 兜不住 | S2 单片提示词 4380 字,**正文仅占 37%**;候选块 1566 字(36%)比规则本身大 3 倍;指令:正文 = 1.7:1 |
| 召回事实重复 | context 7 条长行中 **3 对高重叠**,一对 Jaccard=**1.00**;2081 字先砍到 678 字,**砍之前没去重**,预算全买重复 |
| 数字处理 | 07-21 篇中文数字串 15 个:`两千/万十/十两/两两/一十亿`;另一篇 91 个。不只中→阿拉伯,还有**口吃混入数字**和**量级歧义**("两千"=2000 还是 2千万,草稿自己在问) |
### 4.2 【P1】S2 拆成两步(Kai 第 8 点)— 最高优先
现在 S2 一次干四件事:去口语 + 候选裁决 + 消化背景 + 满足闸门。**这是一个已复现 bug 的根因**:
严格重试提示词写「专业术语必须**原样出现**」,把已确认的实体替换又还原了。
证据:单片隔离测试 A 组替换成功(张泰海→张成海、老尚→单永强),但完整 run(`retried=2`)
`cleaned.txt` 里 张泰海/老尚/老善/麦威/金山 全部未替换。
```
S2a 去口语化 仅 CLEAN_RULES + 正文(无候选、无背景)→ 保真闸门只管这一步
S2b 实体/数字归一 A 组由代码确定性替换;B 组一次专注 LLM 裁决(带关系事实,允许弃权)
```
- 闸门只校验 S2a。S2b **有意改写 token**,用闸门校它本身矛盾 → 改成白名单校验:
S2b 输出与输入的差异必须能被映射表解释,否则拒绝(比闸门更严且不冲突)。
- flash-lite 吐 token 快,两次调用不影响总时延(Kai 明确认可)。
### 4.3 【P1】A 组代码化确定性替换
`settled` 的映射不应依赖 LLM 服从指令。边界(**必须实现,否则是净负**):
- 仅当目标实体本次 `in_play` 时才替换,否则降级 B 组
(`金山→宝山基金` 是特定语境确认,全局替换危险:金山也可能是地名/金山办公)
- 排除 `common_word`(`光辉`/`公司` 这类)
- 避免嵌套误替(`白帆`→`白帆生物` 不能产生 `白帆生物生物`,用负向 lookahead)
### 4.4 【P1】背景去重压缩(Kai 第 7 点)
- 去重放在**截断之前**(Jaccard > 0.45 合并同义句)
- 散文式记忆压成紧凑表,例:
`刘冀|张科领弋 基金投资人|活跃|最近 2026-06-26|连接 基金/LP/白帆CDMO/桂林三金`
### 4.5 【P2】关系图谱 + 紧密度衰减(Kai 第 3、4 点)
```sql
relation(a, b, kind, weight, first_seen, last_seen, n)
kind: 任职 | 同事 | 项目参与 | 共同出现 | 出资
```
来源优先级:卡片 `company`/`related_people` → 纪要 `people`/`companies` 共现(带日期)
→ Hindsight 实体 → (可选)LLM 抽 typed 关系。建议先做确定性来源,评估够不够再上 LLM。
`in_play` 从「90 天硬截断 + 计数」改为**指数衰减** `w = Σ exp(-Δdays/λ)`,λ≈120 天,
直接进候选排序,替掉现在粗糙的 `boost=0/1`。
### 4.6 【P2】音通道放宽 + 字通道独立(Kai 第 1 点)
**关键认知修正**:`张泰海→张成海` 中 tai/cheng 音距很远,**不是靠音捞回来的**,
是靠"等长 + 同首字 + 只差一字"的字形锚定。所以要两条独立通道并行取较高者,
而不是把一切压进一个音近分数:
- **音通道放宽**:现有折叠 zh/z、ch/c、sh/s、n/l、f/h、ng、en/eng、in/ing。
需补:送气丢失(b/p、d/t、g/k)、尖团(j/z、q/c、x/s)、r/l、u/ü、ai/ei、ao/ou;
声调默认不计权(口音下不可靠)。注意 char→pinyin 回还本身会引入误差。
- **字通道**:首字锚定 + 尾字锚定 + n-1 字相同,**任意长度**(现在只做 ≥3 字且首字相同)。
- 建议用加权评分替掉 6 个互相独立的 `RULE_MIN` 阈值(那套是打补丁堆出来的,规则一多互相漏),
结构性 gate(称谓字字面相同)保留。
### 4.7 【P2】数字归一(Kai 第 5 点)
- 无歧义中文数字 → 阿拉伯(`八千万`→8000万、`二零二一`→2021)
- 清理口吃残留(`一一百八`、`两五千`、`万十`、`两两`)
- **量级歧义不猜**:写成 `两千[?万/千万]` 并进「待你确认」
- **闸门必须认识中文数字**,否则转换会被判成数字丢失
### 4.8 【P1】纪要目标重新定位(Kai 第 6 点)
要改掉我此前的默认倾向。DRAFT_PROMPT 以**梳理关键内容与线索**为目标,明确写:
转写原文已保留在 `transcript:`,遗漏可事后按 Kai 指示补救;宁可写"此处录音模糊、金额量级不明",
不要为凑完整而编。**闸门继续留在 S2a**(防的是清洗阶段静默删信息,是另一回事),
但成稿阶段不再以逐字对齐为目标。
Kai 原话要点:「哪怕人类回听也有很大概率不能完全还原,所以撰写主要是总结梳理关键内容和线索,
不要追求逐字还原」。
### 4.9 【P0】必须承认的边界(不要再花时间攻)
- `徐总`:三个候选(徐劲前/邹洵/许俊彦)全与桂林三金相关,**结构性歧义**,
正确输出是 `徐总(疑似 [[徐劲前]] 或 [[邹洵]])`,不是硬选。
- `趟鬼`:零信息。
- `"张泰海、张博"` 是同一人说两次还是两个人 —— 音频不可恢复。
- Kai 明确说过:「有些模糊问题,你是永远无法解决的」。目标是
**不含糊的直接落定 + 含糊的带证据出候选集**,不是让管线替他决定。
---
## 5. 建议实施顺序
1. **第一段(当天可回归)**:4.2 S2 拆分 + 4.3 A 组代码化 + 4.4 背景去重 + 4.8 成稿定位
—— 这四条直接解释了 Kai 看到的全部症状,改动最小、收益最确定。
2. **第二段**:4.7 数字归一(含闸门认中文数字)。
3. **第三段**:4.5 关系图谱 + 衰减权重(工程量最大)、4.6 音/字双通道重构。
### 回归验证要求(不可省)
- `asr-lab/fixtures` 12 篇 + 11 篇人工定稿卡回归。
- **过度纠错率必须保持 0**(历史横评实测为 0),否则确定性替换是净负。
- 保真三项(num/latin/term recall)在 S2a 上保持 1.00。
- 07-21 录音的 13 个面形召回不低于 12/13。
- 命令:`dry-run` 看 `cleaned.txt` 与 `draft.md` 的差异,判断问题在 S2 还是 S4。
---
## 6. 未提交 / 遗留
- `asr-pipeline/` 与 `asr-lab/` 所在的 `claude-workspace` **不是 git 仓库**,无版本控制,改动前建议手工备份。
- `state/lexicon.db` 含 Kai 12 条人工确认,**删库会丢**(`lexicon sync` 只能重建卡片+术语表部分,
`origin='learned'` 的丢失后无法自动恢复)。备份它。
- `dry-run/` 下有 3 份历史产物,可删。
- skill 文档 `asr-refine-pipeline.md` 已更新到「独立程序 + qwen3cam + S0–S4」,
但**尚未反映本轮 lexicon 与 S2 拆分方案**,实施后需同步。
- 已知未修 bug:见 4.2(严格重试还原已确认替换)。
# 2026-07-26 Codex 接手更新
当前实现已开始迁移到 v2 分阶段架构:纯去口语化与实体/数字规范化分离,确认映射由程序
确定性执行,Hindsight recall 在进入 prompt 前按实体压缩去重,并增加可复用的 SQLite
关系、互动、观察、模糊音和人工确认队列表。详细设计见 `docs/architecture-v2.md`。
四项核心能力已经接入生产判断链:历史纪要增量关系图、45 天半衰期的近期/频率权重、
由确认纠错学习的定向音节混淆,以及带类型/置信度/原值记录的确定性数字规范化。个人
模糊音只对当前关系语境中的实体生效,不影响全局术语库。
移动交付层已加入:每份生产草稿(不论实体候选是否触发)都会把 `## 待你确认` 转成
逐项 Web 问题。顺序为通话背景确认(可填正确背景请求重跑草稿)、缺失声纹注册、内容
不确定项、草稿整体审核;通过既有 Hermes 签名 webhook 把一次性页面地址发到 Telegram。
页面可展开完整私有粗稿。全部回答后将审计记录留在 state,由独立 oneshot 调用 Sonnet,
通过正式 schema/Summary/来源校验后只把正式稿写到 `minutes/` 并 push。确认的匿名
speaker 会立即写入 frontmatter 并调用 10099 声纹注册,失败则留给每日任务重试。
SQLite 以不同正式会议为单位统计实体;未建卡人物/公司达到 2 场时在 Web 提议建保守
存根,只有 Kai 同意才创建。页面完成后 10 秒退出,打开后空闲 45 分钟退出,最长运行
2 小时;服务不随开机常驻。`dry-run` 已禁止写入人工确认队列。
+47 -42
View File
@@ -1,51 +1,56 @@
# ASR Pipeline on tools-pve # Meeting ASR
独立的录音转写、精修、人工审核和导出服务。WebUI 与 Agent API 共用 `192.168.50.49:8788`;worker 调用 unRaid 上的 Qwen3-ASR/CAM++ 服务以及 ZenMux 模型 API。 独立的会议/通话录音转写与整理 Web 应用(`asrservice` + `asrpipe`)。支持本地 GPU 声纹服务(`qwen3cam`)与云端多模型语音识别(`soniox`、`gemini`、`openai`、`grok`),可通过 Arcane 部署到本地 VM(`tools-pve`)或外网 VPS。
## 数据边界 ## 核心架构
- `/mnt/truenas/personal/ASR-Pipeline` → 容器 `/data`:原始录音、转写、ASR 原始响应、草稿、正式稿、日志和 SQLite 在线备份。 1. **Web 控制与审核面(`asrservice.web`)**
- `/opt/asr-pipeline/state` → 容器 `/state`:活动 SQLite、worker heartbeat、revision queue。SQLite WAL 不放在 NFS。 - 极简零构建服务端渲染 HTML + 原生 JS(深浅色自适应)。
- `/srv/asr/inbox`:指向 TrueNAS inbox 的宿主机链接,供 `scp`/`sftp` 上传。 - 上传录音时可按任务切换 ASR 引擎(`qwen3cam` / `soniox` / `gemini` / `openai` / `grok`)。
- Secret 只存在于 Arcane Project 环境和 tools-pve 的 `0600` 恢复副本,不进入 Git。 - 任务详情页内置支持 HTTP Range 的音频播放器、失败一键切换模型重试、草稿与纪要 Markdown 在线编辑。
- `/review/` 提供低置信度词语、关系确认与说话人身份确认(本地 `qwen3cam` 自动注册声纹至 `:10099`,线上多说话人模型直接更新草稿与词库而不依赖 `:10099`)。
- `/lexicon` 提供热词、同音纠错、人物关系管理与 Obsidian 词库一键同步。
- `/settings` 展示引擎是否就绪。网关地址和 API 密钥只显示「已配置 / 未配置」,页面不回显内网地址或密钥片段。
- 浏览器界面默认关闭。必须设置 `ASR_WEB_PASSWORD` 后才能上传、审核和改配置。未设置口令时页面只提示需要配置该变量,不会整站可写。`ASR_WEB_ALLOW_ANON=1` 仅用于跑在本机且不可被其他主机访问的调试进程。状态变更还要带 CSRF 令牌。`/api/v1/*` 继续使用 `ASR_AGENT_TOKEN`。
SSH 大文件建议先上传为 `.part`,完成后在同一目录原子改名为受支持扩展名: 2. **后台消费与修订队列(`asrservice.worker`)**
- 监控 `ASR_DATA_DIR/inbox`(容器内 `/data/inbox`)并认领 `intake.db` 任务。旧的 `meeting-watch.py` 仍监听 `recordings/inbox` 并桥接到 vault `.asr-inbox`。两者共用 `ASR_STATE_DIR/consumer.lock`,第二个进程直接退出,避免同一盘录音被消费两次。
- 调用统一多引擎层 `asrpipe.providers.transcribe_audio()` 生成带结构化注释头的原始转写。
- 执行 `asr-refine draft`:结合 SQLite 词库、Obsidian Vault(`People/`、`Companies/`、`Projects/`、`minutes/`、`_glossary.md`)与 Hindsight(`personal-context`)进行分段去口语化、保真检查与会议纪要生成。
- 监控 `state/revision-queue/` 处理 Web 审核提交的修订重跑请求,确认通过后发布正式纪要至 Obsidian 并可选执行 Git 自动提交推送。
3. **语音识别(ASR)与大模型网关边界**
- **LLM 后处理**:彻底移除 ZenMux,统一通过 **LiteLLM**(内网 `http://192.168.50.49:4000/v1` / 外网 `https://api.opfo.xyz/v1`)调用:
- `ASR_CLEAN_MODEL` / `ASR_KEYWORD_MODEL`:默认 `aistudio/gemini-flash-lite-latest`
- `ASR_DRAFT_MODEL`:默认 `codex/gpt-sol-latest`
- **ASR 引擎**:
- `qwen3cam`(内网默认):对接局域网 GPU `Qwen3-ASR 1.7B + CAM++`(`http://192.168.50.100:10099/asr`),支持声纹特征比对与增量注册。
- `soniox`(线上高精度首选):因 LiteLLM 同步 `/v1/audio/transcriptions` 不支持 Soniox 异步 Files API、结构化 `context` 热词注入与逐词 `speaker` 分离且有 180s 超时限制,由 `asrpipe.providers` 通过 `SONIOX_API_KEY` 直连 Soniox `stt-async-v5`(自动注入 SQLite 词库热词与任务背景,完成后自动清理云端临时音频)。
- `gemini`:通过 LiteLLM 统一网关调用 `aistudio/gemini-3.5-transcribe`(长音频自动 `ffmpeg` 切片)。
- `openai`:通过 LiteLLM 统一网关调用 `openai/gpt-4o-transcribe-diarize`(支持 `diarized_json` 角色分离与 >23MB 自动压缩切片)。
- `grok`:通过 `XAI_API_KEY` 调用 xAI `grok-voice-transcribe-2.0`(支持多说话人分离)。
## 目录结构
- `asrpipe/`:核心转写引擎适配(`providers.py`)、词库召回(`lexicon.py`)、分段去口语化(`segment.py`、`refine.py`)、审核流(`review.py`)与 Obsidian/Hindsight 集成。
- `asrservice/`:容器化 Web 服务(`web.py`)、后台 Worker(`worker.py`)、SQLite 任务状态机(`db.py`)、Webhook 通知(`notify.py`)与健康检查(`healthcheck.py`)。
- `recordings/bin/`:录音监听与 CLI 转写脚本(兼容 `MEETING_ASR_BACKEND` 多引擎调度)。
- `termbank/`:专业词库源数据。
- `tests/`:核心管线、多引擎契约与 Web 服务单元测试。
- `Dockerfile` & `compose.yaml`:面向 Arcane 部署的单镜像双角色(`asr-web` + `asr-worker`)容器编排。
## 本地开发与测试
```bash ```bash
scp call.m4a tools@192.168.50.49:/srv/asr/inbox/call.m4a.part python3 -m venv .venv
ssh tools@192.168.50.49 'mv /srv/asr/inbox/call.m4a.part /srv/asr/inbox/call.m4a' .venv/bin/pip install -e .
ASR_NOTIFY=0 ASR_GIT_SYNC=0 .venv/bin/python -m unittest discover -s tests -v
``` ```
## Agent API ## Arcane 部署(内网 VM / 外网 VPS)
所有 `/api/v1/*` 路由使用 `Authorization: Bearer $ASR_AGENT_TOKEN`。 1. 复制 `.env.example` 为 `.env`,按环境配置:
- **内网 VM(`tools-pve`)**:`MEETING_ASR_BACKEND=qwen3cam`(同时可配 `SONIOX_API_KEY` 等以便在 Web UI 按需切换)、`LITELLM_BASE_URL=http://192.168.50.49:4000/v1`、`HINDSIGHT_URL=http://192.168.50.47:8888`。
```bash - **外网 VPS**:`MEETING_ASR_BACKEND=soniox`(或 `gemini` / `openai` / `grok`)、`LITELLM_BASE_URL=https://api.opfo.xyz/v1`,并且必须设置 `ASR_WEB_PASSWORD`。代码不会按部署位置自动加锁。HTTPS 站点再设 `ASR_WEB_COOKIE_SECURE=1`。
curl -X POST http://192.168.50.49:8788/api/v1/jobs \ 2. `ASR_GIT_SYNC` 与 `ASR_NOTIFY` 缺省为关。要推送 vault 或发通知时显式设为 `1`,并配置 `ASR_NOTIFY_BACKEND`。
-H "Authorization: Bearer $ASR_AGENT_TOKEN" \ 3. 使用 `compose.yaml` 启动 `asr-web`(默认 `:8788`)与 `asr-worker`。`deploy/systemd/` 是现网单元快照,路径仍指向旧工作目录,不要原样安装到本仓库。
-H 'X-Filename: call.m4a' -H 'X-Title: 项目通话' \
-H 'X-Context: 参与人和项目背景' --data-binary @call.m4a
curl -H "Authorization: Bearer $ASR_AGENT_TOKEN" \
http://192.168.50.49:8788/api/v1/jobs/JOB_ID
```
路由:
- `POST /api/v1/jobs`:上传并创建异步任务。
- `GET /api/v1/jobs?limit=&offset=&status=`:分页查询。
- `GET /api/v1/jobs/{id}`:任务详情、事件时间线与产物链接。
- `GET /api/v1/jobs/{id}/artifacts/{transcript|draft|result}`:读取产物。
- `GET /api/v1/jobs/{id}/reviews`:待人工确认项。
- `POST /api/v1/jobs/{id}/reviews/{question_id}`:JSON `{"answer":"..."}`。
- `POST /api/v1/jobs/{id}/retry`:重试失败任务。
`/healthz` 仅表示 Web 进程存活;`/readyz` 同时检查数据库与 worker heartbeat。
## 通知
`ASR_NOTIFY_BACKEND=none|telegram|hermes`。Telegram 需要 `TELEGRAM_BOT_TOKEN` 与 `TELEGRAM_CHAT_ID`;Hermes 需要一个明确存在的签名 webhook URL 与共享密钥。发送失败只影响提醒,不改变任务结果。
## 运维
项目由 Arcane Git Sync 管理,自动同步、自动 pull 和自动 redeploy 均关闭。Compose 变更先在 Git 中验证,再对 tools-pve 的准确 Sync 手动同步。活动数据库每小时通过 SQLite backup API 写到 `/data/backups/*.latest`。
@@ -209,7 +209,9 @@ def cmd_doctor(a):
print("credentials") print("credentials")
try: try:
k = CFG.api_key() k = CFG.api_key()
chk("api key", len(k) > 20, f"len={len(k)} source={'env' if os.environ.get('ANTHROPIC_AUTH_TOKEN') else CFG.env_file.name}") env_set = any(os.environ.get(x) for x in (
"LITELLM_API_KEY", "ASR_LLM_API_KEY", "OPENAI_API_KEY", "ANTHROPIC_AUTH_TOKEN"))
chk("api key", len(k) > 10, f"len={len(k)} source={'env' if env_set else CFG.env_file.name}")
except Exception as e: except Exception as e:
chk("api key", False, str(e)) chk("api key", False, str(e))
+380 -68
View File
@@ -1,25 +1,30 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
"""Configuration for the standalone ASR post-processing pipeline. """Configuration for the standalone ASR post-processing pipeline and web service.
This program deliberately has **no dependency on Claude Code, opencode, or any Model endpoints, provider selections, and API keys are stored in SQLite
agent CLI**. It talks to the model provider over plain HTTPS and does its own file (`app_setting` table in `ASR_STATE_DIR/intake.db`) so they can be managed
and git handling, so it can run from systemd, cron, or a shell with nothing but dynamically from the Web UI without restarting containers. Environment variables
python3 and a network route. serve as initial bootstrap defaults when a setting is not yet in SQLite.
Everything is overridable by environment variable so the systemd unit stays the
single place where deployment-specific values live. Credentials are read from the
same 600-perm env file the legacy unit uses, so token rotation remains a one-file
change.
""" """
from __future__ import annotations from __future__ import annotations
import os import os
import sqlite3
import threading
import time
from dataclasses import dataclass, field from dataclasses import dataclass, field
from pathlib import Path from pathlib import Path
from typing import Any, Callable
PKG = Path(__file__).resolve().parent PKG = Path(__file__).resolve().parent
ROOT = PKG.parent # OBSIDIAN/asr-pipeline ROOT = PKG.parent
OBSIDIAN = ROOT.parent # OBSIDIAN project dir OBSIDIAN = Path(os.environ.get("ASR_WORKSPACE", str(ROOT)))
_CACHE_LOCK = threading.Lock()
_SETTINGS_CACHE: dict[str, str] = {}
_SETTINGS_TS: float = 0.0
_SETTINGS_DB_PATH: Path | None = None
_CACHE_TTL = 1.0
def _env(name: str, default: str) -> str: def _env(name: str, default: str) -> str:
@@ -41,65 +46,368 @@ def _envi(name: str, default: int) -> int:
return default return default
def _state_db_path() -> Path:
state_dir = Path(_env("ASR_STATE_DIR", str(ROOT / "state")))
return state_dir / "intake.db"
def _ensure_settings_table(con: sqlite3.Connection) -> None:
con.execute("""
CREATE TABLE IF NOT EXISTS app_setting(
key TEXT PRIMARY KEY,
value TEXT NOT NULL DEFAULT '',
updated_at REAL NOT NULL
)
""")
def load_db_settings(force: bool = False) -> dict[str, str]:
"""Read `app_setting` rows from `intake.db` with a 1s in-memory cache."""
global _SETTINGS_CACHE, _SETTINGS_TS, _SETTINGS_DB_PATH
db_path = _state_db_path()
now = time.time()
with _CACHE_LOCK:
if (
not force
and _SETTINGS_DB_PATH == db_path
and (now - _SETTINGS_TS) < _CACHE_TTL
):
return dict(_SETTINGS_CACHE)
if not db_path.exists():
_SETTINGS_CACHE = {}
_SETTINGS_TS = now
_SETTINGS_DB_PATH = db_path
return {}
try:
con = sqlite3.connect(db_path, timeout=5)
con.row_factory = sqlite3.Row
_ensure_settings_table(con)
rows = con.execute("SELECT key, value FROM app_setting").fetchall()
con.close()
_SETTINGS_CACHE = {str(r["key"]): str(r["value"]) for r in rows}
_SETTINGS_TS = now
_SETTINGS_DB_PATH = db_path
return dict(_SETTINGS_CACHE)
except Exception:
return dict(_SETTINGS_CACHE)
def get_setting(key: str, default: str = "") -> str:
data = load_db_settings()
val = data.get(key)
return val if val not in (None, "") else default
def set_settings(updates: dict[str, Any]) -> None:
"""Persist multiple settings into `intake.db` and invalidate cache."""
global _SETTINGS_TS
db_path = _state_db_path()
db_path.parent.mkdir(parents=True, exist_ok=True)
now = time.time()
con = sqlite3.connect(db_path, timeout=15)
try:
con.execute("PRAGMA journal_mode=WAL")
_ensure_settings_table(con)
for k, v in updates.items():
if v is None:
continue
con.execute(
"INSERT INTO app_setting(key, value, updated_at) VALUES(?,?,?) "
"ON CONFLICT(key) DO UPDATE SET value=excluded.value, updated_at=excluded.updated_at",
(str(k), str(v), now),
)
con.commit()
finally:
con.close()
with _CACHE_LOCK:
_SETTINGS_TS = 0.0
if "CFG" in globals():
for k in updates:
CFG.__dict__.pop(f"_override_{k}", None)
load_db_settings(force=True)
def bootstrap_default_settings() -> None:
"""Seed `app_setting` from environment variables for keys not yet stored."""
db_path = _state_db_path()
db_path.parent.mkdir(parents=True, exist_ok=True)
now = time.time()
defaults = {
"asr_provider": _env("ASR_PROVIDER", _env("MEETING_ASR_BACKEND", "qwen3cam")).lower(),
"diarize_url": _env("ASR_DIARIZE_URL", _env("MEETING_DIARIZE_URL", "http://192.168.50.100:10099/asr")),
"voiceprint_enroll_url": _env("ASR_VOICEPRINT_ENROLL_URL", "http://192.168.50.100:10099/voiceprints/enroll"),
"litellm_base_url": _env("LITELLM_BASE_URL", _env("ASR_LLM_BASE_URL", "http://192.168.50.49:4000/v1")).rstrip("/"),
"litellm_api_key": _env("LITELLM_API_KEY", _env("ASR_LLM_API_KEY", _env("OPENAI_API_KEY", ""))),
"clean_model": _env("ASR_CLEAN_MODEL", "aistudio/gemini-flash-lite-latest"),
"keyword_model": _env("ASR_KEYWORD_MODEL", "aistudio/gemini-flash-lite-latest"),
"draft_model": _env("ASR_DRAFT_MODEL", "codex/gpt-sol-latest"),
"gemini_asr_model": _env("ASR_GEMINI_MODEL", "aistudio/gemini-3.5-transcribe"),
"openai_asr_model": _env("ASR_OPENAI_MODEL", "openai/gpt-4o-transcribe-diarize"),
"soniox_api_base": _env("SONIOX_API_BASE", "https://api.soniox.com/v1").rstrip("/"),
"soniox_api_key": _env("SONIOX_API_KEY", ""),
"soniox_model": _env("SONIOX_MODEL", "stt-async-v5"),
"grok_api_base": _env("XAI_API_BASE", "https://api.x.ai/v1").rstrip("/"),
"grok_api_key": _env("XAI_API_KEY", _env("GROK_API_KEY", "")),
"grok_asr_model": _env("ASR_GROK_MODEL", "grok-voice-transcribe-2.0"),
"hindsight_api": _env("HINDSIGHT_API", "http://192.168.50.47:8888").rstrip("/"),
"hindsight_bank": _env("HINDSIGHT_BANK", "personal-context"),
"use_hindsight": "1" if _env("ASR_HINDSIGHT", "1") != "0" else "0",
"use_termbank": "1" if _env("ASR_TERMBANK", "1") != "0" else "0",
"git_sync": "0" if _env("ASR_GIT_SYNC", "0") == "0" else "1",
}
con = sqlite3.connect(db_path, timeout=15)
try:
con.execute("PRAGMA journal_mode=WAL")
_ensure_settings_table(con)
for k, v in defaults.items():
con.execute(
"INSERT OR IGNORE INTO app_setting(key, value, updated_at) VALUES(?,?,?)",
(k, str(v), now),
)
# If key was seeded empty earlier but env now provides a non-empty secret, populate it
if v and k.endswith("_api_key"):
con.execute(
"UPDATE app_setting SET value=?, updated_at=? WHERE key=? AND value=''",
(str(v), now, k),
)
con.commit()
finally:
con.close()
load_db_settings(force=True)
class _SettingAttr:
"""Descriptor that resolves from instance override -> SQLite `app_setting` -> env fallback."""
def __init__(
self,
key: str,
fallback: Callable[[], str],
transform: Callable[[str], Any] | None = None,
) -> None:
self.key = key
self.fallback = fallback
self.transform = transform
self.override_attr = f"_override_{key}"
def __get__(self, instance: Any, owner: Any = None) -> Any:
if instance is None:
return self
if self.override_attr in instance.__dict__:
return instance.__dict__[self.override_attr]
raw = get_setting(self.key, self.fallback())
return self.transform(raw) if self.transform else raw
def __set__(self, instance: Any, value: Any) -> None:
instance.__dict__[self.override_attr] = value
def __delete__(self, instance: Any) -> None:
instance.__dict__.pop(self.override_attr, None)
def _default_vault_path() -> Path:
state_vault = _state_db_path().parent / "vault"
if (state_vault / ".git").exists():
return state_vault
env_vault = _env("ASR_VAULT", "")
if env_vault:
p = Path(env_vault)
if (p / ".git").exists() or not state_vault.exists():
return p
return Path.home() / "obsidian-vault"
@dataclass @dataclass
class Config: class Config:
# ---- paths ---------------------------------------------------------- # ---- paths ----------------------------------------------------------
vault: Path = field(default_factory=lambda: Path( vault = _SettingAttr(
_env("ASR_VAULT", str(Path.home() / "obsidian-vault")))) "vault_dir",
obsidian: Path = OBSIDIAN lambda: str(_default_vault_path()),
lambda s: Path(s),
)
obsidian: Path = field(default_factory=lambda: Path(
_env("ASR_WORKSPACE", str(ROOT))))
root: Path = ROOT root: Path = ROOT
# ---- models --------------------------------------------------------- # ---- SQLite-backed dynamic model & provider settings ----------------
# Cleaning: measured against claude-haiku-4.5 on a 118k-char English-mixed call, litellm_base_url = _SettingAttr(
# gemini-3.5-flash-lite tripped the fidelity gate 19/87 times vs haiku's 78/87 "litellm_base_url",
# (haiku's failures were 69x "over-compression" — it behaves like a summarizer lambda: _env("LITELLM_BASE_URL", _env("ASR_LLM_BASE_URL", "http://192.168.50.49:4000/v1")),
# and deleted terms such as AUTOIMMUNE/ADCS/CD), while being 2x faster and lambda s: s.rstrip("/"),
# 2.9x cheaper. Hence flash-lite is the default cleaner. )
clean_model: str = _env("ASR_CLEAN_MODEL", "google/gemini-3.5-flash-lite") clean_model = _SettingAttr(
# Drafting: one call, needs judgement and structure. sonnet-5.5 replaces the "clean_model",
# legacy headless opus-4-8. lambda: _env("ASR_CLEAN_MODEL", "aistudio/gemini-flash-lite-latest"),
draft_model: str = _env("ASR_DRAFT_MODEL", "anthropic/claude-sonnet-5.5") )
keyword_model: str = _env("ASR_KEYWORD_MODEL", "google/gemini-3.5-flash-lite") draft_model = _SettingAttr(
"draft_model",
lambda: _env("ASR_DRAFT_MODEL", "codex/gpt-sol-latest"),
)
keyword_model = _SettingAttr(
"keyword_model",
lambda: _env("ASR_KEYWORD_MODEL", "aistudio/gemini-flash-lite-latest"),
)
endpoint: str = _env("ASR_ENDPOINT", "https://zenmux.ai/api/v1/chat/completions") asr_provider = _SettingAttr(
# ZenMux returned HTTP 429 at ~12 concurrent requests and was stable at 6. "asr_provider",
concurrency: int = _envi("CONCURRENCY", 6) lambda: _env("ASR_PROVIDER", _env("MEETING_ASR_BACKEND", "qwen3cam")),
http_timeout: float = _envf("HTTP_TIMEOUT", 420.0) lambda s: s.strip().lower() or "qwen3cam",
max_retries: int = _envi("MAX_RETRIES", 5) )
diarize_url = _SettingAttr(
"diarize_url",
lambda: _env("ASR_DIARIZE_URL", _env("MEETING_DIARIZE_URL", "http://192.168.50.100:10099/asr")),
)
voiceprint_enroll_url = _SettingAttr(
"voiceprint_enroll_url",
lambda: _env("ASR_VOICEPRINT_ENROLL_URL", "http://192.168.50.100:10099/voiceprints/enroll"),
)
soniox_api_key = _SettingAttr(
"soniox_api_key",
lambda: _env("SONIOX_API_KEY", ""),
)
soniox_api_base = _SettingAttr(
"soniox_api_base",
lambda: _env("SONIOX_API_BASE", "https://api.soniox.com/v1"),
lambda s: s.rstrip("/"),
)
soniox_model = _SettingAttr(
"soniox_model",
lambda: _env("SONIOX_MODEL", "stt-async-v5"),
)
gemini_asr_model = _SettingAttr(
"gemini_asr_model",
lambda: _env("ASR_GEMINI_MODEL", "aistudio/gemini-3.5-transcribe"),
)
openai_asr_model = _SettingAttr(
"openai_asr_model",
lambda: _env("ASR_OPENAI_MODEL", "openai/gpt-4o-transcribe-diarize"),
)
grok_api_key = _SettingAttr(
"grok_api_key",
lambda: _env("XAI_API_KEY", _env("GROK_API_KEY", "")),
)
grok_api_base = _SettingAttr(
"grok_api_base",
lambda: _env("XAI_API_BASE", "https://api.x.ai/v1"),
lambda s: s.rstrip("/"),
)
grok_asr_model = _SettingAttr(
"grok_asr_model",
lambda: _env("ASR_GROK_MODEL", "grok-voice-transcribe-2.0"),
)
hindsight_api = _SettingAttr(
"hindsight_api",
lambda: _env("HINDSIGHT_API", "http://192.168.50.47:8888"),
lambda s: s.rstrip("/"),
)
hindsight_bank = _SettingAttr(
"hindsight_bank",
lambda: _env("HINDSIGHT_BANK", "personal-context"),
)
use_hindsight = _SettingAttr(
"use_hindsight",
lambda: _env("ASR_HINDSIGHT", "1"),
lambda s: str(s).lower() not in ("0", "false", "no", "off"),
)
use_termbank = _SettingAttr(
"use_termbank",
lambda: _env("ASR_TERMBANK", "1"),
lambda s: str(s).lower() not in ("0", "false", "no", "off"),
)
git_sync = _SettingAttr(
"git_sync",
lambda: _env("ASR_GIT_SYNC", "0"),
lambda s: str(s).lower() not in ("0", "false", "no", "off"),
)
@property
def endpoint(self) -> str:
if "_override_endpoint" in self.__dict__:
return self.__dict__["_override_endpoint"]
explicit = os.environ.get("ASR_ENDPOINT")
if explicit:
return explicit
return f"{self.litellm_base_url}/chat/completions"
@endpoint.setter
def endpoint(self, value: str) -> None:
self.__dict__["_override_endpoint"] = value
@property
def transcriptions_endpoint(self) -> str:
if "_override_transcriptions_endpoint" in self.__dict__:
return self.__dict__["_override_transcriptions_endpoint"]
explicit = os.environ.get("ASR_TRANSCRIPTIONS_ENDPOINT")
if explicit:
return explicit
return f"{self.litellm_base_url}/audio/transcriptions"
@transcriptions_endpoint.setter
def transcriptions_endpoint(self, value: str) -> None:
self.__dict__["_override_transcriptions_endpoint"] = value
concurrency: int = field(default_factory=lambda: _envi("CONCURRENCY", 6))
http_timeout: float = field(default_factory=lambda: _envf("HTTP_TIMEOUT", 420.0))
max_retries: int = field(default_factory=lambda: _envi("MAX_RETRIES", 5))
segment_seconds: int = field(default_factory=lambda: _envi(
"SEGMENT_SECONDS", int(_env("MEETING_ASR_SEGMENT_SECONDS", "540"))))
# ---- chunking ------------------------------------------------------- # ---- chunking -------------------------------------------------------
chunk_budget: int = _envi("CHUNK_BUDGET", 1800) chunk_budget: int = field(default_factory=lambda: _envi("CHUNK_BUDGET", 1800))
chunk_overlap: int = _envi("CHUNK_OVERLAP", 200) chunk_overlap: int = field(default_factory=lambda: _envi("CHUNK_OVERLAP", 200))
collapse_noise: bool = _env("ASR_COLLAPSE_NOISE", "1") != "0" collapse_noise: bool = field(
default_factory=lambda: _env("ASR_COLLAPSE_NOISE", "1") != "0")
# ---- fidelity gate -------------------------------------------------- # ---- fidelity gate --------------------------------------------------
gate_enabled: bool = _env("ASR_GATE", "1") != "0" gate_enabled: bool = field(
gate_min_ratio: float = _envf("GATE_MIN_RATIO", 0.55) default_factory=lambda: _env("ASR_GATE", "1") != "0")
gate_max_ratio: float = _envf("GATE_MAX_RATIO", 1.35) gate_min_ratio: float = field(
default_factory=lambda: _envf("GATE_MIN_RATIO", 0.55))
gate_max_ratio: float = field(
default_factory=lambda: _envf("GATE_MAX_RATIO", 1.35))
# ---- knowledge sources ---------------------------------------------- # ---- knowledge sources ----------------------------------------------
use_termbank: bool = _env("ASR_TERMBANK", "1") != "0" use_keyword_recall: bool = field(
use_hindsight: bool = _env("ASR_HINDSIGHT", "1") != "0" default_factory=lambda: _env("ASR_KEYWORD_RECALL", "1") != "0")
use_keyword_recall: bool = _env("ASR_KEYWORD_RECALL", "1") != "0" use_lexicon: bool = field(
use_lexicon: bool = _env("ASR_LEXICON", "1") != "0" default_factory=lambda: _env("ASR_LEXICON", "1") != "0")
use_normalize: bool = _env("ASR_NORMALIZE", "1") != "0" use_normalize: bool = field(
use_evidence: bool = _env("ASR_EVIDENCE", "1") != "0" default_factory=lambda: _env("ASR_NORMALIZE", "1") != "0")
hindsight_api: str = _env("HINDSIGHT_API", "http://192.168.50.47:8888") use_evidence: bool = field(
hindsight_bank: str = _env("HINDSIGHT_BANK", "personal-context") default_factory=lambda: _env("ASR_EVIDENCE", "1") != "0")
hindsight_timeout: float = _envf("HINDSIGHT_TIMEOUT", 12.0) hindsight_timeout: float = field(
flag_min_score: float = _envf("FLAG_MIN_SCORE", 0.86) default_factory=lambda: _envf("HINDSIGHT_TIMEOUT", 12.0))
review_min_score: float = _envf("REVIEW_MIN_SCORE", 0.82) flag_min_score: float = field(
review_max_score: float = _envf("REVIEW_MAX_SCORE", 0.96) default_factory=lambda: _envf("FLAG_MIN_SCORE", 0.86))
review_host: str = _env("ASR_REVIEW_HOST", "0.0.0.0") review_min_score: float = field(
review_public_host: str = _env("ASR_REVIEW_PUBLIC_HOST", "192.168.50.145") default_factory=lambda: _envf("REVIEW_MIN_SCORE", 0.82))
review_port: int = _envi("REVIEW_PORT", 8765) review_max_score: float = field(
review_idle_seconds: int = _envi("REVIEW_IDLE_SECONDS", 2700) default_factory=lambda: _envf("REVIEW_MAX_SCORE", 0.96))
review_max_seconds: int = _envi("REVIEW_MAX_SECONDS", 7200) review_host: str = field(
default_factory=lambda: _env("ASR_REVIEW_HOST", "0.0.0.0"))
review_public_host: str = field(
default_factory=lambda: _env("ASR_REVIEW_PUBLIC_HOST", "192.168.50.145"))
review_port: int = field(default_factory=lambda: _envi("REVIEW_PORT", 8765))
review_idle_seconds: int = field(
default_factory=lambda: _envi("REVIEW_IDLE_SECONDS", 2700))
review_max_seconds: int = field(
default_factory=lambda: _envi("REVIEW_MAX_SECONDS", 7200))
# ---- behaviour ------------------------------------------------------ # ---- behaviour ------------------------------------------------------
git_sync: bool = _env("ASR_GIT_SYNC", "1") != "0" notify: bool = field(
notify: bool = _env("ASR_NOTIFY", "1") != "0" default_factory=lambda: _env("ASR_NOTIFY", "0") != "0")
cache_llm: bool = _env("ASR_CACHE", "0") != "0" # off in production cache_llm: bool = field(
default_factory=lambda: _env("ASR_CACHE", "0") != "0")
def __post_init__(self) -> None:
# Clear dataclass-populated default descriptors so __get__ is invoked dynamically
for k in list(self.__dict__.keys()):
if isinstance(getattr(type(self), k, None), _SettingAttr):
del self.__dict__[k]
@property @property
def inbox(self) -> Path: def inbox(self) -> Path:
@@ -120,7 +428,6 @@ class Config:
@property @property
def pending_index(self) -> Path: def pending_index(self) -> Path:
# Legacy compatibility only; the Web queue replaced the Vault draft index.
return self.vault / "minutes" / "_drafts" / "_待确认.md" return self.vault / "minutes" / "_drafts" / "_待确认.md"
@property @property
@@ -150,15 +457,19 @@ class Config:
@property @property
def notifier(self) -> Path: def notifier(self) -> Path:
return self.obsidian / "bin" / "meeting-status-notify.py" return Path(_env("ASR_NOTIFIER", str(ROOT / "scripts" / "meeting-status-notify.py")))
@property @property
def env_file(self) -> Path: def env_file(self) -> Path:
"""600-perm credentials file, shared with the legacy systemd unit.""" """600-perm credentials file."""
return Path(_env("ASR_ENV_FILE", str(self.obsidian / "bin" / ".asr-refine.env"))) return Path(_env("ASR_ENV_FILE", str(ROOT / ".env")))
def api_key(self) -> str: def api_key(self) -> str:
for k in ("ZENMUX_API_KEY", "ANTHROPIC_AUTH_TOKEN"): db_key = get_setting("litellm_api_key", "")
if db_key:
return db_key
for k in ("LITELLM_API_KEY", "ASR_LLM_API_KEY", "OPENAI_API_KEY",
"ANTHROPIC_AUTH_TOKEN"):
v = os.environ.get(k) v = os.environ.get(k)
if v: if v:
return v return v
@@ -166,13 +477,14 @@ class Config:
if p.exists(): if p.exists():
for line in p.read_text(errors="ignore").splitlines(): for line in p.read_text(errors="ignore").splitlines():
line = line.strip() line = line.strip()
if line.startswith("ANTHROPIC_AUTH_TOKEN="): for prefix in ("LITELLM_API_KEY=", "ASR_LLM_API_KEY=",
val = line.split("=", 1)[1].strip().strip("'\"") "OPENAI_API_KEY=", "ANTHROPIC_AUTH_TOKEN="):
if val: if line.startswith(prefix):
return val val = line.split("=", 1)[1].strip().strip("'\"")
if val:
return val
raise RuntimeError( raise RuntimeError(
"no API key: set ZENMUX_API_KEY/ANTHROPIC_AUTH_TOKEN or provide " "no API key: configure LiteLLM API Key in /settings or set LITELLM_API_KEY")
f"{p} with ANTHROPIC_AUTH_TOKEN=…")
CFG = Config() CFG = Config()
@@ -2,6 +2,8 @@
"""Persistent mobile control plane for recording intake and human review.""" """Persistent mobile control plane for recording intake and human review."""
from __future__ import annotations from __future__ import annotations
import hashlib
import hmac
import html import html
import json import json
import os import os
@@ -14,7 +16,7 @@ from http import HTTPStatus
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path from pathlib import Path
from . import review from . import review, webauth
from .config import CFG from .config import CFG
from .log import log from .log import log
@@ -166,6 +168,7 @@ def _shell(title: str, body: str, script: str = "") -> bytes:
return f"""<!doctype html> return f"""<!doctype html>
<html lang="zh-CN"><head> <html lang="zh-CN"><head>
<meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"> <meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1">
<meta name="csrf" content="{html.escape(webauth.csrf_token(), quote=True)}">
<meta name="color-scheme" content="light"><meta http-equiv="Cache-Control" content="no-store"> <meta name="color-scheme" content="light"><meta http-equiv="Cache-Control" content="no-store">
<title>{html.escape(title)}</title> <title>{html.escape(title)}</title>
<style> <style>
@@ -236,6 +239,7 @@ document.querySelector('#upload').addEventListener('submit',e=>{
const x=new XMLHttpRequest(), b=document.querySelector('#send'), p=document.querySelector('#progress'), out=document.querySelector('#result'); const x=new XMLHttpRequest(), b=document.querySelector('#send'), p=document.querySelector('#progress'), out=document.querySelector('#result');
b.disabled=true;p.hidden=false;out.textContent='正在上传…'; b.disabled=true;p.hidden=false;out.textContent='正在上传…';
x.open('POST','/api/upload');x.setRequestHeader('Content-Type','application/octet-stream'); x.open('POST','/api/upload');x.setRequestHeader('Content-Type','application/octet-stream');
x.setRequestHeader('X-CSRF-Token', document.querySelector('meta[name=csrf]')?.content || '');
x.setRequestHeader('X-Filename',encodeURIComponent(f.name)); x.setRequestHeader('X-Filename',encodeURIComponent(f.name));
x.setRequestHeader('X-Title',encodeURIComponent(document.querySelector('#title').value)); x.setRequestHeader('X-Title',encodeURIComponent(document.querySelector('#title').value));
x.setRequestHeader('X-Context',encodeURIComponent(document.querySelector('#context').value)); x.setRequestHeader('X-Context',encodeURIComponent(document.querySelector('#context').value));
@@ -293,6 +297,7 @@ def _review_page(action: str, job: dict | None = None) -> bytes:
body = f"""<section class="panel"><div class="meta">{html.escape(title)} · 剩余 {len(rows)} 个 · 问题 #{row['id']}</div> body = f"""<section class="panel"><div class="meta">{html.escape(title)} · 剩余 {len(rows)} 个 · 问题 #{row['id']}</div>
<h1>{html.escape(row['question'])}</h1>{source_html}{draft_html} <h1>{html.escape(row['question'])}</h1>{source_html}{draft_html}
<form method="post" action="{html.escape(action, quote=True)}"> <form method="post" action="{html.escape(action, quote=True)}">
<input type="hidden" name="csrf" value="{html.escape(webauth.csrf_token(), quote=True)}">
<input type="hidden" name="id" value="{row['id']}">{buttons} <input type="hidden" name="id" value="{row['id']}">{buttons}
<div class="custom"><input name="custom" maxlength="120" autocomplete="off" <div class="custom"><input name="custom" maxlength="120" autocomplete="off"
placeholder="{html.escape(placeholder, quote=True)}"><button type="submit">提交</button></div> placeholder="{html.escape(placeholder, quote=True)}"><button type="submit">提交</button></div>
@@ -342,12 +347,43 @@ class Handler(BaseHTTPRequestHandler):
return True, _job(match.group(1)) return True, _job(match.group(1))
return False, None return False, None
def _browser_ok(self) -> bool:
if webauth.allow_anon():
return True
if not webauth.password():
return False
return webauth.cookie_matches(self.headers.get("Cookie", ""))
def _login_page(self, err: str = "") -> bytes:
if not webauth.password() and not webauth.allow_anon():
err = err or "未配置 ASR_WEB_PASSWORD,控制面已关闭。"
token = html.escape(webauth.csrf_token(), quote=True)
alert = f"<p>{html.escape(err)}</p>" if err else ""
body = (
f'<section class="panel"><h1>访问验证</h1>{alert}'
f'<form method="post" action="/login">'
f'<input type="hidden" name="csrf" value="{token}">'
f'<label>访问口令</label><input name="password" type="password" required>'
f'<div class="actions"><button type="submit">登录</button></div></form></section>'
)
return _shell("登录", body)
def do_GET(self): def do_GET(self):
path = urllib.parse.urlsplit(self.path).path path = urllib.parse.urlsplit(self.path).path
if path == "/":
return self._send(_home())
if path == "/health": if path == "/health":
return self._json({"ok": True, "pending": len(review.pending(100))}) return self._json({"ok": True, "pending": len(review.pending(100))})
if path in {"/login", "/logout"}:
if path == "/logout":
self.send_response(HTTPStatus.SEE_OTHER)
self.send_header("Location", "/login")
self.send_header("Set-Cookie", "asr_session=; Path=/; HttpOnly; SameSite=Strict; Max-Age=0")
self.end_headers()
return
return self._send(self._login_page())
if not self._browser_ok():
return self._send(self._login_page(), HTTPStatus.UNAUTHORIZED)
if path == "/":
return self._send(_home())
if path == "/api/jobs": if path == "/api/jobs":
jobs = _jobs() jobs = _jobs()
for j in jobs: for j in jobs:
@@ -372,9 +408,9 @@ class Handler(BaseHTTPRequestHandler):
def _upload(self): def _upload(self):
try: try:
length = int(self.headers.get("Content-Length", "0")) length = int(self.headers.get("Content-Length", "0") or 0)
except ValueError: except ValueError:
length = 0 return self._json({"error": "bad content-length"}, HTTPStatus.BAD_REQUEST)
if length <= 0 or length > MAX_UPLOAD: if length <= 0 or length > MAX_UPLOAD:
return self._json({"error": "invalid upload size"}, return self._json({"error": "invalid upload size"},
HTTPStatus.REQUEST_ENTITY_TOO_LARGE) HTTPStatus.REQUEST_ENTITY_TOO_LARGE)
@@ -408,7 +444,7 @@ class Handler(BaseHTTPRequestHandler):
while remaining: while remaining:
data = self.rfile.read(min(CHUNK, remaining)) data = self.rfile.read(min(CHUNK, remaining))
if not data: if not data:
raise EOFError("upload ended early") raise EOFError("incomplete upload")
f.write(data) f.write(data)
remaining -= len(data) remaining -= len(data)
f.flush() f.flush()
@@ -426,16 +462,28 @@ class Handler(BaseHTTPRequestHandler):
update_job(job_id, "queued", "录音已落盘,等待 watcher 接收") update_job(job_id, "queued", "录音已落盘,等待 watcher 接收")
return self._json({"ok": True, "job_id": job_id, return self._json({"ok": True, "job_id": job_id,
"stored_name": stored}, HTTPStatus.CREATED) "stored_name": stored}, HTTPStatus.CREATED)
except EOFError:
part.unlink(missing_ok=True)
update_job(job_id, "failed", "上传未完成")
return self._json({"error": "incomplete upload"}, HTTPStatus.BAD_REQUEST)
except Exception as e: except Exception as e:
part.unlink(missing_ok=True) part.unlink(missing_ok=True)
update_job(job_id, "failed", f"上传失败:{e}") update_job(job_id, "failed", "上传失败")
return self._json({"error": str(e)}, HTTPStatus.INTERNAL_SERVER_ERROR) log(f"upload failed: {e}")
return self._json({"error": "upload failed"}, HTTPStatus.INTERNAL_SERVER_ERROR)
def _answer(self): def _answer(self):
try: try:
length = min(int(self.headers.get("Content-Length", "0")), 4096) try:
length = int(self.headers.get("Content-Length", "0") or 0)
except ValueError as exc:
raise ValueError("bad content-length") from exc
if length < 0 or length > 4096:
raise ValueError("bad content-length")
form = urllib.parse.parse_qs( form = urllib.parse.parse_qs(
self.rfile.read(length).decode("utf-8")) self.rfile.read(length).decode("utf-8"))
if not webauth.csrf_matches((form.get("csrf") or [""])[0]):
raise ValueError("csrf")
qid = int(form.get("id", ["0"])[0]) qid = int(form.get("id", ["0"])[0])
answer = (form.get("custom", [""])[0].strip() answer = (form.get("custom", [""])[0].strip()
or form.get("answer", [""])[0].strip()) or form.get("answer", [""])[0].strip())
@@ -460,7 +508,31 @@ class Handler(BaseHTTPRequestHandler):
def do_POST(self): def do_POST(self):
path = urllib.parse.urlsplit(self.path).path path = urllib.parse.urlsplit(self.path).path
if path == "/login":
try:
length = int(self.headers.get("Content-Length", "0") or 0)
except ValueError:
return self._json({"error": "bad content-length"}, HTTPStatus.BAD_REQUEST)
if length < 0:
return self._json({"error": "bad content-length"}, HTTPStatus.BAD_REQUEST)
length = min(length, 4096)
form = urllib.parse.parse_qs(self.rfile.read(length).decode("utf-8", "ignore"))
pw = form.get("password", [""])[0]
if webauth.password() and hmac.compare_digest(pw, webauth.password()):
self.send_response(HTTPStatus.SEE_OTHER)
self.send_header("Location", "/")
self.send_header(
"Set-Cookie",
f"asr_session={webauth.session_token()}; Path=/; HttpOnly; SameSite=Strict; Max-Age=86400",
)
self.end_headers()
return
return self._send(self._login_page("口令错误或未配置 ASR_WEB_PASSWORD"), HTTPStatus.UNAUTHORIZED)
if not self._browser_ok():
return self._json({"error": "unauthorized"}, HTTPStatus.UNAUTHORIZED)
if path in ("/api/upload", "/hook/intake"): if path in ("/api/upload", "/hook/intake"):
if not webauth.csrf_matches(self.headers.get("X-CSRF-Token", "")):
return self._json({"error": "csrf"}, HTTPStatus.FORBIDDEN)
return self._upload() return self._upload()
if self._review_job()[0]: if self._review_job()[0]:
return self._answer() return self._answer()
@@ -44,14 +44,16 @@ from pathlib import Path
from .config import CFG from .config import CFG
LAB = CFG.root def _vault() -> Path:
BANK = CFG.termbank_dir return CFG.vault
CACHE = CFG.cache_dir / "flagger"
VAULT = CFG.vault
HS_API = CFG.hindsight_api
HS_BANK = CFG.hindsight_bank def _bank() -> Path:
HS_TIMEOUT = CFG.hindsight_timeout return CFG.termbank_dir
def _cache() -> Path:
return CFG.cache_dir / "flagger"
HAN = re.compile(r"[\u4e00-\u9fff]+") HAN = re.compile(r"[\u4e00-\u9fff]+")
HAN_FULL = re.compile(r"^[\u4e00-\u9fff]+$") HAN_FULL = re.compile(r"^[\u4e00-\u9fff]+$")
@@ -264,16 +266,16 @@ def _dir_sig(paths) -> str:
def load_pool(use_termbank=True) -> dict[str, Cand]: def load_pool(use_termbank=True) -> dict[str, Cand]:
"""Vault entities (cards + glossary) ∪ optional domain term bank.""" """Vault entities (cards + glossary) ∪ optional domain term bank."""
CACHE.mkdir(parents=True, exist_ok=True) _cache().mkdir(parents=True, exist_ok=True)
if CFG.use_lexicon: if CFG.use_lexicon:
try: try:
from . import lexicon from . import lexicon
lexicon.sync_from_vault() lexicon.sync_from_vault()
except Exception: except Exception:
pass pass
sig = _dir_sig([VAULT / "People", VAULT / "Companies", VAULT / "minutes", sig = _dir_sig([_vault() / "People", _vault() / "Companies", _vault() / "minutes",
CFG.lexicon_db]) CFG.lexicon_db])
cpath = CACHE / f"pool6-{sig}-{int(use_termbank)}.json" cpath = _cache() / f"pool6-{sig}-{int(use_termbank)}.json"
if cpath.exists(): if cpath.exists():
d = json.loads(cpath.read_text()) d = json.loads(cpath.read_text())
return {k: Cand(**v) for k, v in d.items()} return {k: Cand(**v) for k, v in d.items()}
@@ -331,7 +333,7 @@ def load_pool(use_termbank=True) -> dict[str, Cand]:
ents = {} ents = {}
if not ents: # lexicon disabled/broken if not ents: # lexicon disabled/broken
for sub, kind in (("People", "人名"), ("Companies", "公司")): for sub, kind in (("People", "人名"), ("Companies", "公司")):
d = VAULT / sub d = _vault() / sub
if not d.exists(): if not d.exists():
continue continue
for f in d.glob("*.md"): for f in d.glob("*.md"):
@@ -346,7 +348,7 @@ def load_pool(use_termbank=True) -> dict[str, Cand]:
add(form, c.canonical, c.kind, "variant", rule, c.owner, c.role, anchor) add(form, c.canonical, c.kind, "variant", rule, c.owner, c.role, anchor)
if use_termbank: if use_termbank:
tb = BANK / "terms.json" tb = _bank() / "terms.json"
if tb.exists(): if tb.exists():
for term, e in json.loads(tb.read_text()).items(): for term, e in json.loads(tb.read_text()).items():
# Only the on-domain, distinctive slices are worth phonetic search; # Only the on-domain, distinctive slices are worth phonetic search;
@@ -361,7 +363,7 @@ def load_pool(use_termbank=True) -> dict[str, Cand]:
def load_allowlist() -> set[str]: def load_allowlist() -> set[str]:
p = BANK / "allowlist.txt" p = _bank() / "allowlist.txt"
return set(p.read_text().split()) if p.exists() else set() return set(p.read_text().split()) if p.exists() else set()
@@ -388,13 +390,13 @@ def _minutes_entities(text: str) -> set[str]:
def _corpus() -> list[tuple[str, set[str], str]]: def _corpus() -> list[tuple[str, set[str], str]]:
CACHE.mkdir(parents=True, exist_ok=True) _cache().mkdir(parents=True, exist_ok=True)
sig = _dir_sig([VAULT / "minutes"]) sig = _dir_sig([_vault() / "minutes"])
cpath = CACHE / f"corpus-{sig}.json" cpath = _cache() / f"corpus-{sig}.json"
if cpath.exists(): if cpath.exists():
return [(a, set(b), c) for a, b, c in json.loads(cpath.read_text())] return [(a, set(b), c) for a, b, c in json.loads(cpath.read_text())]
out = [] out = []
for f in sorted((VAULT / "minutes").glob("*.md")): for f in sorted((_vault() / "minutes").glob("*.md")):
txt = f.read_text(errors="ignore") txt = f.read_text(errors="ignore")
m = re.search(r"^date:\s*(\d{4}-\d{2}-\d{2})", txt, re.M) m = re.search(r"^date:\s*(\d{4}-\d{2}-\d{2})", txt, re.M)
out.append((f.name, _minutes_entities(txt), m.group(1) if m else "")) out.append((f.name, _minutes_entities(txt), m.group(1) if m else ""))
@@ -414,11 +416,11 @@ def hindsight_recall(seed: list[str], extra_query: str = "") -> tuple[set[str],
return set(), [] return set(), []
query = (" ".join(sorted(seed)) + " " + extra_query).strip() or "会议 相关人物 公司" query = (" ".join(sorted(seed)) + " " + extra_query).strip() or "会议 相关人物 公司"
body = json.dumps({"query": query, "max_tokens": 1500}).encode() body = json.dumps({"query": query, "max_tokens": 1500}).encode()
url = f"{HS_API}/v1/default/banks/{HS_BANK}/memories/recall" url = f"{CFG.hindsight_api}/v1/default/banks/{CFG.hindsight_bank}/memories/recall"
try: try:
req = urllib.request.Request(url, data=body, req = urllib.request.Request(url, data=body,
headers={"Content-Type": "application/json"}) headers={"Content-Type": "application/json"})
with urllib.request.urlopen(req, timeout=HS_TIMEOUT) as r: with urllib.request.urlopen(req, timeout=CFG.hindsight_timeout) as r:
d = json.loads(r.read()) d = json.loads(r.read())
except Exception: except Exception:
return set(), [] return set(), []
@@ -157,7 +157,8 @@ class Entity:
def connect() -> sqlite3.Connection: def connect() -> sqlite3.Connection:
CFG.state_dir.mkdir(parents=True, exist_ok=True) CFG.state_dir.mkdir(parents=True, exist_ok=True)
con = sqlite3.connect(CFG.lexicon_db) con = sqlite3.connect(CFG.lexicon_db, timeout=30)
con.execute("PRAGMA journal_mode=WAL")
con.row_factory = sqlite3.Row con.row_factory = sqlite3.Row
con.executescript(SCHEMA) con.executescript(SCHEMA)
# One-time compatibility migration for the 158 corrections learned before the # One-time compatibility migration for the 158 corrections learned before the
@@ -298,7 +299,7 @@ def sync_from_vault(force: bool = False) -> dict:
now = time.time() now = time.time()
n_ent = n_alias = 0 n_ent = n_alias = 0
con.execute("DELETE FROM alias WHERE origin IN ('card','glossary')") con.execute("DELETE FROM alias WHERE origin IN ('card','glossary')")
con.execute("DELETE FROM entity") con.execute("DELETE FROM entity WHERE card <> ''")
con.execute("DELETE FROM relation WHERE source LIKE 'card:%'") con.execute("DELETE FROM relation WHERE source LIKE 'card:%'")
for sub, kind in (("People", "人名"), ("Companies", "公司"), ("Projects", "项目")): for sub, kind in (("People", "人名"), ("Companies", "公司"), ("Projects", "项目")):
d = CFG.vault / sub d = CFG.vault / sub
@@ -524,3 +525,320 @@ def stats() -> dict:
d["synced_at"] = r["v"] if r else "" d["synced_at"] = r["v"] if r else ""
con.close() con.close()
return d return d
# ------------------------------------------------------------------ Web UI CRUD
def list_aliases(q: str = "", origin_filter: str = "", limit: int = 200, offset: int = 0) -> list[dict]:
con = connect()
sql = "SELECT * FROM alias WHERE 1=1"
args: list = []
if q.strip():
pat = f"%{q.strip()}%"
sql += " AND (alias LIKE ? OR canonical LIKE ? OR note LIKE ?)"
args.extend([pat, pat, pat])
if origin_filter == "learned":
sql += " AND origin <> 'card'"
elif origin_filter == "card":
sql += " AND origin = 'card'"
sql += " ORDER BY confirmed DESC, created_at DESC, alias ASC LIMIT ? OFFSET ?"
args.extend((max(1, min(limit, 500)), max(0, offset)))
rows = [dict(r) for r in con.execute(sql, args).fetchall()]
con.close()
return rows
def count_aliases(q: str = "", origin_filter: str = "") -> int:
con = connect()
sql = "SELECT count(*) FROM alias WHERE 1=1"
args: list = []
if q.strip():
pat = f"%{q.strip()}%"
sql += " AND (alias LIKE ? OR canonical LIKE ? OR note LIKE ?)"
args.extend([pat, pat, pat])
if origin_filter == "learned":
sql += " AND origin <> 'card'"
elif origin_filter == "card":
sql += " AND origin = 'card'"
n = int(con.execute(sql, args).fetchone()[0])
con.close()
return n
def update_alias(
old_alias: str,
old_canonical: str,
new_alias: str,
new_canonical: str,
*,
kind: str = "",
confirmed: int = 1,
note: str = "",
) -> bool:
con = connect()
row = con.execute(
"SELECT * FROM alias WHERE alias=? AND canonical=?",
(old_alias, old_canonical),
).fetchone()
origin = row["origin"] if row else "manual"
hits = row["hits"] if row else 0
created_at = row["created_at"] if row else time.time()
if not kind:
er = con.execute("SELECT kind FROM entity WHERE canonical=?", (new_canonical,)).fetchone()
kind = er["kind"] if er else (row["kind"] if row else guess_kind(new_canonical))
con.execute(
"DELETE FROM alias WHERE alias=? AND canonical=?",
(old_alias, old_canonical),
)
con.execute(
"INSERT OR REPLACE INTO alias(alias,canonical,kind,origin,confirmed,hits,created_at,note) "
"VALUES(?,?,?,?,?,?,?,?)",
(new_alias, new_canonical, kind, origin, int(bool(confirmed)), hits, created_at, note),
)
con.commit()
con.close()
return True
def delete_alias(alias: str, canonical: str) -> bool:
con = connect()
cur = con.execute("DELETE FROM alias WHERE alias=? AND canonical=?", (alias, canonical))
con.commit()
deleted = cur.rowcount > 0
con.close()
return deleted
def list_corrections(q: str = "", limit: int = 200, offset: int = 0) -> list[dict]:
con = connect()
sql = "SELECT * FROM correction WHERE 1=1"
args: list = []
if q.strip():
pat = f"%{q.strip()}%"
sql += " AND (surface LIKE ? OR canonical LIKE ? OR note LIKE ?)"
args.extend([pat, pat, pat])
sql += " ORDER BY confirmed DESC, n DESC, last_seen DESC LIMIT ? OFFSET ?"
args.extend((max(1, min(limit, 500)), max(0, offset)))
rows = [dict(r) for r in con.execute(sql, args).fetchall()]
con.close()
return rows
def count_corrections(q: str = "") -> int:
con = connect()
sql = "SELECT count(*) FROM correction WHERE 1=1"
args: list = []
if q.strip():
pat = f"%{q.strip()}%"
sql += " AND (surface LIKE ? OR canonical LIKE ? OR note LIKE ?)"
args.extend([pat, pat, pat])
n = int(con.execute(sql, args).fetchone()[0])
con.close()
return n
def update_correction(
old_surface: str,
old_canonical: str,
new_surface: str,
new_canonical: str,
*,
confirmed: int = 1,
note: str = "",
) -> bool:
con = connect()
row = con.execute(
"SELECT * FROM correction WHERE surface=? AND canonical=?",
(old_surface, old_canonical),
).fetchone()
n = row["n"] if row else 1
now = time.time()
con.execute("DELETE FROM correction WHERE surface=? AND canonical=?", (old_surface, old_canonical))
con.execute(
"DELETE FROM phonetic_confusion WHERE heard=? AND intended=?",
(old_surface, old_canonical),
)
con.execute(
"INSERT OR REPLACE INTO correction(surface,canonical,n,confirmed,last_seen,note) "
"VALUES(?,?,?,?,?,?)",
(new_surface, new_canonical, n, int(bool(confirmed)), now, note),
)
con.execute(
"INSERT OR REPLACE INTO phonetic_confusion(heard,intended,n,confirmed,last_seen,note) "
"VALUES(?,?,?,?,?,?)",
(new_surface, new_canonical, n, int(bool(confirmed)), now, note),
)
con.commit()
con.close()
return True
def delete_correction(surface: str, canonical: str) -> bool:
con = connect()
cur = con.execute("DELETE FROM correction WHERE surface=? AND canonical=?", (surface, canonical))
con.execute("DELETE FROM phonetic_confusion WHERE heard=? AND intended=?", (surface, canonical))
con.commit()
deleted = cur.rowcount > 0
con.close()
return deleted
def list_entities(q: str = "", kind: str = "", limit: int = 200, offset: int = 0) -> list[dict]:
con = connect()
sql = "SELECT * FROM entity WHERE 1=1"
args: list = []
if q.strip():
pat = f"%{q.strip()}%"
sql += " AND (canonical LIKE ? OR company LIKE ? OR role LIKE ? OR related LIKE ?)"
args.extend([pat, pat, pat, pat])
if kind.strip():
sql += " AND kind=?"
args.append(kind.strip())
sql += " ORDER BY (CASE WHEN card='' THEN 0 ELSE 1 END), canonical ASC LIMIT ? OFFSET ?"
args.extend((max(1, min(limit, 500)), max(0, offset)))
rows = [dict(r) for r in con.execute(sql, args).fetchall()]
con.close()
return rows
def count_entities(q: str = "", kind: str = "") -> int:
con = connect()
sql = "SELECT count(*) FROM entity WHERE 1=1"
args: list = []
if q.strip():
pat = f"%{q.strip()}%"
sql += " AND (canonical LIKE ? OR company LIKE ? OR role LIKE ? OR related LIKE ?)"
args.extend([pat, pat, pat, pat])
if kind.strip():
sql += " AND kind=?"
args.append(kind.strip())
n = int(con.execute(sql, args).fetchone()[0])
con.close()
return n
def upsert_entity(
canonical: str,
kind: str = "人名",
*,
company: str = "",
role: str = "",
related: str = "",
card: str = "",
) -> None:
con = connect()
con.execute(
"INSERT OR REPLACE INTO entity(canonical,kind,company,role,related,card,synced_at) "
"VALUES(?,?,?,?,?,?,?)",
(canonical, kind or guess_kind(canonical), company, role, related, card, time.time()),
)
con.commit()
con.close()
def update_entity(
old_canonical: str,
new_canonical: str,
kind: str,
*,
company: str = "",
role: str = "",
related: str = "",
) -> bool:
con = connect()
row = con.execute("SELECT * FROM entity WHERE canonical=?", (old_canonical,)).fetchone()
card = row["card"] if row else ""
con.execute("DELETE FROM entity WHERE canonical=?", (old_canonical,))
con.execute(
"INSERT OR REPLACE INTO entity(canonical,kind,company,role,related,card,synced_at) "
"VALUES(?,?,?,?,?,?,?)",
(new_canonical, kind or guess_kind(new_canonical), company, role, related, card, time.time()),
)
if old_canonical != new_canonical:
con.execute("UPDATE alias SET canonical=? WHERE canonical=?", (new_canonical, old_canonical))
con.execute("UPDATE correction SET canonical=? WHERE canonical=?", (new_canonical, old_canonical))
con.commit()
con.close()
return True
def delete_entity(canonical: str) -> bool:
con = connect()
cur = con.execute("DELETE FROM entity WHERE canonical=?", (canonical,))
con.commit()
deleted = cur.rowcount > 0
con.close()
return deleted
def list_relations(q: str = "", limit: int = 200, offset: int = 0) -> list[dict]:
con = connect()
sql = "SELECT * FROM relation WHERE 1=1"
args: list = []
if q.strip():
pat = f"%{q.strip()}%"
sql += " AND (subject LIKE ? OR predicate LIKE ? OR object LIKE ? OR source LIKE ?)"
args.extend([pat, pat, pat, pat])
sql += " ORDER BY active DESC, confidence DESC, last_seen DESC LIMIT ? OFFSET ?"
args.extend((max(1, min(limit, 500)), max(0, offset)))
rows = [dict(r) for r in con.execute(sql, args).fetchall()]
con.close()
return rows
def count_relations(q: str = "") -> int:
con = connect()
sql = "SELECT count(*) FROM relation WHERE 1=1"
args: list = []
if q.strip():
pat = f"%{q.strip()}%"
sql += " AND (subject LIKE ? OR predicate LIKE ? OR object LIKE ? OR source LIKE ?)"
args.extend([pat, pat, pat, pat])
n = int(con.execute(sql, args).fetchone()[0])
con.close()
return n
def update_relation(
old_subject: str,
old_predicate: str,
old_object: str,
old_source: str,
subject: str,
predicate: str,
object_: str,
*,
confidence: float = 0.8,
active: int = 1,
) -> bool:
con = connect()
row = con.execute(
"SELECT * FROM relation WHERE subject=? AND predicate=? AND object=? AND source=?",
(old_subject, old_predicate, old_object, old_source),
).fetchone()
first_seen = row["first_seen"] if row else time.time()
source = old_source or "manual"
con.execute(
"DELETE FROM relation WHERE subject=? AND predicate=? AND object=? AND source=?",
(old_subject, old_predicate, old_object, old_source),
)
con.execute(
"INSERT OR REPLACE INTO relation(subject,predicate,object,confidence,source,first_seen,last_seen,active) "
"VALUES(?,?,?,?,?,?,?,?)",
(subject, predicate, object_, float(confidence), source, first_seen, time.time(), int(bool(active))),
)
con.commit()
con.close()
return True
def delete_relation(subject: str, predicate: str, object_: str, source: str) -> bool:
con = connect()
cur = con.execute(
"DELETE FROM relation WHERE subject=? AND predicate=? AND object=? AND source=?",
(subject, predicate, object_, source),
)
con.commit()
deleted = cur.rowcount > 0
con.close()
return deleted
+19 -26
View File
@@ -1,19 +1,7 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
"""HTTPS client for the model provider. No agent CLI, no SDK. """HTTPS client for LiteLLM / OpenAI-compatible model gateways. No agent CLI, no SDK.
Deliberately built on urllib so the program has zero third-party HTTP Built on urllib so the program has zero third-party HTTP dependencies.
dependencies: it must keep working inside a bare systemd unit years from now.
Behaviours encoded from measurement rather than guesswork:
* concurrency 6 (HTTP 429 appears at ~12)
* HTTP 402 "Payment Required" is transient on ZenMux — credit seems reserved
against max_tokens, so a burst of large-budget calls can trip it and then
succeed moments later; it gets a much longer backoff than a rate limit
* `temperature` is rejected outright by claude-sonnet-5.5 / opus-5 but accepted by
haiku-4.5, sonnet-4.6, gemini and qwen, so it cannot be inferred from the
vendor prefix; the client self-heals on the 400 and remembers
* an empty completion (reasoning models burning the whole budget on hidden
thoughts) is retried rather than silently propagated as ""
""" """
from __future__ import annotations from __future__ import annotations
@@ -30,14 +18,22 @@ from .config import CFG
from .log import log from .log import log
PRICES = { PRICES = {
"aistudio/gemini-flash-lite-latest": (0.3, 2.5),
"vertex/gemini-flash-lite-latest": (0.3, 2.5),
"gemini-flash-lite-latest": (0.3, 2.5),
"google/gemini-3.5-flash-lite": (0.3, 2.5), "google/gemini-3.5-flash-lite": (0.3, 2.5),
"google/gemini-3.5-flash": (1.5, 9.0), "aistudio/gemini-flash-latest": (1.5, 9.0),
"vertex/gemini-flash-latest": (1.5, 9.0),
"aistudio/gemini-pro-latest": (2.5, 12.0),
"codex/gpt-sol-latest": (2.0, 8.0),
"openai/gpt-sol-latest": (2.0, 8.0),
"codex/gpt-luna-latest": (1.5, 6.0),
"xai/grok-latest": (2.0, 10.0),
"anthropic/claude-haiku-4.5": (1.0, 5.0), "anthropic/claude-haiku-4.5": (1.0, 5.0),
"anthropic/claude-sonnet-5": (2.0, 10.0), "anthropic/claude-sonnet-5": (2.0, 10.0),
"anthropic/claude-sonnet-5.5": (2.0, 10.0), "anthropic/claude-sonnet-5.5": (2.0, 10.0),
"anthropic/claude-sonnet-4.6": (3.0, 15.0), "anthropic/claude-sonnet-4.6": (3.0, 15.0),
"anthropic/claude-opus-5": (5.0, 25.0), "anthropic/claude-opus-5": (5.0, 25.0),
"qwen/qwen3.7-plus": (0.1373076, 0.5492304),
} }
_sem = threading.BoundedSemaphore(CFG.concurrency) _sem = threading.BoundedSemaphore(CFG.concurrency)
@@ -45,16 +41,16 @@ _NO_TEMPERATURE: set[str] = {
"anthropic/claude-sonnet-5", "anthropic/claude-sonnet-5",
"anthropic/claude-sonnet-5.5", "anthropic/claude-sonnet-5.5",
"anthropic/claude-opus-5", "anthropic/claude-opus-5",
"codex/gpt-sol-latest",
"codex/gpt-luna-latest",
} }
_lock = threading.Lock() _lock = threading.Lock()
_key: list[str] = [] _key: list[str] = []
def _auth() -> str: def _auth() -> str:
with _lock: # Read on every call so a key saved from /settings is used without a restart.
if not _key: return CFG.api_key()
_key.append(CFG.api_key())
return _key[0]
@dataclass @dataclass
@@ -143,7 +139,7 @@ def chat(model: str, prompt: str, *, system: str | None = None,
except Exception: except Exception:
pass pass
last = f"HTTP{e.code}: {detail[:200]}" last = f"HTTP{e.code}: {detail[:200]}"
if e.code == 400 and "temperature" in detail: if e.code == 400 and "temperature" in detail.lower():
with _lock: with _lock:
_NO_TEMPERATURE.add(model) _NO_TEMPERATURE.add(model)
payload = payload_of() payload = payload_of()
@@ -151,13 +147,12 @@ def chat(model: str, prompt: str, *, system: str | None = None,
cpath = _cache_path(model, payload) cpath = _cache_path(model, payload)
retries += 1 retries += 1
continue continue
if e.code not in (402, 408, 429, 500, 502, 503, 504): if e.code not in (408, 429, 500, 502, 503, 504):
if usage: if usage:
usage.failures += 1 usage.failures += 1
raise RuntimeError(f"{model} {last}") from e raise RuntimeError(f"{model} {last}") from e
retries += 1 retries += 1
delay = (min(20 * (attempt + 1), 120) if e.code == 402 delay = min(2 ** attempt + random.uniform(0, 1.5), 45)
else min(2 ** attempt + random.uniform(0, 1.5), 45))
log(f"llm retry {attempt+1}/{CFG.max_retries} {model} {last} " log(f"llm retry {attempt+1}/{CFG.max_retries} {model} {last} "
f"sleep {delay:.0f}s", level="WARN") f"sleep {delay:.0f}s", level="WARN")
time.sleep(delay) time.sleep(delay)
@@ -203,8 +198,6 @@ def chat_many(model: str, prompts: list[str], **kw) -> list[str]:
except BaseException as e: # noqa: BLE001 - recorded, decided by caller except BaseException as e: # noqa: BLE001 - recorded, decided by caller
errors.append((i, e)) errors.append((i, e))
if errors: if errors:
# Partial failure must not silently yield empty chunks: the caller falls
# back to the original text for these indices, which the gate then flags.
for i, e in errors: for i, e in errors:
log(f"chunk {i} failed: {e}", level="ERROR") log(f"chunk {i} failed: {e}", level="ERROR")
raise PartialFailure(out, [i for i, _ in errors]) raise PartialFailure(out, [i for i, _ in errors])
@@ -0,0 +1,25 @@
"""One machine-wide lock for processes that consume the recording inbox."""
from __future__ import annotations
import fcntl
import os
from pathlib import Path
def acquire_recording_lock(path: Path):
"""Return an open file holding an exclusive lock, or exit if another consumer has it.
The handle must stay referenced for the life of the process.
"""
path.parent.mkdir(parents=True, exist_ok=True)
handle = path.open("a+")
try:
fcntl.flock(handle, fcntl.LOCK_EX | fcntl.LOCK_NB)
except OSError:
handle.close()
raise SystemExit(f"another recording consumer holds {path}")
handle.seek(0)
handle.truncate()
handle.write(str(os.getpid()))
handle.flush()
return handle
@@ -93,7 +93,7 @@ KEYWORD_PROMPT = """从下面的会议转写摘选中,提取用于检索背景
# (「谭凯(白帆生物总经理)」、「此项目负责人为 [[冯昆仑]](背景确认)」) even when the # (「谭凯(白帆生物总经理)」、「此项目负责人为 [[冯昆仑]](背景确认)」) even when the
# recording never said them. The facts were often correct, but a meeting note that # recording never said them. The facts were often correct, but a meeting note that
# cannot distinguish "said on the call" from "known from notes" is unsafe to finalize. # cannot distinguish "said on the call" from "known from notes" is unsafe to finalize.
DRAFT_PROMPT = """你是会议录音整理助手,正在根据**带来源编号的事实卡片**产出「粗整理草稿」。 DRAFT_PROMPT = """你是会议录音整理助手,正在根据下方整理要点写出会议草稿。
严格遵循: 严格遵循:
1. 只产出粗整理草稿,**不是**正式纪要。 1. 只产出粗整理草稿,**不是**正式纪要。
@@ -133,8 +133,11 @@ DRAFT_PROMPT = """你是会议录音整理助手,正在根据**带来源编号
(例如「徐总(疑似 [[徐劲前]] 或 [[邹洵]])」),而不是笼统写「身份不明」。 (例如「徐总(疑似 [[徐劲前]] 或 [[邹洵]])」),而不是笼统写「身份不明」。
13. 判定简称指向谁时使用背景里的任职信息,这属于**辨识**而非越界断言, 13. 判定简称指向谁时使用背景里的任职信息,这属于**辨识**而非越界断言,
不需要加〔背景〕;但如果你要额外陈述这个人的职务、公司归属本身,那就要加〔背景〕。 不需要加〔背景〕;但如果你要额外陈述这个人的职务、公司归属本身,那就要加〔背景〕。
14. 对匿名说话人,除非事实卡片已经给出有证据的候选,否则只写“身份待确认”; 14. 对匿名说话人,除非整理要点已经给出有证据的候选,否则只写“身份待确认”;
不要从背景中枚举可能的人名让用户猜。 不要从背景中枚举可能的人名让用户猜。
15. 正文和标题不要出现「卡片」「提示词」「系统指令」「粗整理」这些内部说法。
记录里没有的信息写「记录未写明」,不要写「卡片未提供」或「卡片提及」。
标题用业务名称,不要带「草稿」二字。
{speaker_note} {speaker_note}
@@ -153,7 +156,7 @@ companies: [有把握的公司…]
transcript: {transcript} transcript: {transcript}
{extra_fm}--- {extra_fm}---
事实卡片: 整理要点:
{body} {body}
只输出 markdown 草稿全文,不要任何额外说明。""" 只输出 markdown 草稿全文,不要任何额外说明。"""
@@ -166,6 +169,20 @@ SPEAKER_NOTE = {
"- 不同 speaker_N 是不同真人(CAM++ 已分),但可能过度聚类(同一人被拆成多个 N)," "- 不同 speaker_N 是不同真人(CAM++ 已分),但可能过度聚类(同一人被拆成多个 N),"
"按内容合并同一真人。\n" "按内容合并同一真人。\n"
"- frontmatter 的 people 优先填**声纹命名的真人**。"), "- frontmatter 的 people 优先填**声纹命名的真人**。"),
"soniox": (
"转写来自 Soniox 线上分角色模型:\n"
"- `speaker_N` 为自动区分的说话人标签,按内容与背景推断身份并标「疑似」,"
"待人工在 Web 审核中确认。"),
"openai": (
"转写来自 OpenAI Transcribe 模型:\n"
"- 若含 `speaker_N` 标签,为单场自动区分的说话人,按内容推断身份并标「疑似」;"
"若无说话人标签则按内容推断发言归属。"),
"gemini": (
"转写来自 Gemini 3.5 Transcribe 模型:\n"
"- 按内容推断发言归属,不要臆造未出现的 speaker 标签。"),
"grok": (
"转写来自 Grok Voice Transcribe 2.0 分角色模型:\n"
"- `speaker_N` 为自动区分的说话人标签,按内容推断身份并标「疑似」。"),
"vibevoice": "转写带 S# 说话人标签,仅作软提示,块间编号不保证一致。", "vibevoice": "转写带 S# 说话人标签,仅作软提示,块间编号不保证一致。",
"qwen3-plain": "转写**无说话人信息**,据内容推断发言归属,不要臆造 speaker 标签。", "qwen3-plain": "转写**无说话人信息**,据内容推断发言归属,不要臆造 speaker 标签。",
} }
@@ -0,0 +1,938 @@
#!/usr/bin/env python3
"""Unified ASR provider abstraction for local GPU (qwen3cam) and online models
(Soniox, Gemini 3.5 Transcribe via LiteLLM, OpenAI Transcribe via LiteLLM, Grok).
"""
from __future__ import annotations
import json
import mimetypes
import os
import re
import shutil
import subprocess
import tempfile
import time
import urllib.error
import urllib.request
import uuid
from dataclasses import dataclass, field
import http.client
from pathlib import Path
from typing import Any
from .config import CFG
from .log import log
class EmptyAudioError(RuntimeError):
"""Raised when an audio file contains silence/noise or produces zero ASR speech tokens."""
pass
PROVIDERS: dict[str, dict[str, str]] = {
"qwen3cam": {
"label": "内网 Qwen3-ASR 1.7B + CAM++ (分角色 + 声纹)",
"default_model": "qwen3-asr-1.7b-cam++",
"mode": "lan-diarized-voiceprint",
},
"soniox": {
"label": "Soniox Async v5 (线上分角色 + 词库热词注入)",
"default_model": "stt-async-v5",
"mode": "cloud-diarized",
},
"gemini": {
"label": "Gemini 3.5 Transcribe (LiteLLM 统一路由)",
"default_model": "aistudio/gemini-3.5-transcribe",
"mode": "litellm-segmented",
},
"openai": {
"label": "OpenAI GPT-4o Transcribe Diarize (LiteLLM 统一路由)",
"default_model": "openai/gpt-4o-transcribe-diarize",
"mode": "litellm-diarized",
},
"grok": {
"label": "Grok Voice Transcribe 2.0 (线上分角色)",
"default_model": "grok-voice-transcribe-2.0",
"mode": "cloud-diarized",
},
}
@dataclass
class TranscribeResult:
provider: str
model: str
transcript_text: str
raw_response: Any
diarized: bool = True
voiceprint_session: str = ""
duration_seconds: int = 0
meta: dict[str, Any] = field(default_factory=dict)
def normalize_provider_name(name: str | None) -> str:
raw = (name or CFG.asr_provider or "qwen3cam").strip().lower()
if raw in ("paraformer", "qwen3", "qwen3-cam"):
return "qwen3cam"
if raw in ("gemini-3.5-transcribe", "aistudio/gemini-3.5-transcribe",
"vertex/gemini-3.5-transcribe"):
return "gemini"
if raw in ("openai-diarize", "gpt-4o-transcribe-diarize", "gpt-4o-transcribe",
"gpt-transcribe"):
return "openai"
if raw in ("xai", "grok-voice-transcribe-2.0"):
return "grok"
return raw if raw in PROVIDERS else "qwen3cam"
def provider_status() -> list[dict[str, Any]]:
"""Return readiness metadata for each ASR provider without exposing secrets."""
has_litellm = False
try:
has_litellm = bool(CFG.api_key())
except Exception:
has_litellm = False
return [
{
"id": "qwen3cam",
"label": PROVIDERS["qwen3cam"]["label"],
"model": PROVIDERS["qwen3cam"]["default_model"],
"ready": bool(CFG.diarize_url),
"endpoint": CFG.diarize_url,
"via": "LAN GPU (:10099)",
},
{
"id": "soniox",
"label": PROVIDERS["soniox"]["label"],
"model": CFG.soniox_model,
"ready": bool(CFG.soniox_api_key),
"endpoint": CFG.soniox_api_base,
"via": "SONIOX_API_KEY 直连",
},
{
"id": "gemini",
"label": PROVIDERS["gemini"]["label"],
"model": CFG.gemini_asr_model,
"ready": has_litellm,
"endpoint": CFG.transcriptions_endpoint,
"via": "LiteLLM 统一网关",
},
{
"id": "openai",
"label": PROVIDERS["openai"]["label"],
"model": CFG.openai_asr_model,
"ready": has_litellm,
"endpoint": CFG.transcriptions_endpoint,
"via": "LiteLLM 统一网关",
},
{
"id": "grok",
"label": PROVIDERS["grok"]["label"],
"model": CFG.grok_asr_model,
"ready": bool(CFG.grok_api_key),
"endpoint": CFG.grok_api_base + "/stt",
"via": "XAI_API_KEY 直连",
},
]
def collect_hotwords(limit: int = 120) -> list[str]:
"""Collect canonical entity names and glossary terms for ASR context biasing."""
terms: list[str] = []
seen: set[str] = set()
def add(word: str) -> None:
w = word.strip()
if not w or len(w) < 2 or len(w) > 40 or w in seen:
return
if re.match(r"^\d{4}-\d{2}|^_|speaker_", w):
return
seen.add(w)
terms.append(w)
try:
from . import lexicon
con = lexicon.connect()
for row in con.execute(
"SELECT canonical FROM entity ORDER BY synced_at DESC LIMIT ?",
(limit,)).fetchall():
add(str(row["canonical"]))
for row in con.execute(
"SELECT DISTINCT canonical FROM alias WHERE confirmed=1 LIMIT ?",
(limit,)).fetchall():
add(str(row["canonical"]))
con.close()
except Exception:
pass
if CFG.glossary.exists():
for raw in CFG.glossary.read_text(errors="ignore").splitlines():
s = raw.strip()
if not s.startswith(("-", "*")) or re.search(r"[→⇒]|->", s):
continue
s = re.split(r"\s[—–-]\s|[::]",
re.sub(r"^[-*+\d\.\)\s]+", "", s), maxsplit=1)[0].strip()
s = re.sub(r"\[\[|\]\]", "", s).strip()
add(s)
if len(terms) >= limit:
break
return terms[:limit]
def _normalize_speaker_label(raw: Any, mapping: dict[str, str]) -> str:
s = str(raw if raw is not None else "").strip()
if not s or s == "?":
return "speaker_0"
if re.match(r"^[\u4e00-\u9fff]{2,8}$", s):
return s
if re.match(r"^speaker_\d+$", s):
return s
m = re.match(r"^[sS](?:peaker)?[_ -]?(\d+)$", s)
if m:
return f"speaker_{m.group(1)}"
if s.isdigit():
# Soniox often uses 1-based speaker numbers ("1", "2"); normalize to speaker_0..N-1
if s not in mapping:
mapping[s] = f"speaker_{len(mapping)}"
return mapping[s]
if s not in mapping:
mapping[s] = f"speaker_{len(mapping)}"
return mapping[s]
def format_diarized_transcript(
segments: list[dict[str, Any]],
*,
provider: str,
model: str,
header_title: str,
voiceprint_session: str = "",
matched_names: list[str] | None = None,
) -> str:
if not segments:
raise EmptyAudioError(
f"ASR ({provider}) 未识别到有效语音片段(音频可能为静音、设备未拾音或音量过低)"
)
speaker_map: dict[str, str] = {}
turns: list[list[str]] = []
speakers_seen: set[str] = set()
for seg in segments:
raw_spk = seg.get("speaker")
text = str(seg.get("text") or "").strip()
if not text:
continue
spk = (_normalize_speaker_label(raw_spk, speaker_map)
if provider != "qwen3cam"
else str(raw_spk if raw_spk is not None else "speaker_0"))
speakers_seen.add(spk)
if turns and turns[-1][0] == spk:
turns[-1][1] += text
else:
turns.append([spk, text])
if not turns:
raise EmptyAudioError(
f"ASR ({provider}) 未识别到任何有效文字(音频可能为静音或底噪)"
)
lines = [f"<!-- asr_provider: {provider} model: {model} -->"]
if voiceprint_session or provider == "qwen3cam":
lines.append(
f"<!-- voiceprint_session: {voiceprint_session} "
f"clusters: {len(speakers_seen)} -->")
hdr = f"# 转写({header_title})"
if matched_names:
hdr += " | 声纹命名: " + ", ".join(matched_names)
lines.append(hdr)
lines.append("")
lines.extend(f"【{spk}】{txt}\n" for spk, txt in turns)
return "\n".join(lines).strip() + "\n"
def transcript_from_qwen3cam_response(payload: dict[str, Any]) -> str:
segments = payload.get("segments")
if not isinstance(segments, list) or not segments:
raise EmptyAudioError("ASR (qwen3cam) 未识别到有效语音片段(音频可能为静音、设备未拾音或音量过低)")
vp = payload.get("voiceprint") or {}
sid = str(vp.get("session_id") or "")
matched = [
f"{m.get('speaker_local')}={m.get('person_name')}"
for m in vp.get("matches", [])
if m.get("status") == "matched"
]
return format_diarized_transcript(
segments,
provider="qwen3cam",
model="qwen3-asr-1.7b-cam++",
header_title="Qwen3-ASR 1.7B + CAM++ 分角色 + 声纹",
voiceprint_session=sid,
matched_names=matched,
)
def transcript_from_soniox_response(
payload: dict[str, Any],
*,
model: str = "stt-async-v5",
) -> str:
"""Convert Soniox token-level or segment-level diarized response into turns."""
tokens = payload.get("tokens")
segments: list[dict[str, Any]] = []
if isinstance(tokens, list) and tokens:
cur_speaker: Any = None
cur_pieces: list[str] = []
for tok in tokens:
txt = str(tok.get("text") or "")
if not txt:
continue
spk = tok.get("speaker")
if cur_speaker is not None and spk != cur_speaker and cur_pieces:
segments.append({"speaker": cur_speaker,
"text": "".join(cur_pieces).strip()})
cur_pieces = []
cur_speaker = spk if spk is not None else (cur_speaker or "1")
cur_pieces.append(txt)
if cur_pieces:
segments.append({"speaker": cur_speaker or "1",
"text": "".join(cur_pieces).strip()})
elif isinstance(payload.get("segments"), list) and payload["segments"]:
segments = payload["segments"]
elif isinstance(payload.get("text"), str) and payload["text"].strip():
segments = [{"speaker": "speaker_0", "text": payload["text"].strip()}]
return format_diarized_transcript(
segments,
provider="soniox",
model=model,
header_title=f"Soniox {model} 分角色",
)
def transcript_from_openai_response(
payload: dict[str, Any],
*,
model: str,
) -> str:
"""Convert OpenAI / LiteLLM transcription response (diarized or plain) into text."""
segments = payload.get("segments")
if isinstance(segments, list) and segments and any(
s.get("speaker") is not None for s in segments if isinstance(s, dict)):
return format_diarized_transcript(
segments,
provider="openai",
model=model,
header_title=f"OpenAI {model} 分角色",
)
text = str(payload.get("text") or payload.get("transcript") or "").strip()
if not text and isinstance(segments, list):
text = "".join(str(s.get("text") or "") for s in segments).strip()
if not text:
raise RuntimeError(f"OpenAI ASR ({model}) returned empty transcript")
if re.search(r"【[^】]+】", text):
return (
f"<!-- asr_provider: openai model: {model} -->\n"
f"# 转写(OpenAI {model})\n\n{text}\n"
)
return (
f"<!-- asr_provider: openai model: {model} -->\n"
f"--- 第1段 [00:00 - 结束] ---\n{text}\n"
)
def transcript_from_grok_response(
payload: dict[str, Any],
*,
model: str = "grok-voice-transcribe-2.0",
) -> str:
segments = payload.get("segments") or payload.get("utterances") or payload.get("turns")
if isinstance(segments, list) and segments:
norm_segs = []
for s in segments:
if not isinstance(s, dict):
continue
spk = s.get("speaker")
if spk is None:
spk = s.get("speaker_id")
if spk is None:
spk = "speaker_0"
txt = str(s.get("text") or s.get("transcript") or "").strip()
if txt:
norm_segs.append({"speaker": spk, "text": txt})
if norm_segs:
return format_diarized_transcript(
norm_segs,
provider="grok",
model=model,
header_title=f"Grok {model} 分角色",
)
words = payload.get("words")
if isinstance(words, list) and words and any(
isinstance(w, dict) and w.get("speaker") is not None for w in words):
norm_segs = []
cur_spk = None
buf: list[str] = []
for w in words:
if not isinstance(w, dict):
continue
txt = str(w.get("word") or w.get("text") or "")
spk = w.get("speaker")
if cur_spk is not None and spk != cur_spk and buf:
norm_segs.append({"speaker": cur_spk, "text": "".join(buf).strip()})
buf = []
cur_spk = spk if spk is not None else (cur_spk or "0")
buf.append(txt)
if buf:
norm_segs.append({"speaker": cur_spk or "0", "text": "".join(buf).strip()})
if norm_segs:
return format_diarized_transcript(
norm_segs,
provider="grok",
model=model,
header_title=f"Grok {model} 分角色",
)
text = str(payload.get("text") or payload.get("transcript") or "").strip()
if not text:
raise RuntimeError(f"Grok ASR ({model}) returned empty transcript")
return format_diarized_transcript(
[{"speaker": "speaker_0", "text": text}],
provider="grok",
model=model,
header_title=f"Grok {model}",
)
def probe_duration_seconds(audio: Path) -> int:
try:
out = subprocess.check_output([
"ffprobe", "-v", "error", "-show_entries", "format=duration",
"-of", "default=nk=1:nw=1", str(audio),
], text=True, timeout=30).strip()
return max(1, int(float(out) + 0.999)) if out else 0
except Exception:
return 0
def _multipart_framing(
fields: dict[str, str],
file_field: str,
file_path: Path,
) -> tuple[bytes, bytes, str, int]:
"""Build multipart head/tail without reading the audio into memory."""
boundary = f"----ASRBoundary{uuid.uuid4().hex}"
safe_name = file_path.name.replace('"', "").replace("\r", "").replace("\n", "") or "audio"
head = bytearray()
for name, val in fields.items():
head.extend(f"--{boundary}\r\n".encode())
head.extend(f'Content-Disposition: form-data; name="{name}"\r\n\r\n'.encode())
head.extend(str(val).encode("utf-8"))
head.extend(b"\r\n")
mime = mimetypes.guess_type(safe_name)[0] or "application/octet-stream"
head.extend(f"--{boundary}\r\n".encode())
head.extend(
f'Content-Disposition: form-data; name="{file_field}"; '
f'filename="{safe_name}"\r\n'.encode())
head.extend(f"Content-Type: {mime}\r\n\r\n".encode())
tail = f"\r\n--{boundary}--\r\n".encode()
total = len(head) + file_path.stat().st_size + len(tail)
return bytes(head), tail, f"multipart/form-data; boundary={boundary}", total
def _open_http(url: str, timeout: int) -> tuple[http.client.HTTPConnection, str]:
parsed = urllib.parse.urlsplit(url)
if parsed.scheme not in {"http", "https"} or not parsed.hostname:
raise RuntimeError(f"unsupported ASR URL: {url}")
port = parsed.port or (443 if parsed.scheme == "https" else 80)
if parsed.scheme == "https":
conn: http.client.HTTPConnection = http.client.HTTPSConnection(
parsed.hostname, port, timeout=timeout)
else:
conn = http.client.HTTPConnection(parsed.hostname, port, timeout=timeout)
path = parsed.path or "/"
if parsed.query:
path = f"{path}?{parsed.query}"
return conn, path
def _http_multipart(
url: str,
*,
fields: dict[str, str],
file_field: str,
file_path: Path,
headers: dict[str, str] | None = None,
timeout: int = 900,
retries: int = 3,
) -> dict[str, Any]:
"""POST audio as multipart, streaming the file in 1 MiB chunks."""
head, tail, content_type, total = _multipart_framing(fields, file_field, file_path)
last_exc: Exception | None = None
raw = ""
for attempt in range(1, max(1, retries) + 1):
conn, path = _open_http(url, timeout)
try:
conn.putrequest("POST", path)
conn.putheader("Content-Type", content_type)
conn.putheader("Content-Length", str(total))
for key, value in (headers or {}).items():
conn.putheader(key, value)
conn.endheaders()
conn.send(head)
with file_path.open("rb") as stream:
while True:
chunk = stream.read(1024 * 1024)
if not chunk:
break
conn.send(chunk)
conn.send(tail)
resp = conn.getresponse()
raw = resp.read().decode("utf-8", errors="ignore")
status = resp.status
if status in (502, 503, 504) and attempt < retries:
last_exc = RuntimeError(f"ASR HTTP {status}")
time.sleep(2 * attempt)
continue
if status >= 400:
raise RuntimeError(f"ASR HTTP {status}: {raw[:300]}")
break
except RuntimeError:
raise
except (http.client.HTTPException, ConnectionError, TimeoutError, OSError) as exc:
last_exc = exc
if attempt < retries:
log(f"multipart upload transient error (attempt {attempt}/{retries}): {exc}")
time.sleep(2 * attempt)
continue
raise RuntimeError(f"ASR upload connection error after {retries} attempts: {exc}") from exc
finally:
conn.close()
else:
raise RuntimeError(f"ASR upload failed: {last_exc}")
try:
return json.loads(raw)
except ValueError as exc:
raise RuntimeError(f"ASR returned invalid JSON: {raw[:200]}") from exc
def _http_json(
method: str,
url: str,
*,
payload: dict[str, Any] | None = None,
headers: dict[str, str] | None = None,
timeout: int = 120,
retries: int = 3,
) -> dict[str, Any]:
data = json.dumps(payload, ensure_ascii=False).encode("utf-8") if payload is not None else None
req_headers = {"Accept": "application/json", **(headers or {})}
if data is not None:
req_headers["Content-Type"] = "application/json"
last_exc: Exception | None = None
for attempt in range(1, max(1, retries) + 1):
req = urllib.request.Request(url, data=data, headers=req_headers, method=method)
try:
with urllib.request.urlopen(req, timeout=timeout) as resp:
raw = resp.read().decode("utf-8", errors="ignore")
break
except urllib.error.HTTPError as exc:
detail = exc.read().decode("utf-8", errors="ignore")[:300]
if exc.code in (502, 503, 504) and attempt < retries:
last_exc = exc
time.sleep(1.5 * attempt)
continue
raise RuntimeError(f"ASR HTTP {exc.code}: {detail}") from exc
except (urllib.error.URLError, http.client.HTTPException, ConnectionError, TimeoutError, OSError) as exc:
last_exc = exc
if attempt < retries:
time.sleep(1.5 * attempt)
continue
raise RuntimeError(f"ASR HTTP request failed after {retries} attempts: {exc}") from exc
else:
raise RuntimeError(f"ASR HTTP request failed: {last_exc}")
if not raw.strip():
return {}
try:
return json.loads(raw)
except ValueError as exc:
raise RuntimeError(f"ASR returned invalid JSON: {raw[:200]}") from exc
def _compress_or_split_audio(
audio: Path,
tmpdir: Path,
*,
max_bytes: int = 23 * 1024 * 1024,
segment_seconds: int = 2400,
force_split: bool = False,
) -> list[tuple[Path, int, int]]:
"""Return [(segment_path, start_sec, end_sec), ...] suitable for upstream API."""
duration = probe_duration_seconds(audio)
if not force_split and audio.stat().st_size <= max_bytes and audio.suffix.lower() in {
".mp3", ".m4a", ".wav", ".ogg", ".webm", ".flac"}:
return [(audio, 0, duration)]
if not shutil.which("ffmpeg"):
if audio.stat().st_size <= max_bytes:
return [(audio, 0, duration)]
raise RuntimeError("audio exceeds size limit and ffmpeg is not installed")
if not force_split:
compressed = tmpdir / "compressed.mp3"
subprocess.run([
"ffmpeg", "-hide_banner", "-loglevel", "error", "-y",
"-i", str(audio), "-ar", "16000", "-ac", "1", "-b:a", "48k",
str(compressed),
], check=True, timeout=600)
if compressed.exists() and compressed.stat().st_size <= max_bytes:
return [(compressed, 0, duration)]
total = max(duration, 1)
step = max(60, segment_seconds)
n_segs = (total + step - 1) // step
out: list[tuple[Path, int, int]] = []
for i in range(n_segs):
start = i * step
end = min(total, start + step)
seg = tmpdir / f"segment_{i + 1:03d}.mp3"
subprocess.run([
"ffmpeg", "-hide_banner", "-loglevel", "error", "-y",
"-ss", str(start), "-t", str(step), "-i", str(audio),
"-ar", "16000", "-ac", "1", "-b:a", "64k", str(seg),
], check=True, timeout=600)
out.append((seg, start, end))
return out
def transcribe_qwen3cam(audio: Path, *, model: str = "") -> TranscribeResult:
result = subprocess.run([
"curl", "-sS", "--fail-with-body", "--max-time", "900",
"-F", f"file=@{audio}", "-F", "language=Chinese",
"-F", "save_voiceprint_session=true", CFG.diarize_url,
], capture_output=True, timeout=930)
if result.returncode:
detail = result.stderr.decode("utf-8", errors="ignore")[:240]
raise RuntimeError(f"qwen3cam ASR HTTP failed ({result.returncode}): {detail}")
try:
payload = json.loads(result.stdout)
except ValueError as exc:
raise RuntimeError("qwen3cam returned invalid JSON") from exc
text = transcript_from_qwen3cam_response(payload)
vp = payload.get("voiceprint") or {}
return TranscribeResult(
provider="qwen3cam",
model=model or PROVIDERS["qwen3cam"]["default_model"],
transcript_text=text,
raw_response=payload,
diarized=True,
voiceprint_session=str(vp.get("session_id") or ""),
)
def build_soniox_context(title: str = "", user_context: str = "") -> dict[str, Any]:
ctx: dict[str, Any] = {}
general = [{"key": "domain", "value": "Biotech / Investment / Corporate Meeting"}]
if title.strip():
general.append({"key": "title", "value": title.strip()[:120]})
ctx["general"] = general
if user_context.strip():
ctx["text"] = user_context.strip()[:2000]
terms = collect_hotwords(120)
if terms:
ctx["terms"] = terms
return ctx
def transcribe_soniox(
audio: Path,
*,
model: str = "",
title: str = "",
user_context: str = "",
poll_interval: float = 3.0,
max_wait_seconds: float = 900.0,
) -> TranscribeResult:
key = CFG.soniox_api_key
if not key:
raise RuntimeError("SONIOX_API_KEY is not configured")
base = CFG.soniox_api_base
chosen_model = model or CFG.soniox_model or "stt-async-v5"
headers = {"Authorization": f"Bearer {key}"}
with tempfile.TemporaryDirectory(prefix="asr-soniox-") as tmp:
upload_path = audio
if audio.exists() and audio.stat().st_size > 8 * 1024 * 1024 and shutil.which("ffmpeg"):
compressed = Path(tmp) / "soniox_upload.mp3"
try:
subprocess.run([
"ffmpeg", "-hide_banner", "-loglevel", "error", "-y",
"-i", str(audio), "-ar", "16000", "-ac", "1", "-b:a", "48k",
str(compressed),
], check=True, timeout=600)
if compressed.exists() and 0 < compressed.stat().st_size < audio.stat().st_size:
upload_path = compressed
except Exception as exc:
log(f"soniox audio pre-compress skipped: {exc}")
file_resp = _http_multipart(
f"{base}/files",
fields={},
file_field="file",
file_path=upload_path,
headers=headers,
timeout=600,
)
file_id = str(file_resp.get("id") or "")
if not file_id:
raise RuntimeError(f"Soniox file upload returned no id: {file_resp}")
transcription_id = ""
try:
job_payload: dict[str, Any] = {
"model": chosen_model,
"file_id": file_id,
"enable_speaker_diarization": True,
"enable_language_identification": True,
"language_hints": ["zh", "en"],
"context": build_soniox_context(title, user_context),
}
job_resp = _http_json(
"POST", f"{base}/transcriptions",
payload=job_payload, headers=headers, timeout=60,
)
transcription_id = str(job_resp.get("id") or "")
if not transcription_id:
raise RuntimeError(f"Soniox transcription create returned no id: {job_resp}")
deadline = time.time() + max_wait_seconds
status_resp: dict[str, Any] = {}
while time.time() < deadline:
status_resp = _http_json(
"GET", f"{base}/transcriptions/{transcription_id}",
headers=headers, timeout=60,
)
st = str(status_resp.get("status") or "").lower()
if st == "completed":
break
if st in ("error", "failed"):
err = status_resp.get("error_message") or status_resp
raise RuntimeError(f"Soniox transcription failed: {err}")
time.sleep(poll_interval)
else:
raise RuntimeError("Soniox transcription timed out waiting for completion")
if status_resp.get("tokens") or status_resp.get("segments"):
transcript_payload = status_resp
else:
transcript_payload = _http_json(
"GET", f"{base}/transcriptions/{transcription_id}/transcript",
headers=headers, timeout=120,
)
text = transcript_from_soniox_response(
transcript_payload, model=chosen_model)
return TranscribeResult(
provider="soniox",
model=chosen_model,
transcript_text=text,
raw_response=transcript_payload,
diarized=True,
)
finally:
for cleanup_url in (
f"{base}/transcriptions/{transcription_id}" if transcription_id else "",
f"{base}/files/{file_id}" if file_id else "",
):
if cleanup_url:
try:
_http_json("DELETE", cleanup_url, headers=headers, timeout=15)
except Exception:
pass
def transcribe_gemini(
audio: Path,
*,
model: str = "",
title: str = "",
user_context: str = "",
) -> TranscribeResult:
chosen_model = model or CFG.gemini_asr_model or "aistudio/gemini-3.5-transcribe"
headers = {"Authorization": f"Bearer {CFG.api_key()}"}
with tempfile.TemporaryDirectory(prefix="asr-gemini-") as tmp:
segments = _compress_or_split_audio(
audio, Path(tmp),
max_bytes=18 * 1024 * 1024,
segment_seconds=CFG.segment_seconds,
force_split=probe_duration_seconds(audio) > CFG.segment_seconds + 60,
)
responses: list[dict[str, Any]] = []
parts: list[str] = [f"<!-- asr_provider: gemini model: {chosen_model} -->"]
for idx, (seg_path, start, end) in enumerate(segments, 1):
resp = _http_multipart(
CFG.transcriptions_endpoint,
fields={"model": chosen_model, "response_format": "json"},
file_field="file",
file_path=seg_path,
headers=headers,
timeout=600,
)
responses.append(resp)
seg_text = str(resp.get("text") or resp.get("transcript") or "").strip()
if not seg_text:
raise RuntimeError(f"Gemini segment {idx} returned empty text")
parts.append(
f"--- 第{idx}段 [{start // 60:02d}:{start % 60:02d} - "
f"{end // 60:02d}:{end % 60:02d}] ---\n{seg_text}\n")
return TranscribeResult(
provider="gemini",
model=chosen_model,
transcript_text="\n\n".join(parts).strip() + "\n",
raw_response={"segments": responses},
diarized=False,
)
def transcribe_openai(
audio: Path,
*,
model: str = "",
title: str = "",
user_context: str = "",
) -> TranscribeResult:
chosen_model = model or CFG.openai_asr_model or "openai/gpt-4o-transcribe-diarize"
headers = {"Authorization": f"Bearer {CFG.api_key()}"}
is_diarize = "diarize" in chosen_model.lower()
with tempfile.TemporaryDirectory(prefix="asr-openai-") as tmp:
segments = _compress_or_split_audio(
audio, Path(tmp),
max_bytes=23 * 1024 * 1024,
segment_seconds=2400,
force_split=False,
)
responses: list[dict[str, Any]] = []
all_diarized_segs: list[dict[str, Any]] = []
plain_parts: list[str] = [f"<!-- asr_provider: openai model: {chosen_model} -->"]
hotwords = collect_hotwords(80)
prompt = ""
if not is_diarize and hotwords:
prompt = "会议专有名词参考:" + "、".join(hotwords)[:800]
for idx, (seg_path, start, end) in enumerate(segments, 1):
fields: dict[str, str] = {
"model": chosen_model,
"language": "zh",
"response_format": "diarized_json" if is_diarize else "json",
}
if prompt:
fields["prompt"] = prompt
try:
resp = _http_multipart(
CFG.transcriptions_endpoint,
fields=fields,
file_field="file",
file_path=seg_path,
headers=headers,
timeout=600,
)
except RuntimeError:
if is_diarize:
# Fallback if gateway enforces response_format=json
fields["response_format"] = "json"
resp = _http_multipart(
CFG.transcriptions_endpoint,
fields=fields,
file_field="file",
file_path=seg_path,
headers=headers,
timeout=600,
)
else:
raise
responses.append(resp)
if isinstance(resp.get("segments"), list) and resp["segments"]:
all_diarized_segs.extend(resp["segments"])
else:
txt = str(resp.get("text") or resp.get("transcript") or "").strip()
if txt:
plain_parts.append(
f"--- 第{idx}段 [{start // 60:02d}:{start % 60:02d} - "
f"{end // 60:02d}:{end % 60:02d}] ---\n{txt}\n")
if all_diarized_segs:
text = transcript_from_openai_response(
{"segments": all_diarized_segs}, model=chosen_model)
diarized = True
elif len(responses) == 1:
text = transcript_from_openai_response(responses[0], model=chosen_model)
diarized = "【" in text
else:
if len(plain_parts) <= 1:
raise RuntimeError(f"OpenAI ASR ({chosen_model}) returned no text")
text = "\n\n".join(plain_parts).strip() + "\n"
diarized = False
return TranscribeResult(
provider="openai",
model=chosen_model,
transcript_text=text,
raw_response={"segments": responses} if len(responses) > 1 else responses[0],
diarized=diarized,
)
def transcribe_grok(
audio: Path,
*,
model: str = "",
title: str = "",
user_context: str = "",
) -> TranscribeResult:
key = CFG.grok_api_key
if not key:
raise RuntimeError("XAI_API_KEY is not configured")
chosen_model = model or CFG.grok_asr_model or "grok-voice-transcribe-2.0"
headers = {"Authorization": f"Bearer {key}"}
url = f"{CFG.grok_api_base}/stt"
with tempfile.TemporaryDirectory(prefix="asr-grok-") as tmp:
segments = _compress_or_split_audio(
audio, Path(tmp), max_bytes=23 * 1024 * 1024, segment_seconds=2400)
all_texts: list[str] = []
raw_list: list[dict[str, Any]] = []
for seg_path, _start, _end in segments:
resp = _http_multipart(
url,
fields={"model": chosen_model, "diarize": "true", "language": "zh"},
file_field="file",
file_path=seg_path,
headers=headers,
timeout=600,
)
raw_list.append(resp)
all_texts.append(transcript_from_grok_response(resp, model=chosen_model))
text = all_texts[0] if len(all_texts) == 1 else "\n\n".join(all_texts)
return TranscribeResult(
provider="grok",
model=chosen_model,
transcript_text=text,
raw_response=raw_list[0] if len(raw_list) == 1 else {"chunks": raw_list},
diarized=True,
)
def transcribe_audio(
audio: Path,
*,
provider: str | None = None,
model: str = "",
title: str = "",
user_context: str = "",
) -> TranscribeResult:
"""Dispatch transcription to the selected provider and return normalized result."""
prov = normalize_provider_name(provider)
log(f"transcribe start provider={prov} model={model or 'default'} file={audio.name}")
if prov == "qwen3cam":
return transcribe_qwen3cam(audio, model=model)
if prov == "soniox":
return transcribe_soniox(audio, model=model, title=title, user_context=user_context)
if prov == "gemini":
return transcribe_gemini(audio, model=model, title=title, user_context=user_context)
if prov == "openai":
return transcribe_openai(audio, model=model, title=title, user_context=user_context)
if prov == "grok":
return transcribe_grok(audio, model=model, title=title, user_context=user_context)
raise ValueError(f"unsupported ASR provider: {prov}")
@@ -28,6 +28,10 @@ from .log import log
SOURCE_DETAIL = { SOURCE_DETAIL = {
"qwen3cam": "Qwen3-ASR 1.7B + CAM++ 分角色 + 声纹自动命名", "qwen3cam": "Qwen3-ASR 1.7B + CAM++ 分角色 + 声纹自动命名",
"soniox": "Soniox stt-async-v5 线上分角色转写",
"gemini": "Gemini 3.5 Transcribe (LiteLLM)",
"openai": "OpenAI GPT-4o Transcribe (LiteLLM)",
"grok": "Grok Voice Transcribe 2.0",
"vibevoice": "VibeVoice 分角色", "vibevoice": "VibeVoice 分角色",
"qwen3-plain": "Qwen3-ASR 1.7B", "qwen3-plain": "Qwen3-ASR 1.7B",
} }
@@ -333,6 +337,18 @@ def normalize_chunks(chunks, cleaned_parts: list[str], flags: list[dict],
} }
def scrub_internal_wording(text: str) -> str:
"""Keep pipeline jargon out of text a person will read."""
for src, dst in (
("卡片未提供", "记录未写明"),
("卡片提及", "录音中提到"),
("事实卡片", "整理要点"),
("粗整理草稿", "整理稿"),
):
text = text.replace(src, dst)
return text
def extract_evidence(parts: list[str], usage: Usage) -> str: def extract_evidence(parts: list[str], usage: Usage) -> str:
jobs = [prompts.EVIDENCE_PROMPT.format(chunk_id=i, body=body) jobs = [prompts.EVIDENCE_PROMPT.format(chunk_id=i, body=body)
for i, body in enumerate(parts)] for i, body in enumerate(parts)]
@@ -343,7 +359,7 @@ def extract_evidence(parts: list[str], usage: Usage) -> str:
max_tokens=1400, usage=usage) max_tokens=1400, usage=usage)
except PartialFailure as pf: except PartialFailure as pf:
outs = pf.outputs outs = pf.outputs
return "\n".join(x for x in outs if x.strip()) return scrub_internal_wording("\n".join(x for x in outs if x.strip()))
# ------------------------------------------------------------------ S4 # ------------------------------------------------------------------ S4
@@ -373,12 +389,16 @@ def write_draft(cleaned: str, meta: dict, context: str, date: str,
if ctx: if ctx:
ctx = "本场背景(来自既有笔记与长期记忆;引用其中内容时必须按上面第 3 条加 〔背景〕):\n" \ ctx = "本场背景(来自既有笔记与长期记忆;引用其中内容时必须按上面第 3 条加 〔背景〕):\n" \
+ context + context
detail = SOURCE_DETAIL.get(meta.get("backend", ""), "ASR") backend = meta.get("backend") or meta.get("asr_provider") or ""
model_name = meta.get("asr_model") or ""
detail = SOURCE_DETAIL.get(backend, "ASR")
if model_name and backend in ("soniox", "gemini", "openai", "grok") and model_name not in detail:
detail = f"{detail} ({model_name})"
prompt = prompts.DRAFT_PROMPT.format( prompt = prompts.DRAFT_PROMPT.format(
speaker_note=prompts.SPEAKER_NOTE.get(meta.get("backend", ""), ""), speaker_note=prompts.SPEAKER_NOTE.get(backend, ""),
context=ctx, date=date, source_detail=detail, context=ctx, date=date, source_detail=detail,
transcript=transcript_path, extra_fm=extra, body=cleaned) transcript=transcript_path, extra_fm=extra, body=cleaned)
return chat(CFG.draft_model, prompt, max_tokens=12000, usage=usage) return scrub_internal_wording(chat(CFG.draft_model, prompt, max_tokens=12000, usage=usage))
# ------------------------------------------------------------------ driver # ------------------------------------------------------------------ driver
@@ -185,7 +185,8 @@ def _voiceprint_name(value: str) -> str:
def _confirm_voiceprint(draft_path: str, speaker_id: str, value: str) -> bool: def _confirm_voiceprint(draft_path: str, speaker_id: str, value: str) -> bool:
"""Persist an explicit speaker identity and immediately enroll its session.""" """Persist an explicit speaker identity and, when a voiceprint session exists,
enroll it against the acoustic voiceprint service."""
name = _voiceprint_name(value) name = _voiceprint_name(value)
if not name: if not name:
return False return False
@@ -196,8 +197,6 @@ def _confirm_voiceprint(draft_path: str, speaker_id: str, value: str) -> bool:
return False return False
text = path.read_text(encoding="utf-8", errors="ignore") text = path.read_text(encoding="utf-8", errors="ignore")
session_id = _frontmatter_value(text, "voiceprint_session") session_id = _frontmatter_value(text, "voiceprint_session")
if not session_id:
return False
fm = re.match(r"(?s)^---\n(.*?)\n---", text) fm = re.match(r"(?s)^---\n(.*?)\n---", text)
if not fm: if not fm:
return False return False
@@ -205,27 +204,34 @@ def _confirm_voiceprint(draft_path: str, speaker_id: str, value: str) -> bool:
speaker_line = rf"(?m)^(\s+{re.escape(speaker_id)}:)\s*.*$" speaker_line = rf"(?m)^(\s+{re.escape(speaker_id)}:)\s*.*$"
if re.search(speaker_line, block): if re.search(speaker_line, block):
block = re.sub(speaker_line, rf"\1 {name}", block) block = re.sub(speaker_line, rf"\1 {name}", block)
elif re.search(r"(?m)^voiceprint_speakers:\s*(?:\{\})?\s*$", block):
block = re.sub(
r"(?m)^voiceprint_speakers:\s*(?:\{\})?\s*$",
f"voiceprint_speakers:\n {speaker_id}: {name}", block, count=1)
elif re.search(r"(?m)^voiceprint_speakers:\s*$", block): elif re.search(r"(?m)^voiceprint_speakers:\s*$", block):
block = re.sub( block = re.sub(
r"(?m)^(voiceprint_speakers:\s*)$", r"(?m)^(voiceprint_speakers:\s*)$",
rf"\1\n {speaker_id}: {name}", block, count=1) rf"\1\n {speaker_id}: {name}", block, count=1)
else: else:
block += f"\nvoiceprint_speakers:\n {speaker_id}: {name}" block += f"\nvoiceprint_speakers:\n {speaker_id}: {name}"
if re.search(r"(?m)^voiceprint_enroll:", block): if session_id:
block = re.sub(r"(?m)^voiceprint_enroll:.*$", if re.search(r"(?m)^voiceprint_enroll:", block):
"voiceprint_enroll: true", block) block = re.sub(r"(?m)^voiceprint_enroll:.*$",
else: "voiceprint_enroll: true", block)
block += "\nvoiceprint_enroll: true" else:
block += "\nvoiceprint_enroll: true"
text = text[:fm.start()] + "---\n" + block + "\n---" + text[fm.end():] text = text[:fm.start()] + "---\n" + block + "\n---" + text[fm.end():]
from . import vault from . import vault
vault.write_draft_file(path, text) vault.write_draft_file(path, text)
if not session_id or not CFG.voiceprint_enroll_url:
return False
body = json.dumps({ body = json.dumps({
"session_id": session_id, "speaker_id": int(speaker_id.split("_")[1]), "session_id": session_id, "speaker_id": int(speaker_id.split("_")[1]),
"name": name, "name": name,
}).encode() }).encode()
req = urllib.request.Request( req = urllib.request.Request(
"http://192.168.50.100:10099/voiceprints/enroll", data=body, CFG.voiceprint_enroll_url, data=body,
headers={"Content-Type": "application/json"}) headers={"Content-Type": "application/json"})
try: try:
with urllib.request.urlopen(req, timeout=20) as response: with urllib.request.urlopen(req, timeout=20) as response:
@@ -301,14 +307,14 @@ def _request_revision(draft_path: str) -> bool:
return False return False
def answer(question_id: int, value: str) -> dict: def answer(question_id: int, value: str, *, allow_update: bool = False) -> dict:
con = lexicon.connect() con = lexicon.connect()
row = con.execute("SELECT * FROM review_question WHERE id=?", row = con.execute("SELECT * FROM review_question WHERE id=?",
(question_id,)).fetchone() (question_id,)).fetchone()
if not row: if not row:
con.close() con.close()
raise KeyError(f"review question {question_id} not found") raise KeyError(f"review question {question_id} not found")
if row["status"] != "pending": if row["status"] != "pending" and not allow_update:
result = { result = {
"id": question_id, "answer": row["answer"], "learned": False, "id": question_id, "answer": row["answer"], "learned": False,
"already_answered": True, "already_answered": True,
@@ -317,6 +323,7 @@ def answer(question_id: int, value: str) -> dict:
return result return result
payload = json.loads(row["payload"] or "{}") payload = json.loads(row["payload"] or "{}")
recording = row["recording"] recording = row["recording"]
previous = str(row["answer"] or "")
con.execute( con.execute(
"UPDATE review_question SET status='answered',answer=?,answered_at=? WHERE id=?", "UPDATE review_question SET status='answered',answer=?,answered_at=? WHERE id=?",
(value, time.time(), question_id)) (value, time.time(), question_id))
@@ -337,9 +344,14 @@ def answer(question_id: int, value: str) -> dict:
confidence=1.0, confirmed=1) confidence=1.0, confirmed=1)
saved = False saved = False
voiceprint_enrolled = False voiceprint_enrolled = False
if payload.get("kind") == "voiceprint": if payload.get("kind") == "voiceprint" and not (allow_update and value == previous):
voiceprint_enrolled = _confirm_voiceprint( try:
payload.get("draft_path", ""), payload.get("speaker_id", ""), value) voiceprint_enrolled = _confirm_voiceprint(
payload.get("draft_path", ""), payload.get("speaker_id", ""), value)
except Exception as exc:
from .log import log
log(f"voiceprint enroll failed after answer saved: {exc}", level="WARN")
voiceprint_enrolled = False
if payload.get("draft_path"): if payload.get("draft_path"):
try: try:
saved = _persist_draft_feedback(recording, payload["draft_path"]) saved = _persist_draft_feedback(recording, payload["draft_path"])
@@ -370,3 +382,62 @@ def answer(question_id: int, value: str) -> dict:
"draft_feedback_saved": saved, "voiceprint_enrolled": voiceprint_enrolled, "draft_feedback_saved": saved, "voiceprint_enrolled": voiceprint_enrolled,
"revision_started": revision_started, "revision_started": revision_started,
} }
def recent_answered(limit: int = 30) -> list[dict]:
con = lexicon.connect()
rows = con.execute(
"SELECT * FROM review_question WHERE status='answered' "
"ORDER BY answered_at DESC, id DESC LIMIT ?",
(limit,),
).fetchall()
con.close()
return [dict(r) for r in rows]
def reopen_question(question_id: int) -> bool:
con = lexicon.connect()
cur = con.execute(
"UPDATE review_question SET status='pending', answer='', answered_at=NULL WHERE id=?",
(question_id,),
)
con.commit()
ok = cur.rowcount > 0
con.close()
return ok
def _question_matches(row, transcript_path: str, draft_path: str, job_id: str) -> bool:
payload = json.loads(row["payload"] or "{}")
return bool(
(transcript_path and row["recording"] == transcript_path)
or (draft_path and payload.get("draft_path") == draft_path)
or (job_id and payload.get("job_id") == job_id)
)
def delete_for_job(transcript_path: str = "", draft_path: str = "", job_id: str = "") -> int:
return _delete_questions(transcript_path, draft_path, job_id, pending_only=False)
def delete_pending_for_job(transcript_path: str = "", draft_path: str = "", job_id: str = "") -> int:
"""Drop unanswered questions. Answered rows stay so a failed refine can be retried."""
return _delete_questions(transcript_path, draft_path, job_id, pending_only=True)
def _delete_questions(transcript_path: str, draft_path: str, job_id: str, *, pending_only: bool) -> int:
con = lexicon.connect()
rows = con.execute("SELECT id, recording, payload, status FROM review_question").fetchall()
to_delete: list[int] = []
for r in rows:
if pending_only and r["status"] != "pending":
continue
if _question_matches(r, transcript_path, draft_path, job_id):
to_delete.append(int(r["id"]))
if to_delete:
marks = ",".join("?" for _ in to_delete)
con.execute(f"DELETE FROM review_question WHERE id IN ({marks})", to_delete)
con.commit()
con.close()
return len(to_delete)
@@ -29,6 +29,8 @@ def _url(token: str) -> str:
def request_start(job_id: str = "") -> bool: def request_start(job_id: str = "") -> bool:
"""Ensure control service is running and send a task-scoped review URL.""" """Ensure control service is running and send a task-scoped review URL."""
if os.environ.get("ASR_ROLE"):
return True
try: try:
r = subprocess.run(["systemctl", "--user", "start", UNIT], r = subprocess.run(["systemctl", "--user", "start", UNIT],
capture_output=True, text=True, timeout=15) capture_output=True, text=True, timeout=15)
@@ -54,12 +54,14 @@ def _safe_title(markdown: str) -> str:
return title[:72] or "通话记录" return title[:72] or "通话记录"
def _formal_path(markdown: str) -> Path: def _formal_path(markdown: str, root: Path | None = None) -> Path:
target_dir = root or CFG.minutes
target_dir.mkdir(parents=True, exist_ok=True)
date = _field(markdown, "date") or time.strftime("%Y-%m-%d") date = _field(markdown, "date") or time.strftime("%Y-%m-%d")
base = CFG.minutes / f"{date}-{_safe_title(markdown)}.md" base = target_dir / f"{date}-{_safe_title(markdown)}.md"
transcript = _field(markdown, "transcript") transcript = _field(markdown, "transcript")
if CFG.minutes.exists() and transcript: if target_dir.exists() and transcript:
for path in CFG.minutes.glob("*.md"): for path in target_dir.glob("*.md"):
try: try:
if _field(path.read_text(errors="ignore"), "transcript") == transcript: if _field(path.read_text(errors="ignore"), "transcript") == transcript:
return path return path
@@ -256,13 +258,21 @@ def publish(path: Path) -> bool:
destination = _formal_path(out) destination = _formal_path(out)
vault.write_draft_file(destination, out) vault.write_draft_file(destination, out)
vault_minutes_dir = CFG.vault / "minutes"
vault_destination = (
destination
if destination.parent.resolve() == vault_minutes_dir.resolve()
else (vault_minutes_dir / destination.name)
)
if vault_destination != destination:
vault.write_draft_file(vault_destination, out)
date = _field(out, "date") or time.strftime("%Y-%m-%d") date = _field(out, "date") or time.strftime("%Y-%m-%d")
touched = [destination] touched = [vault_destination]
for name, kind in _answered_card_requests(path): for name, kind in _answered_card_requests(path):
card = _create_card(name, kind, destination, date) card = _create_card(name, kind, vault_destination, date)
if card: if card:
touched.append(card) touched.append(card)
state = vault.sync(touched, f"minutes: {destination.stem}(Web 确认后正式发布)") state = vault.sync(touched, f"minutes: {vault_destination.stem}(Web 确认后正式发布)")
if state == "error": if state == "error":
raise RuntimeError("formal minute commit failed") raise RuntimeError("formal minute commit failed")
@@ -22,6 +22,7 @@ import re
from dataclasses import dataclass, field from dataclasses import dataclass, field
VOICEPRINT_HDR = re.compile(r"^<!--\s*voiceprint_session:") VOICEPRINT_HDR = re.compile(r"^<!--\s*voiceprint_session:")
PROVIDER_HDR = re.compile(r"^<!--\s*asr_provider:\s*(\S+)(?:\s+model:\s*(\S+))?\s*-->", re.M)
SEGMENT_BANNER = re.compile(r"^---\s*第\d+段.*---\s*$") SEGMENT_BANNER = re.compile(r"^---\s*第\d+段.*---\s*$")
TURN_TAG = re.compile(r"^【([^】]+)】\s*(.*)$", re.S) TURN_TAG = re.compile(r"^【([^】]+)】\s*(.*)$", re.S)
VIBE_TAG = re.compile(r"^\[(\d{1,2}:\d{2})\]\s*(S\d+)\s*[::]\s*(.*)$", re.S) VIBE_TAG = re.compile(r"^\[(\d{1,2}:\d{2})\]\s*(S\d+)\s*[::]\s*(.*)$", re.S)
@@ -95,6 +96,11 @@ class Chunk:
def detect_backend(raw: str) -> str: def detect_backend(raw: str) -> str:
head = raw[:500] head = raw[:500]
m_prov = PROVIDER_HDR.search(head)
if m_prov:
prov = m_prov.group(1).strip().lower()
if prov in {"qwen3cam", "soniox", "gemini", "openai", "grok", "vibevoice"}:
return prov
if "CAM++" in head or TURN_TAG.match(raw.strip().splitlines()[-1] if raw.strip() else ""): if "CAM++" in head or TURN_TAG.match(raw.strip().splitlines()[-1] if raw.strip() else ""):
return "qwen3cam" return "qwen3cam"
if re.search(r"^\[\d{1,2}:\d{2}\]\s*S\d+\s*[::]", raw, re.M): if re.search(r"^\[\d{1,2}:\d{2}\]\s*S\d+\s*[::]", raw, re.M):
@@ -108,7 +114,7 @@ def parse(raw: str) -> tuple[list[Turn], dict]:
"""Split a transcript into turns plus header metadata.""" """Split a transcript into turns plus header metadata."""
meta: dict = { meta: dict = {
"voiceprint_session": "", "voiceprint_named": {}, "banners": [], "voiceprint_session": "", "voiceprint_named": {}, "banners": [],
"intake": {}, "intake": {}, "asr_provider": "", "asr_model": "",
} }
lines = raw.splitlines() lines = raw.splitlines()
body_lines = [] body_lines = []
@@ -120,6 +126,11 @@ def parse(raw: str) -> tuple[list[Turn], dict]:
except (TypeError, ValueError): except (TypeError, ValueError):
pass pass
continue continue
m_prov = PROVIDER_HDR.match(ln)
if m_prov:
meta["asr_provider"] = m_prov.group(1) or ""
meta["asr_model"] = m_prov.group(2) or ""
continue
if VOICEPRINT_HDR.match(ln): if VOICEPRINT_HDR.match(ln):
m = re.search(r"voiceprint_session:\s*(\S+)", ln) m = re.search(r"voiceprint_session:\s*(\S+)", ln)
if m: if m:
@@ -137,8 +148,10 @@ def parse(raw: str) -> tuple[list[Turn], dict]:
continue continue
body_lines.append(ln) body_lines.append(ln)
backend = detect_backend(raw) backend = meta["asr_provider"] or str(meta["intake"].get("asr_provider") or "") or detect_backend(raw)
meta["backend"] = backend meta["backend"] = backend
if not meta["asr_model"] and meta["intake"].get("asr_model"):
meta["asr_model"] = str(meta["intake"]["asr_model"])
turns: list[Turn] = [] turns: list[Turn] = []
if backend == "vibevoice": if backend == "vibevoice":
@@ -10,6 +10,7 @@ Ported from the legacy shell implementation with its hard-won details preserved:
""" """
from __future__ import annotations from __future__ import annotations
import os
import re import re
import subprocess import subprocess
import tempfile import tempfile
@@ -90,8 +91,39 @@ def update_pending(date: str, title: str, path: Path) -> None:
def _git(*args, check=False) -> subprocess.CompletedProcess: def _git(*args, check=False) -> subprocess.CompletedProcess:
return subprocess.run(["git", *args], cwd=CFG.vault, capture_output=True, env = dict(os.environ)
text=True, timeout=180, check=check) for key_candidate in (
CFG.state_dir / "ssh" / "id_ed25519_gitea",
Path("/state/ssh/id_ed25519_gitea"),
Path("/root/.ssh/id_ed25519_gitea"),
Path.home() / ".ssh" / "id_ed25519_gitea",
):
if key_candidate.exists():
env.setdefault(
"GIT_SSH_COMMAND",
f"ssh -i {key_candidate} -o StrictHostKeyChecking=accept-new -o IdentitiesOnly=yes",
)
break
return subprocess.run(
["git", "-c", f"safe.directory={CFG.vault}", *args],
cwd=CFG.vault, capture_output=True,
text=True, timeout=180, check=check, env=env,
)
def pull() -> str:
"""Pull latest remote changes into CFG.vault when git_sync is enabled."""
if not CFG.git_sync or not (CFG.vault / ".git").exists():
return "disabled"
try:
r = _git("pull", "--rebase", "--autostash", "-q")
if r.returncode != 0:
log(f"git pull failed: {r.stderr.strip()[:200]}", level="WARN")
return "error"
return "pulled"
except Exception as e:
log(f"git pull error: {e}", level="WARN")
return "error"
def sync(paths: list[Path], message: str) -> str: def sync(paths: list[Path], message: str) -> str:
@@ -99,9 +131,18 @@ def sync(paths: list[Path], message: str) -> str:
if not CFG.git_sync: if not CFG.git_sync:
return "disabled" return "disabled"
try: try:
rel = [str(p.relative_to(CFG.vault)) for p in paths if p.exists()] vault_root = CFG.vault.resolve()
rel = []
for p in paths:
if not p.exists():
continue
try:
rel.append(str(p.resolve().relative_to(vault_root)))
except ValueError:
continue
if not rel: if not rel:
return "noop" return "noop"
_git("pull", "--rebase", "--autostash", "-q")
_git("add", "--", *rel) _git("add", "--", *rel)
if _git("diff", "--cached", "--quiet").returncode == 0: if _git("diff", "--cached", "--quiet").returncode == 0:
log("nothing staged (likely already committed by the auto-sync cron)") log("nothing staged (likely already committed by the auto-sync cron)")
@@ -0,0 +1,57 @@
"""Browser session helpers shared by the new workbench and the legacy control UI.
An empty ``ASR_WEB_PASSWORD`` closes the UI. Set ``ASR_WEB_ALLOW_ANON=1`` only
for a local process that is not reachable from the network.
"""
from __future__ import annotations
import hashlib
import hmac
import os
from http import cookies
from .config import get_setting
def allow_anon() -> bool:
return os.getenv("ASR_WEB_ALLOW_ANON", "").strip().lower() in {
"1", "true", "yes", "on",
}
def password() -> str:
env_pw = os.getenv("ASR_WEB_PASSWORD")
if env_pw not in (None, ""):
return env_pw
return get_setting("web_password", "")
def _secret() -> bytes:
return (password() or os.getenv("ASR_AGENT_TOKEN", "") or "asr-web-unset").encode("utf-8")
def session_token() -> str:
return hmac.new(_secret(), b"asr-web-session-v1", hashlib.sha256).hexdigest()
def csrf_token() -> str:
return hmac.new(_secret(), b"asr-web-csrf-v1", hashlib.sha256).hexdigest()
def cookie_matches(header: str) -> bool:
if not password():
return False
jar = cookies.SimpleCookie()
try:
jar.load(header or "")
except Exception:
return False
morsel = jar.get("asr_session")
return bool(morsel and hmac.compare_digest(morsel.value, session_token()))
def csrf_matches(supplied: str) -> bool:
if not supplied:
return False
return hmac.compare_digest(supplied, csrf_token())
@@ -1 +1 @@
"""Standalone ASR service for web, Agent API, and SSH drop-folder intake.""" """Standalone WebUI, REST API, and background worker service for Meeting ASR."""
@@ -3,23 +3,27 @@ from __future__ import annotations
import os import os
from pathlib import Path from pathlib import Path
from asrpipe.config import CFG
DATA = Path(os.getenv("ASR_DATA_DIR", "/data")) DATA = Path(os.getenv("ASR_DATA_DIR", "/data"))
INBOX = DATA / "inbox" INBOX = DATA / "inbox"
PROCESSING = DATA / "processing" PROCESSING = DATA / "processing"
PROCESSED = DATA / "processed" PROCESSED = DATA / "processed"
FAILED = DATA / "failed" FAILED = DATA / "failed"
TRANSCRIPTS = DATA / "transcripts" TRANSCRIPTS = DATA / "transcripts"
STATE = Path(os.getenv("ASR_STATE_DIR", "/state")) STATE = Path(os.getenv("ASR_STATE_DIR", str(CFG.state_dir)))
DRAFTS = DATA / "artifacts" / "drafts" DRAFTS = Path(os.getenv("ASR_DRAFTS_DIR", str(DATA / "artifacts" / "drafts")))
RESULTS = DATA / "artifacts" / "results" RESULTS = Path(os.getenv("ASR_RESULTS_DIR", str(DATA / "artifacts" / "results")))
LOGS = DATA / "logs" LOGS = DATA / "logs"
META = STATE / "uploads" META = STATE / "uploads"
REVISION_QUEUE = STATE / "revision-queue" REVISION_QUEUE = Path(os.getenv("ASR_REVISION_QUEUE_DIR", str(STATE / "revision-queue")))
DB = STATE / "intake.db" DB = STATE / "intake.db"
SUPPORTED = {".m4a", ".mp3", ".wav", ".mp4", ".aac", ".ogg", ".opus", ".webm", ".flac"} SUPPORTED = {".m4a", ".mp3", ".wav", ".mp4", ".aac", ".ogg", ".opus", ".webm", ".flac"}
PUBLIC_URL = os.getenv("ASR_PUBLIC_URL", "http://192.168.50.49:8788").rstrip("/") PUBLIC_URL = os.getenv("ASR_PUBLIC_URL", "http://192.168.50.49:8788").rstrip("/")
AGENT_TOKEN = os.getenv("ASR_AGENT_TOKEN", "") AGENT_TOKEN = os.getenv("ASR_AGENT_TOKEN", "")
DIARIZE_URL = os.getenv("ASR_DIARIZE_URL", "http://192.168.50.100:10099/asr") WEB_PASSWORD = os.getenv("ASR_WEB_PASSWORD", "")
DIARIZE_URL = os.getenv("ASR_DIARIZE_URL", CFG.diarize_url)
DEFAULT_PROVIDER = os.getenv("ASR_PROVIDER", CFG.asr_provider)
MAX_UPLOAD = int(os.getenv("ASR_UPLOAD_MAX_BYTES", str(4 * 1024**3))) MAX_UPLOAD = int(os.getenv("ASR_UPLOAD_MAX_BYTES", str(4 * 1024**3)))
POLL_SECONDS = max(1, int(os.getenv("ASR_POLL_SECONDS", "5"))) POLL_SECONDS = max(1, int(os.getenv("ASR_POLL_SECONDS", "5")))
STABLE_SECONDS = max(1, int(os.getenv("ASR_STABLE_SECONDS", "10"))) STABLE_SECONDS = max(1, int(os.getenv("ASR_STABLE_SECONDS", "10")))
@@ -27,8 +31,13 @@ MAX_ATTEMPTS = max(1, int(os.getenv("ASR_MAX_ATTEMPTS", "3")))
def ensure_dirs() -> None: def ensure_dirs() -> None:
for path in (INBOX, PROCESSING, PROCESSED, FAILED, TRANSCRIPTS, DRAFTS, # Keep asrpipe CFG aligned with container/service paths when running under asrservice
RESULTS, STATE, LOGS, META, REVISION_QUEUE, DATA / "cache", CFG.state_dir
DATA / "artifacts" / "raw", DATA / "vault" / "minutes", for path in (
DATA / "vault" / "People", DATA / "vault" / "Companies"): INBOX, PROCESSING, PROCESSED, FAILED, TRANSCRIPTS, DRAFTS,
RESULTS, STATE, LOGS, META, REVISION_QUEUE, DATA / "cache",
DATA / "artifacts" / "raw", DATA / "artifacts" / "results",
DATA / "vault" / "minutes", DATA / "vault" / "People",
DATA / "vault" / "Companies", DATA / "vault" / "Projects",
):
path.mkdir(parents=True, exist_ok=True) path.mkdir(parents=True, exist_ok=True)
+148 -22
View File
@@ -1,17 +1,22 @@
from __future__ import annotations from __future__ import annotations
import json
import sqlite3 import sqlite3
import time import time
from typing import Any from typing import Any
from asrpipe.config import CFG, bootstrap_default_settings, get_setting, load_db_settings, set_settings
from .config import DB, ensure_dirs from .config import DB, ensure_dirs
STATUSES = {"uploading", "queued", "transcribing", "refining", "review", STATUSES = {
"publishing", "complete", "failed"} "uploading", "queued", "transcribing", "refining", "review",
"publishing", "complete", "failed",
}
_BOOTSTRAPPED_DB: str = ""
def connect() -> sqlite3.Connection: def connect() -> sqlite3.Connection:
global _BOOTSTRAPPED_DB
ensure_dirs() ensure_dirs()
con = sqlite3.connect(DB, timeout=30) con = sqlite3.connect(DB, timeout=30)
con.row_factory = sqlite3.Row con.row_factory = sqlite3.Row
@@ -26,6 +31,8 @@ def connect() -> sqlite3.Connection:
draft TEXT NOT NULL DEFAULT '', result TEXT NOT NULL DEFAULT '', draft TEXT NOT NULL DEFAULT '', result TEXT NOT NULL DEFAULT '',
source TEXT NOT NULL DEFAULT 'web', sha256 TEXT NOT NULL DEFAULT '', source TEXT NOT NULL DEFAULT 'web', sha256 TEXT NOT NULL DEFAULT '',
attempts INTEGER NOT NULL DEFAULT 0, error_code TEXT NOT NULL DEFAULT '', attempts INTEGER NOT NULL DEFAULT 0, error_code TEXT NOT NULL DEFAULT '',
asr_provider TEXT NOT NULL DEFAULT 'qwen3cam',
asr_model TEXT NOT NULL DEFAULT '',
created_at REAL NOT NULL, updated_at REAL NOT NULL created_at REAL NOT NULL, updated_at REAL NOT NULL
); );
CREATE TABLE IF NOT EXISTS job_event( CREATE TABLE IF NOT EXISTS job_event(
@@ -34,37 +41,73 @@ def connect() -> sqlite3.Connection:
status TEXT NOT NULL, detail TEXT NOT NULL DEFAULT '', created_at REAL NOT NULL status TEXT NOT NULL, detail TEXT NOT NULL DEFAULT '', created_at REAL NOT NULL
); );
CREATE INDEX IF NOT EXISTS ix_job_event_job ON job_event(job_id,id); CREATE INDEX IF NOT EXISTS ix_job_event_job ON job_event(job_id,id);
CREATE TABLE IF NOT EXISTS app_setting(
key TEXT PRIMARY KEY,
value TEXT NOT NULL DEFAULT '',
updated_at REAL NOT NULL
);
""") """)
cols = {r[1] for r in con.execute("PRAGMA table_info(intake_job)")} cols = {r[1] for r in con.execute("PRAGMA table_info(intake_job)")}
default_prov = CFG.asr_provider or "qwen3cam"
if default_prov not in {"qwen3cam", "soniox", "gemini", "openai", "grok"}:
default_prov = "qwen3cam"
additions = { additions = {
"draft": "TEXT NOT NULL DEFAULT ''", "result": "TEXT NOT NULL DEFAULT ''", "draft": "TEXT NOT NULL DEFAULT ''",
"source": "TEXT NOT NULL DEFAULT 'web'", "sha256": "TEXT NOT NULL DEFAULT ''", "result": "TEXT NOT NULL DEFAULT ''",
"attempts": "INTEGER NOT NULL DEFAULT 0", "error_code": "TEXT NOT NULL DEFAULT ''", "source": "TEXT NOT NULL DEFAULT 'web'",
"sha256": "TEXT NOT NULL DEFAULT ''",
"attempts": "INTEGER NOT NULL DEFAULT 0",
"error_code": "TEXT NOT NULL DEFAULT ''",
"asr_provider": f"TEXT NOT NULL DEFAULT '{default_prov}'",
"asr_model": "TEXT NOT NULL DEFAULT ''",
} }
for name, ddl in additions.items(): for name, ddl in additions.items():
if name not in cols: if name not in cols:
con.execute(f"ALTER TABLE intake_job ADD COLUMN {name} {ddl}") con.execute(f"ALTER TABLE intake_job ADD COLUMN {name} {ddl}")
con.commit() con.commit()
if _BOOTSTRAPPED_DB != str(DB):
_BOOTSTRAPPED_DB = str(DB)
try:
bootstrap_default_settings()
except Exception:
pass
return con return con
def create_job(job_id: str, original_name: str, stored_name: str, *, title: str = "", def create_job(
context: str = "", size: int = 0, source: str = "web") -> dict[str, Any]: job_id: str,
original_name: str,
stored_name: str,
*,
title: str = "",
context: str = "",
size: int = 0,
source: str = "web",
asr_provider: str = "",
asr_model: str = "",
) -> dict[str, Any]:
now = time.time() now = time.time()
prov = (asr_provider or CFG.asr_provider or "qwen3cam").strip().lower()
with connect() as con: with connect() as con:
con.execute("INSERT INTO intake_job(id,original_name,stored_name,title,context,size," con.execute(
"status,source,created_at,updated_at) VALUES(?,?,?,?,?,?,?,?,?,?)", "INSERT INTO intake_job(id,original_name,stored_name,title,context,size,"
(job_id, original_name, stored_name, title, context, size, "status,source,asr_provider,asr_model,created_at,updated_at) "
"queued", source, now, now)) "VALUES(?,?,?,?,?,?,?,?,?,?,?,?)",
con.execute("INSERT INTO job_event(job_id,status,detail,created_at) VALUES(?,?,?,?)", (job_id, original_name, stored_name, title, context, size,
(job_id, "queued", "录音已进入队列", now)) "queued", source, prov, asr_model, now, now))
con.execute(
"INSERT INTO job_event(job_id,status,detail,created_at) VALUES(?,?,?,?)",
(job_id, "queued", f"录音已进入队列 (ASR: {prov})", now))
return get_job(job_id) or {} return get_job(job_id) or {}
def update_job(job_id: str, status: str, detail: str = "", **fields: Any) -> bool: def update_job(job_id: str, status: str, detail: str = "", **fields: Any) -> bool:
if status not in STATUSES: if status not in STATUSES:
raise ValueError(f"invalid status: {status}") raise ValueError(f"invalid status: {status}")
allowed = {"transcript", "draft", "result", "sha256", "attempts", "error_code"} allowed = {
"transcript", "draft", "result", "sha256", "attempts",
"error_code", "asr_provider", "asr_model", "title", "context", "stored_name",
}
changes = {k: v for k, v in fields.items() if k in allowed and v is not None} changes = {k: v for k, v in fields.items() if k in allowed and v is not None}
sets = ["status=?", "updated_at=?", "detail=?"] sets = ["status=?", "updated_at=?", "detail=?"]
values: list[Any] = [status, time.time(), detail[:1000]] values: list[Any] = [status, time.time(), detail[:1000]]
@@ -75,11 +118,50 @@ def update_job(job_id: str, status: str, detail: str = "", **fields: Any) -> boo
with connect() as con: with connect() as con:
cur = con.execute(f"UPDATE intake_job SET {','.join(sets)} WHERE id=?", values) cur = con.execute(f"UPDATE intake_job SET {','.join(sets)} WHERE id=?", values)
if cur.rowcount: if cur.rowcount:
con.execute("INSERT INTO job_event(job_id,status,detail,created_at) VALUES(?,?,?,?)", con.execute(
(job_id, status, detail[:1000], time.time())) "INSERT INTO job_event(job_id,status,detail,created_at) VALUES(?,?,?,?)",
(job_id, status, detail[:1000], time.time()))
return bool(cur.rowcount) return bool(cur.rowcount)
def update_job_meta(
job_id: str,
*,
title: str | None = None,
context: str | None = None,
asr_provider: str | None = None,
asr_model: str | None = None,
) -> bool:
sets = ["updated_at=?"]
values: list[Any] = [time.time()]
if title is not None:
sets.append("title=?")
values.append(title[:120])
if context is not None:
sets.append("context=?")
values.append(context[:2000])
if asr_provider is not None:
sets.append("asr_provider=?")
values.append(asr_provider[:40])
if asr_model is not None:
sets.append("asr_model=?")
values.append(asr_model[:80])
values.append(job_id)
with connect() as con:
cur = con.execute(f"UPDATE intake_job SET {','.join(sets)} WHERE id=?", values)
return bool(cur.rowcount)
def delete_job(job_id: str) -> dict[str, Any] | None:
job = get_job(job_id)
if not job:
return None
with connect() as con:
con.execute("DELETE FROM job_event WHERE job_id=?", (job_id,))
con.execute("DELETE FROM intake_job WHERE id=?", (job_id,))
return job
def get_job(job_id: str) -> dict[str, Any] | None: def get_job(job_id: str) -> dict[str, Any] | None:
with connect() as con: with connect() as con:
row = con.execute("SELECT * FROM intake_job WHERE id=?", (job_id,)).fetchone() row = con.execute("SELECT * FROM intake_job WHERE id=?", (job_id,)).fetchone()
@@ -88,18 +170,33 @@ def get_job(job_id: str) -> dict[str, Any] | None:
def find_by_stored(stored_name: str) -> dict[str, Any] | None: def find_by_stored(stored_name: str) -> dict[str, Any] | None:
with connect() as con: with connect() as con:
row = con.execute("SELECT * FROM intake_job WHERE stored_name=? ORDER BY created_at DESC LIMIT 1", row = con.execute(
(stored_name,)).fetchone() "SELECT * FROM intake_job WHERE stored_name=? ORDER BY created_at DESC LIMIT 1",
(stored_name,)).fetchone()
return dict(row) if row else None return dict(row) if row else None
def list_jobs(limit: int = 30, offset: int = 0, status: str = "") -> list[dict[str, Any]]: def list_jobs(
limit: int = 30,
offset: int = 0,
status: str = "",
q: str = "",
provider: str = "",
) -> list[dict[str, Any]]:
limit = min(max(limit, 1), 200) limit = min(max(limit, 1), 200)
offset = max(offset, 0) offset = max(offset, 0)
sql, args = "SELECT * FROM intake_job", [] sql = "SELECT * FROM intake_job WHERE 1=1"
args: list[Any] = []
if status: if status:
sql += " WHERE status=?" sql += " AND status=?"
args.append(status) args.append(status)
if provider:
sql += " AND asr_provider=?"
args.append(provider)
if q.strip():
pat = f"%{q.strip()}%"
sql += " AND (title LIKE ? OR original_name LIKE ? OR context LIKE ? OR detail LIKE ?)"
args.extend([pat, pat, pat, pat])
sql += " ORDER BY created_at DESC LIMIT ? OFFSET ?" sql += " ORDER BY created_at DESC LIMIT ? OFFSET ?"
args.extend((limit, offset)) args.extend((limit, offset))
with connect() as con: with connect() as con:
@@ -118,3 +215,32 @@ def pending_recovery() -> list[dict[str, Any]]:
return [dict(r) for r in con.execute( return [dict(r) for r in con.execute(
"SELECT * FROM intake_job WHERE status IN ('queued','transcribing','refining','publishing') " "SELECT * FROM intake_job WHERE status IN ('queued','transcribing','refining','publishing') "
"ORDER BY created_at").fetchall()] "ORDER BY created_at").fetchall()]
def get_all_settings() -> dict[str, str]:
connect().close()
return load_db_settings(force=True)
def save_settings(updates: dict[str, Any]) -> dict[str, str]:
connect().close()
set_settings(updates)
return load_db_settings(force=True)
__all__ = [
"STATUSES",
"connect",
"create_job",
"update_job",
"update_job_meta",
"delete_job",
"get_job",
"find_by_stored",
"list_jobs",
"events",
"pending_recovery",
"get_all_settings",
"save_settings",
"get_setting",
]
@@ -2,4 +2,6 @@ import time
from .config import STATE from .config import STATE
heartbeat = STATE / "worker.heartbeat" heartbeat = STATE / "worker.heartbeat"
raise SystemExit(0 if heartbeat.exists() and time.time() - heartbeat.stat().st_mtime < 90 else 1) raise SystemExit(
0 if heartbeat.exists() and time.time() - heartbeat.stat().st_mtime < 90 else 1
)
@@ -15,21 +15,28 @@ def send(status: str, filename: str = "", detail: str = "") -> bool:
chat = os.getenv("TELEGRAM_CHAT_ID", "") chat = os.getenv("TELEGRAM_CHAT_ID", "")
if not token or not chat: if not token or not chat:
return False return False
body = parse.urlencode({"chat_id": chat, body = parse.urlencode({
"text": "\n".join(x for x in (status, filename, detail) if x)}).encode() "chat_id": chat,
req = request.Request(f"https://api.telegram.org/bot{token}/sendMessage", "text": "\n".join(x for x in (status, filename, detail) if x),
data=body, method="POST") }).encode()
req = request.Request(
f"https://api.telegram.org/bot{token}/sendMessage",
data=body, method="POST")
elif backend == "hermes": elif backend == "hermes":
url = os.getenv("HERMES_WEBHOOK_URL", "") url = os.getenv("HERMES_WEBHOOK_URL", "")
secret = os.getenv("HERMES_WEBHOOK_SECRET", "") secret = os.getenv("HERMES_WEBHOOK_SECRET", "")
if not url or not secret: if not url or not secret:
return False return False
body = json.dumps({"status": status, "filename": filename, "detail": detail}, body = json.dumps(
ensure_ascii=False).encode() {"status": status, "filename": filename, "detail": detail},
ensure_ascii=False).encode()
sig = hmac.new(secret.encode(), body, hashlib.sha256).hexdigest() sig = hmac.new(secret.encode(), body, hashlib.sha256).hexdigest()
req = request.Request(url, data=body, method="POST", req = request.Request(
headers={"Content-Type": "application/json", url, data=body, method="POST",
"X-Webhook-Signature": sig}) headers={
"Content-Type": "application/json",
"X-Webhook-Signature": sig,
})
else: else:
return False return False
with request.urlopen(req, timeout=10) as response: with request.urlopen(req, timeout=10) as response:
File diff suppressed because it is too large Load Diff
@@ -1,24 +1,28 @@
from __future__ import annotations from __future__ import annotations
import fcntl
import hashlib import hashlib
import json import json
import os import os
import shutil import shutil
import sqlite3 import sqlite3
import subprocess import threading
import time import time
import uuid import uuid
from pathlib import Path from pathlib import Path
from asrpipe import refine, review, revise, vault from asrpipe import graph, lexicon, providers, refine, review, revise, vault
from asrpipe.config import CFG
from asrpipe.locking import acquire_recording_lock
from . import db from . import db
from .config import (DIARIZE_URL, DRAFTS, FAILED, INBOX, MAX_ATTEMPTS, META, from .config import (
POLL_SECONDS, PROCESSED, PROCESSING, PUBLIC_URL, DATA, DRAFTS, FAILED, INBOX, LOGS, MAX_ATTEMPTS,
REVISION_QUEUE, STABLE_SECONDS, STATE, SUPPORTED, TRANSCRIPTS, LOGS, POLL_SECONDS, PROCESSED, PROCESSING, PUBLIC_URL, REVISION_QUEUE,
ensure_dirs) STABLE_SECONDS, STATE, SUPPORTED, TRANSCRIPTS, ensure_dirs,
)
from .notify import send from .notify import send
_RECORDING_LOCK = None
def log(message: str) -> None: def log(message: str) -> None:
ensure_dirs() ensure_dirs()
@@ -35,69 +39,108 @@ def sha256(path: Path) -> str:
def transcript_from_response(payload: dict) -> str: def transcript_from_response(payload: dict) -> str:
segments = payload.get("segments") """Backward-compatible helper for qwen3cam JSON responses."""
if not isinstance(segments, list) or not segments: return providers.transcript_from_qwen3cam_response(payload)
raise RuntimeError("ASR response contains no segments")
vp = payload.get("voiceprint") or {}
sid = str(vp.get("session_id") or "")
matched = [f"{m.get('speaker_local')}={m.get('person_name')}" for m in vp.get("matches", [])
if m.get("status") == "matched"]
speakers = {str(s.get("speaker")) for s in segments if s.get("speaker") is not None}
lines = [f"<!-- voiceprint_session: {sid} clusters: {len(speakers)} -->",
"# 转写(Qwen3-ASR 1.7B + CAM++ 分角色 + 声纹)"
+ (" | 声纹命名: " + ", ".join(matched) if matched else ""), ""]
turns: list[list[str]] = []
for segment in segments:
speaker, text = str(segment.get("speaker", "?")), str(segment.get("text") or "").strip()
if not text:
continue
if turns and turns[-1][0] == speaker:
turns[-1][1] += text
else:
turns.append([speaker, text])
lines.extend(f"【{speaker}】{text}\n" for speaker, text in turns)
return "\n".join(lines).strip() + "\n"
def transcribe(audio: Path, job_id: str) -> Path: def transcribe(
audio: Path,
job_id: str,
*,
provider: str = "",
model: str = "",
title: str = "",
user_context: str = "",
) -> tuple[Path, providers.TranscribeResult]:
res = providers.transcribe_audio(
audio,
provider=provider or CFG.asr_provider,
model=model,
title=title,
user_context=user_context,
)
response_path = TRANSCRIPTS / f"{job_id}.asr.json" response_path = TRANSCRIPTS / f"{job_id}.asr.json"
result = subprocess.run([
"curl", "-sS", "--fail-with-body", "--max-time", "900",
"-F", f"file=@{audio}", "-F", "language=Chinese",
"-F", "save_voiceprint_session=true", DIARIZE_URL,
], capture_output=True, timeout=930)
if result.returncode:
raise RuntimeError(f"ASR HTTP failed ({result.returncode})")
response_path.write_bytes(result.stdout)
try: try:
payload = json.loads(result.stdout) response_path.write_text(
except ValueError as exc: json.dumps(res.raw_response, ensure_ascii=False, indent=2),
raise RuntimeError("ASR returned invalid JSON") from exc encoding="utf-8",
)
except Exception:
pass
transcript = TRANSCRIPTS / f"{job_id}.txt" transcript = TRANSCRIPTS / f"{job_id}.txt"
transcript.write_text(transcript_from_response(payload), encoding="utf-8") transcript.write_text(res.transcript_text, encoding="utf-8")
return transcript return transcript, res
def intake_header(payload: dict) -> str:
blob = json.dumps(payload, ensure_ascii=False).replace("--", "- -")
return f"<!-- intake: {blob} -->"
def replace_intake(raw: str, header: str) -> str:
if raw.startswith("<!-- intake:"):
_first, _sep, rest = raw.partition("\n")
return header + ("\n" + rest if _sep else "")
return header + "\n" + raw
def beat() -> None:
ensure_dirs()
(STATE / "worker.heartbeat").write_text(str(time.time()))
def _beat_loop(stop: threading.Event) -> None:
while not stop.wait(15):
try:
beat()
except Exception:
pass
def process_audio(source: Path, job: dict) -> None: def process_audio(source: Path, job: dict) -> None:
job_id = job["id"] job_id = job["id"]
attempts = int(job.get("attempts") or 0) + 1 attempts = int(job.get("attempts") or 0) + 1
prov = providers.normalize_provider_name(job.get("asr_provider") or CFG.asr_provider)
model = str(job.get("asr_model") or "")
digest = sha256(source) digest = sha256(source)
db.update_job(job_id, "transcribing", "正在进行分角色转写", sha256=digest, prov_label = providers.PROVIDERS.get(prov, {}).get("label", prov)
attempts=attempts, error_code="") db.update_job(
transcript = transcribe(source, job_id) job_id, "transcribing", f"正在转写 ({prov_label})",
intake = {"job_id": job_id, "title": job.get("title", ""), sha256=digest, attempts=attempts, error_code="",
"context": job.get("context", ""), "source": job.get("source", "")} asr_provider=prov, asr_model=model,
)
transcript, res = transcribe(
source, job_id,
provider=prov, model=model,
title=str(job.get("title") or ""),
user_context=str(job.get("context") or ""),
)
intake = {
"job_id": job_id,
"title": job.get("title", ""),
"context": job.get("context", ""),
"source": job.get("source", ""),
"asr_provider": res.provider,
"asr_model": res.model,
}
raw = transcript.read_text(encoding="utf-8") raw = transcript.read_text(encoding="utf-8")
raw = f"<!-- intake: {json.dumps(intake, ensure_ascii=False)} -->\n" + raw raw = replace_intake(raw, intake_header(intake))
transcript.write_text(raw, encoding="utf-8") transcript.write_text(raw, encoding="utf-8")
db.update_job(job_id, "refining", "转写完成,正在生成整理稿", transcript=str(transcript)) db.update_job(
result = refine.refine(raw, date=time.strftime("%Y-%m-%d"), job_id, "refining", f"转写完成 ({res.provider}:{res.model}),正在生成整理稿",
transcript_path=str(transcript), queue_review=True) transcript=str(transcript), asr_provider=res.provider, asr_model=res.model,
)
result = refine.refine(
raw, date=time.strftime("%Y-%m-%d"),
transcript_path=str(transcript), queue_review=True,
)
draft = DRAFTS / f"{job_id}.md" draft = DRAFTS / f"{job_id}.md"
vault.write_draft_file(draft, result.draft) vault.write_draft_file(draft, result.draft)
questions = review.enqueue_draft(str(transcript), draft) questions = review.enqueue_draft(str(transcript), draft)
db.update_job(job_id, "review", f"整理稿已生成,{questions} 个问题待确认", db.update_job(
transcript=str(transcript), draft=str(draft)) job_id, "review", f"整理稿已生成,{questions} 个问题待确认",
transcript=str(transcript), draft=str(draft),
)
target = PROCESSED / source.name target = PROCESSED / source.name
if target.exists(): if target.exists():
target = PROCESSED / f"{source.stem}-{job_id[:8]}{source.suffix}" target = PROCESSED / f"{source.stem}-{job_id[:8]}{source.suffix}"
@@ -106,16 +149,59 @@ def process_audio(source: Path, job: dict) -> None:
f"{PUBLIC_URL}/jobs/{job_id}/review/") f"{PUBLIC_URL}/jobs/{job_id}/review/")
def rerefine_job(job: dict) -> None:
"""Re-run LLM refinement on an existing raw transcript without re-running ASR."""
job_id = job["id"]
transcript = Path(str(job.get("transcript") or ""))
if not transcript.is_file():
raise FileNotFoundError(f"transcript file missing for job {job_id}: {transcript}")
raw = transcript.read_text(encoding="utf-8", errors="ignore")
intake = {
"job_id": job_id,
"title": job.get("title", ""),
"context": job.get("context", ""),
"source": job.get("source", ""),
"asr_provider": job.get("asr_provider") or CFG.asr_provider,
"asr_model": job.get("asr_model") or "",
}
raw = replace_intake(raw, intake_header(intake))
transcript.write_text(raw, encoding="utf-8")
db.update_job(job_id, "refining", "正在基于现有转写重新生成精修草稿")
result = refine.refine(
raw, date=time.strftime("%Y-%m-%d"),
transcript_path=str(transcript), queue_review=True,
)
draft = DRAFTS / f"{job_id}.md"
vault.write_draft_file(draft, result.draft)
review.delete_pending_for_job(
transcript_path=str(transcript),
draft_path=str(job.get("draft") or draft),
job_id=job_id,
)
questions = review.enqueue_draft(str(transcript), draft)
db.update_job(
job_id, "review", f"整理稿已重新生成,{questions} 个问题待确认",
transcript=str(transcript), draft=str(draft),
)
def claim(path: Path) -> tuple[Path, dict]: def claim(path: Path) -> tuple[Path, dict]:
job = db.find_by_stored(path.name) job = db.find_by_stored(path.name)
if not job: if not job:
job_id = uuid.uuid4().hex[:24] job_id = uuid.uuid4().hex[:24]
job = db.create_job(job_id, path.name, path.name, title=path.stem, job = db.create_job(
size=path.stat().st_size, source="ssh") job_id, path.name, path.name, title=path.stem,
size=path.stat().st_size, source="ssh",
asr_provider=CFG.asr_provider,
)
target = PROCESSING / path.name target = PROCESSING / path.name
if target.exists(): if target.exists():
target = PROCESSING / f"{path.stem}-{job['id'][:8]}{path.suffix}" target = PROCESSING / f"{path.stem}-{job['id'][:8]}{path.suffix}"
path.replace(target) path.replace(target)
if target.name != job.get("stored_name"):
db.update_job(job["id"], job["status"], job.get("detail") or "已认领", stored_name=target.name)
job = db.get_job(job["id"]) or job
return target, job return target, job
@@ -131,47 +217,137 @@ def scan_inbox() -> None:
process_audio(source, job) process_audio(source, job)
except Exception as exc: except Exception as exc:
attempts = int((db.get_job(job["id"]) or {}).get("attempts") or 0) attempts = int((db.get_job(job["id"]) or {}).get("attempts") or 0)
retry = attempts < MAX_ATTEMPTS is_empty_audio = isinstance(exc, providers.EmptyAudioError)
db.update_job(job["id"], "queued" if retry else "failed", retry = (attempts < MAX_ATTEMPTS) and not is_empty_audio
"处理失败,等待重试" if retry else "处理失败", db.update_job(
error_code=type(exc).__name__) job["id"], "queued" if retry else "failed",
log(f"job={job['id']} failed type={type(exc).__name__}") f"处理失败,等待重试 ({exc})" if retry else f"处理失败:{exc}",
target = INBOX / source.name if retry else FAILED / source.name error_code=type(exc).__name__,
if source.exists(): )
shutil.move(str(source), target) log(f"job={job['id']} failed type={type(exc).__name__} retry={retry} err={exc}")
try:
if source.exists():
folder = INBOX if retry else FAILED
target = folder / source.name
if target.exists():
target = folder / f"{source.stem}-{job['id'][:8]}{source.suffix}"
shutil.move(str(source), target)
if retry:
old = time.time() - max(STABLE_SECONDS, 60)
os.utime(target, (old, old))
except Exception as move_exc:
log(f"job={job['id']} audio left in processing after failure: {move_exc}")
def _defer_revision(marker: Path, payload: dict, exc: BaseException) -> None:
attempts = int(payload.get("attempts") or 0) + 1
payload["attempts"] = attempts
payload["last_error"] = str(exc)[:300]
job_id = str(payload.get("job_id") or "")
if attempts >= MAX_ATTEMPTS:
marker.unlink(missing_ok=True)
if job_id:
db.update_job(job_id, "failed", f"修订失败:{exc}", error_code=type(exc).__name__)
log(f"revision marker={marker.name} gave up after {attempts}: {exc}")
return
tmp = marker.with_name(f".{marker.name}.{os.getpid()}.tmp")
tmp.write_text(json.dumps(payload, ensure_ascii=False), encoding="utf-8")
tmp.replace(marker)
if job_id:
db.update_job(
job_id, "review",
f"修订失败,将重试 ({attempts}/{MAX_ATTEMPTS}):{exc}",
error_code=type(exc).__name__,
)
log(f"revision marker={marker.name} retry {attempts}/{MAX_ATTEMPTS}: {exc}")
def process_revisions() -> None: def process_revisions() -> None:
for marker in sorted(REVISION_QUEUE.glob("*.json")): for marker in sorted(REVISION_QUEUE.glob("*.json")):
if marker.name.startswith("."):
continue
try: try:
payload = json.loads(marker.read_text(encoding="utf-8")) payload = json.loads(marker.read_text(encoding="utf-8"))
draft = Path(payload["draft"]) except Exception as exc:
log(f"revision marker={marker.name} unreadable: {exc}")
_defer_revision(marker, {}, exc)
continue
try:
action = str(payload.get("action") or "revise")
job_id = str(payload.get("job_id") or "") job_id = str(payload.get("job_id") or "")
if action == "rerefine":
job = db.get_job(job_id)
if job:
rerefine_job(job)
marker.unlink(missing_ok=True)
continue
draft = Path(payload["draft"])
db.update_job(job_id, "publishing", "正在应用人工确认") db.update_job(job_id, "publishing", "正在应用人工确认")
revise.revise(draft) revise.revise(draft)
current = db.get_job(job_id) or {} current = db.get_job(job_id) or {}
if current.get("status") == "complete": if current.get("status") == "complete":
send("ASR 正式记录已生成", current.get("title") or current.get("original_name", ""), send("ASR 正式记录已生成",
current.get("title") or current.get("original_name", ""),
f"{PUBLIC_URL}/jobs/{job_id}/") f"{PUBLIC_URL}/jobs/{job_id}/")
elif current.get("status") != "review": elif current.get("status") != "review":
db.update_job(job_id, "review", "定稿预览已生成,等待发布确认", draft=str(draft)) db.update_job(
job_id, "review", "定稿预览已生成,等待发布确认",
draft=str(draft),
)
marker.unlink(missing_ok=True) marker.unlink(missing_ok=True)
except Exception as exc: except Exception as exc:
log(f"revision marker={marker.name} failed type={type(exc).__name__}") _defer_revision(marker, payload, exc)
def _park_audio(path: Path) -> Path:
dest = INBOX / path.name
if dest.exists():
dest = INBOX / f"{path.stem}-recovered{path.suffix}"
shutil.move(str(path), dest)
old = time.time() - max(STABLE_SECONDS, 60)
os.utime(dest, (old, old))
return dest
def recover() -> None: def recover() -> None:
for job in db.pending_recovery(): """Return audio left in processing/ to the inbox and requeue its job."""
if job["status"] in {"transcribing", "refining"}: if not PROCESSING.exists():
candidate = next((p for p in PROCESSING.iterdir() if p.name == job["stored_name"]), None) return
if candidate: files = [p for p in PROCESSING.iterdir() if p.is_file()]
shutil.move(str(candidate), INBOX / candidate.name) jobs = db.pending_recovery()
db.update_job(job["id"], "queued", "worker 重启后重新排队") claimed: set[str] = set()
for path in files:
match = None
for job in jobs:
if job["id"] in claimed:
continue
stored = str(job.get("stored_name") or "")
if path.name == stored or (job["id"][:8] and job["id"][:8] in path.name):
match = job
break
dest = _park_audio(path)
if match:
claimed.add(match["id"])
db.update_job(
match["id"], "queued", "worker 重启后重新排队",
stored_name=dest.name, error_code="",
)
log(f"recovered job={match['id']} audio={dest.name}")
else:
log(f"recovered orphan audio={dest.name}")
def sync_vault_knowledge() -> None:
try:
vault.pull()
lexicon.sync_from_vault()
graph.sync()
except Exception as exc:
log(f"vault knowledge sync skipped: {exc}")
def backup_databases() -> None: def backup_databases() -> None:
# STATE is local disk; durable copies belong on the NFS data volume. backup_dir = DATA / "backups"
backup_dir = Path("/data/backups")
backup_dir.mkdir(parents=True, exist_ok=True) backup_dir.mkdir(parents=True, exist_ok=True)
for name in ("intake.db", "lexicon.db"): for name in ("intake.db", "lexicon.db"):
source = STATE / name source = STATE / name
@@ -184,22 +360,46 @@ def backup_databases() -> None:
temp.replace(target) temp.replace(target)
def main() -> None: def run_loop() -> None:
os.environ.setdefault("ASR_ROLE", "worker")
os.environ.setdefault("ASR_REVISION_QUEUE_DIR", str(REVISION_QUEUE))
ensure_dirs() ensure_dirs()
lock = (STATE / "worker.lock").open("w") global _RECORDING_LOCK
fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB) lock_path = Path(os.getenv("ASR_CONSUMER_LOCK", str(STATE / "consumer.lock")))
_RECORDING_LOCK = acquire_recording_lock(lock_path)
recover() recover()
(STATE / "worker.heartbeat").write_text(str(time.time())) sync_vault_knowledge()
beat()
stop_beat = threading.Event()
threading.Thread(target=_beat_loop, args=(stop_beat,), name="asr-heartbeat", daemon=True).start()
last_backup = 0.0 last_backup = 0.0
last_vault_sync = time.time()
while True: while True:
scan_inbox() try:
process_revisions() scan_inbox()
if time.time() - last_backup >= 3600: except Exception as exc:
backup_databases() log(f"scan_inbox failed: {exc}")
last_backup = time.time() try:
(STATE / "worker.heartbeat").write_text(str(time.time())) process_revisions()
except Exception as exc:
log(f"process_revisions failed: {exc}")
now = time.time()
if now - last_vault_sync >= 900:
sync_vault_knowledge()
last_vault_sync = now
if now - last_backup >= 3600:
try:
backup_databases()
except Exception as exc:
log(f"backup failed: {exc}")
last_backup = now
beat()
time.sleep(POLL_SECONDS) time.sleep(POLL_SECONDS)
def main() -> None:
run_loop()
if __name__ == "__main__": if __name__ == "__main__":
main() main()
+19
View File
@@ -0,0 +1,19 @@
#!/usr/bin/env bash
# Standalone ASR post-processing entrypoint.
#
# Independent of Claude Code / opencode / any agent runtime: it uses its own venv
# and talks to the model provider over plain HTTPS. Safe to call from systemd,
# cron, or a shell that has nothing but bash.
set -euo pipefail
ROOT="$(cd "$(dirname "$(readlink -f "$0")")/.." && pwd)"
PY="$ROOT/.venv/bin/python"
if [ ! -x "$PY" ]; then
echo "venv missing at $PY" >&2
echo "create it with:" >&2
echo " uv venv $ROOT/.venv --python 3.13" >&2
echo " uv pip install --python $PY pypinyin rapidfuzz numpy" >&2
exit 2
fi
export PYTHONPATH="$ROOT${PYTHONPATH:+:$PYTHONPATH}"
export PYTHONUNBUFFERED=1
exec "$PY" -m asrpipe.cli "$@"
+26 -13
View File
@@ -1,11 +1,24 @@
x-asr-environment: &asr-environment x-asr-environment: &asr-environment
ZENMUX_API_KEY: ${ZENMUX_API_KEY:-} ASR_PROVIDER: ${ASR_PROVIDER:-qwen3cam}
ASR_DRAFT_MODEL: ${ASR_DRAFT_MODEL:-anthropic/claude-sonnet-5.5}
ASR_AGENT_TOKEN: ${ASR_AGENT_TOKEN:-}
ASR_PUBLIC_URL: ${ASR_PUBLIC_URL:-http://192.168.50.49:8788}
ASR_DIARIZE_URL: ${ASR_DIARIZE_URL:-http://192.168.50.100:10099/asr} ASR_DIARIZE_URL: ${ASR_DIARIZE_URL:-http://192.168.50.100:10099/asr}
ASR_VOICEPRINT_ENROLL_URL: ${ASR_VOICEPRINT_ENROLL_URL:-http://192.168.50.100:10099/voiceprints/enroll}
LITELLM_BASE_URL: ${LITELLM_BASE_URL:-http://192.168.50.49:4000/v1}
LITELLM_API_KEY: ${LITELLM_API_KEY:-}
ASR_GEMINI_MODEL: ${ASR_GEMINI_MODEL:-aistudio/gemini-3.5-transcribe}
ASR_OPENAI_MODEL: ${ASR_OPENAI_MODEL:-openai/gpt-4o-transcribe-diarize}
SONIOX_API_KEY: ${SONIOX_API_KEY:-}
SONIOX_MODEL: ${SONIOX_MODEL:-stt-async-v5}
XAI_API_KEY: ${XAI_API_KEY:-}
ASR_GROK_MODEL: ${ASR_GROK_MODEL:-grok-voice-transcribe-2.0}
ASR_CLEAN_MODEL: ${ASR_CLEAN_MODEL:-aistudio/gemini-flash-lite-latest}
ASR_KEYWORD_MODEL: ${ASR_KEYWORD_MODEL:-aistudio/gemini-flash-lite-latest}
ASR_DRAFT_MODEL: ${ASR_DRAFT_MODEL:-codex/gpt-sol-latest}
ASR_AGENT_TOKEN: ${ASR_AGENT_TOKEN:-}
ASR_WEB_PASSWORD: ${ASR_WEB_PASSWORD:-}
ASR_PUBLIC_URL: ${ASR_PUBLIC_URL:-http://192.168.50.49:8788}
HINDSIGHT_API: ${HINDSIGHT_API:-http://192.168.50.47:8888} HINDSIGHT_API: ${HINDSIGHT_API:-http://192.168.50.47:8888}
HINDSIGHT_BANK: ${HINDSIGHT_BANK:-personal-context} HINDSIGHT_BANK: ${HINDSIGHT_BANK:-personal-context}
ASR_GIT_SYNC: ${ASR_GIT_SYNC:-0}
ASR_NOTIFY_BACKEND: ${ASR_NOTIFY_BACKEND:-none} ASR_NOTIFY_BACKEND: ${ASR_NOTIFY_BACKEND:-none}
HERMES_WEBHOOK_URL: ${HERMES_WEBHOOK_URL:-} HERMES_WEBHOOK_URL: ${HERMES_WEBHOOK_URL:-}
HERMES_WEBHOOK_SECRET: ${HERMES_WEBHOOK_SECRET:-} HERMES_WEBHOOK_SECRET: ${HERMES_WEBHOOK_SECRET:-}
@@ -14,8 +27,8 @@ x-asr-environment: &asr-environment
services: services:
asr-web: asr-web:
image: tools-asr-pipeline:2026-09-18 image: ${ASR_IMAGE:-tools-asr-pipeline:latest}
pull_policy: never pull_policy: ${ASR_PULL_POLICY:-never}
build: build:
context: . context: .
dockerfile: Dockerfile dockerfile: Dockerfile
@@ -25,10 +38,10 @@ services:
<<: *asr-environment <<: *asr-environment
ASR_ROLE: web ASR_ROLE: web
ports: ports:
- "192.168.50.49:8788:8788" - "${ASR_BIND_ADDRESS:-192.168.50.49}:${ASR_PORT:-8788}:8788"
volumes: volumes:
- /mnt/truenas/personal/ASR-Pipeline:/data - ${ASR_DATA_VOLUME:-/mnt/truenas/personal/ASR-Pipeline}:/data
- /opt/asr-pipeline/state:/state - ${ASR_STATE_VOLUME:-/opt/asr-pipeline/state}:/state
healthcheck: healthcheck:
test: ["CMD", "curl", "-fsS", "http://127.0.0.1:8788/health"] test: ["CMD", "curl", "-fsS", "http://127.0.0.1:8788/health"]
interval: 30s interval: 30s
@@ -39,8 +52,8 @@ services:
- no-new-privileges:true - no-new-privileges:true
asr-worker: asr-worker:
image: tools-asr-pipeline:2026-09-18 image: ${ASR_IMAGE:-tools-asr-pipeline:latest}
pull_policy: never pull_policy: ${ASR_PULL_POLICY:-never}
container_name: tools-asr-worker container_name: tools-asr-worker
restart: unless-stopped restart: unless-stopped
environment: environment:
@@ -48,8 +61,8 @@ services:
ASR_ROLE: worker ASR_ROLE: worker
command: ["python", "-m", "asrservice.worker"] command: ["python", "-m", "asrservice.worker"]
volumes: volumes:
- /mnt/truenas/personal/ASR-Pipeline:/data - ${ASR_DATA_VOLUME:-/mnt/truenas/personal/ASR-Pipeline}:/data
- /opt/asr-pipeline/state:/state - ${ASR_STATE_VOLUME:-/opt/asr-pipeline/state}:/state
healthcheck: healthcheck:
test: ["CMD", "python", "-m", "asrservice.healthcheck"] test: ["CMD", "python", "-m", "asrservice.healthcheck"]
interval: 30s interval: 30s
@@ -0,0 +1,16 @@
# 现网路径快照。模板参数 %I 与旧 asr-refine 路径保持原样,不要在本仓库直接启用。
[Unit]
Description=Prepare or publish Web-reviewed ASR draft %I
After=network-online.target asr-review-web.service
Wants=network-online.target
[Service]
Type=oneshot
WorkingDirectory=/home/claw/claude-workspace/OBSIDIAN/asr-pipeline
EnvironmentFile=/home/claw/claude-workspace/OBSIDIAN/bin/.asr-refine.env
Environment=PYTHONDONTWRITEBYTECODE=1
Environment=PATH=/usr/local/bin:/usr/bin:/bin
ExecStart=/home/claw/claude-workspace/OBSIDIAN/asr-pipeline/bin/asr-refine revise %I
TimeoutStartSec=900
UMask=0077
MemoryMax=512M
@@ -0,0 +1,31 @@
# 现网路径快照。ExecStart / EnvironmentFile / 工作目录保持原样,不要在本仓库直接启用。
[Unit]
Description=Persistent ASR recording intake and review UI
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
WorkingDirectory=/home/claw/claude-workspace/OBSIDIAN/asr-pipeline
Environment=PYTHONDONTWRITEBYTECODE=1
Environment=PATH=/usr/local/bin:/usr/bin:/bin
ExecStart=/home/claw/claude-workspace/OBSIDIAN/asr-pipeline/.venv/bin/python -m asrpipe.control_web
Environment=PYTHONPATH=/home/claw/claude-workspace/OBSIDIAN/asr-pipeline
Environment=ASR_RECORDINGS_ROOT=/home/claw/claude-workspace/OBSIDIAN/recordings
Restart=on-failure
RestartSec=3
TimeoutStopSec=10
UMask=0077
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=read-only
ReadWritePaths=/home/claw/claude-workspace/OBSIDIAN/asr-pipeline/state
ReadWritePaths=/home/claw/claude-workspace/OBSIDIAN/logs
ReadWritePaths=/home/claw/claude-workspace/OBSIDIAN/recordings/inbox
ReadWritePaths=/home/claw/claude-workspace/OBSIDIAN/recordings/state
MemoryMax=192M
TasksMax=32
[Install]
WantedBy=default.target
@@ -0,0 +1,15 @@
# 现网路径快照。EnvironmentFile 与 ExecStart 指向旧目录,保持原样,不要在本仓库直接启用。
[Unit]
Description=Refine new ASR transcripts into Obsidian drafts (standalone, no agent CLI)
After=network-online.target
[Service]
Type=oneshot
# Credentials only. The program needs no agent CLI, no login state, no OAuth —
# just this token and a network route. Token rotation = edit this one file.
EnvironmentFile=/home/claw/claude-workspace/OBSIDIAN/bin/.asr-refine.env
Environment=PATH=/usr/local/bin:/usr/bin:/bin
ExecStart=/home/claw/claude-workspace/OBSIDIAN/asr-pipeline/bin/asr-refine run-all
# A long meeting is ~90 chunks; S2 is parallel but S4 is one serial strong-model
# call, so allow a generous window for a batch of transcripts.
TimeoutStartSec=3600
@@ -0,0 +1,14 @@
[Unit]
Description=Watch Obsidian .asr-inbox for new finished transcripts
[Path]
# Fires when a finished transcript file is mv'd into .asr-inbox/ (dir mtime changes).
# Markers under .state/ and staging in .incoming/ live in subdirs, so they don't refire.
PathModified=/home/claw/obsidian-vault/.asr-inbox
# 2026-07-25: switched from obsidian-asr-refine.service (headless `claude -p`) to the
# standalone pipeline. The old unit + obsidian-asr-refine.sh are kept intact; to roll
# back, point Unit= back at obsidian-asr-refine.service and daemon-reload.
Unit=obsidian-asr-refine-native.service
[Install]
WantedBy=default.target
@@ -0,0 +1,20 @@
# 现网路径快照。ExecStart 与 MEETING_ASR_BACKEND 保持原样,不要在本仓库直接启用。
[Unit]
Description=Obsidian Meeting Recording Watcher (audio inbox -> ASR -> bridge to .asr-inbox)
After=network-online.target
[Service]
Type=simple
ExecStart=/usr/bin/python3 /home/claw/claude-workspace/OBSIDIAN/recordings/bin/meeting-watch.py
Restart=always
RestartSec=10
WorkingDirectory=/home/claw/claude-workspace/OBSIDIAN/recordings
Environment=PYTHONUNBUFFERED=1
Environment=MEETING_INBOX_BASE=/home/claw/claude-workspace/OBSIDIAN/recordings
Environment=MEETING_WEBHOOK_URL=http://127.0.0.1:8644/webhooks/meeting-recording
# ASR backend trial (2026-06-26): route live pipeline through VibeVoice 分角色.
# Revert: remove this line (or set =qwen3) + daemon-reload + restart.
Environment=MEETING_ASR_BACKEND=qwen3cam
[Install]
WantedBy=default.target
@@ -0,0 +1,190 @@
# ASR Agent 解耦实施项目(v2)
日期:2026-09-18。状态:代码核查后的实施计划,尚未执行生产改造。
上游:[耦合 Review](agent-decoupling-review.md)、[v1 实施文档](agent-decoupling-implementation.md)。本文件保留原文作为历史依据,修订执行设计和验收方式;实际开发以当前代码和本计划为准。本文中的工作项不是已完成变更,也不以旧文档的“已授权”文字代替后续执行请求。
## 1. 项目目标与交付边界
让录音监听与 ASR 管线通过同一份本地通知接口发送状态;目标地址、密钥和后端由独立配置控制。核心整理程序继续使用 HTTPS 模型 API 和文件工作流,不依赖 Claude、Hermes、Codex 的运行时、登录态或私有配置。
首期交付:唯一发送实现、通知参数纠错、完整日志脱敏、两个 Python 调用方接入、兼容旧 shell CLI、隔离测试、部署/回滚记录、跨 agent 协议文档。通知目标尚未核实不阻塞代码交付,但“接口改造完成”与“真实消息送达”必须分别报告。
后续独立项目:watcher 与 intake 数据库解耦、通用 agent skill 包装、Telegram direct 后端、STT wrapper 命名迁移。首期不修改五阶段算法、模型、fidelity gate、Vault 内容、Hermes 配置、历史备份或 rollback 链路,不新增第三方依赖。
## 2. 代码核查结果与 v1 修订
下列行号对应 2026-09-18 核查快照,实施时以函数名定位。路径相对 `OBSIDIAN/`。
| 证据 | 结论及实施影响 |
|---|---|
| `bin/meeting-status-notify.py:17–19,22–48` | 地址写死、运行时读取 Hermes 私有订阅;与 watcher 重复签名发送。应收敛为唯一模块。 |
| `asr-pipeline/asrpipe/vault.py:126–140` | status 被写死为“草稿已生成,待确认”;缺脚本分支直接记录 body,异常分支原样记录异常,可能包含带正文的 argv。修参数时须同时修所有脱敏分支。 |
| `recordings/bin/meeting-watch.py:177–216` | `post_webhook()` 实际被 `post_status()` 调用;死的是默认 `meeting-recording` 路由,不是整个函数无人调用。先替换状态路径,再删通用发送函数。 |
| `recordings/bin/meeting-watch.py:142` | `hashlib.sha256()` 还用于音频 ID,删除通知签名后不能顺手删 `hashlib`。 |
| `asr-pipeline/asrpipe/cli.py:136–147` | dry-run 提前返回,不经过通知分支,不能证明通知降级正常。 |
| `asr-pipeline/asrpipe/cli.py:191–248` | doctor 会同步词库、访问 Hindsight 和调用模型;不作为通知重构的必跑验收。 |
| `asr-pipeline/tests/test_core.py:11–14` | 已隔离 state/vault/runtime,但未隔离 ASR_LOG;新测试必须补齐日志、通知配置和网络隔离。 |
| `asr-pipeline/asrpipe/control_web.py:341–367,457–464` | 有查询/上传/审核接口,没有 watcher 注册或状态更新 API。不能将“统一走现有 HTTP API”作为可直接执行的替换指令。 |
| `asr-pipeline/asrpipe/review_web.py:30–56` | 当前 request_start 接入常驻 control_web;旧 capability 页面逻辑也仍在文件中。协议必须区分真实生产入口与兼容路径,不能把所有审核 URL 都称为一次性地址。 |
| `recordings/bin/meeting-watch.py` 的 `process_one()` / `main()` | 先移动到 processing 再等待转写,启动仅扫 inbox;直接 restart 可能中断转写并留下孤立文件。部署必须排空任务。 |
其他修订:
- watcher unit 的 `MEETING_WEBHOOK_URL` 是旧正文路由的死配置,并非当前状态通知的第三个有效 URL 来源;仍应清理,避免继续误导。
- v1 建议保留失效地址并默认 webhook;v2 改为新模板默认 `none`,不在可执行默认值里保留已知失效 endpoint。已有部署配置必须先备份、显式迁移,不静默覆盖。
- v1 空密钥允许不签名;v2 webhook 缺有效密钥时降级为配置错误且不发送,避免将审核链接投往未认证的目标。未来需要匿名 webhook 时另定义显式协议。
- “通知失败不阻塞”指不影响业务成败;同步 HTTP 仍有等待。本期保留有界等待,不声称实现了异步或可靠消息队列。
- 旧文档的 `.48` SSH/端口结论属于历史记录;本轮没有重新探测远端,也未读取凭证。新地址、密钥和投递结果仍需部署时核实。
- 递归搜索只限活跃源文件及相关 unit,排除 `.venv`、备份和其他服务;不能把整个用户目录中出现 8644 或 HMAC 都认定为本项目失败。
## 3. 目标接口与配置决策
### 3.1 单一实现与加载方式
以 `OBSIDIAN/bin/meeting-status-notify.py` 为唯一配置解析、消息序列化、HMAC 和 HTTP 实现,不再另建插件框架或共享包。该文件必须能被 import,顶层不读取密钥、不发请求,CLI 放在 main guard 内。
建议接口:
```python
def send(status: str, filename: str = "", detail: str = "", *,
backend: str | None = None) -> bool:
...
```
前三个位置参数保持兼容;可选 backend 供 `CFG.notify_backend` 显式覆盖使用。只在对端返回 2xx 时返回 True;静音、配置错误和投递失败均返回 False,并用无敏感信息的原因码区分。函数捕获普通 Exception,不捕获 KeyboardInterrupt/SystemExit。CLI 仍接收 `(status, filename, detail)`,投递失败保持退出码 0;CLI 成功退出不代表送达成功。
两个调用方使用 `importlib.util.spec_from_file_location` 从绝对路径加载该模块,避免 sys.path 全局改动和引入管线依赖。vault 沿用 `CFG.notifier`;watcher 从 `__file__` 的项目目录定位。模块可缓存,配置每次 send 重新读取,配置修改无需重启;首次加载失败不永久缓存失败状态。加载与调用均置于降级保护中。
### 3.2 配置契约
生产文件 `OBSIDIAN/bin/notify-target.json`(0600),另新增不含真实凭证的 `notify-target.example.json`。配置更新采用同目录 0600 临时文件、写入后原子替换;不把密钥放进命令行、终端输出或文档。源码备份和凭证备份分开记录,凭证备份同样 0600,后续纳入版本控制时排除生产配置。
```json
{
"backend": "none",
"webhook": {
"url": "",
"secret": "",
"timeout": 10
}
}
```
| 变量 / 字段 | 语义与优先级 |
|---|---|
| `MEETING_NOTIFY_CONFIG` | 配置路径覆盖;默认脚本同目录的 notify-target.json,不能依赖 cwd。 |
| `send(..., backend=...)` | 非空显式值最高优先;vault 从 `CFG.notify_backend` 传入。 |
| `ASR_NOTIFY_BACKEND` | 两调用方共用的 backend 环境覆盖;其后依次为 JSON、内置 none。 |
| `MEETING_STATUS_WEBHOOK_URL` | 非空值覆盖 JSON URL,保留 watcher 既有状态 URL 配置能力。 |
| `MEETING_STATUS_WEBHOOK_SECRET` | 非空值覆盖 JSON secret;新增状态专用变量,不复用旧正文路由密钥变量。 |
| `ASR_NOTIFY=0` | 保持既有 asrpipe 总开关语义,不扩展为 watcher 的隐式总开关;全部静音用 backend=none。 |
| `webhook.timeout` | 有限正数,约束在 0 < timeout <= 10 秒;非法值不发送。urllib 超时不是全链路严格墙钟上限,尤其 DNS 阶段,文档不得作更强保证。 |
未知 backend、损坏 JSON、非对象配置、无 URL/密钥、非法 timeout:安全原因码 + False,无网络请求。`none` 不要求 webhook 字段有效。backend=webhook 仅支持明确的 HTTP(S) URL,不把旧 `INSECURE_NO_AUTH` 当作有效共享密钥。环境变量只作为有记录的部署覆盖;正常运行不在各 unit 重复配置 endpoint。
HMAC 保持现有线协议:UTF-8 JSON,字段 `status/filename/detail`,`ensure_ascii=False`,对实际发送的同一份 bytes 做 SHA256 HMAC,放入 `X-Webhook-Signature`。拒绝自动重定向,避免把正文转发到其他目标;非 2xx 均判失败。无自动重试,不承诺 exactly-once,也不引入后台线程。
### 3.3 调用语义与日志
- `vault.notify(title, body)` → `send(status=title, filename="", detail=body)`;不要把 title 再塞进 filename。
- watcher 的 `post_status(status, filename, detail="")` 保持三个调用点形式和字段含义。
- legacy shell 仍用原三个位置参数运行脚本。
- `redact_log=True` 时,成功、静音、缺文件、加载失败、配置错误、网络异常各路径都不能出现 body。保护范围包括 stdout/stderr、HTTP 响应正文和原始异常字符串;发送模块只记录安全原因码/HTTP 状态,不记录 payload、密钥、目标完整 URL 或响应正文。
- 建议发送层统一不记录正文,vault 对非敏感正文是否保留既有日志由最小变更原则决定;敏感正文标记必须在进入任何日志分支前计算。
- 配置缺失/静音必须可诊断,但不改变录音、转写、草稿或发布成功状态。
## 4. 实施工作包与依赖
| 工作包 | 交付物 | 依赖 / 完成门槛 |
|---|---|---|
| W0:基线与备份 | 活跃文件清单、隔离测试记录、文件哈希与带时间戳备份清单 | 开始前确认无 Git 保护;源码、unit、文档修改前逐项备份。 |
| W1:唯一发送实现 | notify 脚本、示例配置、临时配置测试 | W0;none/签名/失败/CLI/脱敏全部通过,不触达真实目标。 |
| W2:调用方迁移 | vault.py、config.py、meeting-watch.py、必要的 doctor 静态检查调整 | W1;5 个 vault 调用点、3 个 watcher 调用点及 legacy CLI 兼容性通过。 |
| W3:协议与索引 | agent-protocol.md、HANDOFF/CLAUDE/skill 引用更新 | W2;区分当前与计划接口,移除过时排障入口的误导。 |
| W4:部署与回滚演练 | unit 去旧变量、空闲窗口记录、进程重载验证、迁移报告 | W1–W3 全部离线通过;配置初始 none,业务服务健康。 |
| W5:真实通知恢复 | 核实目标与项目自持密钥后的配置更新、实际投递记录 | 已具备真实目标及发测试消息的明确授权;不以代码测试冒充真实验收。 |
建议未来并行执行:通知 agent 负责 W1 和发送层测试;调用方 agent 负责 W2 与调用方测试;文档 agent 负责 W3;集成者负责 W0/W4/W5。W1/W2 先冻结第 3 节契约,避免同时修改同一测试文件。当前本轮子智能体仅完成只读探索,未执行这些开发包。
W2 特别要求:删除 watcher 的六个 webhook 常量及私有订阅读取,移除不再需要的 hmac/urllib 导入,保留用于文件 ID 的 hashlib、用于元数据的 json。保留 `CFG.notify`、`CFG.notifier`;新增 `CFG.notify_backend` 并明确传递,避免“声明配置字段却没生效”。doctor 的 notifier 存在性可保留,但不得把文件存在描述为通知链路健康。
## 5. 离线测试与验收矩阵
优先新增 `tests/test_notify.py` 与 watcher 适配测试,保留既有 unittest 风格。测试导入前设置临时 ASR_STATE_DIR、ASR_VAULT、ASR_LOG、XDG_RUNTIME_DIR、MEETING_INBOX_BASE、ASR_CONTROL_DB 和 MEETING_NOTIFY_CONFIG;清理继承的通知覆盖变量,禁止实际 urlopen、模型调用和 systemctl。测试不读取生产 notify-target.json 或 Hermes 配置。不要用修改生产 backend 的方式进行验收。
| 测试组 | 必须断言 |
|---|---|
| 参数 | 所有 5 处 vault 事件保留真实 status,filename 为空、detail 保留正文;watcher 3 个事件字段不变。 |
| 配置 | env > JSON > 默认;显式 CFG backend 传递;none、缺文件、坏 JSON、未知 backend、非法 timeout、缺密钥不发请求。 |
| 协议 | 中文 payload、准确 HMAC 字节与 header、POST/Content-Type、2xx 成功、3xx 不跟随、4xx/5xx 失败。 |
| 降级 | URLError、超时、import 失败、缺脚本、send 意外异常均不传到业务调用方。 |
| 脱敏 | 用唯一假 capability URL 作标记,遍历全部分支;标记不得出现在应用日志、stdout、stderr、异常回显中。 |
| 兼容 | CLI 0/1/2/3 参数默认值和旧 shell 三参数调用;投递失败 exit 0,但 send 返回 False。 |
| 配置重载 | 同一已加载模块中原子换临时配置,下次 send 使用新值;失败加载不会永久缓存。 |
| watcher | import 不启动循环;post_status 调用唯一模块;计算文件 ID 等非通知行为保持原状。 |
本轮实测基线:额外设置 `PYTHONDONTWRITEBYTECODE=1`、临时 `ASR_LOG` / `ASR_CACHE_DIR`、`ASR_NOTIFY=0`、`ASR_GIT_SYNC=0` 后,结果为:
```text
Ran 28 tests in 0.122s
OK
```
新增测试还应验证不同 cwd、未设置 PYTHONPATH 时两个入口的加载方式。统一 bootstrap 必须先于任何 `asrpipe` 导入,避免 unittest discovery 顺序导致单例 CFG 捕获另一个测试模块的环境;测试 backend 开关本身时在 mock transport 内局部解除总开关。
测试命令应在一个由测试运行器统一创建的临时环境下执行:
```bash
PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=. .venv/bin/python -m unittest discover -s tests -v
```
上面命令只是入口,隔离环境由运行器或测试 fixture 事先提供,不能裸命令后假定生产日志也已隔离。最终报告保留真实 `Ran N tests ... OK`,不硬编码最终数量。附加静态检查:`bash -n` 检查旧 shell;编译改动 Python 时把 pycache 定向临时目录;限定目标文件检查无 Hermes 私有配置读取、无旧 route 默认值、签名只保留一份。
doctor 和完整 dry-run 均不列入本项目必需验收:前者有数据库写和联网行为,后者不覆盖通知且会调用模型。若另需全流程集成测试,使用合成录音/转写和隔离状态目录,单独记录目的与副作用。
## 6. 部署与回滚
1. 在隔离副本完成代码和测试,部署前重新检查磁盘、备份、unit 与 drop-in 的实际生效设置;不输出含凭证的整个环境。旧报告中的 active/inactive 和队列为空只是一时快照。
2. 暂停新上传/投递的窗口内,确认 inbox/processing、watcher 子进程及 native/revise 作业没有在处理任务。先排空再停止相关进程;只看到 processing 为空不等于没有入队竞态。
3. 若 processing 已有孤立文件,记录并按现有恢复流程处理,不自动移动回 inbox,避免重复转写。未排空则推迟部署,不使用强制 restart。
4. 部署已验证代码和权限为 0600 的本地配置;首次无真实目标时保持 none,并在交付记录明确“通知尚未恢复”。生产凭证一次性迁移时只复制所需字段,不输出密钥;源文件缺失则不阻塞代码部署。
5. watcher unit 删除旧 `MEETING_WEBHOOK_URL`,保留 `MEETING_INBOX_BASE`、`MEETING_ASR_BACKEND` 和既有回滚注释;检查 drop-in 是否仍覆盖旧变量。daemon-reload 后在空闲窗口启动 watcher。
6. 检查常驻 control_web 是否已加载改动模块,安排在无审核提交、无正在上传的请求及 `.uploading/*.part` 活跃写入时重启以加载新代码;oneshot 下一次启动自然加载。不能假定仅重启 watcher 会更新全部 Python 进程。无需改 review/revise unit 的模型凭证设置。
7. 只读检查目标服务 active、进程入口和脱敏日志;真实消息测试留给 W5。新地址更新只更换独立配置,不再改源码/unit。
回滚按同一空闲流程恢复成套文件(发送脚本、调用方、unit、配置),daemon-reload 并启动相关服务;记录恢复的哈希和权限。不要回滚生产数据库或手工复制运行中 SQLite 来“恢复状态”;本期不改变其 schema。旧版本通知本来可能不可用,回滚恢复的是业务行为,不能宣称修复消息投递。源码备份不得清理原有历史 `.bak.*`。
## 7. 协议文档与通用 agent 接入
`docs/agent-protocol.md` 作为通知和 agent 接入契约来源;HANDOFF 保留管线架构,control-web.md 保留现有 HTTP 接口说明。其他索引链接到这些来源,不再复制整套阶段表。
协议至少写清:send/CLI 参数、配置优先级、送达与静音语义、故障定位、敏感日志规则、真实现有的查询/上传路由,以及“审核回答由人完成,agent 不代填确认”。当前 HTTP 服务没有通用公网认证,不能描述为可直接暴露的 agent API;旧一次性页面与当前任务页的边界须准确。
更新 `OBSIDIAN/CLAUDE.md` 与 `HANDOFF.md` 的通知说明;HANDOFF 仍有“粗稿 commit/push”和旧临时 Web 生命周期描述,核实后标注历史或改为当前行为,保留算法历史数据。`~/.claude/skills/memo-inbox/references/webhook-toolset-diagnostics.md` 标历史归档并指向协议;修改前读该 skill,去重其他引用时保留“先粗整理、人工确认后正式发布”等纪律。修改 skill 必须在报告中列明。
文档同步还必须覆盖下列 v1 漏项:
- `~/.claude/skills/memo-inbox/SKILL.md:51–52,217–221`:替换 active 的 `claude -p → minutes/_drafts` 描述及旧 webhook-agent/toolset 必查入口。
- `~/.claude/skills/memo-inbox/references/meeting-recording-inbox.md`:更新或明确归档旧 `.hermes-meetings`、正文 webhook 和私有配置排障路径。
- `~/.claude/skills/memo-inbox/references/asr-refine-pipeline.md:16–20`:修正粗稿入 Vault、commit/push、Mac 审阅的旧时序,保留该技能独有的人工确认纪律。
首期交付跨 agent 可直接读取的协议,不把“创建 Markdown”冒充“已安装通用 skill”。如另做通用 skill,先使用 skill-creator,做薄入口引用上述契约,不复制业务规则、不读取私有 agent 状态、不授予自动审核或发消息的新权限。
## 8. 后续项目:任务状态与数据库解耦
先盘点 watcher、cli、review_web、control_web 的全部 intake_job 读写,明确哪些是同一服务的内部实现、哪些是真正跨进程契约;不要把所有内部函数调用都 HTTP 化。
优先评估提取唯一 intake store/schema 模块,由本机组件共享,先去重复 DDL;如确需 watcher 进程与控制服务独立部署,再引入版本化内部 API。后者必须先设计注册、状态迁移、幂等键、重复/乱序更新、认证和服务不可用时的持久重试。候选路由是未来设计,不是现有接口。网页服务中断不能新增为转录中断条件;未解决断网状态恢复前不移除 watcher 的当前写入路径。
验收至少覆盖:Web 上传与直接投文件、重复 fid、控制服务暂时不可用、转录/桥接失败、进程重启恢复、草稿/发布状态关联、禁止旧状态覆盖新状态。该项目单独评审和交付,不与首期通知上线绑定。
## 9. 完成定义与执行报告
- 代码完成:唯一发送实现、参数正确、完整脱敏、原调用契约兼容、隔离回归通过。
- 部署完成:已排空并加载新代码,配置权限正确,服务健康,有可执行的成套回滚记录。
- 通知恢复:已核实真实 endpoint/共享密钥,并在获得测试消息发送授权后确认端到端投递;只有 HTTP 2xx 时记录“后端接受”,未确认接收端则不写“用户已收到”。
- 待决策:真实 webhook 地址/密钥来源;是否另开 direct 后端项目。direct 不等价于自动复用 Hermes bot token 的授权。
报告列出改动与备份路径、测试原文、参数及脱敏测试名称、部署窗口、未覆盖项、真实通知状态和 skill 变更。对不满足的门槛明确标为未完成,不用“重构完成”掩盖投递尚未恢复。
@@ -0,0 +1,363 @@
# ASR 管线 Agent 解耦 — 重构实施文档
**版本**:v1(2026-09-18)
**上游 review**:`docs/agent-decoupling-review.md`
**执行者**:子 agent(Claude Code / Codex / 其他具备文件与 shell 能力的 agent)
**授权边界**:本文档 Stage 1–4 为已授权范围。超出范围的改动(尤其是 §8 禁止项)必须先回报 Kai 并取得确认。
---
## 0. 给执行 agent 的前置须知(先读完本节再动手)
### 0.1 这套系统正在生产运行,且无 git 保护
`/home/claw/claude-workspace` **不是 git 仓库**(已验证:`git rev-parse` 返回 `fatal: not a git repository`)。这意味着:
- **没有 `git checkout` 可以回滚**。任何改动前必须手动备份。
- 备份约定沿用本项目既有风格(目录中已有 7 个 `*.bak.*` 文件):
```bash
cp -a <file> <file>.bak.$(date +%Y%m%d-%H%M%S)
```
- 磁盘余量充足(43G available),备份成本可忽略。
同时注意:`~/obsidian-vault` **是**独立 git 仓库,管线会向它 commit+push。本次重构**不应触碰 vault 内容**,只改 `OBSIDIAN/` 下的基础设施代码与 systemd unit。
### 0.2 当前 baseline(改动前请先自行复现,作为对照)
```bash
cd /home/claw/claude-workspace/OBSIDIAN/asr-pipeline
PYTHONPATH=. .venv/bin/python -m unittest discover -s tests -v
# 期望:Ran 28 tests ... OK
```
注意:venv 里**没有 pytest**,必须用 `unittest`。这 28 个测试全部离线(`test_core.py` 在 import 前就把 `ASR_STATE_DIR`/`ASR_VAULT` 重定向到 tempdir),可以安全反复跑。
**任何 stage 结束时这 28 个测试必须仍然全绿**,且新增功能需补充测试(见各 stage 的验收标准)。
### 0.3 当前运行状态(改动前实测,供对照)
| 单元 | 状态 | 说明 |
|---|---|---|
| `obsidian-recording-watch.service` | active | 音频监听 + 转录 + 桥接 |
| `obsidian-asr-refine.path` | active | 监听 `.asr-inbox/` |
| `asr-review-web.service` | active | Web 审核/上传 UI,`.145:8765` |
| `obsidian-asr-refine.service` | inactive | 旧 `claude -p` 链路,**保留供 rollback** |
`127.0.0.1:8644` 已确认无监听(`ss -tlnp | grep 8644` 为空)。`.48` ICMP 可达,但 8644 等端口从 `.145` 探测为 closed/filtered,且 **SSH 到 `.48` 被拒(publickey)**——执行 agent 无法自行登录 `.48` 查看其配置。**这是本次重构的关键约束,见 §1.3。**
### 0.4 上游 review 的一处补充(执行时必须一并修掉)
在准备本实施文档时复核代码,发现 review 未记录的一个**真实 bug**,与本次解耦同属一处代码,应合并修复:
`asrpipe/vault.py:135` 把第一个位置参数写死成字符串 `"草稿已生成,待确认"`:
```python
r = subprocess.run(["python3", str(n), "草稿已生成,待确认", title, body], ...)
```
而 `meeting-status-notify.py` 的 argv 契约是 `(status, filename, detail)`。于是所有 5 处 `vault.notify()` 调用的语义都被错位丢弃:
| 调用点 | 调用方本意(第 1 参数) | 实际发出的 status |
|---|---|---|
| `revise.py:273` | "ASR 正式记录已发布" | 草稿已生成,待确认 ❌ |
| `cli.py:109` | "ASR Web 审核启动失败" | 草稿已生成,待确认 ❌ |
| `cli.py:160` | "ASR Web 审核启动失败" | 草稿已生成,待确认 ❌ |
| `review_web.py:49` | "ASR 快速确认" | 草稿已生成,待确认 ❌ |
| `review_web.py:271` | "ASR 快速确认" | 草稿已生成,待确认 ❌ |
即:**"发布成功"和"启动失败"在 Telegram 上都显示为"草稿已生成,待确认"**。调用方传的 `title` 被挤到 `filename` 位、`body` 被挤到 `detail` 位。这个 bug 目前被 8644 不通掩盖着(消息根本没发出去),一旦按本文档恢复通知,它会立刻变成可见的错误告警。**Stage 2 必须连带修掉。**
---
## 1. 重构目标与设计约束
### 1.1 目标
把"管线向人(Kai)发通知"这件事,从**"调用 Hermes 网关某个 IP:port 上的私有 webhook"**,改造成**"调用一个稳定的本地通知接口,Hermes 只是它的一种后端"**。
判据(重构完成后应当成立):
1. Hermes 再次换主机/换端口/换协议时,**只改一个配置文件,不改任何一行 Python、不改任何 systemd unit**。
2. 通知目标不可用时,管线行为与今天一致:**记日志、继续跑、不阻塞、不丢数据**。
3. 通知发送逻辑(HMAC 签名 + POST)在代码库里**只有一份实现**。
4. 管线**不再读取 `~/.hermes/` 下的任何文件**。
5. 上述 4 条对两个调用方(`meeting-watch.py` 和 `asrpipe`)同时成立。
### 1.2 设计风格必须与现有代码一致
这个项目有很清晰的既有风格,重构**必须沿用**而不是另立一套:
- 配置集中在 `asrpipe/config.py` 的 `Config` dataclass,每项都是 `_env("ASR_XXX", default)` 形式,env 可覆盖(见该文件已有的 `use_termbank` / `use_hindsight` / `gate_enabled` 等)。
- 失败降级优先于抛异常:通知失败只 `log(..., level="WARN")`,绝不中断管线(`vault.py:139` 已是此风格,`meeting-status-notify.py` 的 docstring 也明确写了 "Silent no-op (exit 0) ... never block the pipeline")。
- 注释说明"为什么"而非"是什么",且保留实测数据(参考 `config.py` 里 clean_model 的选型注释、`llm.py` 的 temperature 注释)。中文注释与英文注释在本项目中混用,**按所改文件的既有语言为准**。
- 凭证走 600 权限的独立 env 文件,不进代码、不进 git(`bin/.asr-refine.env` 是既有先例)。
### 1.3 关键约束:`.48` 的新地址当前未知
执行 agent **无法** SSH 到 `.48`,因此**无法自行确定 Hermes webhook 的新地址**。
这不阻塞重构——恰恰相反,这正是要做接口层的理由。处理方式:
- Stage 1–3 的全部工作**不需要知道 `.48` 的地址**,只需要把地址变成"一个配置文件里的一个字段"。
- 配置文件的初始值填 `.145` 的旧地址(即当前的 `http://127.0.0.1:8644/...`),并在文件里注释清楚"此地址在 2026-09-15 Hermes 迁移至 .48 后已失效,待 Kai 提供新地址后更新此处"。
- 重构完成后,**由 Kai 填入 `.48` 的真实地址**,或按 Stage 4 的 `direct` 后端绕过 Hermes。
- **禁止猜测端口后写进配置当作"已修复"**。宁可保留失效地址 + 明确注释,也不要写一个看似能跑实际错误的值。
---
## 2. 实施总览
| Stage | 内容 | 依赖 | 是否需要 Kai 输入 |
|---|---|---|---|
| **1** | 建立通知配置层 + 单一发送实现 | — | 否 |
| **2** | 两个调用方收敛到该实现 + 修 argv 错位 bug | 1 | 否 |
| **3** | 清理 systemd 硬编码 + 文档同步 | 2 | 否 |
| **4** | (可选)`direct` 后端,彻底不依赖 Hermes | 3 | 是(需 token 授权决定) |
Stage 1–3 应**一镜到底连续执行**,中间不要停下来问;每个 stage 结束跑一次测试即可。Stage 4 涉及凭证使用决策,**必须停下来问 Kai**。
---
## 3. Stage 1 — 建立通知配置层与单一发送实现
### 3.1 新增配置文件 `OBSIDIAN/bin/notify-target.json`
这是本次重构的**唯一真相源**。结构建议(执行时可按需微调字段名,但必须保持"后端可切换 + 目标可配置"两个语义):
```json
{
"backend": "webhook",
"_comment": "backend: webhook | none | direct(见 Stage 4)。切换后端不需要改代码。",
"webhook": {
"url": "http://127.0.0.1:8644/webhooks/meeting-recording-status",
"_url_comment": "⚠️ 此地址在 2026-09-15 Hermes 由 .145 迁移到 .48 后已失效。待 Kai 提供 .48 上的新地址后更新此字段。保留失效值而非猜测端口,是为了让故障显式可见。",
"secret": "",
"_secret_comment": "HMAC-SHA256 共享密钥。留空则不签名(对端若校验会拒绝)。历史上此值取自 ~/.hermes/webhook_subscriptions.json 的 meeting-recording-status.secret;现在改为本项目自持,不再读 Hermes 私有配置。",
"timeout": 10
}
}
```
权限必须是 600(含密钥):
```bash
chmod 600 /home/claw/claude-workspace/OBSIDIAN/bin/notify-target.json
```
**迁移密钥**:从 `~/.hermes/webhook_subscriptions.json` 的 `meeting-recording-status.secret` 复制一次到上面的 `secret` 字段。这是**一次性拷贝**,之后管线不再读 Hermes 的文件。(该文件当前结构已确认为 `{route: {secret, deliver, deliver_extra:{chat_id}, ...}}`。)
### 3.2 改造 `OBSIDIAN/bin/meeting-status-notify.py` 为唯一实现
保持它现有的**双重身份**——既是 CLI(供 shell 脚本调用),也要能被 Python import(供 `asrpipe` 直接调用,省掉一次 subprocess)。要求:
- 读 `notify-target.json` 决定后端与目标;**删除**对 `~/.hermes/webhook_subscriptions.json` 的读取(`SUBSCRIPTIONS` 常量与 `secret()` 函数)。
- 保留 env 覆盖能力,优先级 `env > notify-target.json > 内置兜底`。env 变量名沿用既有前缀风格(如 `MEETING_STATUS_WEBHOOK_URL` 已在 `meeting-watch.py` 中使用,继续复用同名,不要另起新名)。
- `backend: "none"` 时直接记日志返回,不发网络请求——这让"临时静音"不需要改代码,也让测试可以完全离线。
- 暴露一个可 import 的函数,签名保持与 CLI 契约一致:
```python
def send(status: str, filename: str = "", detail: str = "") -> bool
```
返回是否送达成功;**内部吞掉所有异常**,绝不向调用方抛(维持"never block the pipeline")。
- `__main__` 分支保持 `argv = (status, filename, detail)` 契约不变——`bin/obsidian-asr-refine.sh:45` 仍以此方式调用它(旧 rollback 链路),**不要破坏该契约**。
### 3.3 验收标准
```bash
cd /home/claw/claude-workspace/OBSIDIAN
# 1. backend=none 时静默成功、不发网络请求
python3 - <<'EOF'
import json, pathlib
p = pathlib.Path("bin/notify-target.json")
d = json.loads(p.read_text()); d["backend"] = "none"
p.write_text(json.dumps(d, ensure_ascii=False, indent=2))
EOF
python3 bin/meeting-status-notify.py "测试" "f.mp3" "detail" # 期望:退出码 0,日志说明 backend=none
# 2. backend=webhook 且目标不通时,仍然退出码 0(不阻塞)
# (把 backend 改回 webhook 后重跑,期望打印类似 "skipped: Connection refused",退出码仍为 0)
echo "exit=$?"
# 3. 确认不再引用 Hermes 私有配置
grep -rn "webhook_subscriptions\|\.hermes" bin/meeting-status-notify.py
# 期望:无输出(注释里提及历史来源可以,但不能有真实读取)
# 4. 旧链路 CLI 契约未破坏
bash -n bin/obsidian-asr-refine.sh && echo "refine.sh syntax ok"
```
---
## 4. Stage 2 — 两个调用方收敛 + 修 argv 错位
### 4.1 `asrpipe` 侧
**`asrpipe/config.py`**:
- 新增 `notify_backend` 字段,风格与既有项一致:`notify_backend: str = _env("ASR_NOTIFY_BACKEND", "")`(空 = 交由 `notify-target.json` 决定;显式设置则覆盖)。
- 保留现有 `notify: bool`(`ASR_NOTIFY=0` 的总开关)——这是既有行为,`doctor` 和文档都依赖它,不要删。
- `notifier` property 保留(`cli.py:207` 的 doctor 检查在用),但其含义从"要 subprocess 调用的脚本"变为"通知实现模块的位置"。若改动了 doctor 的检查语义,需同步更新 `cli.py` 的 `chk("notifier", ...)`。
**`asrpipe/vault.py` 的 `notify()`**:
- **修掉 argv 错位 bug**:不再把 `"草稿已生成,待确认"` 写死为 status。调用方传入的第一个参数(语义上就是 status/事件名)必须原样传递。
- 建议直接 import `meeting-status-notify.py` 暴露的 `send()`,取代 `subprocess.run(["python3", ...])`——省一次进程启动,也让类型契约显式。若因文件名带连字符不便 import,可用 `importlib.util.spec_from_file_location`(路径来自 `CFG.notifier`),或把共享逻辑提取为 `OBSIDIAN/bin/notify_lib.py` 再由两边 import。**三种都可接受,选一种并在注释里说明理由。**
- 保留 `redact_log` 参数行为(`review_web.py:271` 传了 `redact_log=True`,因为 body 含一次性 capability URL,**绝不能进日志**)。这是安全相关行为,不要动。
- 保留"notifier 不存在时只记日志"的降级分支。
**5 处调用点**(`revise.py:273`、`cli.py:109`、`cli.py:160`、`review_web.py:49`、`review_web.py:271`):修好 `notify()` 后语义即自动正确,**调用点本身不需要改**。改完后请逐一确认它们发出的 status 与调用方本意一致(对照 §0.4 的表格)。
### 4.2 `meeting-watch.py` 侧
- 删除自建的 HMAC 签名与 POST 逻辑:`webhook_secret()`(L177)、`post_webhook()`(L187)、`post_status()`(L205)。
- `post_status` 改为薄封装,转调 Stage 1 的单一实现;3 个调用点(L272 "开始转录"、L286 "转录完成,正在生成粗整理"、L304 "处理失败")的调用形式**保持不变**。
- 删除 `WEBHOOK_URL` / `WEBHOOK_ROUTE` / `WEBHOOK_SECRET` / `STATUS_WEBHOOK_URL` / `STATUS_WEBHOOK_ROUTE` / `SUBSCRIPTIONS` 这些常量(L32–L37)。
**关于 `meeting-recording` 路由(携带正文那条)**:`post_webhook()` 在当前代码里**只被 `post_status()` 调用过**,即 `meeting-recording` 路由从未真正被 POST(已 grep 确认)。它是旧 Hermes agent 精修架构的残留,`CLAUDE.md` 明确记载该架构"已**取代**"。**删除这条死路径**,不要为它保留接口。(`~/.hermes/webhook_subscriptions.json` 里那条带 `prompt`/`skills: [memo-inbox]` 的 `meeting-recording` 订阅同属旧架构遗留,但那是 Hermes 侧的配置,**不在本次授权改动范围**,只在 §7 报告里提一句即可。)
另外注意 `meeting-recording-status` 订阅带 `deliver_only: True`——它只做消息投递、不唤起 agent。这印证了"状态通知本质上只是一条消息,不需要 agent 参与",也是 Stage 4 `direct` 后端在语义上完全等价的依据。
### 4.3 补充测试(必须新增)
在 `asr-pipeline/tests/test_core.py` 中新增(沿用该文件既有的 `unittest` + `mock` 风格,注意它已在 import 前把 state/vault 重定向到 tempdir):
1. **status 透传测试**(覆盖 §0.4 的 bug):mock 掉实际发送,断言 `vault.notify("ASR 正式记录已发布", "body")` 传出的 status 是 `"ASR 正式记录已发布"`,**不是** `"草稿已生成,待确认"`。这是防止该 bug 回归的关键测试。
2. **降级测试**:目标不可达(mock 抛 `URLError`)时 `vault.notify()` 不抛异常、正常返回。
3. **backend=none 测试**:不产生任何网络调用。
4. **`redact_log` 测试**:`redact_log=True` 时日志中不含 body 原文(防止 capability URL 泄漏进日志)。
### 4.4 验收标准
```bash
cd /home/claw/claude-workspace/OBSIDIAN/asr-pipeline
PYTHONPATH=. .venv/bin/python -m unittest discover -s tests -v
# 期望:28 + 新增用例,全绿
# 单一实现确认:HMAC 签名逻辑只应出现在一个文件里
cd /home/claw/claude-workspace/OBSIDIAN
grep -rn "hmac.new" --include="*.py" . | grep -v -E "__pycache__|\.venv|\.bak\."
# 期望:只有 1 处(notify 实现所在文件)
# 管线不再读 Hermes 私有配置
grep -rn "webhook_subscriptions" --include="*.py" --include="*.sh" . | grep -v -E "__pycache__|\.venv|\.bak\."
# 期望:无真实读取(注释提及历史来源可接受)
# doctor 仍通过(会联网测两个模型,属正常)
asr-pipeline/bin/asr-refine doctor
```
---
## 5. Stage 3 — systemd 与文档同步
### 5.1 systemd
`~/.config/systemd/user/obsidian-recording-watch.service` 删除这一行(第三处硬编码):
```ini
Environment=MEETING_WEBHOOK_URL=http://127.0.0.1:8644/webhooks/meeting-recording
```
理由:该 URL 对应的 `meeting-recording` 路由已在 Stage 2 删除;状态通知目标现由 `notify-target.json` 统一提供,不应再由 unit 注入。
**保留** `MEETING_INBOX_BASE` 与 `MEETING_ASR_BACKEND=qwen3cam`——这两个是真实生效的部署配置,与本次解耦无关(后者还带着"ASR backend trial"的回滚注释,不要动)。
生效并验证:
```bash
systemctl --user daemon-reload
systemctl --user restart obsidian-recording-watch.service
systemctl --user is-active obsidian-recording-watch.service # 期望 active
journalctl --user -u obsidian-recording-watch.service -n 30 --no-pager
```
`asr-review-web.service` 与 `asr-review-revise@.service` **不需要改**(它们通过 `asrpipe` 间接受益;后者的 `EnvironmentFile=.asr-refine.env` 只管模型凭证)。
### 5.2 文档
按"一份权威来源 + 其余引用"的结构收敛(与 vault 侧 `AGENTS.md` 作索引的模式一致):
| 文件 | 动作 |
|---|---|
| `asr-pipeline/docs/agent-protocol.md` | **新建**。本次解耦的权威文档:通知接口契约(status/filename/detail 三段语义)、`notify-target.json` 字段含义、后端切换方式、"Hermes 只是一种后端"的定位、任何 agent 想接入时该读什么。Kai 早在 2026-07-17 就规划过这份共用协议文档但从未落地——这次补上。 |
| `asr-pipeline/HANDOFF.md` | 第 71 行 "Telegram 通知(hermes-gateway :8644 签名 webhook)" 改为指向通知接口层,不再把 Hermes 端口写进设计文档;链接到 `agent-protocol.md`。 |
| `OBSIDIAN/CLAUDE.md` | 管线段落中的 `meeting-status-notify.py` 描述更新为"可切换后端的通知接口";补一行指向 `agent-protocol.md`。保持该文件"只留 Claude + 本服务器专属基础设施"的既有定位,不要把协议细节搬进来。 |
| `~/.claude/skills/memo-inbox/references/webhook-toolset-diagnostics.md` | **标注废弃**。它描述的"webhook agent 直接处理 `meeting-recording` 路由 + 需要 `hermes-webhook`/`file`/`skills` toolset"架构已于 2026-07-25 被取代,该路由在 Stage 2 已删除。建议在文件顶部加醒目废弃声明 + 指向 `agent-protocol.md`,而不是直接删除(保留历史排障脉络)。**注意:这是 skill 目录,改动会影响 Claude 的行为,改完在 §7 里明确告知 Kai。** |
| `~/.claude/skills/memo-inbox/references/asr-refine-pipeline.md` | 五阶段表格等与 `HANDOFF.md` 逐字重复的内容改为链接引用;**保留** skill 独有的纪律性内容("第一轮只出粗整理,不直接写正式纪要"、S3 fidelity gate 存在的理由与实测数据、模型选型不得随意改动的警告)——这些是 Kai 的工作纪律,不是可去重的冗余。 |
### 5.3 验收标准
```bash
# 确认 8644 不再出现在任何活跃代码/配置里(文档中作为历史说明可保留)
cd /home/claw/claude-workspace/OBSIDIAN
grep -rn "8644" --include="*.py" --include="*.sh" . ~/.config/systemd/user/*.service 2>/dev/null \
| grep -v -E "__pycache__|\.venv|\.bak\."
# 期望:仅 notify-target.json 的待更新地址 + 注释
# 端到端(不触碰 vault、不 commit)
asr-pipeline/bin/asr-refine dry-run <任一 .asr-inbox/*.txt>
# 期望:五阶段跑通,产物落 asr-pipeline/dry-run/<slug>/,通知步骤降级但不报错
```
---
## 6. Stage 4 —(可选,需 Kai 决策)`direct` 后端
Stage 1–3 完成后,通知链路仍然经过 Hermes:管线 → Hermes 网关 webhook → Telegram。只要 Hermes 不在线,Kai 就收不到通知。
**可选的彻底解耦**:增加 `backend: "direct"`,直接调 Telegram Bot API(`https://api.telegram.org/bot<token>/sendMessage`),不经过 Hermes。
技术上已确认可行:
- `~/.hermes/.env` 中存在 `TELEGRAM_BOT_TOKEN`;`chat_id`(`1093241065`)在 `webhook_subscriptions.json` 的 `deliver_extra` 里已确认。
- `meeting-recording-status` 订阅标记为 `deliver_only: True`,即 Hermes 对这条消息**只做投递、不唤起 agent**。因此 `direct` 后端在语义上与现状**完全等价**,不损失任何 agent 能力。
**但这需要 Kai 决策,不要擅自实施**,因为:
1. 要把 Telegram bot token 复制进本项目自持的凭证文件(或新增一份 600 env 文件)。这扩大了 token 的存放面,是安全决策,须 Kai 同意。
2. 绕过 Hermes 后,Telegram 侧的消息格式/限流/重试不再由 Hermes 统一处理,需自行承担。
3. Kai 可能有意保留"所有对外消息都经 Hermes"的架构约定。
**执行方式**:Stage 3 完成后,在 §7 报告中列出此选项与上述权衡,**等 Kai 明确答复再动手**。届时若 Kai 同意,`direct` 只是 `notify-target.json` 里加一个后端分支 + 一个 sender 函数,改动很小——这正是 Stage 1 接口层带来的收益。
---
## 7. 完成后的回报要求
Stage 1–3 结束后,向 Kai 报告,包含:
1. **改了哪些文件**,每个文件一句话说明,附备份文件名。
2. **测试结果原文**(`Ran N tests ... OK`)。若有任何失败,**如实说明失败项与原因,不要含糊**。
3. **§0.4 的 argv 错位 bug 已修复的证明**:新增测试的名称与断言内容。
4. **仍然待办的事**,明确区分:
- 需要 Kai 提供 `.48` 上 Hermes webhook 新地址,填入 `notify-target.json` 的 `webhook.url`(在此之前通知仍不通,但已降级为"改一个配置字段")。
- Stage 4 `direct` 后端的决策请求(附 §6 的三点权衡)。
- Hermes 侧 `~/.hermes/webhook_subscriptions.json` 中 `meeting-recording` 那条旧订阅(带 `prompt`/`skills: [memo-inbox]`)属旧架构遗留、已无调用方,建议 Kai 自行清理——**本次未改动,因为不在授权范围**。
5. **动过 `~/.claude/skills/` 的明确告知**(若执行了 §5.2 的 skill 文档改动)——这会影响 Claude 后续行为,Kai 需要知道。
---
## 8. 明确禁止(越界前必须先问)
- ❌ **不要猜 `.48` 的端口后填进配置当作"已修复"**。宁可留失效地址 + 注释。SSH 到 `.48` 已确认被拒,无法自行核实。
- ❌ **不要往 `asrpipe/` 核心五阶段(`segment`/`flagger`/`refine`/`fidelity`/`revise`)里加任何 agent 相关代码**。`cli.py` 与 `config.py` 头部的 "no Claude Code, no opencode, no agent runtime" 声明是这套系统最值得保留的架构资产。
- ❌ **不要改模型选型**(`clean_model` / `draft_model`)。`config.py` 与 `HANDOFF.md` 都记录了实测依据(flash-lite 闸门失败 19/87 vs haiku 78/87,且 haiku 有 69 次过度压缩)。与本次解耦无关。
- ❌ **不要关闭或削弱 S3 fidelity gate**。它防的是静默信息丢失(无闸门时英文术语丢失达 22%、数字 11%)。
- ❌ **不要删除 rollback 链路**(`bin/obsidian-asr-refine.sh` + `obsidian-asr-refine.service`)。它们当前 inactive 但被 `obsidian-asr-refine.path` 的注释和 `HANDOFF.md` 记录为回滚路径。只需保证 `refine.sh:45` 调用 `meeting-status-notify.py` 的 CLI 契约不被破坏。
- ❌ **不要动 `~/obsidian-vault/` 的内容**,也不要在本次重构中产生 vault commit。本次只改 `OBSIDIAN/` 基础设施与 systemd unit。
- ❌ **不要清理那 7 个 `*.bak.*` 文件**。它们是 2026-07-26 重构的历史留存,清理属独立决策,需 Kai 单独批准。
- ❌ **不要引入新的第三方依赖**。`asr-pipeline/.venv` 目前只有 pypinyin / rapidfuzz / numpy,通知功能用标准库 `urllib.request` + `hmac` + `json` 足够(现有实现已证明)。
- ❌ **不要为解耦搭建通用消息总线/插件框架**。当前只有一个通知需求,`notify-target.json` 里一个 `backend` 字段 + 几个分支就够。过度工程化会让下一个维护者更难读。
---
## 9. 执行顺序速查
```
0. 读 §0 全部 → 跑 baseline 测试(28 OK)→ 备份将改动的文件
1. Stage 1:notify-target.json + meeting-status-notify.py 单一实现 → 验收 §3.3
2. Stage 2:config.py / vault.py(修 argv bug)/ meeting-watch.py + 4 个新测试 → 验收 §4.4
3. Stage 3:systemd 去硬编码 + agent-protocol.md + 4 份文档同步 → 验收 §5.3
4. 回报 Kai(§7),等 .48 地址与 Stage 4 决策
```
Stage 1–3 连续执行,不要中途停下来确认;Stage 4 必须等 Kai 答复。
@@ -0,0 +1,98 @@
# ASR 管线 × Agent 耦合点 Review
**日期**:2026-09-18
**背景**:Hermes 网关从 .145 迁移到 .48 后,`meeting-status-notify.py` 因硬编码 `127.0.0.1:8644` 而 Telegram 通知失效(核心转录/整理流程本身不受影响)。这次故障暴露的不是孤立 bug,而是整条管线里 Claude/Hermes 两个 agent 以"硬编码地址/私有配置文件/直接 shell out"方式接入,缺乏稳定接口层。本文档目标:**列出所有耦合点,诊断问题,提出"预留接口 + 通用 skill"的解耦方案**,供下一轮重构使用。不在本文档内直接修改代码。
---
## 1. 涉及文件清单
### 1.1 生产代码(按耦合程度分类)
| 文件 | 耦合对象 | 耦合方式 | 严重度 |
|---|---|---|---|
| `OBSIDIAN/bin/meeting-status-notify.py` | Hermes | 硬编码 `http://127.0.0.1:8644/webhooks/...`;直接读 `~/.hermes/webhook_subscriptions.json` 取 HMAC secret | 🔴 高(本次故障根源) |
| `OBSIDIAN/recordings/bin/meeting-watch.py` | Hermes | `post_webhook()`/`post_status()` **重新实现**了一遍同样的 HMAC 签名 + 硬编码 URL 逻辑(与上面文件重复代码,两处维护) | 🔴 高(重复实现同一故障点) |
| `OBSIDIAN/asr-pipeline/asrpipe/vault.py` (`notify()`) | Hermes(间接) | 不直接硬编码地址,但通过 `CFG.notifier` 固定 `subprocess.run(["python3", meeting-status-notify.py, ...])`,位置参数契约无 schema | 🟡 中(依赖上面两个文件之一才能工作) |
| `OBSIDIAN/asr-pipeline/asrpipe/config.py` (`review_public_host`) | 部署环境 | 默认值硬编码 `192.168.50.145`(本机 IP),可用 env 覆盖 | 🟢 低(有 env 覆盖,且 review-web 确实还在 .145,不是错误,只是"写死默认值"的一贯风格问题) |
| `OBSIDIAN/bin/obsidian-asr-refine.sh`(**已停用**,仅供 rollback) | Claude Code | 直接 shell out `$HOME/.local/bin/claude -p`,依赖交互登录态 | 🟡 中(非活跃路径,但 rollback 会重新引入该耦合) |
| `OBSIDIAN/bin/obsidian-asr-hotwords.py`、`OBSIDIAN/bin/obsidian-asr-intake.sh` | 无(命名遗留) | 调用 `~/bin/hermes-qwen3-asr-smart`——**这是个纯 STT wrapper shell 脚本,跟 Hermes agent 运行时完全无关**,只是历史命名带 "hermes" 前缀,容易被误读为依赖 agent | 🟢 低(认知负担,不是真耦合,建议重命名去混淆) |
| `~/.config/systemd/user/obsidian-recording-watch.service` | Hermes | `Environment=MEETING_WEBHOOK_URL=http://127.0.0.1:8644/...` 在 systemd 层再硬编码一次 | 🔴 高(第三处需要同步修改的地方) |
| `asr-pipeline/asrpipe/llm.py` | Claude(仅模型选择) | 字符串形式引用 `anthropic/claude-sonnet-5` 等模型名做定价/no-temperature 判断 | 🟢 低(纯 API 层配置,非 agent 运行时耦合,正常) |
| `asr-pipeline/asrpipe/cli.py`、`config.py` 文件头注释 | 无 | 明确声明"no Claude Code, no opencode, no agent runtime"——这是**架构上已经做对的部分**,核心管线是纯 HTTPS+文件驱动,值得保留和强化 | 🟢 无耦合(正面案例) |
### 1.2 文档/Skill 层(描述与现状不一致,需要一并清理)
| 文件 | 问题 |
|---|---|
| `~/.claude/skills/memo-inbox/references/webhook-toolset-diagnostics.md` | 描述的是**已被取代的旧架构**:假设有一个"webhook agent"(Hermes)直接处理 `meeting-recording` 路由、需要 `hermes-webhook`/`file`/`skills` toolset、把转录内容发给 Hermes agent 做 memo-inbox 整理。但 `CLAUDE.md` 明确写着"已**取代**旧的 Hermes→Telegram agent 精修",且 `meeting-watch.py` 里 `post_webhook()`(对应 `meeting-recording` 路由,携带正文)从未被实际调用,只有 `post_status()`(状态 ping)在用。**这份 skill 文档描述的故障排查对象已经不存在,但仍在 memo-inbox skill 里被索引,可能误导下一次排障。** |
| `~/.claude/skills/memo-inbox/references/asr-refine-pipeline.md` | 与 `CLAUDE.md`、`HANDOFF.md` 三份文档描述同一条管线,各有侧重但存在信息重复(例如五阶段说明在两份文档里几乎逐字重复)。不是错误,但增加了"改一处忘改另一处"的维护成本——这次 8644 端口就是同一类问题在配置层面的重演。 |
| `OBSIDIAN/asr-pipeline/HANDOFF.md` | 明确写了"Telegram 通知(hermes-gateway :8644 签名 webhook)",是**当前唯一还把 Hermes 网关端口写进设计文档**的地方,本身没错但需要随实现一起更新。 |
| 目前**不存在** `agent-protocol.md`(据 hindsight 记忆,2026-07-17 曾规划把 obsidian skill 拆分出一份 Hermes/Codex/Claude Code 共用的 `agent-protocol.md`,统一七步协议) | 说明"通用协议文档"这个方向此前已经被认可过,但从未真正落地到 ASR 通知这一层,这次故障正好是没有该抽象层的直接后果。 |
---
## 2. 问题诊断
### 2.1 核心问题:通知通道是唯一的"横切关注点",却用点对点硬编码实现
真正跨 Claude/Hermes 边界的交互只有一件事:**管线要把"状态变化"告诉人(Kai),而人在 Telegram 上**。当前实现把这件事等同于"调用 Hermes 网关在某个 IP:port 上开的私有 webhook",导致:
- 通知目标地址在**三个物理位置**各写一份(`meeting-status-notify.py`、`meeting-watch.py`、systemd unit 的 `Environment=`),Hermes 一换主机全部失效,还都要单独改。
- 认证方式依赖读取 **Hermes 自己的私有配置文件** `~/.hermes/webhook_subscriptions.json`,管线对 Hermes 的内部文件结构做了假设——Hermes 的实现细节变了,管线也会跟着坏,而管线本身的 git 仓库里完全看不出这个依赖关系。
- `meeting-watch.py` 里的 `post_webhook`/`post_status` 和 `meeting-status-notify.py` 是**同一逻辑的两份实现**(都做 HMAC 签名 + POST),维护者容易改一处忘改另一处。
### 2.2 次要问题:文档层的"多真相源"
`CLAUDE.md`(基础设施说明)、`HANDOFF.md`(设计交接)、`asr-refine-pipeline.md`(skill 引用)三份文档对同一条管线各写一遍架构图和五阶段表格,`webhook-toolset-diagnostics.md` 甚至在描述一个已经退役的架构分支。这不是本次故障的直接原因,但预示着**如果解耦时只改代码不改文档索引,同样的"多处不同步"问题会在文档层重演**。
### 2.3 正面案例,值得保留的设计
- `asrpipe/` 核心包(S0-S4 五阶段 + fidelity gate + termbank)本身**已经是纯 HTTPS + 文件驱动**,不依赖任何 agent CLI 或登录态,`cli.py`/`config.py` 头部注释明确声明了这个边界——这是解耦该学的样子。
- `asr-review-web`/`control_web.py` 是一个独立 HTTP 服务,路由/状态机/上传都通过标准 HTTP 暴露,没有假设调用方是谁——**这个"HTTP 接口面"是天然可以承载 Claude/Hermes 通用交互的地方**,不需要另起炉灶。
---
## 3. 解耦方案建议(预留接口 + 通用 skill)
### 3.1 接口层:把"通知"做成可插拔的抽象,而不是"调用 Hermes"
建议参考 `config.py` 里其余配置项的风格(`use_termbank`/`use_hindsight` 等 boolean 开关 + env 覆盖),把通知也做成同一等级的可插拔能力:
- 在 `asrpipe/config.py` 增加 `notify_backend`(如 `webhook` / `none` / 未来的 `stdout`/`file` 等),而不是隐式认定"notifier = 某个写死路径的 python 脚本"。
- `meeting-status-notify.py` 改造成**唯一**的通知发送实现(`meeting-watch.py` 不再自己重新实现 HMAC 签名,改为 shell out 到同一份脚本,或提取成共享库函数),并且:
- 目标地址、密钥来源全部走 env / 独立配置文件(例如 `OBSIDIAN/bin/notify-target.json`),**不直接读 `~/.hermes/*`**——管线自己的配置与 Hermes 的私有配置解耦,Hermes 换主机/换协议时只改这一份 target 配置,不改代码。
- 把"目标是 Hermes 网关的 webhook"这件事,退化为该抽象接口的**一种实现**,而不是唯一实现——未来即使 Hermes 换成完全不同的通知方式(比如直接调 Telegram Bot API),也只是换一个 backend,管线代码不用动。
### 3.2 协议层:让 Claude 和 Hermes 通过同一份"通用 skill/协议文档"交互,而不是各自看各的私有细节
现状是 Hermes 一侧靠私有 webhook 配置、Claude 一侧靠读 `CLAUDE.md`/`HANDOFF.md`,两边对同一条管线的认知来自不同文档,容易出现你这次遇到的情况——迁移动作只在 Hermes 侧发生,ASR 侧的假设没人同步更新。
建议:
1. **落地此前已经规划过的 `agent-protocol.md`**(hindsight 记忆里 2026-07-17 就提出过这个方向,但目前尚未创建)。把"任何 agent(Hermes/Claude Code/Codex)如何得知 ASR 管线的状态变化、如何触发人工确认、通知目标怎么配置"写成一份三方共用的协议文档,而不是分散在 `CLAUDE.md` + `HANDOFF.md` + memo-inbox skill 引用里各写一份。
2. 把 `control_web.py` 已有的 HTTP 接口面正式定义为**唯一**对外交互入口:任何 agent 需要查询任务状态/触发处理,都走这个 HTTP API,而不是直接读 sqlite(`intake.db`)或直接改文件。目前 `meeting-watch.py` 直接读写 `CONTROL_DB`(sqlite),这本身也是一种"绕过接口层直连内部状态"的耦合,建议后续统一收敛到 `control_web.py` 的 API 之后。
3. 清理 `webhook-toolset-diagnostics.md`:要么标注为"历史架构,已废弃于 2026-07-25",要么直接删除/归档,避免下次排障时被过时描述误导。
4. `CLAUDE.md` / `HANDOFF.md` / `asr-refine-pipeline.md` 三份重叠内容建议收敛为"一份权威来源 + 其余引用链接"的结构(类似 vault 侧 `AGENTS.md` 作为 SOP 索引、具体细则各自一份的模式),而不是三份都写一遍五阶段表格。
### 3.3 命名去混淆
`~/bin/hermes-qwen3-asr-smart` 建议评估是否重命名(例如 `qwen3-asr-transcribe-wrapper`)——它不依赖 Hermes agent 运行时,纯粹是历史命名,留着容易让人(或下一轮重构的 LLM)误判为真实耦合点而做无用功。
---
## 4. 优先级建议
| 优先级 | 事项 | 原因 |
|---|---|---|
| P0 | 通知 backend 抽象化 + 三处硬编码收敛为一处配置 | 直接对应本次故障,且是唯一真正跨 agent 边界的运行时依赖 |
| P1 | 落地 `agent-protocol.md`,清理 `webhook-toolset-diagnostics.md` 等过时文档 | 防止"文档多真相源"重演同类故障 |
| P2 | `meeting-watch.py` 收敛直连 sqlite 为走 `control_web.py` API | 减少内部状态的多点直连,但当前未造成故障,可延后 |
| P3 | `hermes-qwen3-asr-smart` 重命名 | 纯认知成本,无功能风险,可随手做 |
---
## 5. 明确不建议做的事
- 不建议把核心 `asrpipe/` 五阶段管线本身引入任何 agent 依赖——它当前"纯 HTTPS + 文件驱动"的设计是对的,解耦应该聚焦在通知/协议这一层,不要往核心管线里加 Claude/Hermes 相关代码。
- 不建议为了解耦而过度设计一套通用消息总线/插件系统——当前只有"一个通知需求 + 一个 HTTP 状态查询需求",用简单的 backend 开关 + 现有 HTTP 接口面就够,避免过度工程化。
@@ -0,0 +1,130 @@
# ASR Pipeline v2
## Goal
The pipeline recovers useful meeting facts and traces rather than pretending that
speech can always be reconstructed word for word. The original transcript remains
the audit source. A draft may omit low-value repetition, but must not invent a
subject, number, decision, or relationship.
## Stages
1. S0 parses speaker turns and collapses decoder loops.
2. S1 generates candidates from the untouched transcript. Phonetics, observed ASR
errors, aliases, entity relationships, and recency are independent signals.
3. S2 only removes fillers and repairs punctuation. It receives no Hindsight context
and makes no entity decisions.
4. S3 applies settled mappings deterministically. Flash Lite sees only unresolved
candidates and Chinese-number spans.
5. S4 extracts source-labelled evidence cards per chunk.
6. S5 Sonnet groups those cards by topic and writes a rough draft.
## Personal fact store
`state/lexicon.db` is intentionally reusable outside ASR:
- `entity`, `alias`, `correction`: identity and observed surface forms.
- `relation`: person/company/project relations with source, confidence, and dates.
- `interaction`: contact recency and frequency.
- `observation`: what a transcript emitted and how it was resolved.
- `phonetic_confusion`: confirmed personal accent/ASR confusion pairs.
- `review_question`: transport-independent human review queue.
- `source_document`: incremental indexing state for historical minutes.
Obsidian remains a curated source. Hindsight supplies broad recall. SQLite holds
small deterministic facts that applications can safely query.
## Human review loop
The pipeline never blocks waiting for an answer. Ambiguous high-impact entity
candidates are queued and the draft keeps an uncertainty marker.
```bash
bin/asr-refine review pending
bin/asr-refine review answer 12 万亚坤
```
A confirmed candidate is learned as an alias, correction, and observation. The
candidate-pool cache includes the database timestamp, so the next recording uses it
immediately.
Pi, Telegram, and the web UI are thin transports over this queue:
1. Read pending rows and display the question, source sentence, and choices.
2. Submit `question_id + answer`.
3. Call the same `review.answer()` function or CLI command.
4. Optionally request a replay of S3-S5 for the affected recording.
The production web transport is a persistent mobile control plane:
```bash
bin/asr-refine review serve # normally started by systemd
bin/asr-refine review state # print the active one-time URL
```
- Every generated draft is a Web delivery, even when entity candidate detection
finds no ambiguity. Review order is: inferred call background (with a corrected
background field that requests a redraft), unnamed voiceprints, the draft's
`## 待你确认` list, then one overall review item.
- It listens on `192.168.50.145:8765`. `/` accepts streamed audio uploads,
shows recent job state, and atomically hands completed files to the existing
recording watcher. `/hook/intake` exposes the same raw-upload contract for a
future reverse proxy or external uploader.
- Telegram receives a 256-bit capability URL. The token exists only in Telegram and
the mode-0600 runtime state file; it is redacted from service/request logs.
- The page exposes only one queued question at a time: candidate buttons, an
explicit "无法确认", a short custom answer, and an expandable full-draft view.
- Rough drafts live under `state/drafts/`, outside the Vault and Git. After all
questions are answered, an auditable feedback block is saved privately.
`asr-review-revise@.service` applies those answers with Sonnet, validates the
formal schema/Summary/provenance, then writes and pushes exactly one formal note
under `minutes/`. The private reviewed source moves to `state/published/`.
- A confirmed unnamed `speaker_N` is written to `voiceprint_speakers`, marked for
retry by the daily enrollment job, and immediately submitted to the persistent
`:10099/voiceprints/enroll` API.
- SQLite records one `entity_mention` per entity per formal minute. When the current
minute would bring an uncarded person/company to two distinct minutes, Web offers
a conservative card scaffold. No card or judgment field is created without an
explicit human choice.
- Upload metadata (title and optional user-supplied background) is kept separate
from the acoustic transcript. The title survives into the task slug; the
background is explicitly labelled as user-supplied context for drafting.
- `asr-review-web.service` is enabled at boot and restarts on failure. A successful
private draft ensures it is running and sends its stable review URL through
Telegram. `dry-run` never queues questions.
The service unit lives at `deploy/asr-review-web.service`. Telegram may later gain
inline choice buttons, but neither transport owns facts or writes SQLite directly.
## Confidence policy
- Confirmed: voiceprint, explicit human mapping, or multiple independent strong
signals.
- Probable: context and relation graph agree, but identity is inferred.
- Pending: weak or conflicting evidence, including numbers whose type/unit is not
clear.
Only confirmed facts become deterministic aliases automatically.
## Graph and recency
```bash
bin/asr-refine graph sync
bin/asr-refine graph stats
```
The graph incrementally scans finalized minutes, imports person/company/project
frontmatter and known wikilinks, records meeting interactions, creates low-confidence
co-occurrence edges, and promotes explicit linked relations such as 投资/负责/合作 at a
higher confidence. Deleted or changed notes remove only facts owned by that source.
Candidate context weight combines relation confidence, repeated support, last contact,
and a 45-day half-life. It influences ranking but cannot activate the personal
phonetic-confusion channel outside entities already in the current graph context.
## Numbers
The deterministic number pass classifies percentage, date, money, quantity, and
project/code speech. Every decision records raw value, normalized value, type,
confidence, source span, and nearby context. Ambiguous forms such as `两万五九份`
remain unchanged and are passed to the uncertainty stage instead of being guessed.
@@ -0,0 +1,58 @@
# 录音上传与审核工作台
## 入口
- Web:`http://192.168.50.145:8765/`
- 健康检查:`GET /health`
- 最近任务:`GET /api/jobs`
- 人工确认:`GET /review/`
服务只面向 LAN 与 tailnet,没有实现公网身份认证。未来暴露到公网时,应在反向代理层
增加认证、上传大小限制和请求速率限制,不要直接转发当前端口。
## 工作流
1. 浏览器把录音以原始二进制流上传到隐藏临时目录。
2. 文件完整写入并 `fsync` 后,原子移动到 `recordings/inbox/`。
3. 既有 `obsidian-recording-watch.service` 接管转写、CAM++ 分角色和声纹识别。
4. 转写落入 `.asr-inbox/`,独立整理程序生成私有草稿。
5. Telegram 发送确认链接;Web 回答写入 SQLite。
6. 全部确认后生成正式纪要并写入 Obsidian。
上传时填写的背景单独保存在任务元数据中,进入整理提示时会标为“用户提供的背景”,
不会混入或伪装成 ASR 转写内容。
## Hook
`POST /hook/intake` 与 Web 上传使用相同的流式协议,便于以后在外网部署一个很薄的上传
前端或反向代理:
```bash
curl --fail-with-body \
-X POST \
-H 'Content-Type: application/octet-stream' \
-H 'X-Filename: call.m4a' \
-H 'X-Title: 项目通话' \
-H 'X-Context: 参与人、项目和希望重点确认的事项' \
--data-binary @call.m4a \
http://192.168.50.145:8765/hook/intake
```
响应中的 `job_id` 是任务标识。状态可从 `/api/jobs` 查询。
## 运行
```bash
systemctl --user status asr-review-web.service
systemctl --user restart asr-review-web.service
journalctl --user -u asr-review-web.service -n 100 --no-pager
curl -fsS http://127.0.0.1:8765/health
```
持久数据:
- `asr-pipeline/state/intake.db`:上传任务与状态。
- `recordings/state/uploads/`:上传标题、背景和任务关联。
- `recordings/inbox/.uploading/`:未完成上传;正常完成后为空。
默认单文件上限为 4 GiB,可通过 `ASR_UPLOAD_MAX_BYTES` 调整。
@@ -0,0 +1,80 @@
# 当前流程与独立项目交接
核对日期:2026-11-02,Asia/Shanghai。
## 已验证
原代码:`/home/claw/claude-workspace/OBSIDIAN/asr-pipeline`。
原音频编排:`/home/claw/claude-workspace/OBSIDIAN/recordings/bin`。
原通知脚本:`/home/claw/claude-workspace/OBSIDIAN/bin/meeting-status-notify.py`。
本机 systemd:
| 单元 | 本次状态 | 职责 |
|---|---|---|
| asr-review-web.service | running | 8765 常驻上传、最近任务、审核界面 |
| obsidian-recording-watch.service | running | 监听录音并请求外部 ASR |
| obsidian-asr-refine.path | waiting | 监听 vault/.asr-inbox |
| obsidian-asr-refine-native.service | inactive | 按需整理转写,当前未运行 |
| asr-review-revise@.service | 模板 | 人工确认后生成/发布 |
`GET http://127.0.0.1:8765/health` 返回 `ok=true, pending=3`。
这说明审核队列存在待确认项,不证明三个录音都在推理,也不证明通知正常。
本轮不导出待确认正文,不生成新稿、不注册声纹、不写 vault。
## 代码确认的链路
- 监听单元显式选择 `MEETING_ASR_BACKEND=qwen3cam`。
- 对应脚本默认请求 `http://192.168.50.100:10099/asr`,整文件上传,
`language=Chinese`、`save_voiceprint_session=true`;服务内部切片。
本轮未对 GPU 服务发起真实录音测试,不据此宣称其健康或当前镜像版本。
- 原始转写保留声纹 session 与 speaker;归档后桥接到 vault/.asr-inbox。
- 独立 Python 管线包含模糊音/别名、关系与近期权重、数字归一、纯清洗与纠错分阶段。
- 初稿和中间状态在程序私有目录,正式纪要等待 Web 审核通过。
- 人物/公司卡提议与声纹确认存在代码路径;不等于当前外部注册服务验证通过。
## 运行数据边界
- `asr-pipeline/state/lexicon.db`:人工确认、关系、纠错、审核队列,不能重建后替代。
- `asr-pipeline/state/intake.db`:上传任务与状态。
- `state/drafts/`、`state/published/`:私有稿与发布回执。
- `recordings/{inbox,processing,processed,failed,transcripts,state}`:音频及编排状态。
- `vault/.asr-inbox/.state`:整理处理标记,切换漏迁可能造成重复处理。
- `.asr-refine.env`:私有凭据,未读取值、未复制、未入 Git。
本次只提取源代码、专业词库、现有测试、文档、unit 快照。没有复制生产音频、
个人 SQLite、缓存、会话、凭据或 `.venv`。历史设计文档有过时路径和模型,保留作来源。
## 已发现的风险(尚未改生产)
1. 通知脚本默认 `127.0.0.1:8644` 和 `~/.hermes/webhook_subscriptions.json`。
结合 Hermes 已迁走的资产记录,这是通知失效风险;本次未验证远端 webhook。
2. 本仓库代码的模型缺省已是 LiteLLM(清洗 `aistudio/gemini-flash-lite-latest`,成稿 `codex/gpt-sol-latest`),不是 ZenMux,也不是 Sonnet 5。生产 unit 若仍加载旧 `.asr-refine.env`,以那个文件为准;本次未读取凭据,不把代码缺省当成现网正在用的路由。
3. 路径耦合旧工作目录;本项目配置根目录、通知器、监听数据库已可外置,
但声纹地址、审核 systemd 单元名等仍有历史耦合。`deploy/systemd/` 保持现网快照,没有改 ExecStart。
4. 历史 HANDOFF 中写的是一次性页面。代码里有两个常驻界面:旧 `asrpipe.control_web`(快照单元,默认 8765)和新 `asrservice.web`(compose,默认 8788)。旧文档不作现态。
5. 产物删除和在线编辑会拒绝允许根以外的路径。发布到 `minutes/`、`People/`、`Companies/` 仍没有对照 vault SOP 的不可绕过白名单。本轮不冒充已审计完成。
## 代码已修复、生产未切换
下面这些是仓库里的新行为。现网单元没有改,不能当成已经在 8765/8788 的生产进程上生效。
- 未设置 `ASR_WEB_PASSWORD` 时,两个浏览器界面都拒绝上传、审核和改配置。临时放开只认 `ASR_WEB_ALLOW_ANON=1`。
- 登录后的写操作校验 CSRF。设置页不回显内网地址和密钥片段。
- `asrservice.worker` 与 `recordings/bin/meeting-watch.py` 抢同一把 `consumer.lock`。
- `ASR_GIT_SYNC`、`ASR_NOTIFY` 缺省为关。
- 修订失败会保留队列标记并重试;重新精修失败不会先删掉已回答的审核记录。
- 时间在界面上按 Asia/Shanghai 显示。词库列表按页读取,不再一次只截断前 180 条却不说明总数。
## 后续安全切换
1. 先确定模型路由与通知适配器,检查 vault SOP 与正式输出路径。
2. 建独立 venv、冻结依赖,使用合成数据跑回归,再选真实录音单测。
3. 停接新上传,确认没有转写/整理/审核发布在运行,再停相关 user units。
4. 对 SQLite 使用 backup API 生成一致性备份,保留原件;迁移录音状态与 inbox 标记,
不能对运行中数据库只复制主文件。
5. 路径变更还涉及数据库内草稿/转写绝对路径;要审计和映射,不能仅改 unit。
6. 生成新 unit 并逐个启用,确认只存在一个消费者;保留旧代码和备份以便回滚。
7. 验证上传 → 转写 → 确认链接 → 审核 → 正式纪要 → Git 同步完整链路后再移除旧目录。
本轮未执行上述生产切换,不承诺已独立上线。
@@ -0,0 +1,55 @@
# 独立 WebUI 项目路线
## 定位
独立的会议、电话录音处理应用。WebUI 是主要操作界面,Telegram 只做可选通知,
不依附任何 Agent CLI。当前源码已抽出,但仍是旧实现基线,尚未独立部署上线。
## 目标架构
浏览器 → 应用 API → 持久化任务队列 → 后台 Worker → 外部 ASR / LLM / Hindsight。
SQLite 保存任务、处理阶段、审核、实体词库和修改历史;原音频、中间产物单独保存。
Obsidian 是可选发布目标,不是应用内部的任务队列或运行数据库。
ASR GPU 服务继续位于 unRaid,模型调用优先接入统一 LiteLLM。
应用状态、配置和存储必须通过环境配置,不绑定特定机器或个人工作区。
## 核心页面
1. 任务列表:上传、背景输入、进度、失败原因、重试、最近任务。
2. 任务详情:原音频播放、带说话人与时间轴的原文、处理版本、来源。
3. 审核:背景确认、候选词、说话人身份、关键歧义、最终纪要确认。
4. 词库:实体、别名、模糊音规则、关系和人工确认历史。
5. 设置:ASR、LLM、Hindsight、通知、输出路径和数据保留策略。
## 实施顺序
### 第一阶段:独立运行闭环
- 配置与运行数据分离;迁移个人词库、审核状态时使用一致性备份。
- 将 systemd 路径监听/模板任务改为应用内部持久化队列与 Worker。
- 实现任务互斥、幂等、重启恢复、超时和阶段重试,不重跑已确认步骤。
- 修复独立通知适配器,停止依赖本机 Hermes 状态文件。
- 对照现有 vault SOP,审计输出路径;只有确认后的正式稿可发布。
- 上传、转写、审核、发布全链路通过后,再切换生产消费者。
### 第二阶段:WebUI 完整化
- 保留现有纯 Python 页面作过渡,先明确 API 和状态模型,再选前端框架。
- 增加播放器、原文对照、编辑、版本回退与清晰的阶段状态。
- 把说话人确认、词库维护及上下文确认放入一致的任务工作流。
### 第三阶段:Docker 发布
- API/WebUI 和 Worker 可分别运行;持久卷承载 SQLite 与音频。
- 不在容器内运行 systemd;不依赖宿主 Agent home。
- Obsidian 发布器可持有专用 Git checkout,不直接修改主工作副本。
- 确定 Dockerfile 与验收测试后,再接 GitHub Actions + GHCR 构建发布。
## 验收原则
原文不可覆盖;候选纠错须有依据;背景不伪装为录音事实;重要歧义交人工确认。
以关键内容和行动线索为目标,不追求伪精确的逐字恢复。
合成测试与真实录音端到端验收分开,不能以单元测试通过代替上线验收。
本文件是后续开发计划,本轮不新增推理请求、不切换生产服务、不自动部署容器。
@@ -0,0 +1,13 @@
[build-system]
requires = ["setuptools>=68"]
build-backend = "setuptools.build_meta"
[project]
name = "meeting-asr"
version = "0.2.0"
description = "Standalone meeting transcription, terminology correction and human-reviewed minutes web application"
requires-python = ">=3.11"
dependencies = ["pypinyin>=0.51,<1", "rapidfuzz>=3,<4", "numpy>=1.26,<3"]
[tool.setuptools.packages.find]
include = ["asrpipe*", "asrservice*"]
@@ -0,0 +1,255 @@
#!/usr/bin/env bash
set -euo pipefail
INPUT=${1:?audio file required}
OUT=${2:?output txt required}
ASR_URL=${MEETING_ASR_URL:-http://192.168.50.100:8000/v1/audio/transcriptions}
ASR_MODEL=${MEETING_ASR_MODEL:-qwen3-asr}
SEGMENT_SECONDS=${MEETING_ASR_SEGMENT_SECONDS:-540}
LANGUAGE=${MEETING_ASR_LANGUAGE:-zh}
GLOSSARY=${MEETING_ASR_GLOSSARY:-$HOME/obsidian-vault/minutes/_glossary.md}
# Base dir derived from this script's location (recordings/bin/asr-transcribe.sh -> recordings/),
# overridable via MEETING_INBOX_BASE. Migration-proof.
BASE="${MEETING_INBOX_BASE:-$(cd "$(dirname "$(readlink -f "$0")")/.." && pwd)}"
LOG_DIR="$BASE/logs"
LOG="$LOG_DIR/asr-transcribe.log"
mkdir -p "$LOG_DIR" "$(dirname "$OUT")"
log() {
printf '[%s] %s\n' "$(date '+%Y-%m-%d %H:%M:%S')" "$*" >> "$LOG"
}
if [[ ! -f "$INPUT" ]]; then
echo "input file not found: $INPUT" >&2
exit 2
fi
# ---- Cloud / Unified Provider branch (soniox / gemini / openai / grok) ------
case "${MEETING_ASR_BACKEND:-}" in
soniox|gemini|openai|grok)
PROJECT_ROOT="$(cd "$BASE/.." && pwd)"
log "START backend=${MEETING_ASR_BACKEND} input=$INPUT out=$OUT"
if PYTHONPATH="$PROJECT_ROOT:${PYTHONPATH:-}" python3 - "$INPUT" "$OUT" "${MEETING_ASR_BACKEND}" "${MEETING_ASR_MODEL:-}" <<'PY' 2>> "$LOG"
import sys
from pathlib import Path
from asrpipe.providers import transcribe_audio
audio = Path(sys.argv[1])
out = Path(sys.argv[2])
provider = sys.argv[3]
model = sys.argv[4] if len(sys.argv) > 4 else ""
res = transcribe_audio(audio, provider=provider, model=model)
out.write_text(res.transcript_text, encoding="utf-8")
PY
then
log "DONE backend=${MEETING_ASR_BACKEND} out=$OUT chars=$(wc -c < "$OUT")"
exit 0
fi
echo "${MEETING_ASR_BACKEND} transcription failed (see $LOG)" >&2
log "ERROR backend=${MEETING_ASR_BACKEND} failed"
exit 13
;;
esac
# ---- Backend switch (trial): VibeVoice 分角色后端 ----------------------------
# Default backend stays qwen3 (logic below). Set MEETING_ASR_BACKEND=vibevoice
# (e.g. in obsidian-recording-watch.service) to route the live pipeline through
# vibevoice-transcribe.py, producing a diarized transcript (timestamps + S#).
# Interface is preserved: this branch writes the same single $OUT txt the bridge
# expects. Fully reversible: unset the env / restore the .bak.
if [[ "${MEETING_ASR_BACKEND:-qwen3}" == "vibevoice" ]]; then
VIBE_PY="$BASE/bin/vibevoice-transcribe.py"
CHUNK="${MEETING_ASR_VIBE_CHUNK:-200}"
if [[ ! -f "$VIBE_PY" ]]; then
echo "vibevoice backend selected but $VIBE_PY missing" >&2
exit 2
fi
mkdir -p "$BASE/processing"
VOUT=$(mktemp -d "$BASE/processing/vibe.XXXXXX")
log "START backend=vibevoice input=$INPUT chunk=${CHUNK}s out=$OUT"
base=$(basename "$INPUT"); base="${base%.*}"
if python3 "$VIBE_PY" "$INPUT" --out "$VOUT" --chunk "$CHUNK" >> "$LOG" 2>&1 \
&& [[ -f "$VOUT/$base-diarized.txt" ]]; then
cp "$VOUT/$base-diarized.txt" "$OUT"
log "DONE backend=vibevoice out=$OUT chars=$(wc -c < "$OUT")"
rm -rf "$VOUT"
exit 0
fi
echo "vibevoice transcription failed or produced no output (see $LOG)" >&2
log "ERROR backend=vibevoice produced no diarized.txt"
rm -rf "$VOUT"
exit 11
fi
# ---- Backend: Qwen3-ASR 1.7B + CAM++ 分角色 + 声纹 (:10099) ------------------
# MEETING_ASR_BACKEND=qwen3cam (别名 paraformer 向后兼容) 路由到分角色服务。整文件一次
# 请求(FSMN-VAD 内部切段),无客户端分段。输出带说话人标签;声纹命中显示真名,否则
# speaker_N。可用 env 切换回退。热词/postprocess 均已退役(解码软偏置服务端关闭、
# legacy hotword 不生效、确定性预替换抢跑污染),实体纠错统一在 refine 单点处理。
if [[ "${MEETING_ASR_BACKEND:-qwen3}" == "qwen3cam" || "${MEETING_ASR_BACKEND:-qwen3}" == "paraformer" ]]; then
DIAR_URL="${MEETING_DIARIZE_URL:-${MEETING_PARAFORMER_URL:-http://192.168.50.100:10099/asr}}"
log "START backend=qwen3cam input=$INPUT out=$OUT"
RESP=$(mktemp)
# 中文会议固定 Chinese,避免远场噪声触发粤语/日语漂移;英文缩写仍可正常识别。
args=( -sS --max-time 900 -F "file=@$INPUT" -F 'language=Chinese' -F 'save_voiceprint_session=true' )
if curl "${args[@]}" "$DIAR_URL" > "$RESP" 2>> "$LOG"; then
if python3 - "$RESP" "$OUT" <<'PY'
import json, sys
try:
d = json.load(open(sys.argv[1], encoding='utf-8'))
except Exception as e:
sys.stderr.write(f"qwen3cam json parse failed: {e}\n"); sys.exit(1)
if 'detail' in d or not d.get('segments'):
sys.stderr.write(f"qwen3cam no segments: {str(d)[:160]}\n"); sys.exit(1)
vp = d.get('voiceprint', {})
matched = [f"{m['speaker_local']}={m['person_name']}" for m in vp.get('matches', []) if m.get('status') == 'matched']
sid = vp.get('session_id') or ''
# machine-readable line for the voiceprint auto-enroll tool (survives to .asr-inbox)
nclusters = len({s.get('speaker') for s in d['segments']})
with open(sys.argv[2], 'w', encoding='utf-8') as out:
out.write(f"<!-- voiceprint_session: {sid} clusters: {nclusters} -->\n")
hdr = "# 转写(Qwen3-ASR 1.7B + CAM++ 分角色 + 声纹)"
if matched:
hdr += " | 声纹命名: " + ", ".join(matched)
out.write(hdr + "\n\n")
turns = []
for s in d['segments']:
spk = s.get('speaker', '?'); t = (s.get('text') or '').strip()
if not t:
continue
if turns and turns[-1][0] == spk:
turns[-1][1] += t
else:
turns.append([spk, t])
for spk, t in turns:
out.write(f"【{spk}】{t}\n\n")
PY
then
log "DONE backend=qwen3cam out=$OUT chars=$(wc -c < "$OUT")"
rm -f "$RESP"; exit 0
fi
fi
echo "qwen3cam transcription failed (see $LOG)" >&2
log "ERROR backend=qwen3cam failed"
rm -f "$RESP"; exit 12
fi
# ---- default Qwen3 path follows --------------------------------------------
case "$SEGMENT_SECONDS" in
''|*[!0-9]*) echo "MEETING_ASR_SEGMENT_SECONDS must be integer" >&2; exit 2 ;;
esac
if (( SEGMENT_SECONDS < 480 || SEGMENT_SECONDS > 600 )); then
log "WARN segment seconds $SEGMENT_SECONDS outside recommended 480-600; continuing"
fi
if ! command -v ffprobe >/dev/null 2>&1; then
echo "ffprobe not found" >&2
exit 2
fi
if ! command -v ffmpeg >/dev/null 2>&1; then
echo "ffmpeg not found" >&2
exit 2
fi
if ! command -v curl >/dev/null 2>&1; then
echo "curl not found" >&2
exit 2
fi
if ! command -v python3 >/dev/null 2>&1; then
echo "python3 not found" >&2
exit 2
fi
HOTWORD_GEN="$HOME/claude-workspace/OBSIDIAN/bin/obsidian-asr-hotwords.py"
build_prompt() {
local terms
# Shared generator: cards ∪ glossary entity terms, mapping wrong-sides excluded,
# ranked by mention count. Fall back to a minimal inline reader if unavailable.
if [[ -f "$HOTWORD_GEN" ]]; then
terms=$(python3 "$HOTWORD_GEN" --budget 1800 2>/dev/null | tr ',' '、')
fi
if [[ -z "$terms" ]]; then
terms=$(python3 - "$GLOSSARY" <<'PY'
import re, sys
from pathlib import Path
p = Path(sys.argv[1]).expanduser()
if not p.exists():
sys.exit(0)
out, seen = [], set()
for raw in p.read_text(errors='ignore').splitlines():
s = raw.strip()
if not s.startswith(('-', '*')) or re.search(r'[→⇒]|->', s):
continue # mapping lines carry the WRONG form on the left — never bias to it
s = re.split(r'\s[—–-]\s|[::]', re.sub(r'^[-*+\d\.\)\s]+', '', s), maxsplit=1)[0].strip()
s = re.sub(r'\[\[|\]\]', '', s).strip()
if s and s not in seen and len(s) <= 30:
seen.add(s); out.append(s)
print('、'.join(out[:200]))
PY
)
fi
[[ -z "$terms" ]] && return 0
printf '以下是会议中可能出现的人名、公司、项目、产品和术语,请优先准确识别这些正确写法:%s' "$terms"
}
DURATION=$(ffprobe -v error -show_entries format=duration -of default=nk=1:nw=1 "$INPUT" | awk '{printf "%d", ($1 == int($1) ? $1 : int($1)+1)}')
if [[ -z "$DURATION" || "$DURATION" == "0" ]]; then
echo "failed to detect duration: $INPUT" >&2
exit 3
fi
mkdir -p "$BASE/processing"
TMPDIR=$(mktemp -d "$BASE/processing/asr.XXXXXX")
cleanup() { rm -rf "$TMPDIR"; }
trap cleanup EXIT
PROMPT_TEXT=$(build_prompt || true)
: > "$OUT"
log "START input=$INPUT duration=${DURATION}s segment=${SEGMENT_SECONDS}s out=$OUT"
segments=$(( (DURATION + SEGMENT_SECONDS - 1) / SEGMENT_SECONDS ))
failures=0
for ((i=0; i<segments; i++)); do
start=$(( i * SEGMENT_SECONDS ))
end=$(( start + SEGMENT_SECONDS ))
if (( end > DURATION )); then end=$DURATION; fi
seg="$TMPDIR/segment_$((i+1)).mp3"
txt="$TMPDIR/segment_$((i+1)).txt"
printf -- '--- 第%d段 [%02d:%02d - %02d:%02d] ---\n' "$((i+1))" "$((start/60))" "$((start%60))" "$((end/60))" "$((end%60))" >> "$OUT"
ffmpeg -hide_banner -loglevel error -y -ss "$start" -t "$SEGMENT_SECONDS" -i "$INPUT" -ar 16000 -ac 1 -b:a 64k "$seg"
log "SEGMENT $((i+1))/$segments file=$seg size=$(stat -c%s "$seg")"
curl_args=(
-sS --fail-with-body --max-time 600
-F "model=$ASR_MODEL"
-F "file=@$seg"
)
if [[ -n "$LANGUAGE" ]]; then
curl_args+=( -F "language=$LANGUAGE" )
fi
if [[ -n "$PROMPT_TEXT" ]]; then
curl_args+=( -F "prompt=$PROMPT_TEXT" )
fi
if curl "${curl_args[@]}" "$ASR_URL" > "$txt" 2>> "$LOG"; then
python3 - "$txt" >> "$OUT" <<'PY'
import json, sys
from pathlib import Path
s=Path(sys.argv[1]).read_text(errors='ignore').strip()
try:
obj=json.loads(s)
print((obj.get('text') or obj.get('transcript') or s).strip())
except Exception:
print(s)
PY
else
failures=$((failures+1))
printf '[第%d段转录失败]\n' "$((i+1))" >> "$OUT"
log "ERROR segment $((i+1)) failed"
fi
printf '\n\n' >> "$OUT"
done
if (( failures > 0 )); then
log "DONE_WITH_FAILURES failures=$failures out=$OUT"
exit 10
fi
log "DONE out=$OUT chars=$(wc -c < "$OUT")"
@@ -0,0 +1,362 @@
#!/usr/bin/env python3
import fcntl
import hashlib
import hmac
import json
import os
import shutil
import sqlite3
import subprocess
import sys
import time
import urllib.request
from pathlib import Path
PROJECT = Path(__file__).resolve().parents[2]
BASE = Path(os.environ.get("MEETING_INBOX_BASE", str(PROJECT / "recordings")))
INBOX = BASE / "inbox"
PROCESSING = BASE / "processing"
PROCESSED = BASE / "processed"
FAILED = BASE / "failed"
TRANSCRIPTS = BASE / "transcripts"
STATE = BASE / "state"
LOGS = BASE / "logs"
ASR = Path(os.environ.get("MEETING_ASR_SCRIPT", str(PROJECT / "recordings" / "bin" / "asr-transcribe.sh")))
SEEN_FILE = STATE / "seen.json"
UPLOAD_META = STATE / "uploads"
CONTROL_DB = Path(os.environ.get(
"ASR_CONTROL_DB",
str(Path(os.environ.get("ASR_STATE_DIR", str(PROJECT / "state"))) / "intake.db")))
LOG = LOGS / "meeting-watch.log"
POLL_SECONDS = int(os.environ.get("MEETING_WATCH_POLL_SECONDS", "30"))
STABLE_AGE_SECONDS = int(os.environ.get("MEETING_WATCH_STABLE_AGE_SECONDS", "60"))
WEBHOOK_URL = os.environ.get("MEETING_WEBHOOK_URL", "http://127.0.0.1:8644/webhooks/meeting-recording")
WEBHOOK_ROUTE = os.environ.get("MEETING_WEBHOOK_ROUTE", "meeting-recording")
WEBHOOK_SECRET = os.environ.get("MEETING_WEBHOOK_SECRET", "")
STATUS_WEBHOOK_URL = os.environ.get("MEETING_STATUS_WEBHOOK_URL", "http://127.0.0.1:8644/webhooks/meeting-recording-status")
STATUS_WEBHOOK_ROUTE = os.environ.get("MEETING_STATUS_WEBHOOK_ROUTE", "meeting-recording-status")
SUBSCRIPTIONS = Path.home() / ".hermes" / "webhook_subscriptions.json"
SUPPORTED = {".m4a", ".mp3", ".wav", ".mp4", ".aac", ".ogg", ".opus", ".webm", ".flac"}
def log(msg):
LOGS.mkdir(parents=True, exist_ok=True)
with LOG.open("a", encoding="utf-8") as f:
f.write(time.strftime("[%Y-%m-%d %H:%M:%S] ") + msg + "\n")
def load_seen():
try:
return json.loads(SEEN_FILE.read_text())
except Exception:
return {}
def save_seen(seen):
STATE.mkdir(parents=True, exist_ok=True)
tmp = SEEN_FILE.with_suffix(".tmp")
tmp.write_text(json.dumps(seen, ensure_ascii=False, indent=2), encoding="utf-8")
tmp.replace(SEEN_FILE)
def upload_meta(filename):
try:
return json.loads((UPLOAD_META / f"{filename}.json").read_text(encoding="utf-8"))
except Exception:
return {}
def update_job(meta, status, detail="", transcript=""):
job_id = str(meta.get("job_id") or "")
if not job_id or not CONTROL_DB.exists():
return
try:
con = sqlite3.connect(CONTROL_DB, timeout=15)
fields, vals = ["status=?", "updated_at=?"], [status, time.time()]
if detail:
fields.append("detail=?"); vals.append(detail[:1000])
if transcript:
fields.append("transcript=?"); vals.append(transcript)
vals.append(job_id)
con.execute(f"UPDATE intake_job SET {','.join(fields)} WHERE id=?", vals)
con.commit()
con.close()
except Exception as exc:
log(f"CONTROL_STATUS_FAILED job={job_id} status={status} error={exc}")
def ensure_job(path, fid):
"""Create a visible control-plane task for files dropped outside Web upload."""
job_id = f"fs-{fid}"
now = time.time()
try:
CONTROL_DB.parent.mkdir(parents=True, exist_ok=True)
con = sqlite3.connect(CONTROL_DB, timeout=15)
con.execute("""
CREATE TABLE IF NOT EXISTS intake_job(
id TEXT PRIMARY KEY, original_name TEXT NOT NULL,
stored_name TEXT NOT NULL, title TEXT NOT NULL DEFAULT '',
context TEXT NOT NULL DEFAULT '', size INTEGER NOT NULL DEFAULT 0,
status TEXT NOT NULL, detail TEXT NOT NULL DEFAULT '',
transcript TEXT NOT NULL DEFAULT '', created_at REAL NOT NULL,
updated_at REAL NOT NULL, draft TEXT NOT NULL DEFAULT '',
result TEXT NOT NULL DEFAULT ''
)
""")
columns = {row[1] for row in con.execute("PRAGMA table_info(intake_job)")}
for name in ("draft", "result"):
if name not in columns:
con.execute(
f"ALTER TABLE intake_job ADD COLUMN {name} "
"TEXT NOT NULL DEFAULT ''")
con.execute(
"INSERT OR IGNORE INTO intake_job("
"id,original_name,stored_name,title,context,size,status,detail,"
"transcript,created_at,updated_at,draft,result"
") VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?)",
(job_id, path.name, path.name, path.stem, "", path.stat().st_size,
"queued", "监控目录已接收", "", now, now, "", ""))
con.commit()
con.close()
return {
"job_id": job_id, "title": path.stem, "context": "",
"original_name": path.name, "stored_name": path.name,
}
except Exception as exc:
log(f"CONTROL_JOB_CREATE_FAILED file={path} error={exc}")
return {}
def duration_seconds(path):
try:
out = subprocess.check_output([
"ffprobe", "-v", "error", "-show_entries", "format=duration",
"-of", "default=nk=1:nw=1", str(path)
], text=True, timeout=30).strip()
return int(float(out) + 0.999) if out else None
except Exception:
return None
def file_id(path):
st = path.stat()
h = hashlib.sha256()
h.update(str(path.name).encode())
h.update(str(st.st_size).encode())
h.update(str(int(st.st_mtime)).encode())
if st.st_size <= 256 * 1024 * 1024:
with path.open("rb") as f:
for chunk in iter(lambda: f.read(1024 * 1024), b""):
h.update(chunk)
return h.hexdigest()[:24]
def stable(path):
try:
st1 = path.stat()
if time.time() - st1.st_mtime < STABLE_AGE_SECONDS:
return False
time.sleep(2)
st2 = path.stat()
return st1.st_size == st2.st_size and int(st1.st_mtime) == int(st2.st_mtime)
except FileNotFoundError:
return False
def unique_path(directory, name):
p = directory / name
if not p.exists():
return p
stem, suffix = p.stem, p.suffix
for i in range(1, 10000):
q = directory / f"{stem}-{i}{suffix}"
if not q.exists():
return q
raise RuntimeError(f"cannot allocate unique path for {name}")
def webhook_secret(route=WEBHOOK_ROUTE):
if route == WEBHOOK_ROUTE and WEBHOOK_SECRET:
return WEBHOOK_SECRET
try:
data = json.loads(SUBSCRIPTIONS.read_text(encoding="utf-8"))
return str(data.get(route, {}).get("secret", ""))
except Exception:
return ""
def post_webhook(payload, url=WEBHOOK_URL, route=WEBHOOK_ROUTE, label="WEBHOOK", timeout=30):
data = json.dumps(payload, ensure_ascii=False).encode("utf-8")
headers = {"Content-Type": "application/json"}
secret = webhook_secret(route)
if secret and secret != "INSECURE_NO_AUTH":
headers["X-Webhook-Signature"] = hmac.new(secret.encode(), data, hashlib.sha256).hexdigest()
req = urllib.request.Request(
url,
data=data,
headers=headers,
method="POST",
)
with urllib.request.urlopen(req, timeout=timeout) as resp:
body = resp.read(1000).decode("utf-8", errors="ignore")
log(f"{label} status={resp.status} body={body[:300]!r}")
return resp.status
def post_status(status, filename, detail=""):
payload = {"status": status, "filename": filename, "detail": detail}
try:
return post_webhook(
payload,
url=STATUS_WEBHOOK_URL,
route=STATUS_WEBHOOK_ROUTE,
label="STATUS_WEBHOOK",
timeout=10,
)
except Exception as exc:
log(f"STATUS_WEBHOOK_FAILED status={status} file={filename} error={exc}")
return None
import re
_CONSUMER_LOCK = None
ASR_INBOX = Path(os.environ.get("ASR_VAULT", str(Path.home() / "obsidian-vault"))) / ".asr-inbox"
def bridge_to_asr_inbox(transcript, stem, title=""):
"""Atomically drop the finished transcript into the Obsidian .asr-inbox so the
Claude refine pipeline picks it up. Slug carries date+time for the draft name."""
m = re.match(r"^(\d{4})-(\d{2})-(\d{2})[_-](\d{2})[_-](\d{2})[_-](\d{2})", stem)
if m:
y, mo, d, hh, mm, ss = m.groups()
tail = stem[m.end():].strip("_- ")
safe_tail = re.sub(
r"[^\w一-鿿-]+", "_", title or tail).strip("_-")
safe_tail = re.sub(r"-[0-9a-f]{8}$", "", safe_tail)
label = safe_tail or "录音"
slug = f"{y}-{mo}-{d}-{hh}{mm}{ss}-{label}"
else:
safe = re.sub(r"[^\w一-鿿-]+", "_", stem).strip("_") or "录音"
slug = f"{time.strftime('%Y-%m-%d-%H%M%S')}-{safe}"
incoming = ASR_INBOX / ".incoming"
incoming.mkdir(parents=True, exist_ok=True)
(ASR_INBOX / ".state").mkdir(parents=True, exist_ok=True)
# Collision guard: never overwrite an unprocessed transcript.
dest = ASR_INBOX / f"{slug}.txt"
n = 2
while dest.exists():
dest = ASR_INBOX / f"{slug}-{n}.txt"
n += 1
tmp = incoming / dest.name
shutil.copyfile(str(transcript), str(tmp))
tmp.replace(dest) # atomic within the vault filesystem -> triggers the path unit
return dest
def process(path):
seen = load_seen()
fid = file_id(path)
intake = upload_meta(path.name) or ensure_job(path, fid)
if fid in seen:
log(f"SKIP already seen file={path} id={fid}")
return
PROCESSING.mkdir(parents=True, exist_ok=True)
TRANSCRIPTS.mkdir(parents=True, exist_ok=True)
PROCESSED.mkdir(parents=True, exist_ok=True)
FAILED.mkdir(parents=True, exist_ok=True)
proc = unique_path(PROCESSING, path.name)
try:
shutil.move(str(path), str(proc))
except FileNotFoundError:
log(f"SKIP disappeared before claim file={path}")
return
transcript = TRANSCRIPTS / f"{fid}-{proc.stem}.txt"
dur = duration_seconds(proc)
log(f"PROCESS file={proc} id={fid} duration={dur}")
update_job(intake, "transcribing", f"录音时长约 {dur}s" if dur else "正在转写")
post_status("开始转录", proc.name, f"时长约 {dur}s;完成后会继续发送粗整理。")
try:
subprocess.run([str(ASR), str(proc), str(transcript)], check=True, timeout=None)
if intake:
original = transcript.read_text(encoding="utf-8", errors="ignore")
marker = json.dumps({
"job_id": intake.get("job_id", ""),
"title": intake.get("title", ""),
"context": intake.get("context", ""),
}, ensure_ascii=False)
transcript.write_text(f"<!-- intake: {marker} -->\n" + original,
encoding="utf-8")
done = unique_path(PROCESSED, proc.name)
shutil.move(str(proc), str(done))
post_status("转录完成,正在生成粗整理", done.name, f"转录文件:{transcript}")
# Refinement is handled by the Claude Code pipeline (not the Hermes agent
# webhook). Bridge the finished transcript into the Obsidian .asr-inbox so
# obsidian-asr-refine (systemd path unit) produces a rough-outline draft.
bridged = bridge_to_asr_inbox(
transcript, proc.stem, str(intake.get("title") or ""))
update_job(intake, "refining", "转写完成,正在生成整理稿",
transcript=str(transcript))
log(f"BRIDGED transcript -> {bridged}")
seen[fid] = {"filename": done.name, "audio_path": str(done), "transcript_path": str(transcript), "ts": int(time.time())}
save_seen(seen)
log(f"DONE file={done} transcript={transcript}")
except Exception as exc:
fail = unique_path(FAILED, proc.name)
if proc.exists():
shutil.move(str(proc), str(fail))
err = FAILED / f"{fail.name}.error.txt"
err.write_text(str(exc), encoding="utf-8")
post_status("处理失败", fail.name, f"错误:{exc}")
update_job(intake, "failed", f"处理失败:{exc}")
log(f"FAILED file={path} error={exc}")
def once():
for p in sorted(INBOX.iterdir() if INBOX.exists() else []):
if not p.is_file() or p.suffix.lower() not in SUPPORTED:
continue
if stable(p):
process(p)
def hold_consumer_lock():
"""Share ASR_STATE_DIR/consumer.lock with asrservice.worker. Exit if it is taken."""
path = Path(os.environ.get("ASR_CONSUMER_LOCK", str(CONTROL_DB.parent / "consumer.lock")))
path.parent.mkdir(parents=True, exist_ok=True)
handle = path.open("a+")
try:
fcntl.flock(handle, fcntl.LOCK_EX | fcntl.LOCK_NB)
except OSError:
handle.close()
log(f"EXIT another recording consumer holds {path}")
raise SystemExit(1)
handle.seek(0)
handle.truncate()
handle.write(str(os.getpid()))
handle.flush()
return handle
def main():
global _CONSUMER_LOCK
_CONSUMER_LOCK = hold_consumer_lock()
for d in [INBOX, PROCESSING, PROCESSED, FAILED, TRANSCRIPTS, STATE, LOGS]:
d.mkdir(parents=True, exist_ok=True)
if not ASR.exists():
raise SystemExit(f"missing ASR script: {ASR}")
log("START watcher")
if "--once" in sys.argv:
once()
return
while True:
try:
once()
except Exception as exc:
log(f"LOOP_ERROR {exc}")
time.sleep(POLL_SECONDS)
if __name__ == "__main__":
main()
@@ -0,0 +1,70 @@
#!/usr/bin/env python3
"""
ECAPA-TDNN 声纹 + KMeans 全局说话人统一(在 ASR venv 里跑)。
把 VibeVoice 每块独立的 S# 标签换成全场统一的 G# ID。
用法(由 vibevoice-transcribe.py --unify 调用,也可单独跑):
<venv>/bin/python vibevoice-speaker-unify.py <audio> <segs.json> <out.md> [K]
局限(见 vault: ASR服务横评-2026-06):单麦远场录音里声纹只能可靠锁"主讲人",
次要说话人分不干净。K 默认 4(K>=5 会劈裂主讲人)。
"""
import json, os, sys, subprocess, numpy as np
import torch, soundfile as sf
from speechbrain.inference.speaker import EncoderClassifier
from sklearn.cluster import KMeans
from collections import Counter
audio = sys.argv[1]
segs_path = sys.argv[2]
out_md = sys.argv[3]
K = int(sys.argv[4]) if len(sys.argv) > 4 else 4
MIN_DUR = 1.2
segs = [s for s in json.load(open(segs_path)) if s.get("Content", "").strip()]
work = os.path.join(os.path.dirname(segs_path), "_segwav"); os.makedirs(work, exist_ok=True)
clf = EncoderClassifier.from_hparams(source="speechbrain/spkrec-ecapa-voxceleb",
savedir=os.path.expanduser("~/.cache/ecapa_model"),
run_opts={"device": "cpu"})
emb, idx = [], []
for i, s in enumerate(segs):
if s["End"] - s["Start"] < MIN_DUR:
continue
wp = os.path.join(work, f"{i:04d}.wav")
subprocess.run(["ffmpeg", "-hide_banner", "-loglevel", "error", "-y", "-ss", str(s["Start"]),
"-t", str(s["End"] - s["Start"]), "-i", audio, "-ar", "16000", "-ac", "1", wp], check=True)
try:
wav, sr = sf.read(wp)
if len(wav) < 16000 * 0.8:
continue
e = clf.encode_batch(torch.tensor(wav, dtype=torch.float32).unsqueeze(0)).squeeze().detach().numpy()
emb.append(e); idx.append(i)
except Exception as ex:
print("embed fail", i, ex)
emb = np.array(emb)
X = emb / np.linalg.norm(emb, axis=1, keepdims=True)
print(f"embedded {len(emb)} segments; clustering K={K}", flush=True)
km = KMeans(n_clusters=K, n_init=10, random_state=0).fit(X)
gid = {idx[k]: int(km.labels_[k]) for k in range(len(idx))}
es = sorted(gid)
for i, s in enumerate(segs):
s["G"] = gid.get(i) if i in gid else gid[min(es, key=lambda j: abs(segs[j]["Start"] - segs[i]["Start"]))]
# rank G by talk-time (G0 = most)
t = Counter()
for s in segs: t[s["G"]] += s["End"] - s["Start"]
rank = {old: new for new, (old, _) in enumerate(t.most_common())}
for s in segs: s["G"] = rank[s["G"]]
tt = Counter()
for s in segs: tt[s["G"]] += s["End"] - s["Start"]
total = sum(tt.values()) or 1
print("talk-time:", {f"G{k}": f"{round(v)}s/{v/total*100:.0f}%" for k, v in sorted(tt.items())})
def hms(x): x = int(x); return f"{x//60:02d}:{x%60:02d}"
lines = [f"# {os.path.basename(audio)} · 全局统一说话人 (ECAPA+KMeans K={K})",
"> G0=说话最多 · 括号内为 VibeVoice 原始每块标签 · 单麦远场次要说话人分离有限 · 仅草稿\n"]
for s in sorted(segs, key=lambda x: x["Start"]):
lines.append(f"- `{hms(s['Start'])}` **G{s['G']}** (S{s['Speaker']}): {s['Content']}")
open(out_md, "w").write("\n".join(lines))
print(f"wrote {out_md}", flush=True)
@@ -0,0 +1,162 @@
#!/usr/bin/env python3
"""
VibeVoice 分角色转录(手动按需后端,非自动管线)。
把任意音频转成带说话人标签 + 时间戳的草稿。默认管线仍是 Qwen3(asr-transcribe.sh);
这个工具用于"需要谁说了什么"的场景。
用法:
vibevoice-transcribe.py <audio> [--out DIR] [--chunk 200] [--unify]
输出:
<out>/<basename>-diarized.md # 带 `mm:ss` 时间戳 + S# 说话人(每块独立编号)
<out>/<basename>-diarized.txt # 纯文本版
--unify 时额外 -global.md # ECAPA+KMeans 全局统一说话人(需 venv,单麦只能锁主讲)
注意(踩坑见 vault: ASR服务横评-2026-06):
- 服务端 segments 是 JSON 字符串,需 json.loads;非语音段无 Speaker 键。
- 200s 是最佳分段(被服务端 max_time 限,单请求 ~280s 音频上限)。
- 仅出草稿,正式纪要仍需 Kai 确认(memo-inbox 纪律)。
"""
import json, subprocess, os, math, re, sys, argparse
URL = os.environ.get("VIBE_URL", "http://192.168.50.100:10112/v1/audio/transcriptions")
MODEL = "vibevoice-asr-hf-nf4"
GLOSSARY = os.path.expanduser("~/obsidian-vault/minutes/_glossary.md")
SEG_RE = re.compile(r'\{"Start":\s*([\d.]+)\s*,\s*"End":\s*([\d.]+)\s*,\s*"Speaker":\s*(\d+|null)\s*,\s*"Content":\s*"((?:[^"\\]|\\.)*)"\s*\}')
def build_prompt():
"""Reuse glossary wikilinks + terms as a hotword prompt (same spirit as asr-transcribe.sh)."""
terms = []
p = GLOSSARY
if os.path.exists(p):
for raw in open(p, errors="ignore").read().splitlines():
s = raw.strip()
if not s or s.startswith("#") or s.startswith(">"):
continue
terms += [m.strip() for m in re.findall(r'\[\[([^\]|#]+)', s)]
s = re.sub(r'^[-*+\d\.\)\s]+', '', s)
s = re.split(r'\s[—–-]\s|[::]', s, maxsplit=1)[0].strip()
s = re.sub(r'\[\[|\]\]', '', s).strip()
if s and len(s) <= 80:
terms.append(s)
seen, out = set(), []
for t in terms:
if t and t not in seen:
seen.add(t); out.append(t)
if not out:
return ""
return "以下是会议中可能出现的人名、公司、项目、产品和术语,请优先按这些词纠正识别错误:" + " ".join(out[:200])
def parse_segs(d):
raw = d.get("segments")
arr = raw if isinstance(raw, list) else None
if arr is None and isinstance(raw, str):
try: arr = json.loads(raw)
except Exception: arr = None
if arr is None:
body = re.sub(r'^\s*assistant\s*', '', d.get("text", ""))
arr = [{"Start": float(a), "End": float(b),
"Speaker": (None if c == "null" else int(c)), "Content": w}
for a, b, c, w in SEG_RE.findall(body)]
out = []
for s in arr:
if not isinstance(s, dict):
continue
cc = (s.get("Content") or "").strip()
if not cc:
continue
out.append({"Start": float(s.get("Start", 0)), "End": float(s.get("End", 0)),
"Speaker": s.get("Speaker"), "Content": cc})
return out
def is_loop(cc):
return len(cc) > 200 and cc[:18] and cc.count(cc[:18]) > 8
def hms(x):
x = int(x); return f"{x // 60:02d}:{x % 60:02d}"
def main():
ap = argparse.ArgumentParser()
ap.add_argument("audio")
ap.add_argument("--out", default=None, help="output dir (default: alongside audio)")
ap.add_argument("--chunk", type=int, default=200)
ap.add_argument("--unify", action="store_true", help="also run ECAPA global speaker unification")
args = ap.parse_args()
audio = os.path.abspath(os.path.expanduser(args.audio))
if not os.path.isfile(audio):
sys.exit(f"audio not found: {audio}")
base = re.sub(r'\.[^.]+$', '', os.path.basename(audio))
outdir = os.path.abspath(args.out) if args.out else os.path.dirname(audio)
os.makedirs(outdir, exist_ok=True)
work = os.path.join(outdir, f".{base}.vibework"); os.makedirs(work, exist_ok=True)
prompt = build_prompt()
dur = float(subprocess.check_output(
["ffprobe", "-v", "error", "-show_entries", "format=duration",
"-of", "default=nk=1:nw=1", audio]).decode().strip())
n = math.ceil(dur / args.chunk)
print(f"[vibevoice] {os.path.basename(audio)} dur={dur:.0f}s -> {n} chunks of {args.chunk}s", flush=True)
all_segs = []
for i in range(n):
start = i * args.chunk
length = min(args.chunk, dur - start)
mp3 = os.path.join(work, f"c{i:02d}.mp3")
subprocess.run(["ffmpeg", "-hide_banner", "-loglevel", "error", "-y", "-ss", str(start),
"-t", str(length), "-i", audio, "-ar", "16000", "-ac", "1", "-b:a", "64k", mp3], check=True)
out_json = os.path.join(work, f"c{i:02d}.json")
mnt = min(4096, 1024 + args.chunk * 12)
rc = subprocess.run(["curl", "-sS", "--max-time", "260", "-F", f"model={MODEL}",
"-F", f"file=@{mp3}", "-F", f"prompt={prompt}",
"-F", f"max_new_tokens={mnt}", URL, "-o", out_json]).returncode
try:
d = json.load(open(out_json)); segs = parse_segs(d)
except Exception as e:
print(f" chunk {i+1}/{n} FAIL ({e}) rc={rc}", flush=True); continue
loops = sum(1 for s in segs if is_loop(s["Content"]))
last = max((s["End"] for s in segs), default=0)
for s in segs:
cc = s["Content"]
if len(cc) > 400:
cc = cc[:120] + " …[复读截断]"
all_segs.append({"Start": start + s["Start"], "End": start + s["End"],
"Speaker": s["Speaker"], "Content": cc})
tag = f" ⚠loop×{loops}" if loops else ""
print(f" chunk {i+1}/{n} @{start:.0f}s: {len(segs)} segs, cov->{last:.0f}/{length:.0f}s{tag}", flush=True)
all_segs.sort(key=lambda x: x["Start"])
md = [f"# {base} · VibeVoice 分角色草稿",
f"> 音频 {dur/60:.0f}min · {args.chunk}s 切块 · 说话人 S# **每块独立编号**(块间需对齐) · 仅草稿待确认\n"]
txt = []
for s in all_segs:
spk = f"S{s['Speaker']}" if s["Speaker"] is not None else "·"
md.append(f"- `{hms(s['Start'])}` **{spk}**: {s['Content']}")
txt.append(f"[{hms(s['Start'])}] {spk}: {s['Content']}")
md_path = os.path.join(outdir, f"{base}-diarized.md")
txt_path = os.path.join(outdir, f"{base}-diarized.txt")
open(md_path, "w").write("\n".join(md))
open(txt_path, "w").write("\n".join(txt))
json.dump(all_segs, open(os.path.join(work, "segs.json"), "w"), ensure_ascii=False)
print(f"[vibevoice] {len(all_segs)} segs -> {md_path}", flush=True)
if args.unify:
unify = os.path.join(os.path.dirname(os.path.abspath(__file__)), "vibevoice-speaker-unify.py")
venv_py = os.environ.get("ASR_VENV_PY",
os.path.expanduser("~/claude-workspace/OBSIDIAN/recordings/asr-venv/bin/python"))
if os.path.exists(venv_py) and os.path.exists(unify):
print("[vibevoice] running ECAPA global speaker unification ...", flush=True)
subprocess.run([venv_py, unify, audio, os.path.join(work, "segs.json"),
os.path.join(outdir, f"{base}-global.md")])
else:
print(f"[vibevoice] --unify skipped: venv ({venv_py}) or unify script missing. "
f"See vault ASR服务横评 to rebuild venv.", flush=True)
if __name__ == "__main__":
main()
@@ -1,3 +1,3 @@
pypinyin==0.55.0 pypinyin>=0.51,<1
rapidfuzz==3.14.1 rapidfuzz>=3,<4
numpy==2.3.3 numpy>=1.26,<3
@@ -0,0 +1,51 @@
#!/usr/bin/env python3
"""Send a Telegram status ping through the existing Hermes meeting-recording-status
webhook. Used by the Claude refine step to notify Kai that a draft is ready, reusing
the same signed channel that meeting-watch.py uses for 开始转录/转录完成.
Usage: meeting-status-notify.py "<status>" "<filename>" "<detail>"
Silent no-op (exit 0) if the webhook/secret is unavailable — never block the pipeline.
"""
import hashlib
import hmac
import json
import os
import sys
import urllib.request
from pathlib import Path
URL = os.environ.get("MEETING_STATUS_WEBHOOK_URL", "http://127.0.0.1:8644/webhooks/meeting-recording-status")
ROUTE = os.environ.get("MEETING_STATUS_WEBHOOK_ROUTE", "meeting-recording-status")
SUBSCRIPTIONS = Path(os.environ.get("MEETING_SUBSCRIPTIONS_FILE", str(Path.home() / ".hermes" / "webhook_subscriptions.json")))
def secret():
if os.environ.get("MEETING_STATUS_WEBHOOK_SECRET"):
return os.environ["MEETING_STATUS_WEBHOOK_SECRET"]
try:
data = json.loads(SUBSCRIPTIONS.read_text(encoding="utf-8"))
return str(data.get(ROUTE, {}).get("secret", ""))
except Exception:
return ""
def main():
status = sys.argv[1] if len(sys.argv) > 1 else "通知"
filename = sys.argv[2] if len(sys.argv) > 2 else ""
detail = sys.argv[3] if len(sys.argv) > 3 else ""
payload = json.dumps({"status": status, "filename": filename, "detail": detail},
ensure_ascii=False).encode("utf-8")
headers = {"Content-Type": "application/json"}
sec = secret()
if sec and sec != "INSECURE_NO_AUTH":
headers["X-Webhook-Signature"] = hmac.new(sec.encode(), payload, hashlib.sha256).hexdigest()
try:
req = urllib.request.Request(URL, data=payload, headers=headers, method="POST")
with urllib.request.urlopen(req, timeout=10) as resp:
print(f"status webhook -> {resp.status}")
except Exception as exc:
print(f"status webhook skipped: {exc}")
if __name__ == "__main__":
main()
@@ -0,0 +1,86 @@
from __future__ import annotations
import json
import os
import tempfile
import unittest
from pathlib import Path
TMP = tempfile.TemporaryDirectory()
os.environ["ASR_STATE_DIR"] = str(Path(TMP.name) / "state")
os.environ["ASR_VAULT"] = str(Path(TMP.name) / "vault")
os.environ["ASR_RECORDINGS_ROOT"] = str(Path(TMP.name) / "recordings")
from asrpipe import control_web, review, segment # noqa: E402
class ControlWebTests(unittest.TestCase):
def setUp(self):
self.db_dir = Path(TMP.name) / "control-state"
self.db_dir.mkdir(parents=True, exist_ok=True)
control_web.DB = self.db_dir / "intake.db"
def test_job_state_is_persistent(self):
con = control_web._connect()
now = 100.0
con.execute(
"INSERT INTO intake_job(id,original_name,stored_name,title,context,size,"
"status,created_at,updated_at) VALUES(?,?,?,?,?,?,?,?,?)",
("j1", "call.mp3", "stored.mp3", "项目通话", "背景", 123,
"queued", now, now))
con.commit()
con.close()
self.assertTrue(control_web.update_job(
"j1", "transcribing", "正在转写"))
job = control_web._jobs(1)[0]
self.assertEqual(job["status"], "transcribing")
self.assertEqual(job["detail"], "正在转写")
def test_schema_migrates_task_review_fields(self):
con = control_web._connect()
columns = {row["name"] for row in con.execute(
"PRAGMA table_info(intake_job)")}
con.close()
self.assertIn("draft", columns)
self.assertIn("result", columns)
def test_review_queue_is_scoped_to_one_job(self):
con = control_web._connect()
now = 200.0
con.execute(
"INSERT INTO intake_job(id,original_name,stored_name,title,context,size,"
"status,detail,transcript,created_at,updated_at,draft,result) "
"VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?)",
("job-scope", "scope.wav", "scope.wav", "范围测试", "", 10,
"review", "", "/archive/scope.txt", now, now,
"/draft/scope.md", ""))
con.commit()
con.close()
own = review.enqueue("/archive/scope.txt", "本任务问题?", {
"kind": "draft_question"})
review.enqueue("/archive/other.txt", "其他任务问题?", {
"kind": "draft_question"})
job = control_web._job("job-scope")
rows = control_web._pending_for_job(job)
self.assertEqual([row["id"] for row in rows], [own])
self.assertTrue(control_web._review_url("job-scope").endswith(
"/jobs/job-scope/review/"))
def test_intake_metadata_is_not_transcript_body(self):
metadata = {
"job_id": "abc", "title": "项目通话",
"context": "参与人是张三和李四",
}
raw = (
"<!-- intake: " + json.dumps(metadata, ensure_ascii=False) + " -->\n"
"<!-- voiceprint_session: sess clusters: 1 -->\n"
"# 转写(Qwen3-ASR 1.7B + CAM++ 分角色 + 声纹)\n\n"
"【speaker_0】测试正文。\n")
turns, meta = segment.parse(raw)
self.assertEqual(meta["intake"], metadata)
self.assertEqual(turns[0].text, "测试正文。")
self.assertNotIn("参与人", turns[0].text)
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,409 @@
from __future__ import annotations
import json
import os
import subprocess
import tempfile
import unittest
from pathlib import Path
from unittest import mock
TMP = tempfile.TemporaryDirectory()
os.environ["ASR_STATE_DIR"] = str(Path(TMP.name) / "state")
os.environ["ASR_VAULT"] = str(Path(TMP.name) / "vault")
os.environ["XDG_RUNTIME_DIR"] = TMP.name
from asrpipe import cli, context, flagger, graph, lexicon, numbers, review, revise, vault # noqa: E402
from asrpipe.config import CFG # noqa: E402
from asrpipe.refine import _apply_settled, _queue_ambiguous # noqa: E402
from asrpipe import refine # noqa: E402
from asrpipe.segment import Chunk, Turn # noqa: E402
class CoreTests(unittest.TestCase):
def setUp(self):
CFG.vault.mkdir(parents=True, exist_ok=True)
def test_context_compacts_duplicate_recall(self):
raw = [
"刘冀是张科领弋的基金投资人,目前处于活跃合作状态。",
"刘冀是张科领弋的基金投资人,最近联系日期为2026年6月26日。",
"刘冀是张科领弋的基金投资人,目前处于活跃合作状态。",
]
out = context.compact(["刘冀"], raw, per_entity=2)
self.assertEqual(out.count("目前处于活跃合作状态"), 1)
self.assertLessEqual(out.count("\n- "), 2)
def test_settled_mapping_is_deterministic_and_common_word_is_safe(self):
flags = [
{"span": "老万", "settled": True, "common_word": False,
"candidates": [{"target": "万亚坤"}]},
{"span": "公司", "settled": True, "common_word": True,
"candidates": [{"target": "工时"}]},
]
out, decisions = _apply_settled("老万说公司暂时不投。", flags)
self.assertEqual(out, "万亚坤说公司暂时不投。")
self.assertEqual(decisions[0]["surface"], "老万")
def test_review_answer_becomes_reusable_alias(self):
qid = review.enqueue("fixture-1", "老万是不是万亚坤?", {
"surface": "老万", "candidates": ["万亚坤"],
"sentence": "老万说洛启的事情"})
result = review.answer(qid, "万亚坤")
self.assertTrue(result["learned"])
self.assertIn("万亚坤", [x[0] for x in lexicon.alias_map()["老万"]])
def test_review_answer_is_idempotent(self):
qid = review.enqueue("fixture-idempotent", "是否确认?", {
"surface": "老张", "candidates": ["张成海"]})
first = review.answer(qid, "张成海")
second = review.answer(qid, "另一个人")
self.assertTrue(first["learned"])
self.assertTrue(second["already_answered"])
self.assertEqual(second["answer"], "张成海")
def test_draft_delivery_queues_questions_and_persists_answers(self):
CFG.drafts.mkdir(parents=True, exist_ok=True)
path = CFG.drafts / "web-draft.md"
path.write_text(
"# 草稿\n\n## 通话背景(待确认)\n\n双方讨论项目安排。\n\n"
"## 待你确认\n\n"
"1. 刘总是 [[刘冀]] 还是 [[刘大涛]]?\n"
"2. speaker_0 是谁?\n",
encoding="utf-8")
count = review.enqueue_draft("web-draft-recording", path)
self.assertEqual(count, 4)
rows = [r for r in review.pending(20)
if r["recording"] == "web-draft-recording"]
self.assertEqual(len(rows), 4)
self.assertEqual(json.loads(rows[0]["payload"])["kind"],
"background_check")
question_payload = json.loads(rows[1]["payload"])
self.assertEqual(question_payload["candidates"], ["刘冀", "刘大涛"])
with mock.patch.object(vault, "sync", return_value="disabled"), \
mock.patch.object(review, "_request_revision", return_value=True):
for row, answer_value in zip(
rows, ["背景基本准确", "刘冀", "谭凯", "草稿整体可用"]):
review.answer(row["id"], answer_value)
text = path.read_text(encoding="utf-8")
self.assertIn("Web 人工确认记录", text)
self.assertIn("**回答**:刘冀", text)
self.assertIn("**回答**:草稿整体可用", text)
def test_draft_delivery_prompts_only_substantive_missing_voiceprints(self):
CFG.drafts.mkdir(parents=True, exist_ok=True)
transcripts = Path(TMP.name) / "voiceprint-transcript.txt"
transcripts.write_text(
"【speaker_0】" + "这是高瑶对协议条款和保密范围的详细说明。" * 8 + "\n"
"【speaker_2】嗯。\n"
"【谭凯】好的。\n", encoding="utf-8")
path = CFG.drafts / "voiceprint-draft.md"
path.write_text(
"---\nstatus: 草稿待确认\n"
f"transcript: {transcripts}\n"
"voiceprint_session: test-session\n"
"voiceprint_speakers:\n speaker_1: 谭凯\n---\n\n"
"# 草稿\n\n## 通话背景(待确认)\n\n协议修改沟通。\n\n"
"## 待你确认\n\n1. speaker_0 身份是谁?\n",
encoding="utf-8")
review.enqueue_draft("voiceprint-recording", path)
rows = [r for r in review.pending(20)
if r["recording"] == "voiceprint-recording"]
kinds = [json.loads(r["payload"]).get("kind") for r in rows]
self.assertEqual(kinds.count("voiceprint"), 1)
voice = next(json.loads(r["payload"]) for r in rows
if json.loads(r["payload"]).get("kind") == "voiceprint")
self.assertEqual(voice["speaker_id"], "speaker_0")
self.assertFalse(any("speaker_2" in r["question"] for r in rows))
def test_entity_card_suggestion_counts_distinct_formal_sources(self):
con = lexicon.connect()
con.execute(
"INSERT INTO entity_mention(entity,kind,source,occurred_at) "
"VALUES(?,?,?,?)", ("高瑶", "人名", "minute:older.md", 1))
con.commit()
con.close()
CFG.drafts.mkdir(parents=True, exist_ok=True)
path = CFG.drafts / "entity-suggestion.md"
path.write_text(
"---\npeople: [高瑶]\ncompanies: []\n---\n\n"
"# 草稿\n\n## 通话背景(待确认)\n\n协议沟通。\n\n"
"## 待你确认\n\n", encoding="utf-8")
review.enqueue_draft("entity-suggestion-recording", path)
rows = [r for r in review.pending(20)
if r["recording"] == "entity-suggestion-recording"]
cards = [json.loads(r["payload"]) for r in rows
if json.loads(r["payload"]).get("kind") == "entity_card"]
self.assertEqual(len(cards), 1)
self.assertEqual(cards[0]["entity"], "高瑶")
self.assertEqual(cards[0]["entity_kind"], "人名")
def test_first_review_prepares_preview_without_publishing(self):
CFG.drafts.mkdir(parents=True, exist_ok=True)
path = CFG.drafts / "publish-fixture.md"
path.write_text(
"---\ndate: 2026-07-26\ntype: minutes\n"
"tags: [会议纪要, 草稿, 待确认]\nsource: 录音转录\n"
"status: 草稿待确认\npeople: [谭凯]\ncompanies: []\n"
"transcript: /archive/publish.txt\n---\n\n# 粗整理草稿\n\n"
f"{review.FEEDBACK_START}\n## Web 人工确认记录\n\n"
"1. **问题**:背景?\n - **回答**:准确\n\n"
f"{review.FEEDBACK_END}\n", encoding="utf-8")
formal = """---
date: 2026-07-26
type: minutes
tags: [会议纪要]
source: 录音转录
people: [谭凯]
companies: []
transcript: /archive/publish.txt
---
# 谭凯通话测试
> [!summary] Summary
> 测试正式发布。
"""
with mock.patch.object(revise, "chat", return_value=formal), \
mock.patch("asrpipe.review_web.request_start", return_value=True):
self.assertTrue(revise.revise(path))
self.assertTrue(path.exists())
self.assertIn("status: 待发布确认", path.read_text())
approvals = [
r for r in review.pending(100)
if json.loads(r["payload"]).get("draft_path") == str(path)
and json.loads(r["payload"]).get("kind") == "publish_approval"
]
self.assertEqual(len(approvals), 1)
destination = CFG.minutes / "2026-07-26-谭凯通话测试.md"
self.assertFalse(destination.exists())
with mock.patch.object(review, "_request_revision", return_value=False):
review.answer(approvals[0]["id"], "发布正式稿")
with mock.patch.object(vault, "sync", return_value="pushed"), \
mock.patch.object(vault, "notify"), \
mock.patch.object(graph, "sync", return_value={}):
self.assertTrue(revise.revise(path))
self.assertTrue(destination.exists())
self.assertNotIn("草稿待确认", destination.read_text())
self.assertNotIn("待发布确认", destination.read_text())
self.assertFalse(path.exists())
self.assertTrue((CFG.state_dir / "published" / path.name).exists())
def test_normalize_can_disable_review_side_effects(self):
chunk = Chunk(0, [Turn("", "老万说洛启项目。", 0)])
flags = [{
"span": "老万", "settled": False, "common_word": False,
"sentence": "老万说洛启项目。",
"candidates": [{
"target": "万亚坤", "kind": "人名", "score": 0.9,
"rule": "alias", "source": "alias-card",
}],
}]
usage = refine.Usage()
with mock.patch.object(refine, "_queue_ambiguous") as queue, \
mock.patch.object(refine, "chat_many",
return_value=["老万说洛启项目。"]):
_parts, _decisions, stats = refine.normalize_chunks(
[chunk], [chunk.body], flags, "", "dry-run-fixture", usage,
queue_review=False)
queue.assert_not_called()
self.assertEqual(stats["review_queued"], 0)
def test_atomic_write_and_recovery_lookup(self):
CFG.drafts.mkdir(parents=True, exist_ok=True)
p = CFG.drafts / "fixture.md"
vault.write_draft_file(p, "---\ntranscript: /archive/a.txt\n---\nbody")
self.assertEqual(vault.find_draft_for_transcript("/archive/a.txt"), p)
leftovers = list(CFG.drafts.glob(".fixture.md.*"))
self.assertEqual(leftovers, [])
def test_relation_card_is_structured(self):
lexicon.upsert_relation("刘冀", "投资", "张科领弋", confidence=0.9,
source="test")
card = lexicon.relationship_card({"刘冀"})
self.assertTrue(any("刘冀 投资 张科领弋" in x for x in card))
def test_confirmed_correction_enters_personal_confusion_channel(self):
lexicon.learn_correction("张泰海", "张成海", confirmed=1,
note="ASR fixture")
rows = lexicon.confusion_map()["张泰海"]
self.assertTrue(any(canon == "张成海" and confirmed
for canon, _n, confirmed in rows))
def test_confirmed_correction_teaches_directional_syllable_error(self):
lexicon.learn_correction("张泰海", "张成海", confirmed=1)
flagger._PERSONAL_SYL = None
heard, intended = flagger.to_syls("泰")[0], flagger.to_syls("成")[0]
self.assertGreater(flagger.syl_sim(heard, intended, personal=True), 0.90)
self.assertLess(flagger.syl_sim(heard, intended, personal=False), 0.90)
def test_graph_indexes_minutes_and_recency(self):
for sub in ("People", "Companies", "Projects", "minutes"):
(CFG.vault / sub).mkdir(parents=True, exist_ok=True)
(CFG.vault / "People" / "刘冀.md").write_text(
"---\ncompany: 张科领弋\nrole: 投资人\n---\n", encoding="utf-8")
(CFG.vault / "Companies" / "张科领弋.md").write_text(
"---\n---\n", encoding="utf-8")
(CFG.vault / "Projects" / "杭州基金.md").write_text(
"---\n---\n", encoding="utf-8")
(CFG.vault / "minutes" / "2026-07-25-test.md").write_text(
"---\ndate: 2026-07-25\npeople: [刘冀]\ncompanies: [张科领弋]\n"
"---\n[[刘冀]]参与[[杭州基金]],并与[[张科领弋]]合作。\n",
encoding="utf-8")
result = graph.sync(force=True)
self.assertGreaterEqual(result["relations_written"], 3)
weights = graph.context_weights({"刘冀"})
self.assertIn("张科领弋", weights)
self.assertGreater(weights["张科领弋"], 0)
def test_number_normalization_is_typed_and_conservative(self):
text = "七月十日打款一千万元,BF四幺八二项目占百分之十五,共三十人,两万五九份待核。"
out, decisions = numbers.normalize(text)
self.assertIn("7月10日", out)
self.assertIn("1000万元", out)
self.assertIn("BF4182", out)
self.assertIn("15%", out)
self.assertIn("30人", out)
self.assertIn("两万五九份", out)
self.assertTrue(any(d["raw"] == "两万五九" and d["confidence"] < 0.8
for d in decisions))
def test_generic_discourse_counter_stays_chinese(self):
out, _ = numbers.normalize("我同步两个事,第一个是条款,第二个是公告。")
self.assertEqual(out, "我同步两个事,第一个是条款,第二个是公告。")
def test_financial_scale_without_currency_is_normalized_from_context(self):
out, decisions = numbers.normalize("八月份投资两千万,后续再退出。")
self.assertIn("投资2000万元", out)
self.assertTrue(any(d["kind"] == "money" for d in decisions))
def test_process_lock_remains_held(self):
handle = cli._lock()
lock_path = str(Path(TMP.name) / "asr-pipeline.lock")
code = (
"import fcntl,sys;"
f"f=open({lock_path!r},'w');"
"\ntry: fcntl.flock(f,fcntl.LOCK_EX|fcntl.LOCK_NB)"
"\nexcept OSError: sys.exit(23)"
"\nsys.exit(0)"
)
r = subprocess.run([os.sys.executable, "-c", code])
self.assertEqual(r.returncode, 23)
handle.close()
cli._LOCK_HANDLE = None
def test_staged_pipeline_smoke_without_network(self):
old = (CFG.use_termbank, CFG.use_hindsight, CFG.use_keyword_recall,
CFG.use_normalize, CFG.use_evidence)
CFG.use_termbank = False
CFG.use_hindsight = False
CFG.use_keyword_recall = False
CFG.use_normalize = False
CFG.use_evidence = True
def fake_many(_model, jobs, *, system=None, **_kw):
if system == refine.prompts.CLEAN_SYS:
return ["【谭凯】讨论洛启项目,下一步确认投资条款。" for _ in jobs]
return ["- [片段 0] 决策|洛启项目|下一步确认投资条款|置信度:明确"
for _ in jobs]
try:
with mock.patch.object(refine, "chat_many", side_effect=fake_many), \
mock.patch.object(refine, "chat", return_value="# 粗整理\n\n确认投资条款。"):
result = refine.refine(
"【谭凯】嗯,讨论洛启项目,然后下一步确认投资条款。",
date="2026-07-26", transcript_path="/archive/test.txt")
self.assertIn("确认投资条款", result.draft)
self.assertIn("[片段 0]", result.evidence)
self.assertEqual(result.stats["n_chunks"], 1)
finally:
(CFG.use_termbank, CFG.use_hindsight, CFG.use_keyword_recall,
CFG.use_normalize, CFG.use_evidence) = old
def test_pure_phonetic_guess_does_not_interrupt_human(self):
groups = [{
"span": "也说", "settled": False, "common_word": False,
"sentence": "但是他也说他肯定不会反驳",
"candidates": [{
"target": "叶硕", "kind": "人名", "score": 0.91,
"rule": "full", "source": "lexicon-entity",
"boost": 0, "support": "",
}],
}]
self.assertEqual(_queue_ambiguous("fixture-junk", groups), 0)
def test_context_adjudicator_must_explicitly_request_human_review(self):
groups = [{
"span": "老万", "settled": False, "common_word": False,
"sentence": "老万说洛启项目。",
"candidates": [{
"target": "万亚坤", "kind": "人名", "score": 0.92,
"rule": "alias", "source": "alias-card",
"boost": 1, "support": "关系图",
}],
}]
self.assertEqual(
_queue_ambiguous("fixture-context-decided", groups,
"老万说洛启项目。"), 0)
self.assertEqual(
_queue_ambiguous("fixture-context-uncertain", groups,
"[?万亚坤]说洛启项目。"), 1)
def test_common_words_block_embedded_name_guesses(self):
cases = [
("小问", "UPS是某个小问题", ["万亚坤", "万津"]),
("用介", "包括所有的这些公用介质报告", ["邓永杰"]),
("们总", "我上次问他们总监", ["卢萌"]),
("经签", "所有方案已经签掉", ["徐劲前"]),
("统统", "三批之间统统计一下", ["薛彤彤"]),
("峰总", "雷峰总这边的开发部", ["冯昆仑"]),
]
for i, (surface, sentence, choices) in enumerate(cases):
groups = [{
"span": surface, "settled": False, "common_word": False,
"sentence": sentence,
"candidates": [{
"target": target, "kind": "人名", "score": 0.92,
"rule": "alias", "source": "alias-card",
"boost": 1, "support": "关系图",
} for target in choices],
}]
marker = f"[?{'/'.join(choices)}]"
self.assertEqual(
_queue_ambiguous(f"fixture-common-{i}", groups, marker), 0)
def test_archive_resolution_uses_content_when_timestamp_collides(self):
old_obsidian = CFG.obsidian
root = Path(TMP.name) / "archive-resolution"
archive = root / "recordings" / "transcripts"
archive.mkdir(parents=True, exist_ok=True)
inbox = root / "2026-07-27-095902-录音-2.txt"
inbox.write_text("wav transcript", encoding="utf-8")
mp3 = archive / "aaa-2026-07-27_09_59_02.txt"
wav = archive / "bbb-2026-07-27_09_59_02.txt"
mp3.write_text("mp3 transcript", encoding="utf-8")
wav.write_text("wav transcript", encoding="utf-8")
CFG.obsidian = root
try:
self.assertEqual(
cli.resolve_archive("2026-07-27-095902-录音-2", inbox),
str(wav))
finally:
CFG.obsidian = old_obsidian
def test_observed_neighbour_needs_more_than_recent_weight(self):
groups = [{
"span": "们七", "settled": False, "common_word": False,
"sentence": "你们七月二十号签完",
"candidates": [{
"target": "范梦奇", "kind": "人名", "score": 0.96,
"rule": "observed", "source": "observed-correction",
"via": "范梦奇", "boost": 1.2, "support": "近期权重4.0",
}],
}]
self.assertEqual(_queue_ambiguous("fixture-observed-junk", groups), 0)
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,47 @@
from __future__ import annotations
import os
import subprocess
import sys
import tempfile
import unittest
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
class PortabilityTests(unittest.TestCase):
def probe(self, code, overrides):
env = {**os.environ, **overrides, "PYTHONPATH": str(ROOT),
"PYTHONDONTWRITEBYTECODE": "1"}
return subprocess.check_output([sys.executable, "-c", code],
env=env, text=True).strip()
def test_workspace_and_notifier_are_explicit(self):
with tempfile.TemporaryDirectory() as tmp:
result = self.probe(
"from asrpipe.config import CFG; print(CFG.obsidian); print(CFG.notifier)",
{"ASR_WORKSPACE": tmp, "ASR_NOTIFIER": tmp + "/notify.py"})
self.assertEqual(result.splitlines(), [tmp, tmp + "/notify.py"])
def test_watcher_does_not_assume_claude_workspace(self):
with tempfile.TemporaryDirectory() as tmp:
result = self.probe(
"import runpy; from pathlib import Path; "
"d=runpy.run_path('recordings/bin/meeting-watch.py',run_name='probe'); "
"print(d['CONTROL_DB']); print(d['ASR_INBOX'])",
{"ASR_STATE_DIR": tmp + "/state", "ASR_VAULT": tmp + "/vault"})
self.assertEqual(result.splitlines(),
[tmp + "/state/intake.db", tmp + "/vault/.asr-inbox"])
def test_notifier_accepts_external_webhook_without_sending(self):
result = self.probe(
"import runpy; d=runpy.run_path('scripts/meeting-status-notify.py',run_name='probe'); "
"print(d['URL']); print(d['secret']() == 'synthetic-only')",
{"MEETING_STATUS_WEBHOOK_URL": "http://example.invalid/hook",
"MEETING_STATUS_WEBHOOK_SECRET": "synthetic-only"})
self.assertEqual(result.splitlines(), ["http://example.invalid/hook", "True"])
if __name__ == "__main__":
unittest.main()
@@ -1,51 +1,341 @@
from __future__ import annotations from __future__ import annotations
import importlib
import json import json
import os import os
import tempfile import tempfile
import threading
import unittest import unittest
import urllib.parse
import urllib.request
from http.server import ThreadingHTTPServer
from pathlib import Path from pathlib import Path
from unittest import mock
TMP = tempfile.TemporaryDirectory()
os.environ["ASR_DATA_DIR"] = str(Path(TMP.name) / "data")
os.environ["ASR_STATE_DIR"] = str(Path(TMP.name) / "state")
os.environ["ASR_VAULT"] = str(Path(TMP.name) / "vault")
os.environ["ASR_RECORDINGS_ROOT"] = str(Path(TMP.name) / "recordings")
os.environ["ASR_AGENT_TOKEN"] = "test-agent-token"
os.environ["ASR_NOTIFY"] = "0"
os.environ["ASR_GIT_SYNC"] = "0"
from asrpipe import lexicon, providers, review, segment # noqa: E402
from asrpipe.config import CFG # noqa: E402
from asrservice import config as svc_cfg, db as svc_db, web as svc_web # noqa: E402
class ServiceTests(unittest.TestCase): class ProviderAndServiceTests(unittest.TestCase):
@classmethod def setUp(self):
def setUpClass(cls): self.tmp = tempfile.TemporaryDirectory()
cls.tmp = tempfile.TemporaryDirectory() self.root = Path(self.tmp.name)
os.environ["ASR_DATA_DIR"] = str(Path(cls.tmp.name) / "data") svc_cfg.ensure_dirs()
os.environ["ASR_STATE_DIR"] = str(Path(cls.tmp.name) / "state") svc_db.connect().close()
from asrservice import config, db, worker
cls.config, cls.db, cls.worker = config, db, worker
@classmethod def tearDown(self):
def tearDownClass(cls): self.tmp.cleanup()
cls.tmp.cleanup()
def test_job_events_follow_updates(self): def test_sqlite_settings_override_config_dynamically(self):
row = self.db.create_job("a" * 24, "x.m4a", "x.m4a", source="ssh") svc_db.save_settings({
self.assertEqual(row["status"], "queued") "asr_provider": "soniox",
self.db.update_job(row["id"], "transcribing", "started", attempts=1) "litellm_base_url": "https://api.opfo.xyz/v1",
current = self.db.get_job(row["id"]) "litellm_api_key": "sk-sqlite-dynamic-key",
self.assertEqual(current["attempts"], 1) "clean_model": "aistudio/gemini-flash-latest",
self.assertEqual([e["status"] for e in self.db.events(row["id"])], "draft_model": "openai/gpt-sol-latest",
["queued", "transcribing"]) "soniox_api_key": "soniox-db-secret",
"use_hindsight": "0",
def test_transcript_contract(self):
text = self.worker.transcript_from_response({
"segments": [
{"speaker": "speaker_0", "text": "你好。"},
{"speaker": "speaker_0", "text": "继续。"},
{"speaker": "speaker_1", "text": "收到。"},
],
"voiceprint": {"session_id": "s1", "matches": []},
}) })
self.assertIn("voiceprint_session: s1", text) self.assertEqual(CFG.asr_provider, "soniox")
self.assertIn("【speaker_0】你好。继续。", text) self.assertEqual(CFG.litellm_base_url, "https://api.opfo.xyz/v1")
self.assertEqual(text.count("【speaker_0】"), 1) self.assertEqual(CFG.endpoint, "https://api.opfo.xyz/v1/chat/completions")
self.assertEqual(CFG.transcriptions_endpoint, "https://api.opfo.xyz/v1/audio/transcriptions")
self.assertEqual(CFG.api_key(), "sk-sqlite-dynamic-key")
self.assertEqual(CFG.clean_model, "aistudio/gemini-flash-latest")
self.assertEqual(CFG.draft_model, "openai/gpt-sol-latest")
self.assertEqual(CFG.soniox_api_key, "soniox-db-secret")
self.assertFalse(CFG.use_hindsight)
# Restore default for subsequent tests
svc_db.save_settings({"asr_provider": "qwen3cam", "use_hindsight": "1"})
def test_transcript_rejects_empty_segments(self): def test_soniox_token_diarization_and_segment_parse(self):
with self.assertRaises(RuntimeError): payload = {
self.worker.transcript_from_response({"segments": []}) "tokens": [
{"text": "大家", "speaker": "1"},
{"text": "好,", "speaker": "1"},
{"text": "今天", "speaker": "1"},
{"text": "讨论", "speaker": "1"},
{"text": "预算。", "speaker": "1"},
{"text": "收到,", "speaker": "2"},
{"text": "我来", "speaker": "2"},
{"text": "汇报。", "speaker": "2"},
]
}
text = providers.transcript_from_soniox_response(payload, model="stt-async-v5")
with_intake = '<!-- intake: {"job_id": "abc", "asr_provider": "soniox", "asr_model": "stt-async-v5"} -->\n' + text
parsed_turns, meta = segment.parse(with_intake)
self.assertEqual(meta["asr_provider"], "soniox")
self.assertEqual(meta["asr_model"], "stt-async-v5")
self.assertEqual(meta["backend"], "soniox")
self.assertEqual(len(parsed_turns), 2)
self.assertEqual(parsed_turns[0].speaker, "speaker_0")
self.assertEqual(parsed_turns[0].text, "大家好,今天讨论预算。")
self.assertEqual(parsed_turns[1].speaker, "speaker_1")
self.assertEqual(parsed_turns[1].text, "收到,我来汇报。")
with self.assertRaises(providers.EmptyAudioError):
providers.transcript_from_soniox_response({"tokens": [], "text": ""}, model="stt-async-v5")
def test_soniox_context_includes_lexicon_and_intake(self):
lexicon.learn_alias("lite网关", "LiteLLM", note="test")
ctx = providers.build_soniox_context(
title="网关评审",
user_context="讨论 LiteLLM 与 Soniox 集成",
)
self.assertIn("terms", ctx)
self.assertIn("LiteLLM", ctx["terms"])
self.assertIn("讨论 LiteLLM", ctx.get("text", ""))
def test_openai_and_grok_diarized_transcriptions(self):
audio = self.root / "sample.mp3"
audio.write_bytes(b"ID3-fake-audio")
def fake_openai_multipart(url, *, fields, file_field, file_path, headers=None, timeout=900):
return {
"segments": [
{"speaker": "A", "text": "第一句发言。"},
{"speaker": "A", "text": "继续补充。"},
{"speaker": "B", "text": "第二位发言。"},
]
}
with mock.patch("asrpipe.providers._http_multipart", side_effect=fake_openai_multipart), \
mock.patch("asrpipe.providers.probe_duration_seconds", return_value=30), \
mock.patch.object(providers.CFG, "api_key", return_value="sk-test"):
res = providers.transcribe_openai(
audio,
title="OpenAI 测试",
model="openai/gpt-4o-transcribe-diarize",
)
turns, meta = segment.parse(res.transcript_text)
self.assertEqual(meta["asr_provider"], "openai")
self.assertEqual(len(turns), 2)
self.assertIn("第一句发言。继续补充。", turns[0].text)
self.assertEqual(turns[1].speaker, "speaker_1")
def fake_grok_multipart(url, *, fields, file_field, file_path, headers=None, timeout=900):
return {
"utterances": [
{"speaker": 0, "text": "Grok 说话人零。"},
{"speaker": 1, "text": "Grok 说话人一。"},
]
}
with mock.patch("asrpipe.providers._http_multipart", side_effect=fake_grok_multipart), \
mock.patch("asrpipe.providers.probe_duration_seconds", return_value=30), \
mock.patch.object(providers.CFG, "grok_api_key", "xai-test"):
gres = providers.transcribe_grok(
audio,
title="Grok 测试",
)
gturns, gmeta = segment.parse(gres.transcript_text)
self.assertEqual(gmeta["asr_provider"], "grok")
self.assertEqual(len(gturns), 2)
self.assertEqual(gturns[0].speaker, "speaker_0")
self.assertEqual(gturns[1].speaker, "speaker_1")
def test_online_speaker_confirmation_updates_draft_without_voiceprint_server(self):
drafts_dir = review.CFG.drafts
drafts_dir.mkdir(parents=True, exist_ok=True)
draft = drafts_dir / "online-draft.md"
draft.write_text(
"---\n"
"title: 线上模型会议\n"
"status: draft\n"
"asr_provider: soniox\n"
"voiceprint_session: ''\n"
"voiceprint_speakers: {}\n"
"---\n\n"
"【speaker_1】发言内容\n",
encoding="utf-8",
)
with mock.patch.object(review.CFG, "voiceprint_enroll_url", "http://127.0.0.1:9/unreachable"):
enrolled = review._confirm_voiceprint(str(draft), "speaker_1", "张三")
self.assertFalse(enrolled)
updated = draft.read_text(encoding="utf-8")
self.assertIn("speaker_1: 张三", updated)
def test_lexicon_crud_operations(self):
# Entity CRUD
lexicon.upsert_entity("测试生物", "公司", company="测试集团", role="子公司", related="李四")
ents = lexicon.list_entities(q="测试生物")
self.assertEqual(len(ents), 1)
self.assertEqual(ents[0]["company"], "测试集团")
lexicon.update_entity("测试生物", "测试生物科技", "公司", company="新集团", role="总部", related="李四,王五")
ents2 = lexicon.list_entities(q="测试生物科技")
self.assertEqual(len(ents2), 1)
self.assertEqual(ents2[0]["company"], "新集团")
self.assertTrue(lexicon.delete_entity("测试生物科技"))
self.assertEqual(len(lexicon.list_entities(q="测试生物科技")), 0)
# Alias & Correction CRUD
lexicon.learn_alias("老测试", "测试人", kind="人名", origin="manual", note="初始")
lexicon.update_alias("老测试", "测试人", "小测试", "测试人", kind="人名", confirmed=1, note="已改")
aliases = lexicon.list_aliases(q="小测试")
self.assertEqual(len(aliases), 1)
self.assertEqual(aliases[0]["note"], "已改")
self.assertTrue(lexicon.delete_alias("小测试", "测试人"))
lexicon.learn_correction("误识词", "正词", note="c1")
lexicon.update_correction("误识词", "正词", "新误识词", "正词", confirmed=1, note="c2")
cors = lexicon.list_corrections(q="新误识词")
self.assertEqual(len(cors), 1)
self.assertEqual(cors[0]["note"], "c2")
self.assertTrue(lexicon.delete_correction("新误识词", "正词"))
# Relation CRUD
lexicon.upsert_relation("张三", "任职于", "洛启生物", confidence=0.8, source="manual")
lexicon.update_relation("张三", "任职于", "洛启生物", "manual", "张三", "实控人", "洛启生物", confidence=0.95, active=1)
rels = lexicon.list_relations(q="张三")
self.assertTrue(any(r["predicate"] == "实控人" for r in rels))
self.assertTrue(lexicon.delete_relation("张三", "实控人", "洛启生物", "manual"))
def test_db_web_endpoints_multi_review_and_job_delete(self):
job1_id = "0123456789abcdef01234567"
job2_id = "fedcba987654321001234567"
t1 = svc_cfg.TRANSCRIPTS / f"{job1_id}.txt"
t2 = svc_cfg.TRANSCRIPTS / f"{job2_id}.txt"
t1.write_text("【speaker_0】录音一内容\n", encoding="utf-8")
t2.write_text("【speaker_0】录音二内容\n", encoding="utf-8")
svc_db.create_job(
job_id=job1_id,
original_name="meeting1.m4a",
stored_name=f"20261102-{job1_id}.m4a",
title="第一场尽调会",
context="验证多录音确认",
size=2048,
asr_provider="soniox",
asr_model="stt-async-v5",
)
svc_db.update_job(job1_id, "review", "待确认", transcript=str(t1))
svc_db.create_job(
job_id=job2_id,
original_name="meeting2.m4a",
stored_name=f"20261102-{job2_id}.m4a",
title="第二场投决会",
context="验证批量操作",
size=4096,
asr_provider="qwen3cam",
)
svc_db.update_job(job2_id, "review", "待确认", transcript=str(t2))
q1 = review.enqueue(str(t1), "第一场问题A?", {"kind": "draft_question", "candidates": ["选项A1", "选项A2"]})
q2 = review.enqueue(str(t1), "第一场问题B?", {"kind": "draft_question", "candidates": ["选项B1"]})
q3 = review.enqueue(str(t2), "第二场问题C?", {"kind": "draft_question", "candidates": ["选项C1"]})
os.environ["ASR_WEB_PASSWORD"] = "test-web-password"
os.environ.pop("ASR_WEB_ALLOW_ANON", None)
svc_web.AGENT_TOKEN = "test-agent-token"
server = ThreadingHTTPServer(("127.0.0.1", 0), svc_web.Handler)
port = server.server_address[1]
thread = threading.Thread(target=server.serve_forever, daemon=True)
thread.start()
try:
import http.cookiejar
from asrpipe import webauth
jar = http.cookiejar.CookieJar()
opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(jar))
login_body = urllib.parse.urlencode({"password": "test-web-password"}).encode()
opener.open(urllib.request.Request(
f"http://127.0.0.1:{port}/login", data=login_body, method="POST",
), timeout=5)
def fetch(path: str, headers: dict[str, str] | None = None) -> tuple[int, str]:
req = urllib.request.Request(
f"http://127.0.0.1:{port}{path}",
headers=headers or {},
)
with opener.open(req, timeout=5) as resp:
return resp.status, resp.read().decode("utf-8")
def post_form(path: str, data: dict[str, str]) -> tuple[int, str]:
payload = dict(data)
payload.setdefault("csrf", webauth.csrf_token())
encoded = urllib.parse.urlencode(payload).encode("utf-8")
req = urllib.request.Request(
f"http://127.0.0.1:{port}{path}",
data=encoded,
headers={"Content-Type": "application/x-www-form-urlencoded"},
method="POST",
)
with opener.open(req, timeout=5) as resp:
return resp.status, resp.read().decode("utf-8")
status, body = fetch("/health")
self.assertEqual(status, 200)
self.assertEqual(json.loads(body)["ok"], True)
status, body = fetch(
"/api/v1/providers",
headers={"Authorization": "Bearer test-agent-token"},
)
self.assertEqual(status, 200)
prov_ids = [p["id"] for p in json.loads(body)["providers"]]
self.assertEqual(prov_ids, ["qwen3cam", "soniox", "gemini", "openai", "grok"])
# Multi-recording review page shows both recordings grouped
status, body = fetch("/review/")
self.assertEqual(status, 200)
self.assertIn("第一场尽调会", body)
self.assertIn("第二场投决会", body)
self.assertIn("第一场问题A?", body)
self.assertIn("第二场问题C?", body)
# Batch submit both questions for recording 1
status, body = post_form(f"/jobs/{job1_id}/review/", {
"question_ids": f"{q1},{q2}",
f"preset_{q1}": "选项A1",
f"custom_{q2}": "自定义回答B",
"batch_submit": "1",
})
self.assertEqual(status, 200)
self.assertEqual(len(svc_web.pending(svc_db.get_job(job1_id))), 0)
self.assertEqual(len(svc_web.pending(svc_db.get_job(job2_id))), 1)
# Edit job metadata
status, body = post_form(f"/jobs/{job1_id}/meta", {
"title": "第一场尽调会(已改名)",
"context": "更新后的背景",
"provider": "openai",
"model": "openai/gpt-4o-transcribe-diarize",
"next_action": "save",
})
self.assertEqual(status, 200)
self.assertIn("第一场尽调会(已改名)", body)
# Save settings via Web UI POST
status, body = post_form("/settings", {
"litellm_base_url": "http://192.168.50.49:4000/v1",
"clean_model": "aistudio/gemini-flash-lite-latest",
"keyword_model": "aistudio/gemini-flash-lite-latest",
"draft_model": "codex/gpt-sol-latest",
"asr_provider": "soniox",
"use_hindsight": "1",
})
self.assertEqual(status, 200)
self.assertEqual(CFG.asr_provider, "soniox")
# Delete job 2 and verify its pending review question is also cleaned up
status, _ = post_form(f"/jobs/{job2_id}/delete", {"delete_files": "1"})
self.assertEqual(status, 200)
self.assertIsNone(svc_db.get_job(job2_id))
self.assertFalse(t2.exists())
self.assertNotIn(q3, [r["id"] for r in review.pending(200)])
finally:
server.shutdown()
server.server_close()
if __name__ == "__main__": if __name__ == "__main__":
@@ -0,0 +1,375 @@
"""Regression checks for the review and the 8788 black-box report."""
from __future__ import annotations
import json
import os
import socket
import subprocess
import sys
import tempfile
import threading
import unittest
import urllib.error
import urllib.parse
import urllib.request
from datetime import datetime, timezone
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
from unittest import mock
if "asrservice.config" not in sys.modules:
_boot = tempfile.mkdtemp(prefix="asr-harden-")
os.environ["ASR_DATA_DIR"] = str(Path(_boot) / "data")
os.environ["ASR_STATE_DIR"] = str(Path(_boot) / "state")
os.environ["ASR_VAULT"] = str(Path(_boot) / "vault")
os.environ["ASR_NOTIFY"] = "0"
os.environ["ASR_GIT_SYNC"] = "0"
os.environ["ASR_AGENT_TOKEN"] = "test-agent-token"
from asrpipe import locking, refine, review, webauth # noqa: E402
from asrpipe.config import Config # noqa: E402
from asrpipe import providers # noqa: E402
from asrservice import config as svc_cfg # noqa: E402
from asrservice import db as svc_db # noqa: E402
from asrservice import web as svc_web # noqa: E402
from asrservice import worker # noqa: E402
class _NoRedirect(urllib.request.HTTPRedirectHandler):
def redirect_request(self, req, fp, code, msg, headers, newurl):
raise urllib.error.HTTPError(req.full_url, code, msg, headers, fp)
def _opener(follow=True):
if follow:
return urllib.request.build_opener(urllib.request.HTTPCookieProcessor())
return urllib.request.build_opener(_NoRedirect)
def _start_server():
server = ThreadingHTTPServer(("127.0.0.1", 0), svc_web.Handler)
thread = threading.Thread(target=server.serve_forever, daemon=True)
thread.start()
return server, server.server_address[1]
def _raw(port: int, payload: bytes) -> bytes:
sock = socket.create_connection(("127.0.0.1", port), timeout=5)
try:
sock.sendall(payload)
sock.shutdown(socket.SHUT_WR)
chunks = []
while True:
part = sock.recv(65536)
if not part:
break
chunks.append(part)
return b"".join(chunks)
finally:
sock.close()
class HardeningTests(unittest.TestCase):
def setUp(self):
os.environ.pop("ASR_WEB_ALLOW_ANON", None)
os.environ["ASR_WEB_PASSWORD"] = "test-web-password"
os.environ["ASR_NOTIFY"] = "0"
os.environ["ASR_GIT_SYNC"] = "0"
svc_cfg.ensure_dirs()
svc_db.connect().close()
self._hindsight = Config.use_hindsight.__get__(svc_web.CFG, type(svc_web.CFG))
svc_web.CFG.use_hindsight = False
def tearDown(self):
svc_web.CFG.use_hindsight = self._hindsight
def test_closed_without_password_and_csrf_and_bad_length(self):
os.environ["ASR_WEB_PASSWORD"] = ""
server, port = _start_server()
try:
opener = _opener(False)
with self.assertRaises(urllib.error.HTTPError) as caught:
opener.open(f"http://127.0.0.1:{port}/review/", timeout=5)
self.assertEqual(caught.exception.code, 303)
self.assertIn("/login", caught.exception.headers.get("Location", ""))
page = urllib.request.urlopen(f"http://127.0.0.1:{port}/login", timeout=5).read().decode()
self.assertIn("ASR_WEB_PASSWORD", page)
raw = _raw(port, b"POST /login HTTP/1.0\r\nContent-Length: abc\r\n\r\n")
self.assertIn(b" 400 ", raw)
self.assertNotIn(b"Empty", raw)
finally:
server.shutdown()
server.server_close()
os.environ["ASR_WEB_PASSWORD"] = "test-web-password"
def test_pages_hide_secrets_escape_titles_and_reject_bad_edits(self):
svc_db.save_settings({
"litellm_api_key": "sk-SECRET-abcd-wxyz",
"diarize_url": "http://192.168.50.100:10099/asr",
})
jid = "abc123abc123abc123abc123"
title = "X');window.__xss=1;//"
svc_db.create_job(jid, "录音.wav", "sample.wav", title=title, size=100, source="web")
audio = svc_cfg.PROCESSED / "sample.wav"
audio.write_bytes(b"0123456789" * 10)
draft = svc_cfg.DRAFTS / f"{jid}.md"
draft.write_text("原文一行\n", encoding="utf-8")
result = svc_cfg.RESULTS / "2026-纪要.md"
result.write_text("# 纪要\n", encoding="utf-8")
svc_db.update_job(jid, "complete", "完成", draft=str(draft), result=str(result))
q1 = review.enqueue("rec-a", "第一题?", {"kind": "draft_question", "candidates": ["甲"]})
q2 = review.enqueue("rec-b", "第二题?", {"kind": "draft_question", "candidates": ["乙"]})
server, port = _start_server()
try:
opener = _opener(True)
opener.open(urllib.request.Request(
f"http://127.0.0.1:{port}/login",
data=urllib.parse.urlencode({"password": "test-web-password"}).encode(),
method="POST",
), timeout=5)
home = opener.open(f"http://127.0.0.1:{port}/", timeout=5).read().decode()
self.assertNotIn('onclick="delOne(', home)
self.assertIn("data-del", home)
settings = opener.open(f"http://127.0.0.1:{port}/settings", timeout=5).read().decode()
self.assertNotIn("192.168.50.100", settings)
self.assertNotIn("sk-SECRET", settings)
self.assertIn("已配置", settings)
review_html = opener.open(f"http://127.0.0.1:{port}/review/", timeout=5).read().decode()
forms = review_html.split('<form class="q-item"')
self.assertGreaterEqual(len(forms), 3)
for form in forms[1:]:
head = form.split("</form>", 1)[0]
self.assertLess(head.find('name="single_custom"'), head.find("custom_"))
missing = _raw(
port,
(
"POST /jobs/%s/meta HTTP/1.0\r\nHost: 127.0.0.1\r\n"
"Content-Type: application/x-www-form-urlencoded\r\n"
"Cookie: asr_session=%s\r\nContent-Length: 10\r\n\r\ntitle=nope"
% (jid, webauth.session_token())
).encode(),
)
self.assertIn(b" 403 ", missing)
edit = urllib.parse.urlencode({
"csrf": webauth.csrf_token(),
"target": "../../x",
"content": "x",
}).encode()
req = urllib.request.Request(
f"http://127.0.0.1:{port}/jobs/{jid}/edit", data=edit, method="POST",
)
body = opener.open(req, timeout=5).read().decode()
self.assertIn("未保存", body)
self.assertEqual(draft.read_text(encoding="utf-8"), "原文一行\n")
empty = urllib.parse.urlencode({
"csrf": webauth.csrf_token(), "target": "draft", "content": " ",
}).encode()
body = opener.open(urllib.request.Request(
f"http://127.0.0.1:{port}/jobs/{jid}/edit", data=empty, method="POST",
), timeout=5).read().decode()
self.assertIn("内容为空", body)
header = (
f"POST /jobs/{jid}/edit HTTP/1.0\r\nCookie: asr_session={webauth.session_token()}\r\n"
"Content-Type: application/x-www-form-urlencoded\r\nContent-Length: 3000000\r\n\r\n"
).encode()
huge = _raw(port, header)
self.assertIn(b" 413 ", huge)
art = urllib.request.Request(f"http://127.0.0.1:{port}/jobs/{jid}/artifact/result")
with opener.open(art, timeout=5) as resp:
disp = resp.headers.get("Content-Disposition", "")
text = resp.read().decode()
disp.encode("latin-1")
self.assertIn("filename*=UTF-8''", disp)
self.assertIn("filename=\"download.md\"", disp)
self.assertIn("纪要", text)
ranged = urllib.request.Request(
f"http://127.0.0.1:{port}/jobs/{jid}/audio",
headers={"Range": "bytes=80-10"},
)
try:
opener.open(ranged, timeout=5)
self.fail("reverse range should be rejected")
except urllib.error.HTTPError as exc:
self.assertEqual(exc.code, 416)
self.assertIn("bytes */100", exc.headers.get("Content-Range", ""))
suffix = urllib.request.Request(
f"http://127.0.0.1:{port}/jobs/{jid}/audio",
headers={"Range": "bytes=-10"},
)
with opener.open(suffix, timeout=5) as resp:
self.assertEqual(resp.status, 206)
self.assertEqual(resp.headers.get("Content-Range"), "bytes 90-99/100")
self.assertEqual(len(resp.read()), 10)
try:
opener.open(f"http://127.0.0.1:{port}/no-such-page", timeout=5)
self.fail("missing page should be 404")
except urllib.error.HTTPError as exc:
self.assertEqual(exc.code, 404)
self.assertIn("页面不存在", exc.read().decode())
finally:
server.shutdown()
server.server_close()
review.delete_for_job(transcript_path="rec-a")
review.delete_for_job(transcript_path="rec-b")
self.assertIsNotNone(q1)
self.assertIsNotNone(q2)
def test_busy_retry_does_not_move_processing_audio(self):
jid = "fedcba9876543210fedcba98"
name = "busy.wav"
svc_db.create_job(jid, name, name, title="进行中", size=4, source="web")
audio = svc_cfg.PROCESSING / name
audio.write_bytes(b"wave")
svc_db.update_job(jid, "transcribing", "正在转写")
server, port = _start_server()
try:
opener = _opener(True)
opener.open(urllib.request.Request(
f"http://127.0.0.1:{port}/login",
data=urllib.parse.urlencode({"password": "test-web-password"}).encode(),
method="POST",
), timeout=5)
body = urllib.parse.urlencode({
"csrf": webauth.csrf_token(),
"title": "进行中",
"provider": "not-a-real-engine",
"next_action": "retry",
}).encode()
page = opener.open(urllib.request.Request(
f"http://127.0.0.1:{port}/jobs/{jid}/meta", data=body, method="POST",
), timeout=5).read().decode()
self.assertIn("未重新排队", page)
self.assertIn("未知引擎", page)
self.assertTrue(audio.is_file())
self.assertFalse((svc_cfg.INBOX / name).exists())
self.assertEqual(svc_db.get_job(jid)["status"], "transcribing")
finally:
server.shutdown()
server.server_close()
def test_revision_failure_keeps_marker_and_answers(self):
jid = "a1b2c3d4e5f60718293a4b5c"
transcript = svc_cfg.TRANSCRIPTS / f"{jid}.txt"
transcript.write_text("【speaker_0】原文\n", encoding="utf-8")
draft = svc_cfg.DRAFTS / f"{jid}.md"
draft.write_text("---\ntype: minutes\n---\n# 稿\n", encoding="utf-8")
svc_db.create_job(jid, "a.wav", "a.wav", title="重精修", size=1, source="web")
svc_db.update_job(jid, "review", "待确认", transcript=str(transcript), draft=str(draft))
qid = review.enqueue(str(transcript), "已确认的问题?", {
"kind": "draft_question", "draft_path": str(draft), "candidates": ["保留"],
})
review.answer(qid, "保留")
pending_id = review.enqueue(str(transcript), "仍待确认?", {
"kind": "draft_question", "draft_path": str(draft), "candidates": ["甲"],
})
with mock.patch("asrservice.worker.refine.refine", side_effect=RuntimeError("llm down")):
with self.assertRaises(RuntimeError):
worker.rerefine_job(svc_db.get_job(jid))
answered = review.recent_answered(20)
self.assertIn(qid, [row["id"] for row in answered])
self.assertIn(pending_id, [row["id"] for row in review.pending(50)])
marker = svc_cfg.REVISION_QUEUE / f"revise-{jid}.json"
marker.write_text(json.dumps({
"action": "revise", "job_id": jid, "draft": "/missing/draft.md",
}), encoding="utf-8")
worker.process_revisions()
self.assertTrue(marker.is_file())
payload = json.loads(marker.read_text(encoding="utf-8"))
self.assertEqual(payload["attempts"], 1)
self.assertEqual(svc_db.get_job(jid)["status"], "review")
def test_recover_processing_and_shared_lock_and_stream(self):
jid = "00112233445566778899aabb"
name = f"stuck-{jid[:8]}.wav"
audio = svc_cfg.PROCESSING / name
audio.write_bytes(b"stuck")
svc_db.create_job(jid, name, name, title="卡住", size=5, source="web")
svc_db.update_job(jid, "transcribing", "中断")
worker.recover()
self.assertFalse(audio.exists())
parked = svc_cfg.INBOX / name
self.assertTrue(parked.is_file())
self.assertEqual(svc_db.get_job(jid)["status"], "queued")
worker.beat()
self.assertTrue((svc_cfg.STATE / "worker.heartbeat").is_file())
lock_path = Path(tempfile.mkdtemp()) / "consumer.lock"
handle = locking.acquire_recording_lock(lock_path)
try:
with self.assertRaises(SystemExit):
locking.acquire_recording_lock(lock_path)
finally:
handle.close()
blob = b"STREAMTEST" + b"q" * (256 * 1024)
audio_path = Path(tempfile.mkdtemp()) / "stream.wav"
audio_path.write_bytes(blob)
seen = {}
class Capture(BaseHTTPRequestHandler):
def do_POST(self):
n = int(self.headers["Content-Length"])
body = self.rfile.read(n)
seen["n"] = len(body)
seen["head"] = b"STREAMTEST" in body
seen["tail"] = body.endswith(b"q" + b"\r\n--" + body.split(b"----ASRBoundary", 1)[0][:0] or b"q")
self.send_response(200)
self.send_header("Content-Type", "application/json")
self.end_headers()
self.wfile.write(b'{"ok": true}')
def log_message(self, *_):
pass
cap = ThreadingHTTPServer(("127.0.0.1", 0), Capture)
threading.Thread(target=cap.serve_forever, daemon=True).start()
try:
out = providers._http_multipart(
f"http://127.0.0.1:{cap.server_address[1]}/up",
fields={"model": "unit"},
file_field="file",
file_path=audio_path,
timeout=10,
retries=1,
)
finally:
cap.shutdown()
cap.server_close()
self.assertTrue(out["ok"])
self.assertGreater(seen["n"], len(blob))
self.assertTrue(seen["head"])
def test_wording_timezone_git_default_and_lexicon_page(self):
self.assertEqual(
refine.scrub_internal_wording("卡片未提供职务,卡片提及张三"),
"记录未写明职务,录音中提到张三",
)
stamp = datetime(2026, 11, 2, 13, 8, 26, tzinfo=timezone.utc).timestamp()
self.assertEqual(svc_web.fmt_ts(stamp), "2026-11-02 21:08:26")
header = worker.intake_header({"context": "a-->b } c"})
self.assertEqual(header.count("-->"), 1)
self.assertTrue(header.endswith("-->"))
self.assertNotIn("a-->", header)
self.assertTrue(header.startswith("<!-- intake:"))
for i in range(3):
from asrpipe import lexicon
lexicon.learn_alias(f"别名{i}", f"规范{i}", origin="manual")
from asrpipe import lexicon
self.assertGreaterEqual(lexicon.count_aliases(q="别名"), 3)
page = lexicon.list_aliases(q="别名", limit=2, offset=0)
self.assertEqual(len(page), 2)
env = os.environ.copy()
env.pop("ASR_GIT_SYNC", None)
env.pop("ASR_NOTIFY", None)
env["ASR_STATE_DIR"] = tempfile.mkdtemp()
env["PYTHONPATH"] = str(Path(__file__).resolve().parents[1])
out = subprocess.check_output(
[sys.executable, "-c",
"from asrpipe.config import Config; c=Config(); print(int(c.git_sync), int(c.notify))"],
env=env, text=True,
).strip()
self.assertEqual(out, "0 0")
if __name__ == "__main__":
unittest.main()
+23
View File
@@ -0,0 +1,23 @@
# Byparr
Private LAN browser-fetch service on `tools-pve` for pages that reject ordinary
HTTP clients.
- API docs: `http://192.168.50.49:8191/docs`
- FlareSolverr-compatible endpoint: `POST http://192.168.50.49:8191/v1`
- Authentication: none; the port is bound only to the host LAN address
- Persistent state: none
Example:
```bash
curl -sS http://192.168.50.49:8191/v1 \
-H 'Content-Type: application/json' \
-d '{"cmd":"request.get","url":"https://example.com","maxTimeout":120000}'
```
The upstream image is pinned to an immutable `linux/amd64` digest. Browser
scratch data and cache are RAM-backed because Byparr performs frequent writes.
The local TCP healthcheck deliberately avoids Byparr's `/health` endpoint,
which opens an external page on every check.
+27
View File
@@ -0,0 +1,27 @@
services:
byparr:
image: ghcr.io/thephaseless/byparr@sha256:51a629fa3dcde0cee1a780922e1a572c4a5be0b5a0cb825db73d4e63059275ff
container_name: tools-byparr
restart: unless-stopped
shm_size: 1gb
stdin_open: true
tty: true
environment:
LOG_LEVEL: INFO
BROWSER_LOCALE: en-US
BROWSER_TIMEZONE: America/Los_Angeles
ports:
- "192.168.50.49:8191:8191"
tmpfs:
- /tmp:exec,mode=1777,size=1G
- /cache:exec,mode=1777,size=2G
healthcheck:
test:
- CMD
- /app/.venv/bin/python
- -c
- import socket; s=socket.create_connection(('127.0.0.1',8191),5); s.close()
interval: 30s
timeout: 10s
retries: 3
start_period: 30s
+14
View File
@@ -0,0 +1,14 @@
LISTEN_ADDRESS=192.168.50.49
LITELLM_PORT=4000
POSTGRES_PASSWORD=
LITELLM_MASTER_KEY=
LITELLM_SALT_KEY=
ZENMUX_SUBSCRIPTION_API_KEY=
ZENMUX_PAYG_API_KEY=
OPENROUTER_API_KEY=
BAILIAN_CN_API_KEY=
BAILIAN_CN_API_BASE=https://ws-v68kvjnmhkiejaxo.cn-beijing.maas.aliyuncs.com/api/v1
GOOGLE_AI_STUDIO_API_KEY=
VERTEXAI_ENABLED=false
VERTEXAI_PROJECT=
VERTEXAI_LOCATION=global
+3
View File
@@ -0,0 +1,3 @@
__pycache__/
*.pyc
.env
+329
View File
@@ -0,0 +1,329 @@
# tools-pve 统一模型代理
## ZenMux 订阅退役(2026-11-02)
本节覆盖下文历史的订阅优先策略。`zenmux_subscription_enabled: false`,Compose 不再
传入订阅 Key;即使旧环境中仍有该变量,renderer 也忽略它。
- Claude、DeepSeek、GLM、Kimi、MiMo:OpenRouter 优先,ZenMux BYOK/PAYG 最后备用。
- `auto/...`、无前缀兼容名使用上述顺序;旧 `zenmux/...` 为避免客户端中断保留,
现在也遵循主备,不代表仍使用 ZenMux 订阅。新配置推荐用 `auto/...`。
- `zenmux-payg/...` 显式指定 BYOK,仍可直调;不能给显式 provider 路由虚构优先级。
- Gemini 旧 ZenMux 兼容名优先使用已有 Google 路由,BYOK 最后。
- OpenAI 官方 API、Codex OAuth、xAI OAuth、Google 路由顺序不变。
- 没有其他同模型后端的路由,只能用保留的 BYOK,不做跨型号伪回退。
- BYOK 按量付费,不再受已取消订阅配额保护;跨 provider 缓存不共享。
OpenRouter Claude 合成直连探针在两台生产主机均返回条款限制类 403,
不是订阅退役是否生效的判断依据,也不能宣称首选上游已通过。
## OpenAI 官方 API 与订阅回退(2026-10-29)
官方 API key 由用户生成,经过明确授权,仅加入本 tools-pve 项目的 Arcane 私有
`OPENAI_API_KEY`。不写入 Git、不复制到 HK、不替换客户端 virtual key 或 OAuth。
官方上游固定为 `https://api.openai.com/v1`,不能因 LiteLLM provider 同为 openai
就误归入 ZenMux。未配置 key 时不生成这些路由和 fallback。
| 首选订阅路由 | 官方 API 备用路由 | 当前实际模型 |
| --- | --- | --- |
| `codex/gpt-luna-latest` | `openai/gpt-luna-latest` | `gpt-6-luna` |
| `codex/gpt-sol-latest` | `openai/gpt-sol-latest` | `gpt-6.1-sol` |
| `codex/gpt-astra-latest` | `openai/gpt-astra-latest` | `gpt-6-astra` |
| `codex/gpt-6-sol` | `openai/gpt-6-sol` | `gpt-6-sol` |
| `codex/gpt-5.5` | `openai/gpt-5.5` | `gpt-5.5` |
原不带provider的GPT兼容别名也保留同型号fallback。renderer从chatgpt表生成
对应的API别名,因此既有latest维护更新后,两侧不会各自残留不同型号。
`openai/...`显式路由只走官方API,不回到订阅,不改变Grok/Claude/Google/开放模型的主备。
订阅429、认证或暂时故障可以回退;不自动跨型号降级,失败冷却仍为60秒。
跨后端不共享prompt cache,也不承诺中途流断线或previous_response_id跨后端续接。
需要保留完整上下文以便调用方重试,不自动重放已经执行的工具。
API专有补充:`openai/gpt-terra-latest`(目前6.1 Terra);
`openai/gpt-transcribe`、`openai/gpt-4o-transcribe`、
`openai/gpt-4o-mini-transcribe`、`openai/gpt-4o-transcribe-diarize`;
`openai/text-embedding-3-small`、`openai/text-embedding-3-large`;
`openai/gpt-4o-mini-tts`;`openai/gpt-image-2.5-flare`、
`openai/gpt-image-2.5-sunburst`。图像路线可配置不代表做过昂贵出图验收,
各接口按实际测试结果记录。没有修改现有ASR热词、Hindsight embedding或客户端默认模型。
Terra与其他API专有模型暂按models.yaml显式维护,不擅自扩张既有Hermes自动维护授权。
现有lan-agents应用key增补这些API路线;Hindsight等受限key只增补其原先已允许的
GPT路线对应备用模型,不开放图像、语音等无关能力。Claude专用key不变。
赠金需绑定到此key所属API组织,并在该订阅账期末到期,不结转。赠金额度先消耗,
余额不足后若有已购额度可继续消费;没有任何额度则返回429 credit_balance_exhausted。
代理不能凭普通project key保证“只花赠金”,不替用户开启充值。这里提供的是
标准API请求的主备,不把官方赠金解释为可给Codex CLI的API-key模式增加额度。
依据:[OpenAI赠金说明](https://developers.openai.com/api/docs/guides/chatgpt-api-credits)、
[LiteLLM回退](https://docs.litellm.ai/docs/proxy/reliability)。
本轮验收:18项配置测试通过;官方Luna/Sol/Terra的Responses、Luna Chat、
Codex Luna Responses完整SSE通过(正常请求fallbacks=0);embedding small、
TTS与合成语音的gpt-transcribe/mini-transcribe通过。图像未实际生成,
diarize与embedding large只配置且模型目录可见,不能据此声称完成任务验收。
`verify_openai_fallback.py`在独立容器exec进程中对Chat与Responses各注入
429/401/503,六例均收到真实官方API的短回复;不是人为耗尽订阅,也不修改
生产worker的函数或真实OAuth token。复测命令(会有少量API用量):
```bash
ssh tools@192.168.50.49 'sudo -n docker exec -i tools-litellm python -' < verify_openai_fallback.py
```
Arcane Sync及Up终态success;PostgreSQL启动时间仍是
`2026-10-12T05:46:12.472887407Z`,数据库及OAuth卷未重建。
## 部署状态
2026-10-09:已通过 Arcane 上线,LiteLLM 1.104.2 与 PostgreSQL 均 healthy。
UI HTTP 200;应用 virtual key 的 OpenRouter MiMo 请求返回 OK。
ZenMux Sonnet 5.5 原生 Messages 正常;重复请求证实写入与读取
21,502 prompt-cache tokens。Codex/xAI 已独立授权,SDK 原生请求验证
Luna、Sol 6.1 和 Grok 4.7 返回 OK。用户更新后的 PAYG key 原生 Claude 请求 HTTP 200。
Google AI Studio key 模型目录 HTTP 200,使用上游提供的三个 latest 别名。
17 条路线已加载;原客户端 key 权限已更新,Luna/Sol 的 gateway Responses SSE、
Luna 非流式 Chat Completions、Grok/Gemini 三条 Chat 路线和 PAYG Messages 均实测 OK。
本项目对其他模型的自动 latest 维护尚未实现。
- Environment:`ff859844-8393-4f87-8caf-054df735a65d`。
- Project:`06052b47-f640-4e23-968d-bfc25a8a6e7a`,名称 `tools-litellm`。
- Git Sync:`49148e73-65ca-43a6-a859-efc9cc750b93`;Auto Sync 关闭。
- 宿主机项目目录:`/var/lib/docker/volumes/arcane-agent_agent-data/_data/projects/litellm`。
- 管理凭据:.145 的 `/home/claw/codex-workspace/server-management/.secrets/tools-litellm.env`。
- 普通客户端配置:同目录 `tools-litellm-client.env`;两者均为 0600,不提交 Git。
## 架构与入口
部署在 `192.168.50.49`,LiteLLM 与独立 PostgreSQL 17。
- OpenAI 兼容客户端:`http://192.168.50.49:4000/v1`。
- Anthropic SDK:`http://192.168.50.49:4000`,请求 `/v1/messages`。
- 管理界面:`http://192.168.50.49:4000/ui`。
- PostgreSQL 不发布主机端口。
- named volume:`tools-litellm-database`、`tools-litellm-oauth`。
只使用单个代理 worker,降低 OAuth refresh token 竞争的风险。
所有配置从 Git 生成,不允许 UI 中的模型配置成为另一份权威。
未提供 key 或未完成 OAuth 登录的路线不会进入运行配置。
OAuth 判定检查 auth.json 中 access_token,不仅检查文件存在性:
Codex 设备授权期间会先写入冷却占位文件,这不代表已经授权。
## Provider 与缓存
ZenMux Claude 使用 Anthropic Messages 上游:
`https://zenmux.ai/api/anthropic`。
LiteLLM 的 `anthropic/anthropic/claude-sonnet-5.5` 第一个前缀是
LiteLLM provider,第二个前缀是 ZenMux 的模型 ID,不是笔误。
订阅路线为 `claude-sonnet-latest`、`claude-opus-latest`、
`claude-haiku-latest`;PAYG 路线在同名末尾加 `-payg`。
不自动由订阅切换到 PAYG,也不设置跨模型 fallback。
Claude 客户端优先使用原生 `/v1/messages`,保留 `cache_control`、
稳定的 tools/system 前缀以及原样回传的 thinking/signature。
不能保证每次 cache hit;首次创建、最小缓存长度、TTL 和上游路由都会影响结果。
上线验收需重复发送足够长且完全相同的前缀,检查
`cache_creation_input_tokens` 和 `cache_read_input_tokens`,不能只看 HTTP 200。
不把 LiteLLM 的响应缓存当作 Anthropic prompt cache。
实测 ZenMux Sonnet 5.5 会拒绝强制 `tool_choice.type=tool/any`,
错误由上游返回;使用支持的 `auto` 模式,不在代理中静默改写调用者的要求。
开放模型已改为 ZenMux 订阅优先、OpenRouter 回退;`models.yaml` 来自实际 provider 目录。
除 Google 自带 latest 映射由其上游维护外,其他 `latest` 都是本项目手动维护的稳定别名。
Google AI Studio 使用独立 key 与模型表,已实际查询模型目录,
不把 Google OAuth/ADC 当成 AI Studio API key。
Google 三条模型为 gemini-flash-latest、gemini-flash-lite-latest、gemini-pro-latest。
用户新 provider key 的权威位置是 Arcane tools-litellm Project `.env`;
不要用 .145 首次生成的 tools-litellm.env 整体覆盖它,那份文件可能还含旧 provider 值。
该本机文件主要用于 master key 管理与数据库凭据恢复;提供商配置必须先从 Arcane 实查。
## 开源模型主备路由(2026-10-12)
| 客户端稳定别名 | 两侧具体模型 |
| --- | --- |
| `deepseek-flash-latest` | `deepseek/deepseek-v4.1-flash` |
| `deepseek-pro-latest` | `deepseek/deepseek-v4-pro` |
| `glm-latest` | `z-ai/glm-5.3-prime` |
| `kimi-latest` | `moonshotai/kimi-k3` |
| `mimo-flash-latest` | `xiaomi/mimo-v2.6-flash` |
这些原有别名先调用 ZenMux `/api/v1`,失败后由 Router 转向同名末尾加
`-openrouter` 的备用组;不是随机分流,不切到 PAYG,也不改变 Claude/OAuth。
DeepSeek Pro 原 OpenRouter 日期版 `v4-pro-0813` 改为双方共有的 `v4-pro`,
两侧返回模型 ID 已核验一致,但不能仅据 ID 保证供应商权重实现完全相同。
只配置现有五组,不把整个供应商目录自动开放。
缺 ZenMux key 时原别名仍直接走 OpenRouter;缺 OpenRouter key 时不生成悬空
fallback。普通 virtual key 需同时允许主组与备用组,现有 lan-agents key
通过 `/key/update` 增补五个备用别名,无需更换 key。
部署所用 SDK 的回环 mock 已验证 402、403、429、500 自动回退。
显式部署级异常策略让 429/500 在第一次失败后冷却 60 秒;402/403 仍会在
下一请求尝试主组,不宣称已按订阅窗口熔断。Router 默认最多一次重试,
不是耗尽后无限等待。两家共享订阅额度的跨模型熔断尚未实现。
请求开始前的失败已覆盖;流式输出中途断开不承诺无缝拼接或恰好一次执行工具。
ZenMux 恢复后重新优先使用它。OpenRouter 会产生其账户费用;缓存不跨
供应商共享。上游工具/推理参数兼容性仍按调用者实际用法验收,不能只凭
短文本成功宣称所有 Agent 工作流均兼容。
参考:[LiteLLM Fallbacks](https://docs.litellm.ai/docs/proxy/reliability)。
## Hermes 两个 profile 的集中接入
Hermes 使用三种协议入口:Claude 经 Anthropic Messages,Gemini/Grok/开源模型
经 Chat Completions,Codex 订阅经 Responses。不要为了单一 URL 把所有模型强行
桥接成同一种协议;`/v1/responses` 和 `/v1/messages` 均由同一代理提供。
补充路线 `gemini-flash-zenmux` 保留 Hermes 原 ZenMux Gemini 3.8 Flash 数据去向,
不自动换到 Google ADC。`gpt-work-sol`、`gpt-review-5.5`、`gpt-astra-latest` 分别
保留原 gpt-6-sol、gpt-5.5、gpt-6-astra;三者均在独立 Codex OAuth 实时目录中。
它们是兼容现有分工,不以过期型号或“最新总更好”为由静默更换。
`glm-flash-latest` 补齐原选择器中的 GLM 5.3 Flash,并使用同一 ZenMux/OpenRouter 主备模式。
用户同日确认的最终策略:default 用 `grok-latest`,work 用 `gpt-sol-latest`;
辅助全部用 `gpt-luna-latest`(一般 low、任务拆解 medium),审阅及后台审阅用
`gpt-sol-latest` high。当前分别映射 Grok 4.7、GPT-6 Luna、GPT-6.1 Sol。
原 Haiku/Flash/GPT-5.5 分工为迁移阶段,不再作为 Hermes 的当前策略。
旧兼容路线留给历史客户端,Hermes GPT 选择器只列 Luna/Sol/Astra latest。
MoA 仅按需启动,参考模型为 Sonnet 5.5/Grok/Luna,Astra 聚合。
work 的旧 Sonnet 5 参考槽统一到已有 Sonnet 5.5 路线。旧直连 provider 只保留
回滚兼容,新默认与任务槽不再引用;已打开 session 的 provider/model 覆盖不自动改写。
## 密钥
真实 `.env` 权限 `0600`,不提交 Git;模板见 `.env.example`。
ZenMux subscription/PAYG 的 key 需要明确确认用途,不能按外观推断。
数据库密码使用无特殊 URL 字符的随机值,避免 `DATABASE_URL` 编码问题。
`LITELLM_MASTER_KEY` 以 `sk-` 开头,salt 独立随机生成。
UI 使用数据库管理员 `kai@litellm.local`(只是登录标识,不发送邮件),
设置 `disable_env_credential_login: true`,禁用 UI_PASSWORD 以及 master key 的网页登录。
用户已修改首次临时密码,临时文件已删除。数据库保存密码哈希,
用户的新密码不写入环境变量,也不由 Agent 保存。
master key 仍能管理 API,用于恢复权限,不再充当网页登录密码。
不能直接用这份配置初始化空库后期待默认 admin 登录:必须预先通过
`/user/new` 建立管理员,再 `/user/update` 设置临时密码,随后启用限制。
日常客户端应使用管理界面创建的应用专属 virtual key,不共享 master key。
不要开启输出请求全文的 debug 日志。
## 独立 OAuth 登录
不复制 Codex CLI、Grok CLI 或 Hermes 的 refresh token。
在目标主机项目目录执行:
```bash
bash oauth-login.sh codex
```
打开终端显示的设备验证网址并输入验证码。
xAI 需要本机浏览器回调。先在浏览器所在电脑保持 SSH 隧道:
```bash
ssh -N -L 56121:127.0.0.1:56121 tools@192.168.50.49
```
然后在 tools-pve 的另一个终端运行:
```bash
bash oauth-login.sh xai
```
打开输出的授权 URL;若 56121 被占用,先关闭占用者再重试,
不要把临时改用的端口当作已有隧道可达。
登录 helper 使用 host network 仅供本地 OAuth 回调,生产代理仍使用独立 bridge。
确认账号实际支持的模型 ID 后,在 `models.yaml` 的 `chatgpt`/`xai` 表添加别名。
OAuth 映射为 gpt-luna-latest -> gpt-6-luna、gpt-sol-latest -> gpt-6.1-sol、
grok-latest -> grok-4.7,独立账号目录与 SDK 请求均已验证。
Codex 路线由 `chatgpt/` 访问,优先 `/v1/responses`;
原生 Responses 使用消息列表形式的 input 和 stream=true,按 SSE 读取到 response.completed。
常规 OpenAI SDK 客户端也可用 /v1/chat/completions,Luna 非流式桥接已实测 OK。
实测直接给字符串 input 会被 Codex backend 拒绝;不要把已授权误写成协议完全兼容。
xAI 路线使用 `xai/` 与 `use_xai_oauth: true`,不能设置 XAI API key 覆盖它。
新增 token 后需重启代理,运行时 refresh 会写回同一个持久卷。
## 部署及验收
正常走 Arcane Git Sync;Manager 已恢复,本项目已通过控制面部署。
如用户明确同意临时直接部署,将本目录放到 tools-pve,准备 `.env` 后执行:
```bash
docker compose config --quiet
docker compose up -d
docker compose ps
```
恢复 Arcane 后应纳管这套已有卷和项目,避免生成第二套数据库。
上线必须验证健康、带 virtual key 的模型列表、实际请求、stream 和工具调用;
OAuth 与 Claude cache hit 分别验收,未通过不能写成可用。
回滚只停止新增项目或回退配置/image pin,不删除 named volume。
定期 `pg_dump` 并保留独立备份,数据库卷不是备份。
参考:[LiteLLM ChatGPT OAuth](https://docs.litellm.ai/docs/providers/chatgpt)、
[xAI OAuth 官方源码](https://github.com/BerriAI/litellm/blob/main/litellm/llms/xai/oauth.py)、
[ZenMux prompt cache](https://zenmux.ai/docs/guide/advanced/prompt-cache.html)。
## Google Agent Platform 与检索模型
2026-10-09 用户确认 Pro 赠金已绑定结算账号后,接入项目
`gen-lang-client-0312436655`,区域 `global`。复用此前已授权的用户 ADC,
保存在既有 `tools-litellm-oauth` 卷的 `/tokens/google/adc.json`(0600),
不进 Git;容器通过 `GOOGLE_APPLICATION_CREDENTIALS` 读取并刷新凭据。
Arcane Project `.env` 设置 `VERTEXAI_ENABLED=true`、`VERTEXAI_PROJECT`、
`VERTEXAI_LOCATION`。默认关闭该路线,避免仅有项目名就自动发起付费调用。
| 对外路由 | 实际模型 | 验收 |
| --- | --- | --- |
| `gemini-flash-vertex` | `gemini-3.8-flash` | 原生及普通客户端 Chat 均成功 |
| `gemini-flash-lite-vertex` | `gemini-3.5-flash-lite` | 原生及普通客户端 Chat 均成功 |
| `gemini-pro-vertex` | `gemini-3.1-pro-preview` | 原生及普通客户端 Chat 均成功 |
| `gemini-embedding-2-studio` | AI Studio `gemini-embedding-2` | `/v1/embeddings` 两条输入返回两条 1024 维向量,index 正确 |
前三条不冒充 Google 厂商自动 latest 别名;保持与 AI Studio 既有路由分开。
Google 官方说明 AI Pro 月赠金可用于 Agent Platform,但本次没有从账单
实证某次请求扣的是赠金;兑换状态由用户确认。赠金耗尽后可能继续产生付费,
当前没有基于 Google 实时赠金余额的硬停机制,不设置自动跨 provider fallback。
AI Studio 当前账号目录含 `gemini-embedding-001`、`gemini-embedding-2-preview`、
`gemini-embedding-2`,没有独立 rerank 模型。优先用已经 GA 的 Embedding 2。
原生 Agent Platform `gemini-embedding-2:embedContent` 已实测返回 1024 维。
但 LiteLLM 1.104.2 的 `vertex_ai/gemini-embedding-2` 两条输入只返回一条向量,
故没有开放该代理路由供批量检索使用;不要把 API HTTP 200 当成批量验收。
Google rerank 是 Discovery Engine 的独立 Ranking API,不属于 AI Studio
embedding 接口。候选 `semantic-ranker-default@latest`、`semantic-ranker-fast@latest`;
005 为较新模型版本,官方页面 latest 指向说明与表格存在矛盾,不能仅凭别名
断言实际模型版本。当前项目两条路线均返回 403:Discovery Engine API 未启用。
尚未启用或纳入 LiteLLM,可到 Google Console 启用 `discoveryengine.googleapis.com`
后复测中文排序、权限、价格及延迟。当前 LiteLLM 已包含原生 Vertex rerank 适配。
Hindsight 0.10.1 当前生产仍用 unRaid TEI 的 `BAAI/bge-m3` embedding 和
`Alibaba-NLP/gte-multilingual-reranker-base` rerank,Cohere/rrf 是 fallback。
该版本原生 Google embedding provider 会把 Vertex Embedding 2 拆成逐条请求,
并验证输入与输出数量一致,因此后续可优先原生 Google ADC 路线,而非有问题
的 LiteLLM Vertex embedding 路线。不要直接改生产 provider:即使都选 1024 维,
BGE 与 Gemini 的向量空间不同,必须重嵌入并重建索引。rerank 切换不要求重建向量。
embedding 模型禁止使用自动追踪 latest 的别名,否则升级会混用旧向量。
参考:[Google Pro 赠金](https://developers.google.com/program/plans-and-pricing)、
[Gemini embedding](https://ai.google.dev/gemini-api/docs/embeddings)、
[Google Ranking API](https://docs.cloud.google.com/generative-ai-app-builder/docs/ranking)、
[Hindsight 配置](https://hindsight.vectorize.io/developer/configuration)。
# 个人语音输入热词路由(2026-10-25)
独立别名 `gemini-transcribe-personal` 使用 `gemini/gemini-3.5-transcribe`,经 Google AI Studio 凭据转发。客户端调用现有 `/v1/audio/transcriptions`,无需自行传词表。该别名附带 `personal_vocabulary: true`;启动适配器仅对明确标记的 Gemini 转写请求注入原生 `generation_config.transcription_config.custom_vocabulary` 和 `mode: smart`,普通路由不变。SMART 与 HK 保持相同的功能意图,但 AI Studio 使用小写字符串 `smart`,不是 Vertex 的大写 `SMART` 和 camelCase 字段。
当前个人词表仅包含 10 个人名、10 个公司名。按用户要求,从 Obsidian People/Companies 已确认卡片取规范名,以正式工作笔记的独立文档频次和 Hindsight 实体 mention_count 交叉排序;不加入产品、项目、文档标题、职务说明或错误别名。旧的 SQLite 100 词自动抽取已被替代,因为它把文档式长名称作为热词,出现了过度引导。
导出与私有频次审计使用 server-management 的 `scripts/export-entity-dictation-vocabulary.py`;具体名单与卡片证据不提交本配置仓库。私有文件仍为 tools-pve `/opt/litellm/personal-vocabulary.json`(0600)。每次请求读取文件;若通过原子替换文件更新,单文件 bind mount 需重启 LiteLLM 重新挂载。没有自动扩词任务,后续按实际误识别逐条复测。
语音输入使用 `response_format=json`,与 HK 的 Spokenly 设置一致;当前 text 请求虽 HTTP 200 仍返回 JSON 包装,不选用。不支持词级时间戳、字幕及说话人分离。Google 目前不允许 SMART/原生热词与说话人分离或词级时间戳组合。不要把这条路由用于需要 diarization 的会议流程。
LiteLLM 1.104.2 原生适配未映射热词/SMART,因此 `personal_transcription.py` 使用受路由开关约束的启动适配。升级 LiteLLM 后必须重跑 `scripts/test-personal-transcription-wire.py`,确认私有开关不泄漏上游、普通请求不变、Google 请求体中存在原生词表和 `mode: smart`。不要将 mock 验证等同于识别质量提升。
首次上线通过两次合成词 HTTP mock 验证,以及一秒合成静音的真实 Google HTTP 200 测试;未使用私人录音,未改会议配置。LAN 客户端 key 已单独增加本别名权限,其他受限 key 需按需授权。
同日启用 SMART 后,再次通过 mock 确认 native mode/词库和普通请求隔离;既有客户端 key 的静音及 Google 公开语音样本 JSON 转写均返回 200。只重启 tools-litellm,会议 worker/web 与 Postgres 启动时间未变。中文效果仍需真实使用评估。
@@ -0,0 +1,119 @@
"""Bridge OpenAI file uploads to Beijing Bailian's native ASR API."""
import base64
import io
import json
from pathlib import Path
from urllib.parse import urlsplit
import wave
from litellm.llms.base_llm.audio_transcription.transformation import AudioTranscriptionRequestData
from litellm.llms.openai.transcriptions.whisper_transformation import OpenAIWhisperAudioTranscriptionConfig
from litellm.litellm_core_utils.audio_utils.utils import process_audio_file
from litellm.types.utils import TranscriptionResponse, TranscriptionUsageDurationObject
MODEL = 'qwen-audio-3.1-asr-flash'
FORMATS = {'audio/wav':'wav','audio/x-wav':'wav','audio/wave':'wav',
'audio/mpeg':'mp3','audio/mp3':'mp3','audio/flac':'flac','audio/x-flac':'flac',
'audio/mp4':'m4a','audio/m4a':'m4a','audio/x-m4a':'m4a',
'audio/ogg':'ogg','audio/webm':'webm','video/webm':'webm','audio/aac':'aac'}
VOCABULARY_PATHS = frozenset({'/etc/litellm/personal-vocabulary.json',
'/credentials/personal-vocabulary.json'})
def vocabulary(path):
if path not in VOCABULARY_PATHS:
raise ValueError('Unexpected personal vocabulary path')
words = json.loads(Path(path).read_text())
if not isinstance(words,list) or not 1 <= len(words) <= 100:
raise ValueError('Personal vocabulary must contain 1-100 terms')
if any(not isinstance(w,str) or not w.strip() or len(w)>100 for w in words):
raise ValueError('Invalid personal vocabulary term')
return {w.strip():3 for w in words}
class BailianTranscriptionConfig(OpenAIWhisperAudioTranscriptionConfig):
@property
def has_native_transcription_endpoint(self):
return True
def get_supported_openai_params(self,model):
return ['language','response_format','prompt']
def map_openai_params(self,non_default_params,optional_params,model,drop_params):
result=super().map_openai_params(non_default_params,optional_params,model,drop_params)
if result.get('response_format') not in (None,'json','text'):
raise ValueError('Bailian dictation supports json/text without timestamps')
return result
def validate_environment(self,headers,model,messages,optional_params,litellm_params,
api_key=None,api_base=None):
if not api_key:raise ValueError('Beijing Bailian API key required')
return {**headers,'Authorization':'Bearer '+api_key,'Content-Type':'application/json',
'X-DashScope-SSE':'disable'}
def get_complete_url(self,api_base,api_key,model,optional_params,litellm_params,stream=None):
parsed=urlsplit(api_base or '')
if (model != MODEL or parsed.scheme != 'https' or
not (parsed.hostname or '').endswith('.cn-beijing.maas.aliyuncs.com') or
parsed.path.rstrip('/') != '/api/v1' or parsed.port not in (None,443) or
parsed.username or parsed.password or parsed.query or parsed.fragment):
raise ValueError('Bailian ASR restricted to Beijing workspace native endpoints')
if stream:raise ValueError('Streaming not enabled on this dictation route')
return api_base.rstrip('/')+'/services/aigc/multimodal-generation/generation'
def transform_audio_transcription_request(self,model,audio_file,optional_params,litellm_params):
if model != MODEL or optional_params.get('timestamp_granularities'):
raise ValueError('Unsupported Bailian dictation model or timestamps')
audio=process_audio_file(audio_file)
mime=audio.content_type.split(';',1)[0].lower()
if mime not in FORMATS:raise ValueError('Unsupported Bailian audio MIME type')
if len(audio.file_content)>10_000_000:raise ValueError('Bailian inline audio exceeds 10 MB')
parameters={'format':FORMATS[mime]}
if parameters['format']=='wav':
with wave.open(io.BytesIO(audio.file_content),'rb') as wav:
parameters['sample_rate']=str(wav.getframerate())
language=optional_params.get('language')
if language:parameters['language_hints']=[language]
# LiteLLM classifies DashScope as OpenAI-compatible and nests custom kwargs.
extra=optional_params.get('extra_body') or {}
path=optional_params.get('bailian_vocabulary_path') or extra.get('bailian_vocabulary_path')
if path:parameters['vocabulary']=vocabulary(path)
prompt=optional_params.get('prompt')
messages=[]
if prompt:
if not isinstance(prompt,str) or len(prompt)>400:
raise ValueError('Bailian context prompt must be at most 400 characters')
messages.append({'role':'user','content':[{'type':'input_text','text':prompt}]})
# Always preserve the original bytes; only WAV metadata supplies sample rate.
uri_mime='audio/mp3' if parameters['format']=='mp3' else mime
data_uri='data:'+uri_mime+';base64,'+base64.b64encode(audio.file_content).decode()
if len(data_uri)>10_000_000:
raise ValueError('Bailian inline audio exceeds 10 MB after Base64 encoding')
messages.append({'role':'user','content':[{'type':'input_audio','input_audio':{
'data':data_uri}}]})
return AudioTranscriptionRequestData(data={'model':MODEL,'input':{'messages':messages},
'parameters':parameters})
def transform_audio_transcription_response(self,raw_response):
body=raw_response.json()
text=body.get('output',{}).get('text',body.get('text'))
if not isinstance(text,str):raise ValueError('Bailian returned no transcript')
result=TranscriptionResponse(text=text)
duration=(body.get('usage') or {}).get('duration')
if duration is not None:
result.usage=TranscriptionUsageDurationObject(type='duration',seconds=duration)
result['duration']=duration
return result
def install():
import litellm
from litellm.utils import ProviderConfigManager
original=ProviderConfigManager.get_provider_audio_transcription_config
if getattr(original,'_bailian_asr_adapter',False):return
def select(model,provider):
if provider==litellm.LlmProviders.DASHSCOPE and model==MODEL:
return BailianTranscriptionConfig()
return original(model,provider)
select._bailian_asr_adapter=True
ProviderConfigManager.get_provider_audio_transcription_config=staticmethod(select)
+75
View File
@@ -0,0 +1,75 @@
name: tools-litellm
services:
postgres:
image: postgres:17-alpine@sha256:b0f9560a2de083e2cc7382e75f808c7381a32852a7ec49117deedb300e552b24
container_name: tools-litellm-postgres
restart: unless-stopped
environment:
POSTGRES_DB: litellm
POSTGRES_USER: litellm
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?required}
TZ: Asia/Shanghai
volumes:
- database:/var/lib/postgresql/data
healthcheck:
test: [CMD-SHELL, 'pg_isready -U litellm -d litellm']
interval: 10s
timeout: 5s
retries: 10
logging:
driver: json-file
options: {max-size: 10m, max-file: "3"}
litellm:
image: ghcr.io/berriai/litellm:main-stable@sha256:3d88bd757134a00e8c00237b85a021ab1f6d54abb9c1df750f411dbe7256a500
container_name: tools-litellm
restart: unless-stopped
depends_on:
postgres:
condition: service_healthy
ports:
- '${LISTEN_ADDRESS:-192.168.50.49}:${LITELLM_PORT:-4000}:4000'
environment:
TZ: Asia/Shanghai
DATABASE_URL: postgresql://litellm:${POSTGRES_PASSWORD:?required}@postgres:5432/litellm
LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY:?required}
LITELLM_SALT_KEY: ${LITELLM_SALT_KEY:?required}
ZENMUX_PAYG_API_KEY: ${ZENMUX_PAYG_API_KEY:-}
OPENROUTER_API_KEY: ${OPENROUTER_API_KEY:-}
OPENAI_API_KEY: ${OPENAI_API_KEY:-}
BAILIAN_CN_API_KEY: ${BAILIAN_CN_API_KEY:-}
BAILIAN_CN_API_BASE: ${BAILIAN_CN_API_BASE:-https://ws-v68kvjnmhkiejaxo.cn-beijing.maas.aliyuncs.com/api/v1}
GOOGLE_AI_STUDIO_API_KEY: ${GOOGLE_AI_STUDIO_API_KEY:-}
VERTEXAI_ENABLED: ${VERTEXAI_ENABLED:-false}
VERTEXAI_PROJECT: ${VERTEXAI_PROJECT:-}
VERTEXAI_LOCATION: ${VERTEXAI_LOCATION:-global}
GOOGLE_APPLICATION_CREDENTIALS: /tokens/google/adc.json
CHATGPT_TOKEN_DIR: /tokens/chatgpt
XAI_OAUTH_TOKEN_DIR: /tokens/xai
LITELLM_LOG: WARNING
volumes:
- ./models.yaml:/etc/litellm/models.yaml:ro
- ./render_config.py:/etc/litellm/render_config.py:ro
- ./provider_policy.py:/etc/litellm/provider_policy.py:ro
- ./personal_transcription.py:/etc/litellm/personal_transcription.py:ro
- ./bailian_transcription.py:/etc/litellm/bailian_transcription.py:ro
- /opt/litellm/personal-vocabulary.json:/etc/litellm/personal-vocabulary.json:ro
- oauth:/tokens
entrypoint: [/bin/sh, -c]
command:
- 'python /etc/litellm/render_config.py && exec python /etc/litellm/personal_transcription.py --config /tmp/litellm.yaml --host 0.0.0.0 --port 4000 --num_workers 1'
healthcheck:
test: [CMD, python, -c, 'import urllib.request; urllib.request.urlopen("http://127.0.0.1:4000/health/liveliness", timeout=5)']
interval: 30s
timeout: 10s
retries: 6
start_period: 120s
logging:
driver: json-file
options: {max-size: 10m, max-file: "3"}
volumes:
database:
name: tools-litellm-database
oauth:
name: tools-litellm-oauth

Some files were not shown because too many files have changed in this diff Show More