同步香港 LiteLLM 镜像与多提供商路由

servers/bwh-hk-cmi/litellm/README.md

@@ -1,4 +1,4 @@
-# Hong Kong CMI LiteLLM Vertex AI Relay
+# 香港 CMI LiteLLM 多提供商代理
 
 - 公网 OpenAI-compatible 入口为 `https://api.opfo.xyz/v1`，由 Caddy 终止 TLS 后通过外部 Docker 网络 `hk-web` 反向代理到 `litellm-api:1049`。
 - LiteLLM 不使用 host network，也不向宿主机发布端口；容器内监听 `0.0.0.0:1049` 仅供 `hk-web` 上的 Caddy 访问。
@@ -8,4 +8,11 @@
 - 对外模型别名包括 `gemini-3.6-flash`、`gemini-3.5-flash-lite`、
   `google/gemini-embedding-2` 和 `google/gemini-3.8-flash`；后两者仍通过
   Vertex AI ADC 路由。
-- 后续 Vertex Anthropic 模型在 `config.yaml` 增加独立模型项。
+- 2026-10-25 同步 tools-pve 已验证的 LiteLLM 1.104.2 固定镜像，以及 `models.yaml` 中的提供商映射。`latest` 是维护的别名，不是自动发现所有厂商的新模型。
+- `provider_routes.py` 是同日 tools-pve 的路由生成器快照；更新提供商逻辑时应同步比对两端。HK 的 `render_config.py` 删除数据库管理、OAuth 和个人词库路线，保留现有主密钥认证。
+- Arcane Project 本地变量增加 `ZENMUX_SUBSCRIPTION_API_KEY`、`ZENMUX_PAYG_API_KEY`、`OPENROUTER_API_KEY`、`GOOGLE_AI_STUDIO_API_KEY`。只有相应凭据存在时才生成该组路由；真实值不进入 Git。
+- Claude 使用 ZenMux 原生 Anthropic Messages 路径，订阅与 `-payg` 显式分开；开源模型使用 ZenMux 订阅优先、同模型 OpenRouter 后备。不自动回退至 ZenMux PAYG。
+- Google AI Studio 的 `gemini-transcribe` 是普通转写路线，不含个人热词；个人词库、OAuth 会话和会议处理程序没有复制至 HK。
+- Vertex 继续使用 HK 原有 ADC。旧 `gemini-3.6-flash` 别名保留，但指向当前映射的 `gemini-3.8-flash`；其余旧别名保留。
+- 当前实际配置由启动脚本生成至 `/tmp/litellm.yaml`；旧 `config.yaml` 仅作为历史参考，不再挂载。
+- 升级只重建 LiteLLM，不变更 Caddy、HAProxy、其他项目及 ADC 卷。失败时回退本次 Git 提交并同步，再按需要 Redeploy，禁止删除卷。

servers/bwh-hk-cmi/litellm/compose.yaml

@@ -14,7 +14,7 @@ services:
         printf '%s' "$${GOOGLE_ADC_B64}" | base64 -d > /credentials/application_default_credentials.json
 
   litellm:
-    image: ghcr.io/berriai/litellm:main-stable@sha256:625981c83410a3ea68eb0697590a57ec1d764d634514d54fa5db0591077ee839
+    image: ghcr.io/berriai/litellm:main-stable@sha256:3d88bd757134a00e8c00237b85a021ab1f6d54abb9c1df750f411dbe7256a500
     restart: unless-stopped
     depends_on:
       adc-init:
@@ -28,10 +28,19 @@ services:
       VERTEXAI_PROJECT: ${VERTEXAI_PROJECT}
       VERTEXAI_LOCATION: ${VERTEXAI_LOCATION:-global}
       LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY}
+      VERTEXAI_ENABLED: "true"
+      ZENMUX_SUBSCRIPTION_API_KEY: ${ZENMUX_SUBSCRIPTION_API_KEY:-}
+      ZENMUX_PAYG_API_KEY: ${ZENMUX_PAYG_API_KEY:-}
+      OPENROUTER_API_KEY: ${OPENROUTER_API_KEY:-}
+      GOOGLE_AI_STUDIO_API_KEY: ${GOOGLE_AI_STUDIO_API_KEY:-}
     volumes:
-      - ./config.yaml:/app/config.yaml:ro
+      - ./models.yaml:/etc/litellm/models.yaml:ro
+      - ./provider_routes.py:/etc/litellm/provider_routes.py:ro
+      - ./render_config.py:/etc/litellm/render_config.py:ro
       - google-adc:/credentials:ro
-    command: ["--config", "/app/config.yaml", "--host", "0.0.0.0", "--port", "${LITELLM_PORT:-1049}"]
+    entrypoint: ["/bin/sh", "-c"]
+    command:
+      - python /etc/litellm/render_config.py && exec litellm --config /tmp/litellm.yaml --host 0.0.0.0 --port ${LITELLM_PORT:-1049} --num_workers 1
     logging:
       driver: json-file
       options:

servers/bwh-hk-cmi/litellm/models.yaml

@@ -0,0 +1,43 @@
+# 除 Google 自带 latest 别名外，本项目映射不会自行追踪上游发布。
+zenmux:
+  claude-sonnet-latest: anthropic/claude-sonnet-5.5
+  claude-opus-latest: anthropic/claude-opus-5.5
+  claude-haiku-latest: anthropic/claude-haiku-5.5
+openrouter:
+  deepseek-flash-latest: deepseek/deepseek-v4.1-flash
+  deepseek-pro-latest: deepseek/deepseek-v4-pro
+  glm-latest: z-ai/glm-5.3-prime
+  glm-flash-latest: z-ai/glm-5.3-flash
+  kimi-latest: moonshotai/kimi-k3
+  mimo-flash-latest: xiaomi/mimo-v2.6-flash
+# These existing public aliases are subscription-first, not load-balanced.
+zenmux_open_models:
+  deepseek-flash-latest: deepseek/deepseek-v4.1-flash
+  deepseek-pro-latest: deepseek/deepseek-v4-pro
+  glm-latest: z-ai/glm-5.3-prime
+  glm-flash-latest: z-ai/glm-5.3-flash
+  kimi-latest: moonshotai/kimi-k3
+  mimo-flash-latest: xiaomi/mimo-v2.6-flash
+zenmux_chat:
+  gemini-flash-zenmux: google/gemini-3.8-flash
+google_ai_studio:
+  gemini-flash-latest: gemini-flash-latest
+  gemini-flash-lite-latest: gemini-flash-lite-latest
+  gemini-pro-latest: gemini-pro-latest
+  gemini-embedding-2-studio: gemini-embedding-2
+google_vertex:
+  gemini-flash-vertex: gemini-3.8-flash
+  gemini-flash-lite-vertex: gemini-3.5-flash-lite
+  gemini-pro-vertex: gemini-3.1-pro-preview
+  google-rerank-default: semantic-ranker-default@latest
+  google-rerank-fast: semantic-ranker-fast@latest
+# 已有 Hindsight 部署用过的候选模型；本代理仍需独立授权与请求验收。
+# renderer 在实际 token 出现前排除这些路线。
+chatgpt:
+  gpt-luna-latest: gpt-6-luna
+  gpt-sol-latest: gpt-6.1-sol
+  gpt-work-sol: gpt-6-sol
+  gpt-review-5.5: gpt-5.5
+  gpt-astra-latest: gpt-6-astra
+xai:
+  grok-latest: grok-4.7

servers/bwh-hk-cmi/litellm/provider_routes.py

@@ -0,0 +1,96 @@
+"""Render only credential-ready routes; credentials remain environment references."""
+import os
+import json
+from pathlib import Path
+
+import yaml
+
+
+def oauth_ready(path):
+    try:
+        record = json.loads(path.read_text())
+        return isinstance(record, dict) and bool(record.get('access_token'))
+    except (OSError, ValueError):
+        return False
+
+
+def build_config(catalog, env, exists):
+    routes = []
+    fallbacks = []
+
+    def add(alias, model, key=None, **params):
+        if key:
+            params['api_key'] = 'os.environ/' + key
+        routes.append({'model_name': alias, 'litellm_params': {'model': model, **params}})
+
+    for key, suffix in [('ZENMUX_SUBSCRIPTION_API_KEY', ''), ('ZENMUX_PAYG_API_KEY', '-payg')]:
+        if env.get(key):
+            for alias, model in catalog['zenmux'].items():
+                # Strip only the LiteLLM provider prefix, retaining ZenMux's vendor prefix.
+                add(alias + suffix, 'anthropic/' + model, key,
+                    api_base='https://zenmux.ai/api/anthropic')
+    if env.get('ZENMUX_SUBSCRIPTION_API_KEY'):
+        for alias, model in catalog.get('zenmux_chat', {}).items():
+            add(alias, 'openai/' + model, 'ZENMUX_SUBSCRIPTION_API_KEY',
+                api_base='https://zenmux.ai/api/v1')
+    if env.get('OPENROUTER_API_KEY'):
+        for alias, model in catalog['openrouter'].items():
+            primary = catalog.get('zenmux_open_models', {}).get(alias)
+            if primary and env.get('ZENMUX_SUBSCRIPTION_API_KEY'):
+                if primary != model:
+                    raise ValueError('Cross-provider model mismatch: ' + alias)
+                add(alias, 'openai/' + primary, 'ZENMUX_SUBSCRIPTION_API_KEY',
+                    api_base='https://zenmux.ai/api/v1', max_retries=0)
+                routes[-1]['model_info'] = {'cooldown_time': 60,
+                    'allowed_fails_policy': {'RateLimitErrorAllowedFails': 0,
+                        'AuthenticationErrorAllowedFails': 0,
+                        'InternalServerErrorAllowedFails': 0,
+                        'ServiceUnavailableErrorAllowedFails': 0,
+                        'TimeoutErrorAllowedFails': 0}}
+                add(alias + '-openrouter', 'openrouter/' + model, 'OPENROUTER_API_KEY')
+                fallbacks.append({alias: [alias + '-openrouter']})
+            else:
+                add(alias, 'openrouter/' + model, 'OPENROUTER_API_KEY')
+    elif env.get('ZENMUX_SUBSCRIPTION_API_KEY'):
+        for alias, model in catalog.get('zenmux_open_models', {}).items():
+            add(alias, 'openai/' + model, 'ZENMUX_SUBSCRIPTION_API_KEY',
+                api_base='https://zenmux.ai/api/v1', max_retries=0)
+    if env.get('GOOGLE_AI_STUDIO_API_KEY'):
+        add('gemini-transcribe-personal', 'gemini/gemini-3.5-transcribe',
+            'GOOGLE_AI_STUDIO_API_KEY', personal_vocabulary=True)
+        for alias, model in catalog['google_ai_studio'].items():
+            add(alias, 'gemini/' + model, 'GOOGLE_AI_STUDIO_API_KEY')
+    if env.get('VERTEXAI_ENABLED', '').lower() == 'true' and env.get('VERTEXAI_PROJECT'):
+        for alias, model in catalog.get('google_vertex', {}).items():
+            add(alias, 'vertex_ai/' + model,
+                vertex_project='os.environ/VERTEXAI_PROJECT',
+                vertex_location='os.environ/VERTEXAI_LOCATION')
+    for provider, token_var, default_dir in [
+        ('chatgpt', 'CHATGPT_TOKEN_DIR', '/tokens/chatgpt'),
+        ('xai', 'XAI_OAUTH_TOKEN_DIR', '/tokens/xai'),
+    ]:
+        if exists(Path(env.get(token_var, default_dir)) / 'auth.json'):
+            for alias, model in catalog[provider].items():
+                extra = {'use_xai_oauth': True} if provider == 'xai' else {}
+                add(alias, provider + '/' + model, **extra)
+    return {
+        'model_list': routes,
+        'general_settings': {
+            'master_key': 'os.environ/LITELLM_MASTER_KEY',
+            'database_url': 'os.environ/DATABASE_URL',
+            'store_model_in_db': False,
+            'disable_env_credential_login': True,
+        },
+        'litellm_settings': {'drop_params': False, 'set_verbose': False},
+        # Only explicitly approved open-model routes have cross-provider fallbacks.
+        'router_settings': {'num_retries': 1, 'timeout': 180, 'fallbacks': fallbacks},
+    }
+
+
+if __name__ == '__main__':
+    catalog = yaml.safe_load(Path('/etc/litellm/models.yaml').read_text())
+    config = build_config(catalog, os.environ, oauth_ready)
+    output = Path('/tmp/litellm.yaml')
+    output.write_text(yaml.safe_dump(config, allow_unicode=True, sort_keys=False))
+    output.chmod(0o600)
+    print('Configured routes:', ', '.join(route['model_name'] for route in config['model_list']))

servers/bwh-hk-cmi/litellm/render_config.py

@@ -0,0 +1,41 @@
+"""HK reuses tools-pve provider routing without OAuth, database, or private vocabulary."""
+import os
+from pathlib import Path
+
+import yaml
+
+from provider_routes import build_config
+
+
+def build_hk_config(catalog, env):
+    config = build_config(catalog, env, lambda path: False)
+    config['general_settings'] = {'master_key': 'os.environ/LITELLM_MASTER_KEY'}
+    routes = config['model_list']
+    routes[:] = [route for route in routes if route['model_name'] != 'gemini-transcribe-personal']
+    if env.get('GOOGLE_AI_STUDIO_API_KEY'):
+        routes.append({'model_name': 'gemini-transcribe', 'litellm_params': {
+            'model': 'gemini/gemini-3.5-transcribe',
+            'api_key': 'os.environ/GOOGLE_AI_STUDIO_API_KEY'}})
+    if env.get('VERTEXAI_ENABLED', '').lower() == 'true' and env.get('VERTEXAI_PROJECT'):
+        # Keep old client aliases; the old 3.6 alias now follows the verified Flash map.
+        legacy = {
+            'gemini-3.6-flash': catalog['google_vertex']['gemini-flash-vertex'],
+            'gemini-3.5-flash-lite': catalog['google_vertex']['gemini-flash-lite-vertex'],
+            'google/gemini-embedding-2': 'gemini-embedding-2',
+            'google/gemini-3.8-flash': catalog['google_vertex']['gemini-flash-vertex'],
+        }
+        for alias, model in legacy.items():
+            routes.append({'model_name': alias, 'litellm_params': {
+                'model': 'vertex_ai/' + model,
+                'vertex_project': 'os.environ/VERTEXAI_PROJECT',
+                'vertex_location': 'os.environ/VERTEXAI_LOCATION'}})
+    return config
+
+
+if __name__ == '__main__':
+    catalog = yaml.safe_load(Path('/etc/litellm/models.yaml').read_text())
+    config = build_hk_config(catalog, os.environ)
+    output = Path('/tmp/litellm.yaml')
+    output.write_text(yaml.safe_dump(config, allow_unicode=True, sort_keys=False))
+    output.chmod(0o600)
+    print('Configured routes:', ', '.join(route['model_name'] for route in config['model_list']))

servers/bwh-hk-cmi/litellm/test_config.py

@@ -0,0 +1,36 @@
+import unittest
+from pathlib import Path
+
+import yaml
+
+from render_config import build_hk_config
+
+
+class RoutingTests(unittest.TestCase):
+    def test_all_providers(self):
+        catalog = yaml.safe_load(Path(__file__).with_name('models.yaml').read_text())
+        env = {key: 'synthetic' for key in ['ZENMUX_SUBSCRIPTION_API_KEY',
+            'ZENMUX_PAYG_API_KEY', 'OPENROUTER_API_KEY', 'GOOGLE_AI_STUDIO_API_KEY',
+            'VERTEXAI_PROJECT']}
+        env['VERTEXAI_ENABLED'] = 'true'
+        result = build_hk_config(catalog, env)
+        routes = {r['model_name']: r['litellm_params'] for r in result['model_list']}
+        self.assertEqual(len(routes), len(result['model_list']))
+        self.assertNotIn('gemini-transcribe-personal', routes)
+        self.assertNotIn('grok-latest', routes)
+        self.assertNotIn('gpt-sol-latest', routes)
+        self.assertEqual(routes['gemini-transcribe']['model'], 'gemini/gemini-3.5-transcribe')
+        self.assertEqual(routes['gemini-3.6-flash']['model'], 'vertex_ai/gemini-3.8-flash')
+        self.assertEqual(len(result['router_settings']['fallbacks']), 6)
+        self.assertEqual(result['general_settings'], {'master_key': 'os.environ/LITELLM_MASTER_KEY'})
+        self.assertNotIn('synthetic', yaml.safe_dump(result))
+
+    def test_missing_credentials(self):
+        catalog = yaml.safe_load(Path(__file__).with_name('models.yaml').read_text())
+        result = build_hk_config(catalog, {})
+        self.assertEqual(result['model_list'], [])
+        self.assertEqual(result['router_settings']['fallbacks'], [])
+
+
+if __name__ == '__main__':
+    unittest.main()